Sep 10 19:15:27.328682 ip-10-0-129-114 systemd[1]: Starting Kubernetes Kubelet...
Sep 10 19:15:27.690819 ip-10-0-129-114 kubenswrapper[2575]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 10 19:15:27.690819 ip-10-0-129-114 kubenswrapper[2575]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 10 19:15:27.690819 ip-10-0-129-114 kubenswrapper[2575]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 10 19:15:27.690819 ip-10-0-129-114 kubenswrapper[2575]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 10 19:15:27.691449 ip-10-0-129-114 kubenswrapper[2575]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 10 19:15:27.692430 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.692330    2575 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 10 19:15:27.696113 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.696098    2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 19:15:27.696113 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.696112    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 19:15:27.696174 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.696115    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 19:15:27.696174 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.696119    2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 19:15:27.696174 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.696122    2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 19:15:27.696174 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.696125    2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 19:15:27.696174 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.696129    2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 19:15:27.696174 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.696132    2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 19:15:27.696174 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.696136    2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 19:15:27.696174 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.696139    2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 19:15:27.696174 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.696142    2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 19:15:27.696174 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.696144    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 19:15:27.696174 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.696147    2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 19:15:27.696174 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.696150    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 19:15:27.696174 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.696153    2575 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 19:15:27.696174 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.696156    2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 19:15:27.696174 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.696160    2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 19:15:27.696174 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.696164    2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 19:15:27.696174 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.696167    2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 19:15:27.696174 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.696169    2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 19:15:27.696174 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.696172    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 19:15:27.696633 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.696179    2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 19:15:27.696633 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.696182    2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 19:15:27.696633 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.696185    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 19:15:27.696633 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.696188    2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 19:15:27.696633 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.696190    2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 19:15:27.696633 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.696193    2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 19:15:27.696633 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.696195    2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 19:15:27.696633 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.696198    2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 19:15:27.696633 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.696200    2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 19:15:27.696633 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.696203    2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 19:15:27.696633 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.696206    2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 19:15:27.696633 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.696208    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 19:15:27.696633 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.696211    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 19:15:27.696633 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.696214    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 19:15:27.696633 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.696218    2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 19:15:27.696633 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.696220    2575 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 19:15:27.696633 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.696223    2575 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 19:15:27.696633 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.696225    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 19:15:27.696633 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.696227    2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 19:15:27.696633 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.696230    2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 19:15:27.697177 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.696233    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 19:15:27.697177 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.696235    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 19:15:27.697177 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.696239    2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 19:15:27.697177 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.696242    2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 19:15:27.697177 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.696245    2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 19:15:27.697177 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.696247    2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 19:15:27.697177 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.696250    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 19:15:27.697177 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.696252    2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 19:15:27.697177 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.696959    2575 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 19:15:27.697177 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.696965    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 19:15:27.697177 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.696968    2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 19:15:27.697177 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.696971    2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 19:15:27.697177 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.696974    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 19:15:27.697177 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.696978    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 19:15:27.697177 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.696981    2575 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 19:15:27.697177 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.696984    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 19:15:27.697177 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.696987    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 19:15:27.697177 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.696989    2575 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 19:15:27.697177 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.696992    2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 19:15:27.697930 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.696994    2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 19:15:27.697930 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.696997    2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 19:15:27.697930 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.696999    2575 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 19:15:27.697930 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.697001    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 19:15:27.697930 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.697004    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 19:15:27.697930 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.697006    2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 19:15:27.697930 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.697009    2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 19:15:27.697930 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.697013    2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 19:15:27.697930 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.697016    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 19:15:27.697930 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.697019    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 19:15:27.697930 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.697022    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 19:15:27.697930 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.697024    2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 19:15:27.697930 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.697028    2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 19:15:27.697930 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.697031    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 19:15:27.697930 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.697035    2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 19:15:27.697930 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.697038    2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 19:15:27.697930 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.697041    2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 19:15:27.697930 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.697043    2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 19:15:27.697930 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.697046    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 19:15:27.698474 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.697049    2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 19:15:27.698474 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.697051    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 19:15:27.698474 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.697054    2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 19:15:27.698474 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.697056    2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 19:15:27.698474 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.697059    2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 19:15:27.698474 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.697061    2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 19:15:27.698474 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.697064    2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 19:15:27.698474 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.697066    2575 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 19:15:27.698474 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.697098    2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 19:15:27.698474 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.697133    2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 19:15:27.698474 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.697137    2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 19:15:27.698474 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.697139    2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 19:15:27.698474 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.697143    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 19:15:27.698474 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.697226    2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 19:15:27.698474 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.697231    2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 19:15:27.698474 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.697236    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 19:15:27.698474 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.697240    2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 19:15:27.698474 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.697244    2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 19:15:27.698474 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.697249    2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 19:15:27.698474 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.697254    2575 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 19:15:27.699842 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.699827    2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 19:15:27.699842 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.699842    2575 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 19:15:27.699904 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.699847    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 19:15:27.699904 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.699850    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 19:15:27.699904 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.699854    2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 19:15:27.699904 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.699857    2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 19:15:27.699904 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.699861    2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 19:15:27.699904 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.699864    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 19:15:27.699904 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.699867    2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 19:15:27.699904 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.699869    2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 19:15:27.699904 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.699872    2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 19:15:27.699904 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.699875    2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 19:15:27.699904 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.699877    2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 19:15:27.699904 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.699880    2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 19:15:27.699904 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.699883    2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 19:15:27.699904 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.699885    2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 19:15:27.699904 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.699888    2575 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 19:15:27.699904 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.699891    2575 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 19:15:27.699904 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.699893    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 19:15:27.699904 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.699896    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 19:15:27.699904 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.699899    2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 19:15:27.699904 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.699909    2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 19:15:27.700494 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.699913    2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 19:15:27.700494 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.699915    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 19:15:27.700494 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.699918    2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 19:15:27.700494 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.699921    2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 19:15:27.700494 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.699924    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 19:15:27.700494 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.699926    2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 19:15:27.700494 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.699929    2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 19:15:27.700494 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.699931    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 19:15:27.700494 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.699934    2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 19:15:27.700494 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.699936    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 19:15:27.700494 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.699939    2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 19:15:27.700494 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.699941    2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 19:15:27.700494 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.699944    2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 19:15:27.700494 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.699947    2575 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 19:15:27.700494 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.699949    2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 19:15:27.700494 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.699952    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 19:15:27.700494 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.699954    2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 19:15:27.700494 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.699956    2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 19:15:27.700494 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.699959    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 19:15:27.700494 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.699962    2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 19:15:27.701023 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.699964    2575 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 19:15:27.701023 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.699967    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 19:15:27.701023 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.699970    2575 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 19:15:27.701023 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.699972    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 19:15:27.701023 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.699974    2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 19:15:27.701023 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.699977    2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 19:15:27.701023 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.699979    2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 19:15:27.701023 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.699982    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 19:15:27.701023 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.699984    2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 19:15:27.701023 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.699987    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 19:15:27.701023 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.699989    2575 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 19:15:27.701023 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.699992    2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 19:15:27.701023 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.699995    2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 19:15:27.701023 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.699999    2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 19:15:27.701023 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700003    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 19:15:27.701023 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700006    2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 19:15:27.701023 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700009    2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 19:15:27.701023 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700012    2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 19:15:27.701023 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700015    2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 19:15:27.701023 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700018    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 19:15:27.701525 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700020    2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 19:15:27.701525 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700025    2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 19:15:27.701525 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700028    2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 19:15:27.701525 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700031    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 19:15:27.701525 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700033    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 19:15:27.701525 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700036    2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 19:15:27.701525 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700038    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 19:15:27.701525 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700041    2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 19:15:27.701525 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700044    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 19:15:27.701525 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700047    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 19:15:27.701525 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700051    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 19:15:27.701525 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700053    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 19:15:27.701525 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700056    2575 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 19:15:27.701525 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700058    2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 19:15:27.701525 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700061    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 19:15:27.701525 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700064    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 19:15:27.701525 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700066    2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 19:15:27.701525 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700069    2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 19:15:27.701525 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700071    2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 19:15:27.701985 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700075    2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 19:15:27.701985 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700080    2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 19:15:27.701985 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700087    2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 19:15:27.701985 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700090    2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 19:15:27.701985 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700093    2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 19:15:27.701985 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700097    2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 19:15:27.701985 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700100    2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 19:15:27.701985 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700103    2575 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 19:15:27.701985 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700106    2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 19:15:27.701985 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700108    2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 19:15:27.701985 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700111    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 19:15:27.701985 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700114    2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 19:15:27.701985 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700116    2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 19:15:27.701985 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700119    2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 19:15:27.701985 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700121    2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 19:15:27.701985 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700124    2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 19:15:27.701985 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700126    2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 19:15:27.701985 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700129    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 19:15:27.701985 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700510    2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 19:15:27.702466 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700516    2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 19:15:27.702466 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700519    2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 19:15:27.702466 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700521    2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 19:15:27.702466 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700525    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 19:15:27.702466 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700527    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 19:15:27.702466 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700532    2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 19:15:27.702466 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700535    2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 19:15:27.702466 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700538    2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 19:15:27.702466 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700541    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 19:15:27.702466 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700544    2575 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 19:15:27.702466 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700547    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 19:15:27.702466 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700550    2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 19:15:27.702466 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700553    2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 19:15:27.702466 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700555    2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 19:15:27.702466 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700559    2575 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 19:15:27.702466 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700562    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 19:15:27.702466 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700564    2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 19:15:27.702466 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700567    2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 19:15:27.702466 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700571    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 19:15:27.702466 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700574    2575 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 19:15:27.702965 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700577    2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 19:15:27.702965 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700579    2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 19:15:27.702965 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700582    2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 19:15:27.702965 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700584    2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 19:15:27.702965 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700587    2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 19:15:27.702965 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700590    2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 19:15:27.702965 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700593    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 19:15:27.702965 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700595    2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 19:15:27.702965 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700597    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 19:15:27.702965 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700600    2575 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 19:15:27.702965 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700602    2575 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 19:15:27.702965 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700605    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 19:15:27.702965 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700607    2575 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 19:15:27.702965 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700611    2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 19:15:27.702965 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700614    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 19:15:27.702965 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700617    2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 19:15:27.702965 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700626    2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 19:15:27.702965 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700628    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 19:15:27.702965 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700631    2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 19:15:27.702965 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700634    2575 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 19:15:27.703454 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700636    2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 19:15:27.703454 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700639    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 19:15:27.703454 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700641    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 19:15:27.703454 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700644    2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 19:15:27.703454 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700648    2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 19:15:27.703454 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700651    2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 19:15:27.703454 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700654    2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 19:15:27.703454 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700656    2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 19:15:27.703454 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700659    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 19:15:27.703454 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700661    2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 19:15:27.703454 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700664    2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 19:15:27.703454 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700668    2575 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 19:15:27.703454 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700671    2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 19:15:27.703454 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700673    2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 19:15:27.703454 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700676    2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 19:15:27.703454 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700678    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 19:15:27.703454 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700681    2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 19:15:27.703454 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700683    2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 19:15:27.703454 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700686    2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 19:15:27.703454 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700688    2575 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 19:15:27.703943 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700691    2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 19:15:27.703943 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700693    2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 19:15:27.703943 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700696    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 19:15:27.703943 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700698    2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 19:15:27.703943 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700701    2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 19:15:27.703943 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700703    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 19:15:27.703943 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700706    2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 19:15:27.703943 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700708    2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 19:15:27.703943 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700711    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 19:15:27.703943 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700714    2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 19:15:27.703943 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700716    2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 19:15:27.703943 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700719    2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 19:15:27.703943 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700721    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 19:15:27.703943 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700723    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 19:15:27.703943 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700726    2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 19:15:27.703943 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700729    2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 19:15:27.703943 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700733    2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 19:15:27.703943 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700735    2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 19:15:27.703943 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700738    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 19:15:27.704415 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700740    2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 19:15:27.704415 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700743    2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 19:15:27.704415 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700745    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 19:15:27.704415 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700748    2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 19:15:27.704415 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700751    2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 19:15:27.704415 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700754    2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 19:15:27.704415 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700756    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 19:15:27.704415 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700759    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 19:15:27.704415 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700761    2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 19:15:27.704415 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700764    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 19:15:27.704415 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700766    2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 19:15:27.704415 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700769    2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 19:15:27.704415 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700773    2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 19:15:27.704415 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700776    2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 19:15:27.704415 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700778    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 19:15:27.704415 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700780    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 19:15:27.704415 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700783    2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 19:15:27.704415 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700785    2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 19:15:27.704415 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.700788    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 19:15:27.704415 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.700861    2575 flags.go:64] FLAG: --address="0.0.0.0"
Sep 10 19:15:27.704894 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.700906    2575 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 10 19:15:27.704894 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.700935    2575 flags.go:64] FLAG: --anonymous-auth="true"
Sep 10 19:15:27.704894 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.700940    2575 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 10 19:15:27.704894 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.700945    2575 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 10 19:15:27.704894 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.700948    2575 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 10 19:15:27.704894 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.700952    2575 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 10 19:15:27.704894 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.700956    2575 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 10 19:15:27.704894 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.700959    2575 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 10 19:15:27.704894 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.700962    2575 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 10 19:15:27.704894 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.700967    2575 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 10 19:15:27.704894 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.700971    2575 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 10 19:15:27.704894 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.700974    2575 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 10 19:15:27.704894 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.700976    2575 flags.go:64] FLAG: --cgroup-root=""
Sep 10 19:15:27.704894 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.700979    2575 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 10 19:15:27.704894 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.700982    2575 flags.go:64] FLAG: --client-ca-file=""
Sep 10 19:15:27.704894 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.700989    2575 flags.go:64] FLAG: --cloud-provider="external"
Sep 10 19:15:27.704894 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.700992    2575 flags.go:64] FLAG: --cluster-dns="[]"
Sep 10 19:15:27.704894 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.700996    2575 flags.go:64] FLAG: --cluster-domain=""
Sep 10 19:15:27.704894 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.700999    2575 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 10 19:15:27.704894 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.701002    2575 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 10 19:15:27.704894 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.701005    2575 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 10 19:15:27.704894 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.701008    2575 flags.go:64] FLAG: --container-log-max-files="5"
Sep 10 19:15:27.704894 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.701012    2575 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 10 19:15:27.704894 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.701014    2575 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 10 19:15:27.705493 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.701018    2575 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 10 19:15:27.705493 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.701021    2575 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 10 19:15:27.705493 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.701023    2575 flags.go:64] FLAG: --contention-profiling="false"
Sep 10 19:15:27.705493 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.701026    2575 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 10 19:15:27.705493 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.701029    2575 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 10 19:15:27.705493 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.701033    2575 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 10 19:15:27.705493 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.701036    2575 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 10 19:15:27.705493 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.701040    2575 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 10 19:15:27.705493 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.701043    2575 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 10 19:15:27.705493 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.701045    2575 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 10 19:15:27.705493 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.701048    2575 flags.go:64] FLAG: --enable-load-reader="false"
Sep 10 19:15:27.705493 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.701051    2575 flags.go:64] FLAG: --enable-server="true"
Sep 10 19:15:27.705493 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.701054    2575 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 10 19:15:27.705493 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.701059    2575 flags.go:64] FLAG: --event-burst="100"
Sep 10 19:15:27.705493 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.701062    2575 flags.go:64] FLAG: --event-qps="50"
Sep 10 19:15:27.705493 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.701065    2575 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 10 19:15:27.705493 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.701068    2575 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 10 19:15:27.705493 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.701071    2575 flags.go:64] FLAG: --eviction-hard=""
Sep 10 19:15:27.705493 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.701075    2575 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 10 19:15:27.705493 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.701078    2575 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 10 19:15:27.705493 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.701081    2575 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 10 19:15:27.705493 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.701084    2575 flags.go:64] FLAG: --eviction-soft=""
Sep 10 19:15:27.705493 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.701086    2575 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 10 19:15:27.705493 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.701090    2575 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 10 19:15:27.705493 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.701093    2575 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 10 19:15:27.706155 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.701096    2575 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 10 19:15:27.706155 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.701099    2575 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 10 19:15:27.706155 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.701102    2575 flags.go:64] FLAG: --fail-swap-on="true"
Sep 10 19:15:27.706155 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.701104    2575 flags.go:64] FLAG: --feature-gates=""
Sep 10 19:15:27.706155 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.701108    2575 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 10 19:15:27.706155 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.701110    2575 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 10 19:15:27.706155 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.701113    2575 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 10 19:15:27.706155 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.701116    2575 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 10 19:15:27.706155 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.701119    2575 flags.go:64] FLAG: --healthz-port="10248"
Sep 10 19:15:27.706155 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.701122    2575 flags.go:64] FLAG: --help="false"
Sep 10 19:15:27.706155 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.701125    2575 flags.go:64] FLAG: --hostname-override="ip-10-0-129-114.ec2.internal"
Sep 10 19:15:27.706155 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.701128    2575 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 10 19:15:27.706155 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.701131    2575 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 10 19:15:27.706155 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.701133    2575 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 10 19:15:27.706155 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.701136    2575 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 10 19:15:27.706155 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.701140    2575 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 10 19:15:27.706155 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.701143    2575 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 10 19:15:27.706155 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.701146    2575 flags.go:64] FLAG: --image-service-endpoint=""
Sep 10 19:15:27.706155 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.701148    2575 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 10 19:15:27.706155 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.701151    2575 flags.go:64] FLAG: --kube-api-burst="100"
Sep 10 19:15:27.706155 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.701155    2575 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 10 19:15:27.706155 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.701158    2575 flags.go:64] FLAG: --kube-api-qps="50"
Sep 10 19:15:27.706155 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.701160    2575 flags.go:64] FLAG: --kube-reserved=""
Sep 10 19:15:27.706155 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.701163    2575 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 10 19:15:27.706766 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.701166    2575 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 10 19:15:27.706766 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.701169    2575 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 10 19:15:27.706766 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.701172    2575 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 10 19:15:27.706766 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.701175    2575 flags.go:64] FLAG: --lock-file=""
Sep 10 19:15:27.706766 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.701178    2575 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 10 19:15:27.706766 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.701181    2575 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 10 19:15:27.706766 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.701185    2575 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 10 19:15:27.706766 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.701191    2575 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 10 19:15:27.706766 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.701193    2575 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 10 19:15:27.706766 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.701196    2575 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 10 19:15:27.706766 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.701199    2575 flags.go:64] FLAG: --logging-format="text"
Sep 10 19:15:27.706766 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.701202    2575 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 10 19:15:27.706766 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.701205    2575 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 10 19:15:27.706766 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.701208    2575 flags.go:64] FLAG: --manifest-url=""
Sep 10 19:15:27.706766 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.701211    2575 flags.go:64] FLAG: --manifest-url-header=""
Sep 10 19:15:27.706766 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.701215    2575 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 10 19:15:27.706766 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.701218    2575 flags.go:64] FLAG: --max-open-files="1000000"
Sep 10 19:15:27.706766 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.701222    2575 flags.go:64] FLAG: --max-pods="110"
Sep 10 19:15:27.706766 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.701224    2575 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 10 19:15:27.706766 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.701227    2575 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 10 19:15:27.706766 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.701230    2575 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 10 19:15:27.706766 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.701233    2575 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 10 19:15:27.706766 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.701236    2575 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 10 19:15:27.706766 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.701239    2575 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 10 19:15:27.706766 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.701242    2575 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 10 19:15:27.707350 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.701249    2575 flags.go:64] FLAG: --node-status-max-images="50"
Sep 10 19:15:27.707350 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.701251    2575 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 10 19:15:27.707350 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.701254    2575 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 10 19:15:27.707350 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.701257    2575 flags.go:64] FLAG: --pod-cidr=""
Sep 10 19:15:27.707350 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.701263    2575 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8ccbacc999dbf9abad8f4009240188312b683f36ced4763eed166c1500c164b1"
Sep 10 19:15:27.707350 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.701268    2575 flags.go:64] FLAG: --pod-manifest-path=""
Sep 10 19:15:27.707350 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.701271    2575 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 10 19:15:27.707350 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.701274    2575 flags.go:64] FLAG: --pods-per-core="0"
Sep 10 19:15:27.707350 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.701277    2575 flags.go:64] FLAG: --port="10250"
Sep 10 19:15:27.707350 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.701280    2575 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 10 19:15:27.707350 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.701283    2575 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0d6160050960b9d49"
Sep 10 19:15:27.707350 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.701286    2575 flags.go:64] FLAG: --qos-reserved=""
Sep 10 19:15:27.707350 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.701289    2575 flags.go:64] FLAG: --read-only-port="10255"
Sep 10 19:15:27.707350 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.701294    2575 flags.go:64] FLAG: --register-node="true"
Sep 10 19:15:27.707350 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.701297    2575 flags.go:64] FLAG: --register-with-taints=""
Sep 10 19:15:27.707350 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.701300    2575 flags.go:64] FLAG: --registry-burst="10"
Sep 10 19:15:27.707350 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.701303    2575 flags.go:64] FLAG: --registry-qps="5"
Sep 10 19:15:27.707350 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.701306    2575 flags.go:64] FLAG: --reserved-cpus=""
Sep 10 19:15:27.707350 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.701309    2575 flags.go:64] FLAG: --reserved-memory=""
Sep 10 19:15:27.707350 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.701312    2575 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 10 19:15:27.707350 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.701316    2575 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 10 19:15:27.707350 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.701319    2575 flags.go:64] FLAG: --rotate-certificates="false"
Sep 10 19:15:27.707350 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.701321    2575 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 10 19:15:27.707350 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.701324    2575 flags.go:64] FLAG: --runonce="false"
Sep 10 19:15:27.707350 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.701327    2575 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 10 19:15:27.707979 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.701330    2575 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 10 19:15:27.707979 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.701333    2575 flags.go:64] FLAG: --seccomp-default="false"
Sep 10 19:15:27.707979 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.701336    2575 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 10 19:15:27.707979 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.701339    2575 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 10 19:15:27.707979 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.701342    2575 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 10 19:15:27.707979 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.701345    2575 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 10 19:15:27.707979 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.701348    2575 flags.go:64] FLAG: --storage-driver-password="root"
Sep 10 19:15:27.707979 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.701351    2575 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 10 19:15:27.707979 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.701354    2575 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 10 19:15:27.707979 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.701356    2575 flags.go:64] FLAG: --storage-driver-user="root"
Sep 10 19:15:27.707979 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.701368    2575 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 10 19:15:27.707979 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.701385    2575 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 10 19:15:27.707979 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.701390    2575 flags.go:64] FLAG: --system-cgroups=""
Sep 10 19:15:27.707979 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.701393    2575 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 10 19:15:27.707979 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.701398    2575 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 10 19:15:27.707979 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.701401    2575 flags.go:64] FLAG: --tls-cert-file=""
Sep 10 19:15:27.707979 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.701405    2575 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 10 19:15:27.707979 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.701408    2575 flags.go:64] FLAG: --tls-min-version=""
Sep 10 19:15:27.707979 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.701411    2575 flags.go:64] FLAG: --tls-private-key-file=""
Sep 10 19:15:27.707979 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.701414    2575 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 10 19:15:27.707979 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.701418    2575 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 10 19:15:27.707979 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.701421    2575 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 10 19:15:27.707979 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.701424    2575 flags.go:64] FLAG: --v="2"
Sep 10 19:15:27.707979 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.701428    2575 flags.go:64] FLAG: --version="false"
Sep 10 19:15:27.707979 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.701431    2575 flags.go:64] FLAG: --vmodule=""
Sep 10 19:15:27.709024 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.701435    2575 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 10 19:15:27.709024 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.701438    2575 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 10 19:15:27.709024 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.701548    2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 19:15:27.709024 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.701552    2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 19:15:27.709024 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.701556    2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 19:15:27.709024 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.701558    2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 19:15:27.709024 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.701561    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 19:15:27.709024 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.701564    2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 19:15:27.709024 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.701566    2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 19:15:27.709024 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.701569    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 19:15:27.709024 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.701573    2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 19:15:27.709024 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.701575    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 19:15:27.709024 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.701578    2575 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 19:15:27.709024 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.701581    2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 19:15:27.709024 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.701585    2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 19:15:27.709024 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.701588    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 19:15:27.709024 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.701591    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 19:15:27.709024 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.701594    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 19:15:27.709024 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.701596    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 19:15:27.709850 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.701599    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 19:15:27.709850 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.701602    2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 19:15:27.709850 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.701606    2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 19:15:27.709850 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.701609    2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 19:15:27.709850 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.701612    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 19:15:27.709850 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.701614    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 19:15:27.709850 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.701617    2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 19:15:27.709850 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.701620    2575 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 19:15:27.709850 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.701625    2575 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 19:15:27.709850 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.701629    2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 19:15:27.709850 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.701633    2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 19:15:27.709850 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.701637    2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 19:15:27.709850 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.701641    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 19:15:27.709850 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.701644    2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 19:15:27.709850 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.701647    2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 19:15:27.709850 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.701649    2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 19:15:27.709850 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.701652    2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 19:15:27.709850 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.701655    2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 19:15:27.709850 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.701658    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 19:15:27.709850 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.701661    2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 19:15:27.710732 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.701664    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 19:15:27.710732 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.701666    2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 19:15:27.710732 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.701669    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 19:15:27.710732 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.701672    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 19:15:27.710732 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.701674    2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 19:15:27.710732 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.701677    2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 19:15:27.710732 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.701679    2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 19:15:27.710732 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.701682    2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 19:15:27.710732 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.701685    2575 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 19:15:27.710732 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.701688    2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 19:15:27.710732 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.701690    2575 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 19:15:27.710732 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.701693    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 19:15:27.710732 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.701695    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 19:15:27.710732 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.701698    2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 19:15:27.710732 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.701702    2575 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 19:15:27.710732 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.701705    2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 19:15:27.710732 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.701707    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 19:15:27.710732 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.701710    2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 19:15:27.710732 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.701712    2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 19:15:27.710732 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.701715    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 19:15:27.711666 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.701719    2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 19:15:27.711666 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.701722    2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 19:15:27.711666 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.701725    2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 19:15:27.711666 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.701727    2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 19:15:27.711666 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.701730    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 19:15:27.711666 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.701732    2575 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 19:15:27.711666 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.701735    2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 19:15:27.711666 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.701739    2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 19:15:27.711666 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.701742    2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 19:15:27.711666 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.701745    2575 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 19:15:27.711666 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.701747    2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 19:15:27.711666 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.701750    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 19:15:27.711666 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.701753    2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 19:15:27.711666 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.701755    2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 19:15:27.711666 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.701758    2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 19:15:27.711666 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.701761    2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 19:15:27.711666 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.701763    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 19:15:27.711666 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.701766    2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 19:15:27.711666 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.701768    2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 19:15:27.711666 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.701770    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 19:15:27.712204 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.701773    2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 19:15:27.712204 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.701775    2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 19:15:27.712204 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.701778    2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 19:15:27.712204 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.701780    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 19:15:27.712204 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.701783    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 19:15:27.712204 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.701785    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 19:15:27.712204 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.701790    2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 19:15:27.712204 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.701793    2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 19:15:27.712204 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.701795    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 19:15:27.712204 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.701798    2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 19:15:27.712204 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.701800    2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 19:15:27.712204 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.701803    2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 19:15:27.712204 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.701807    2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 19:15:27.712204 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.701809    2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 19:15:27.712204 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.701812    2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 19:15:27.712204 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.701815    2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 19:15:27.712204 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.701817    2575 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 19:15:27.712204 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.701820    2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 19:15:27.712204 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.701823    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 19:15:27.712204 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.701825    2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 19:15:27.713048 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.701828    2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 19:15:27.713048 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.701830    2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 19:15:27.713048 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.702396    2575 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 10 19:15:27.713048 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.710255    2575 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 10 19:15:27.713048 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.710271    2575 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 10 19:15:27.713048 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.710298    2575 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 10 19:15:27.713048 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.710307    2575 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 10 19:15:27.713048 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.710412    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 19:15:27.713048 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.710421    2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 19:15:27.713048 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.710426    2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 19:15:27.713048 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.710430    2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 19:15:27.713048 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.710435    2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 19:15:27.713048 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.710439    2575 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 19:15:27.713048 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.710444    2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 19:15:27.713048 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.710449    2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 19:15:27.713704 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.710454    2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 19:15:27.713704 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.710459    2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 19:15:27.713704 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.710463    2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 19:15:27.713704 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.710469    2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 19:15:27.713704 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.710474    2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 19:15:27.713704 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.710479    2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 19:15:27.713704 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.710484    2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 19:15:27.713704 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.710490    2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 19:15:27.713704 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.710495    2575 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 19:15:27.713704 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.710499    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 19:15:27.713704 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.710504    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 19:15:27.713704 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.710508    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 19:15:27.713704 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.710513    2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 19:15:27.713704 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.710518    2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 19:15:27.713704 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.710523    2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 19:15:27.713704 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.710527    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 19:15:27.713704 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.710532    2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 19:15:27.713704 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.710536    2575 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 19:15:27.713704 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.710541    2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 19:15:27.714211 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.710545    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 19:15:27.714211 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.710549    2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 19:15:27.714211 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.710555    2575 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 19:15:27.714211 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.710559    2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 19:15:27.714211 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.710563    2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 19:15:27.714211 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.710568    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 19:15:27.714211 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.710572    2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 19:15:27.714211 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.710577    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 19:15:27.714211 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.710581    2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 19:15:27.714211 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.710585    2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 19:15:27.714211 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.710589    2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 19:15:27.714211 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.710594    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 19:15:27.714211 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.710598    2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 19:15:27.714211 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.710602    2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 19:15:27.714211 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.710607    2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 19:15:27.714211 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.710611    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 19:15:27.714211 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.710615    2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 19:15:27.714211 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.710619    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 19:15:27.714211 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.710623    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 19:15:27.714211 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.710628    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 19:15:27.714858 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.710632    2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 19:15:27.714858 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.710636    2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 19:15:27.714858 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.710640    2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 19:15:27.714858 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.710644    2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 19:15:27.714858 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.710648    2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 19:15:27.714858 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.710653    2575 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 19:15:27.714858 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.710657    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 19:15:27.714858 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.710662    2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 19:15:27.714858 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.710666    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 19:15:27.714858 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.710670    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 19:15:27.714858 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.710674    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 19:15:27.714858 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.710678    2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 19:15:27.714858 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.710683    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 19:15:27.714858 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.710687    2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 19:15:27.714858 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.710691    2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 19:15:27.714858 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.710696    2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 19:15:27.714858 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.710701    2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 19:15:27.714858 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.710705    2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 19:15:27.714858 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.710709    2575 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 19:15:27.714858 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.710713    2575 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 19:15:27.715334 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.710717    2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 19:15:27.715334 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.710721    2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 19:15:27.715334 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.710726    2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 19:15:27.715334 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.710729    2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 19:15:27.715334 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.710733    2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 19:15:27.715334 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.710737    2575 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 19:15:27.715334 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.710741    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 19:15:27.715334 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.710744    2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 19:15:27.715334 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.710748    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 19:15:27.715334 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.710752    2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 19:15:27.715334 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.710756    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 19:15:27.715334 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.710760    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 19:15:27.715334 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.710764    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 19:15:27.715334 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.710768    2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 19:15:27.715334 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.710773    2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 19:15:27.715334 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.710777    2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 19:15:27.715334 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.710783    2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 19:15:27.715334 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.710789    2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 19:15:27.715334 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.710794    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 19:15:27.715984 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.710798    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 19:15:27.715984 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.710802    2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 19:15:27.715984 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.710806    2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 19:15:27.715984 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.710810    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 19:15:27.715984 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.710815    2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 19:15:27.715984 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.710820    2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 19:15:27.715984 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.710824    2575 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 19:15:27.715984 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.710829    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 19:15:27.715984 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.710833    2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 19:15:27.715984 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.710848    2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 19:15:27.715984 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.710855    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 19:15:27.715984 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.710860    2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 19:15:27.715984 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.710864    2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 19:15:27.715984 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.710872    2575 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 10 19:15:27.716337 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.711047    2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 19:15:27.716337 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.711056    2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 19:15:27.716337 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.711061    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 19:15:27.716337 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.711066    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 19:15:27.716337 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.711071    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 19:15:27.716337 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.711075    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 19:15:27.716337 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.711080    2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 19:15:27.716337 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.711084    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 19:15:27.716337 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.711088    2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 19:15:27.716337 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.711092    2575 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 19:15:27.716337 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.711097    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 19:15:27.716337 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.711101    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 19:15:27.716337 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.711105    2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 19:15:27.716337 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.711109    2575 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 19:15:27.716337 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.711113    2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 19:15:27.716337 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.711120    2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 19:15:27.716337 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.711126    2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 19:15:27.716337 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.711131    2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 19:15:27.716337 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.711135    2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 19:15:27.716337 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.711139    2575 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 19:15:27.716885 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.711143    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 19:15:27.716885 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.711147    2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 19:15:27.716885 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.711152    2575 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 19:15:27.716885 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.711156    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 19:15:27.716885 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.711160    2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 19:15:27.716885 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.711164    2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 19:15:27.716885 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.711169    2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 19:15:27.716885 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.711174    2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 19:15:27.716885 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.711179    2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 19:15:27.716885 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.711183    2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 19:15:27.716885 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.711188    2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 19:15:27.716885 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.711194    2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 19:15:27.716885 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.711199    2575 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 19:15:27.716885 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.711204    2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 19:15:27.716885 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.711208    2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 19:15:27.716885 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.711213    2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 19:15:27.716885 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.711217    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 19:15:27.716885 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.711222    2575 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 19:15:27.716885 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.711226    2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 19:15:27.717387 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.711232    2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 19:15:27.717387 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.711238    2575 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 19:15:27.717387 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.711243    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 19:15:27.717387 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.711248    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 19:15:27.717387 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.711252    2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 19:15:27.717387 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.711256    2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 19:15:27.717387 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.711260    2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 19:15:27.717387 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.711264    2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 19:15:27.717387 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.711268    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 19:15:27.717387 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.711273    2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 19:15:27.717387 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.711277    2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 19:15:27.717387 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.711281    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 19:15:27.717387 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.711285    2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 19:15:27.717387 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.711289    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 19:15:27.717387 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.711294    2575 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 19:15:27.717387 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.711298    2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 19:15:27.717387 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.711302    2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 19:15:27.717387 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.711306    2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 19:15:27.717387 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.711311    2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 19:15:27.717387 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.711315    2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 19:15:27.717879 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.711319    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 19:15:27.717879 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.711322    2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 19:15:27.717879 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.711327    2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 19:15:27.717879 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.711332    2575 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 19:15:27.717879 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.711336    2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 19:15:27.717879 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.711340    2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 19:15:27.717879 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.711344    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 19:15:27.717879 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.711348    2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 19:15:27.717879 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.711352    2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 19:15:27.717879 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.711356    2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 19:15:27.717879 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.711360    2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 19:15:27.717879 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.711365    2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 19:15:27.717879 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.711386    2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 19:15:27.717879 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.711391    2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 19:15:27.717879 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.711395    2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 19:15:27.717879 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.711400    2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 19:15:27.717879 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.711404    2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 19:15:27.717879 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.711408    2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 19:15:27.717879 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.711412    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 19:15:27.717879 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.711416    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 19:15:27.718703 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.711421    2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 19:15:27.718703 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.711425    2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 19:15:27.718703 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.711428    2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 19:15:27.718703 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.711432    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 19:15:27.718703 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.711436    2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 19:15:27.718703 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.711440    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 19:15:27.718703 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.711445    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 19:15:27.718703 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.711449    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 19:15:27.718703 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.711454    2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 19:15:27.718703 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.711458    2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 19:15:27.718703 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.711462    2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 19:15:27.718703 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.711466    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 19:15:27.718703 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.711470    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 19:15:27.718703 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.711475    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 19:15:27.718703 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.711479    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 19:15:27.718703 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.711484    2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 19:15:27.718703 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.711488    2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 19:15:27.718703 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.711492    2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 19:15:27.718703 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.711496    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 19:15:27.718703 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:27.711500    2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 19:15:27.719531 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.711508    2575 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 10 19:15:27.719531 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.711651    2575 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 10 19:15:27.719531 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.714469    2575 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 10 19:15:27.719531 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.715767    2575 server.go:1019] "Starting client certificate rotation"
Sep 10 19:15:27.719531 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.715874    2575 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 10 19:15:27.719531 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.716597    2575 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 10 19:15:27.719531 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.718601    2575 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 10 19:15:27.721252 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.721235    2575 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 10 19:15:27.731062 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.731046    2575 log.go:25] "Validated CRI v1 runtime API"
Sep 10 19:15:27.733807 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.733793    2575 log.go:25] "Validated CRI v1 image API"
Sep 10 19:15:27.734623 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.734602    2575 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 10 19:15:27.738173 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.738153    2575 fs.go:135] Filesystem UUIDs: map[395f103e-9de0-4e32-97aa-7fe6754989e6:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 f3737722-c856-4e4a-9379-1394ad258de1:/dev/nvme0n1p4]
Sep 10 19:15:27.738253 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.738172    2575 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 10 19:15:27.743094 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.742980    2575 manager.go:216] Machine: {Timestamp:2026-09-10 19:15:27.742528331 +0000 UTC m=+0.309323571 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3098369 MemoryCapacity:33164484608 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2195d8af8f097317e608031a86777d SystemUUID:ec2195d8-af8f-0973-17e6-08031a86777d BootID:60300fe1-992a-4607-80d9-3571bfac8911 Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582242304 Type:vfs Inodes:4048399 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:1f:68:1e:b3:b7 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:1f:68:1e:b3:b7 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:76:dc:55:98:1d:05 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164484608 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 10 19:15:27.743094 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.743087    2575 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 10 19:15:27.743254 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.743175    2575 manager.go:232] Version: {KernelVersion:5.14.0-570.138.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260901-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 10 19:15:27.746132 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.746107    2575 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 10 19:15:27.746280 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.746133    2575 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-129-114.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 10 19:15:27.747439 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.747427    2575 topology_manager.go:138] "Creating topology manager with none policy"
Sep 10 19:15:27.747496 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.747444    2575 container_manager_linux.go:309] "Creating device plugin manager"
Sep 10 19:15:27.747496 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.747455    2575 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 10 19:15:27.748177 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.748165    2575 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 10 19:15:27.749788 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.749776    2575 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 10 19:15:27.752662 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.752648    2575 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 10 19:15:27.753839 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.753826    2575 state_mem.go:36] "Initialized new in-memory state store"
Sep 10 19:15:27.753969 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.753958    2575 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 10 19:15:27.754039 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.753995    2575 kubelet.go:486] "Attempting to sync node with API server"
Sep 10 19:15:27.754039 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.754014    2575 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 10 19:15:27.754039 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.754029    2575 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 10 19:15:27.754154 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.754043    2575 kubelet.go:398] "Adding apiserver pod source"
Sep 10 19:15:27.754154 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.754054    2575 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 10 19:15:27.754441 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.754428    2575 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 10 19:15:27.754441 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.754450    2575 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 10 19:15:27.756466 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.756449    2575 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 10 19:15:27.758889 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.758875    2575 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 10 19:15:27.758928 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.758907    2575 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 10 19:15:27.759040 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.759031    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 10 19:15:27.759069 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.759045    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 10 19:15:27.759069 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.759052    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 10 19:15:27.759069 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.759058    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 10 19:15:27.759069 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.759066    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 10 19:15:27.759169 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.759071    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 10 19:15:27.759629 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.759620    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 10 19:15:27.759660 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.759632    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 10 19:15:27.759660 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.759641    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 10 19:15:27.759660 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.759649    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 10 19:15:27.759740 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.759670    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 10 19:15:27.759740 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.759678    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 10 19:15:27.759740 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.759700    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 10 19:15:27.759740 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.759705    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 10 19:15:27.763284 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.763267    2575 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-114.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 10 19:15:27.763464 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:15:27.763446    2575 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-129-114.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 10 19:15:27.763530 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.763520    2575 server.go:1268] "Started kubelet"
Sep 10 19:15:27.763615 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.763568    2575 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 10 19:15:27.763615 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.763583    2575 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 10 19:15:27.763722 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.763637    2575 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 10 19:15:27.764286 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:15:27.764263    2575 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 10 19:15:27.764338 ip-10-0-129-114 systemd[1]: Started Kubernetes Kubelet.
Sep 10 19:15:27.766281 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.766258    2575 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 10 19:15:27.768628 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.767689    2575 server.go:310] "Adding debug handlers to kubelet server"
Sep 10 19:15:27.769999 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:15:27.769980    2575 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 10 19:15:27.771661 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:15:27.770799    2575 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-114.ec2.internal.18d40b9e98755e0c  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-114.ec2.internal,UID:ip-10-0-129-114.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-129-114.ec2.internal,},FirstTimestamp:2026-09-10 19:15:27.76349646 +0000 UTC m=+0.330291683,LastTimestamp:2026-09-10 19:15:27.76349646 +0000 UTC m=+0.330291683,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-114.ec2.internal,}"
Sep 10 19:15:27.774141 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.774124    2575 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 10 19:15:27.774141 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.774136    2575 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 10 19:15:27.774323 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:15:27.774307    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-114.ec2.internal\" not found"
Sep 10 19:15:27.774706 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.774687    2575 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 10 19:15:27.774822 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.774808    2575 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 10 19:15:27.774912 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.774902    2575 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 10 19:15:27.774965 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.774935    2575 factory.go:55] Registering systemd factory
Sep 10 19:15:27.774965 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.774948    2575 factory.go:223] Registration of the systemd container factory successfully
Sep 10 19:15:27.775081 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.774987    2575 reconstruct.go:97] "Volume reconstruction finished"
Sep 10 19:15:27.775081 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.775001    2575 reconciler.go:29] "Reconciler: start to sync state"
Sep 10 19:15:27.775162 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.775126    2575 factory.go:153] Registering CRI-O factory
Sep 10 19:15:27.775162 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.775141    2575 factory.go:223] Registration of the crio container factory successfully
Sep 10 19:15:27.775251 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.775201    2575 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 10 19:15:27.775251 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.775219    2575 factory.go:103] Registering Raw factory
Sep 10 19:15:27.775251 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.775234    2575 manager.go:1196] Started watching for new ooms in manager
Sep 10 19:15:27.775629 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.775613    2575 manager.go:349] Starting recovery of all containers
Sep 10 19:15:27.784501 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:15:27.784475    2575 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-129-114.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 10 19:15:27.784501 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.784490    2575 manager.go:354] Recovery completed
Sep 10 19:15:27.784640 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:15:27.784618    2575 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 10 19:15:27.788482 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.788469    2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 19:15:27.790321 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.790299    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-114.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 19:15:27.790418 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.790335    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-114.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 19:15:27.790418 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.790349    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-114.ec2.internal" event="NodeHasSufficientPID"
Sep 10 19:15:27.790998 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.790983    2575 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 10 19:15:27.790998 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.790997    2575 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 10 19:15:27.791082 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.791016    2575 state_mem.go:36] "Initialized new in-memory state store"
Sep 10 19:15:27.791082 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.790989    2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 19:15:27.791726 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.791711    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-114.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 19:15:27.791789 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.791741    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-114.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 19:15:27.791789 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.791755    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-114.ec2.internal" event="NodeHasSufficientPID"
Sep 10 19:15:27.793188 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.793094    2575 policy_none.go:49] "None policy: Start"
Sep 10 19:15:27.793222 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.793193    2575 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 10 19:15:27.793222 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.793203    2575 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 10 19:15:27.795074 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.795063    2575 policy_none.go:47] "Start"
Sep 10 19:15:27.802751 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:15:27.802683    2575 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-114.ec2.internal.18d40b9e9a0ea535  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-114.ec2.internal,UID:ip-10-0-129-114.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-129-114.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-129-114.ec2.internal,},FirstTimestamp:2026-09-10 19:15:27.790318901 +0000 UTC m=+0.357114130,LastTimestamp:2026-09-10 19:15:27.790318901 +0000 UTC m=+0.357114130,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-114.ec2.internal,}"
Sep 10 19:15:27.814276 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:15:27.814216    2575 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-114.ec2.internal.18d40b9e9a0efcf5  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-114.ec2.internal,UID:ip-10-0-129-114.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-129-114.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-129-114.ec2.internal,},FirstTimestamp:2026-09-10 19:15:27.790341365 +0000 UTC m=+0.357136591,LastTimestamp:2026-09-10 19:15:27.790341365 +0000 UTC m=+0.357136591,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-114.ec2.internal,}"
Sep 10 19:15:27.824004 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.823983    2575 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-n7wqg"
Sep 10 19:15:27.826232 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:15:27.826169    2575 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-114.ec2.internal.18d40b9e9a0f2f61  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-114.ec2.internal,UID:ip-10-0-129-114.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-129-114.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-129-114.ec2.internal,},FirstTimestamp:2026-09-10 19:15:27.790354273 +0000 UTC m=+0.357149497,LastTimestamp:2026-09-10 19:15:27.790354273 +0000 UTC m=+0.357149497,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-114.ec2.internal,}"
Sep 10 19:15:27.830242 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.830225    2575 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-n7wqg"
Sep 10 19:15:27.855588 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.832014    2575 manager.go:337] "Starting Device Plugin manager"
Sep 10 19:15:27.855588 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:15:27.832050    2575 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 10 19:15:27.855588 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.832062    2575 server.go:87] "Starting device plugin registration server"
Sep 10 19:15:27.855588 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.832407    2575 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 10 19:15:27.855588 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.832419    2575 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 10 19:15:27.855588 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.832624    2575 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 10 19:15:27.855588 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.832688    2575 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 10 19:15:27.855588 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.832696    2575 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 10 19:15:27.855588 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:15:27.833133    2575 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 10 19:15:27.855588 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:15:27.833181    2575 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-129-114.ec2.internal\" not found"
Sep 10 19:15:27.869781 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.869755    2575 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 10 19:15:27.870923 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.870899    2575 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 10 19:15:27.871001 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.870955    2575 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 10 19:15:27.871001 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.870977    2575 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 10 19:15:27.871096 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:15:27.871051    2575 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 10 19:15:27.873327 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.873306    2575 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 19:15:27.933475 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.933456    2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 19:15:27.934356 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.934338    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-114.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 19:15:27.934445 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.934391    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-114.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 19:15:27.934445 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.934409    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-114.ec2.internal" event="NodeHasSufficientPID"
Sep 10 19:15:27.934445 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.934433    2575 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-129-114.ec2.internal"
Sep 10 19:15:27.944115 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.944061    2575 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-129-114.ec2.internal"
Sep 10 19:15:27.944115 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:15:27.944080    2575 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-129-114.ec2.internal\": node \"ip-10-0-129-114.ec2.internal\" not found"
Sep 10 19:15:27.971222 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.971195    2575 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-114.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-129-114.ec2.internal"]
Sep 10 19:15:27.971293 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.971267    2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 19:15:27.972898 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.972884    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-114.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 19:15:27.972969 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.972909    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-114.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 19:15:27.972969 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.972923    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-114.ec2.internal" event="NodeHasSufficientPID"
Sep 10 19:15:27.974860 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:15:27.974845    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-114.ec2.internal\" not found"
Sep 10 19:15:27.975219 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.975209    2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 19:15:27.975335 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.975322    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-114.ec2.internal"
Sep 10 19:15:27.975411 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.975350    2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 19:15:27.975411 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.975345    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a5967616edfbe6b806f38063e88e3534-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-114.ec2.internal\" (UID: \"a5967616edfbe6b806f38063e88e3534\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-114.ec2.internal"
Sep 10 19:15:27.975488 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.975406    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a5967616edfbe6b806f38063e88e3534-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-114.ec2.internal\" (UID: \"a5967616edfbe6b806f38063e88e3534\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-114.ec2.internal"
Sep 10 19:15:27.976010 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.975994    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-114.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 19:15:27.976074 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.976025    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-114.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 19:15:27.976074 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.976036    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-114.ec2.internal" event="NodeHasSufficientPID"
Sep 10 19:15:27.976137 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.975996    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-114.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 19:15:27.976137 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.976095    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-114.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 19:15:27.976137 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.976107    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-114.ec2.internal" event="NodeHasSufficientPID"
Sep 10 19:15:27.978276 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.978259    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-114.ec2.internal"
Sep 10 19:15:27.978389 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.978287    2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 19:15:27.979107 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.979092    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-114.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 19:15:27.979177 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.979122    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-114.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 19:15:27.979177 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:27.979135    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-114.ec2.internal" event="NodeHasSufficientPID"
Sep 10 19:15:28.005847 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:15:28.005830    2575 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-114.ec2.internal\" not found" node="ip-10-0-129-114.ec2.internal"
Sep 10 19:15:28.009958 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:15:28.009943    2575 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-114.ec2.internal\" not found" node="ip-10-0-129-114.ec2.internal"
Sep 10 19:15:28.075263 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:15:28.075240    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-114.ec2.internal\" not found"
Sep 10 19:15:28.076327 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:28.076314    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/3eeea337b91a216332a11d902b8fa0ef-config\") pod \"kube-apiserver-proxy-ip-10-0-129-114.ec2.internal\" (UID: \"3eeea337b91a216332a11d902b8fa0ef\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-114.ec2.internal"
Sep 10 19:15:28.076391 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:28.076349    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a5967616edfbe6b806f38063e88e3534-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-114.ec2.internal\" (UID: \"a5967616edfbe6b806f38063e88e3534\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-114.ec2.internal"
Sep 10 19:15:28.076391 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:28.076367    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a5967616edfbe6b806f38063e88e3534-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-114.ec2.internal\" (UID: \"a5967616edfbe6b806f38063e88e3534\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-114.ec2.internal"
Sep 10 19:15:28.076469 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:28.076427    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a5967616edfbe6b806f38063e88e3534-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-114.ec2.internal\" (UID: \"a5967616edfbe6b806f38063e88e3534\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-114.ec2.internal"
Sep 10 19:15:28.076469 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:28.076445    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a5967616edfbe6b806f38063e88e3534-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-114.ec2.internal\" (UID: \"a5967616edfbe6b806f38063e88e3534\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-114.ec2.internal"
Sep 10 19:15:28.175474 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:15:28.175439    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-114.ec2.internal\" not found"
Sep 10 19:15:28.176582 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:28.176568    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/3eeea337b91a216332a11d902b8fa0ef-config\") pod \"kube-apiserver-proxy-ip-10-0-129-114.ec2.internal\" (UID: \"3eeea337b91a216332a11d902b8fa0ef\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-114.ec2.internal"
Sep 10 19:15:28.176631 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:28.176607    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/3eeea337b91a216332a11d902b8fa0ef-config\") pod \"kube-apiserver-proxy-ip-10-0-129-114.ec2.internal\" (UID: \"3eeea337b91a216332a11d902b8fa0ef\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-114.ec2.internal"
Sep 10 19:15:28.276095 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:15:28.276048    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-114.ec2.internal\" not found"
Sep 10 19:15:28.309285 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:28.309269    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-114.ec2.internal"
Sep 10 19:15:28.312893 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:28.312876    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-114.ec2.internal"
Sep 10 19:15:28.376344 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:15:28.376324    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-114.ec2.internal\" not found"
Sep 10 19:15:28.476778 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:15:28.476759    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-114.ec2.internal\" not found"
Sep 10 19:15:28.577194 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:15:28.577153    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-114.ec2.internal\" not found"
Sep 10 19:15:28.619329 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:28.619312    2575 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 19:15:30.181484 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.181465    2575 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-114.ec2.internal"
Sep 10 19:15:30.196900 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.196884    2575 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 10 19:15:30.198509 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.198497    2575 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-114.ec2.internal"
Sep 10 19:15:30.208382 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.208367    2575 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 10 19:15:30.222541 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.222519    2575 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 10 19:15:30.222635 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.222620    2575 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 10 19:15:30.222694 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.222655    2575 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 10 19:15:30.261836 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.261813    2575 apiserver.go:52] "Watching apiserver"
Sep 10 19:15:30.275534 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.275514    2575 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 10 19:15:30.275854 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.275835    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-bbmp5","kube-system/kube-apiserver-proxy-ip-10-0-129-114.ec2.internal","openshift-cluster-node-tuning-operator/tuned-g4hld","openshift-image-registry/node-ca-nh8r2","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-114.ec2.internal","openshift-multus/multus-additional-cni-plugins-wt6fs","openshift-multus/multus-tt257","openshift-multus/network-metrics-daemon-wx2q6","kube-system/global-pull-secret-syncer-24lb9","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tx2mc","openshift-dns/node-resolver-vmvhn","openshift-network-diagnostics/network-check-target-9rkjw","openshift-network-operator/iptables-alerter-qs9mw","openshift-ovn-kubernetes/ovnkube-node-rrxfj"]
Sep 10 19:15:30.278031 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.278015    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-tt257"
Sep 10 19:15:30.280432 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.280402    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 10 19:15:30.280510 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.280419    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 10 19:15:30.280887 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.280740    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 10 19:15:30.280887 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.280748    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 10 19:15:30.280887 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.280785    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-nr8q4\""
Sep 10 19:15:30.281201 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.281186    2575 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 10 19:15:30.281434 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.281417    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-vmvhn"
Sep 10 19:15:30.282869 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.282849    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 10 19:15:30.283035 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.283022    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-fs6r4\""
Sep 10 19:15:30.283109 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.283094    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 10 19:15:30.283531 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.283519    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-g4hld"
Sep 10 19:15:30.284940 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.284927    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-5d8ss\""
Sep 10 19:15:30.285068 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.285054    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 10 19:15:30.285102 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.285058    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 10 19:15:30.285521 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.285506    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d896f9fb-21e2-4845-ab2d-a855a43b0a17-etc-sysconfig\") pod \"tuned-g4hld\" (UID: \"d896f9fb-21e2-4845-ab2d-a855a43b0a17\") " pod="openshift-cluster-node-tuning-operator/tuned-g4hld"
Sep 10 19:15:30.285566 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.285541    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xcw5d\" (UniqueName: \"kubernetes.io/projected/d896f9fb-21e2-4845-ab2d-a855a43b0a17-kube-api-access-xcw5d\") pod \"tuned-g4hld\" (UID: \"d896f9fb-21e2-4845-ab2d-a855a43b0a17\") " pod="openshift-cluster-node-tuning-operator/tuned-g4hld"
Sep 10 19:15:30.285566 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.285548    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-nh8r2"
Sep 10 19:15:30.285566 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.285559    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d896f9fb-21e2-4845-ab2d-a855a43b0a17-etc-systemd\") pod \"tuned-g4hld\" (UID: \"d896f9fb-21e2-4845-ab2d-a855a43b0a17\") " pod="openshift-cluster-node-tuning-operator/tuned-g4hld"
Sep 10 19:15:30.285683 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.285574    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d896f9fb-21e2-4845-ab2d-a855a43b0a17-host\") pod \"tuned-g4hld\" (UID: \"d896f9fb-21e2-4845-ab2d-a855a43b0a17\") " pod="openshift-cluster-node-tuning-operator/tuned-g4hld"
Sep 10 19:15:30.285683 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.285591    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/09299baf-b713-431a-9b8b-dfbe54f74b69-host-run-netns\") pod \"multus-tt257\" (UID: \"09299baf-b713-431a-9b8b-dfbe54f74b69\") " pod="openshift-multus/multus-tt257"
Sep 10 19:15:30.285683 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.285631    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/09299baf-b713-431a-9b8b-dfbe54f74b69-hostroot\") pod \"multus-tt257\" (UID: \"09299baf-b713-431a-9b8b-dfbe54f74b69\") " pod="openshift-multus/multus-tt257"
Sep 10 19:15:30.285683 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.285653    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/09299baf-b713-431a-9b8b-dfbe54f74b69-multus-conf-dir\") pod \"multus-tt257\" (UID: \"09299baf-b713-431a-9b8b-dfbe54f74b69\") " pod="openshift-multus/multus-tt257"
Sep 10 19:15:30.285683 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.285674    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g7skd\" (UniqueName: \"kubernetes.io/projected/d093d956-9c62-4201-a4ae-60187edb6fbf-kube-api-access-g7skd\") pod \"node-resolver-vmvhn\" (UID: \"d093d956-9c62-4201-a4ae-60187edb6fbf\") " pod="openshift-dns/node-resolver-vmvhn"
Sep 10 19:15:30.285897 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.285700    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d896f9fb-21e2-4845-ab2d-a855a43b0a17-etc-sysctl-d\") pod \"tuned-g4hld\" (UID: \"d896f9fb-21e2-4845-ab2d-a855a43b0a17\") " pod="openshift-cluster-node-tuning-operator/tuned-g4hld"
Sep 10 19:15:30.285897 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.285721    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d896f9fb-21e2-4845-ab2d-a855a43b0a17-sys\") pod \"tuned-g4hld\" (UID: \"d896f9fb-21e2-4845-ab2d-a855a43b0a17\") " pod="openshift-cluster-node-tuning-operator/tuned-g4hld"
Sep 10 19:15:30.285897 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.285750    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d896f9fb-21e2-4845-ab2d-a855a43b0a17-etc-tuned\") pod \"tuned-g4hld\" (UID: \"d896f9fb-21e2-4845-ab2d-a855a43b0a17\") " pod="openshift-cluster-node-tuning-operator/tuned-g4hld"
Sep 10 19:15:30.285897 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.285774    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/09299baf-b713-431a-9b8b-dfbe54f74b69-cnibin\") pod \"multus-tt257\" (UID: \"09299baf-b713-431a-9b8b-dfbe54f74b69\") " pod="openshift-multus/multus-tt257"
Sep 10 19:15:30.285897 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.285794    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/09299baf-b713-431a-9b8b-dfbe54f74b69-host-var-lib-kubelet\") pod \"multus-tt257\" (UID: \"09299baf-b713-431a-9b8b-dfbe54f74b69\") " pod="openshift-multus/multus-tt257"
Sep 10 19:15:30.285897 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.285812    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/09299baf-b713-431a-9b8b-dfbe54f74b69-host-run-multus-certs\") pod \"multus-tt257\" (UID: \"09299baf-b713-431a-9b8b-dfbe54f74b69\") " pod="openshift-multus/multus-tt257"
Sep 10 19:15:30.285897 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.285834    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d896f9fb-21e2-4845-ab2d-a855a43b0a17-run\") pod \"tuned-g4hld\" (UID: \"d896f9fb-21e2-4845-ab2d-a855a43b0a17\") " pod="openshift-cluster-node-tuning-operator/tuned-g4hld"
Sep 10 19:15:30.285897 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.285850    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d896f9fb-21e2-4845-ab2d-a855a43b0a17-tmp\") pod \"tuned-g4hld\" (UID: \"d896f9fb-21e2-4845-ab2d-a855a43b0a17\") " pod="openshift-cluster-node-tuning-operator/tuned-g4hld"
Sep 10 19:15:30.285897 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.285886    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nqnn5\" (UniqueName: \"kubernetes.io/projected/09299baf-b713-431a-9b8b-dfbe54f74b69-kube-api-access-nqnn5\") pod \"multus-tt257\" (UID: \"09299baf-b713-431a-9b8b-dfbe54f74b69\") " pod="openshift-multus/multus-tt257"
Sep 10 19:15:30.286198 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.285931    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d093d956-9c62-4201-a4ae-60187edb6fbf-tmp-dir\") pod \"node-resolver-vmvhn\" (UID: \"d093d956-9c62-4201-a4ae-60187edb6fbf\") " pod="openshift-dns/node-resolver-vmvhn"
Sep 10 19:15:30.286198 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.285981    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d896f9fb-21e2-4845-ab2d-a855a43b0a17-etc-sysctl-conf\") pod \"tuned-g4hld\" (UID: \"d896f9fb-21e2-4845-ab2d-a855a43b0a17\") " pod="openshift-cluster-node-tuning-operator/tuned-g4hld"
Sep 10 19:15:30.286198 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.286023    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d896f9fb-21e2-4845-ab2d-a855a43b0a17-var-lib-kubelet\") pod \"tuned-g4hld\" (UID: \"d896f9fb-21e2-4845-ab2d-a855a43b0a17\") " pod="openshift-cluster-node-tuning-operator/tuned-g4hld"
Sep 10 19:15:30.286198 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.286064    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/09299baf-b713-431a-9b8b-dfbe54f74b69-multus-cni-dir\") pod \"multus-tt257\" (UID: \"09299baf-b713-431a-9b8b-dfbe54f74b69\") " pod="openshift-multus/multus-tt257"
Sep 10 19:15:30.286198 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.286088    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/09299baf-b713-431a-9b8b-dfbe54f74b69-host-run-k8s-cni-cncf-io\") pod \"multus-tt257\" (UID: \"09299baf-b713-431a-9b8b-dfbe54f74b69\") " pod="openshift-multus/multus-tt257"
Sep 10 19:15:30.286198 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.286119    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/09299baf-b713-431a-9b8b-dfbe54f74b69-multus-daemon-config\") pod \"multus-tt257\" (UID: \"09299baf-b713-431a-9b8b-dfbe54f74b69\") " pod="openshift-multus/multus-tt257"
Sep 10 19:15:30.286198 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.286142    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/09299baf-b713-431a-9b8b-dfbe54f74b69-etc-kubernetes\") pod \"multus-tt257\" (UID: \"09299baf-b713-431a-9b8b-dfbe54f74b69\") " pod="openshift-multus/multus-tt257"
Sep 10 19:15:30.286198 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.286162    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d093d956-9c62-4201-a4ae-60187edb6fbf-hosts-file\") pod \"node-resolver-vmvhn\" (UID: \"d093d956-9c62-4201-a4ae-60187edb6fbf\") " pod="openshift-dns/node-resolver-vmvhn"
Sep 10 19:15:30.286525 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.286205    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d896f9fb-21e2-4845-ab2d-a855a43b0a17-etc-modprobe-d\") pod \"tuned-g4hld\" (UID: \"d896f9fb-21e2-4845-ab2d-a855a43b0a17\") " pod="openshift-cluster-node-tuning-operator/tuned-g4hld"
Sep 10 19:15:30.286525 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.286237    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d896f9fb-21e2-4845-ab2d-a855a43b0a17-etc-kubernetes\") pod \"tuned-g4hld\" (UID: \"d896f9fb-21e2-4845-ab2d-a855a43b0a17\") " pod="openshift-cluster-node-tuning-operator/tuned-g4hld"
Sep 10 19:15:30.286525 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.286259    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/09299baf-b713-431a-9b8b-dfbe54f74b69-multus-socket-dir-parent\") pod \"multus-tt257\" (UID: \"09299baf-b713-431a-9b8b-dfbe54f74b69\") " pod="openshift-multus/multus-tt257"
Sep 10 19:15:30.286525 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.286284    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d896f9fb-21e2-4845-ab2d-a855a43b0a17-lib-modules\") pod \"tuned-g4hld\" (UID: \"d896f9fb-21e2-4845-ab2d-a855a43b0a17\") " pod="openshift-cluster-node-tuning-operator/tuned-g4hld"
Sep 10 19:15:30.286525 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.286311    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/09299baf-b713-431a-9b8b-dfbe54f74b69-system-cni-dir\") pod \"multus-tt257\" (UID: \"09299baf-b713-431a-9b8b-dfbe54f74b69\") " pod="openshift-multus/multus-tt257"
Sep 10 19:15:30.286525 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.286351    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/09299baf-b713-431a-9b8b-dfbe54f74b69-cni-binary-copy\") pod \"multus-tt257\" (UID: \"09299baf-b713-431a-9b8b-dfbe54f74b69\") " pod="openshift-multus/multus-tt257"
Sep 10 19:15:30.286525 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.286382    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/09299baf-b713-431a-9b8b-dfbe54f74b69-host-var-lib-cni-bin\") pod \"multus-tt257\" (UID: \"09299baf-b713-431a-9b8b-dfbe54f74b69\") " pod="openshift-multus/multus-tt257"
Sep 10 19:15:30.286525 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.286406    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/09299baf-b713-431a-9b8b-dfbe54f74b69-host-var-lib-cni-multus\") pod \"multus-tt257\" (UID: \"09299baf-b713-431a-9b8b-dfbe54f74b69\") " pod="openshift-multus/multus-tt257"
Sep 10 19:15:30.287052 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.287037    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 10 19:15:30.287161 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.287147    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 10 19:15:30.287257 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.287245    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 10 19:15:30.287298 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.287292    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-dvz6x\""
Sep 10 19:15:30.287932 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.287917    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-rrxfj"
Sep 10 19:15:30.289325 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.289310    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 10 19:15:30.289613 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.289596    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 10 19:15:30.289684 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.289612    2575 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 10 19:15:30.289684 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.289655    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 10 19:15:30.289784 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.289714    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 10 19:15:30.289836 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.289780    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 10 19:15:30.289836 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.289830    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 10 19:15:30.289918 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.289850    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-vdpbj\""
Sep 10 19:15:30.290121 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.290109    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-wt6fs"
Sep 10 19:15:30.291786 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.291771    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 10 19:15:30.292288 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.292051    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 10 19:15:30.292288 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.292130    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-x9bc4\""
Sep 10 19:15:30.292441 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.292372    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-24lb9"
Sep 10 19:15:30.292484 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:15:30.292454    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-24lb9" podUID="ac571b4d-dbf7-4e62-8d4d-c8c2a527deb3"
Sep 10 19:15:30.294603 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.294589    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-qs9mw"
Sep 10 19:15:30.296237 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.296206    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 10 19:15:30.296309 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.296220    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-5v2sv\""
Sep 10 19:15:30.296309 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.296283    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 10 19:15:30.296402 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.296390    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 10 19:15:30.296813 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.296797    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wx2q6"
Sep 10 19:15:30.296901 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:15:30.296876    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wx2q6" podUID="bfed5912-60ab-48fc-91c2-10d325fd6428"
Sep 10 19:15:30.299106 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.299091    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tx2mc"
Sep 10 19:15:30.300939 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.300920    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 10 19:15:30.301034 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.301009    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 10 19:15:30.301178 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.301130    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-wnpqz\""
Sep 10 19:15:30.301266 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.301229    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 10 19:15:30.301582 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.301558    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-bbmp5"
Sep 10 19:15:30.303216 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.303197    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-jg9j2\""
Sep 10 19:15:30.303309 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.303251    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 10 19:15:30.303405 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.303390    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 10 19:15:30.303854 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.303839    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9rkjw"
Sep 10 19:15:30.306775 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:15:30.304083    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9rkjw" podUID="cbd42c54-3649-4d2d-8f94-d2789f7f66c7"
Sep 10 19:15:30.320786 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.320766    2575 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-4w482"
Sep 10 19:15:30.328327 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.328312    2575 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-4w482"
Sep 10 19:15:30.340427 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.340371    2575 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-09 19:10:29 +0000 UTC" deadline="2028-03-21 01:58:43.818272271 +0000 UTC"
Sep 10 19:15:30.340427 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.340427    2575 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13374h43m13.477849776s"
Sep 10 19:15:30.382430 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.382415    2575 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 10 19:15:30.386881 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.386855    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d896f9fb-21e2-4845-ab2d-a855a43b0a17-etc-modprobe-d\") pod \"tuned-g4hld\" (UID: \"d896f9fb-21e2-4845-ab2d-a855a43b0a17\") " pod="openshift-cluster-node-tuning-operator/tuned-g4hld"
Sep 10 19:15:30.386962 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.386895    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d896f9fb-21e2-4845-ab2d-a855a43b0a17-etc-kubernetes\") pod \"tuned-g4hld\" (UID: \"d896f9fb-21e2-4845-ab2d-a855a43b0a17\") " pod="openshift-cluster-node-tuning-operator/tuned-g4hld"
Sep 10 19:15:30.386962 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.386923    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b4070c77-f931-4973-b076-035a9e33e600-socket-dir\") pod \"aws-ebs-csi-driver-node-tx2mc\" (UID: \"b4070c77-f931-4973-b076-035a9e33e600\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tx2mc"
Sep 10 19:15:30.386962 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.386950    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cf162519-8424-43b0-8de5-a19bc237a9d0-cni-binary-copy\") pod \"multus-additional-cni-plugins-wt6fs\" (UID: \"cf162519-8424-43b0-8de5-a19bc237a9d0\") " pod="openshift-multus/multus-additional-cni-plugins-wt6fs"
Sep 10 19:15:30.387107 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.386975    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c7511dfe-c588-4be1-bd16-1dce34740809-etc-openvswitch\") pod \"ovnkube-node-rrxfj\" (UID: \"c7511dfe-c588-4be1-bd16-1dce34740809\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrxfj"
Sep 10 19:15:30.387107 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.386999    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/09299baf-b713-431a-9b8b-dfbe54f74b69-cni-binary-copy\") pod \"multus-tt257\" (UID: \"09299baf-b713-431a-9b8b-dfbe54f74b69\") " pod="openshift-multus/multus-tt257"
Sep 10 19:15:30.387107 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.387016    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d896f9fb-21e2-4845-ab2d-a855a43b0a17-etc-modprobe-d\") pod \"tuned-g4hld\" (UID: \"d896f9fb-21e2-4845-ab2d-a855a43b0a17\") " pod="openshift-cluster-node-tuning-operator/tuned-g4hld"
Sep 10 19:15:30.387107 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.387016    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d896f9fb-21e2-4845-ab2d-a855a43b0a17-etc-kubernetes\") pod \"tuned-g4hld\" (UID: \"d896f9fb-21e2-4845-ab2d-a855a43b0a17\") " pod="openshift-cluster-node-tuning-operator/tuned-g4hld"
Sep 10 19:15:30.387107 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.387042    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/09299baf-b713-431a-9b8b-dfbe54f74b69-host-var-lib-cni-bin\") pod \"multus-tt257\" (UID: \"09299baf-b713-431a-9b8b-dfbe54f74b69\") " pod="openshift-multus/multus-tt257"
Sep 10 19:15:30.387107 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.387076    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/09299baf-b713-431a-9b8b-dfbe54f74b69-host-var-lib-cni-bin\") pod \"multus-tt257\" (UID: \"09299baf-b713-431a-9b8b-dfbe54f74b69\") " pod="openshift-multus/multus-tt257"
Sep 10 19:15:30.387107 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.387089    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-24wkn\" (UniqueName: \"kubernetes.io/projected/bfed5912-60ab-48fc-91c2-10d325fd6428-kube-api-access-24wkn\") pod \"network-metrics-daemon-wx2q6\" (UID: \"bfed5912-60ab-48fc-91c2-10d325fd6428\") " pod="openshift-multus/network-metrics-daemon-wx2q6"
Sep 10 19:15:30.387449 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.387116    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cf162519-8424-43b0-8de5-a19bc237a9d0-system-cni-dir\") pod \"multus-additional-cni-plugins-wt6fs\" (UID: \"cf162519-8424-43b0-8de5-a19bc237a9d0\") " pod="openshift-multus/multus-additional-cni-plugins-wt6fs"
Sep 10 19:15:30.387449 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.387139    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cf162519-8424-43b0-8de5-a19bc237a9d0-cnibin\") pod \"multus-additional-cni-plugins-wt6fs\" (UID: \"cf162519-8424-43b0-8de5-a19bc237a9d0\") " pod="openshift-multus/multus-additional-cni-plugins-wt6fs"
Sep 10 19:15:30.387449 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.387169    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c7511dfe-c588-4be1-bd16-1dce34740809-host-run-netns\") pod \"ovnkube-node-rrxfj\" (UID: \"c7511dfe-c588-4be1-bd16-1dce34740809\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrxfj"
Sep 10 19:15:30.387449 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.387219    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d896f9fb-21e2-4845-ab2d-a855a43b0a17-etc-systemd\") pod \"tuned-g4hld\" (UID: \"d896f9fb-21e2-4845-ab2d-a855a43b0a17\") " pod="openshift-cluster-node-tuning-operator/tuned-g4hld"
Sep 10 19:15:30.387449 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.387253    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d896f9fb-21e2-4845-ab2d-a855a43b0a17-host\") pod \"tuned-g4hld\" (UID: \"d896f9fb-21e2-4845-ab2d-a855a43b0a17\") " pod="openshift-cluster-node-tuning-operator/tuned-g4hld"
Sep 10 19:15:30.387449 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.387307    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d896f9fb-21e2-4845-ab2d-a855a43b0a17-host\") pod \"tuned-g4hld\" (UID: \"d896f9fb-21e2-4845-ab2d-a855a43b0a17\") " pod="openshift-cluster-node-tuning-operator/tuned-g4hld"
Sep 10 19:15:30.387449 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.387319    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/09299baf-b713-431a-9b8b-dfbe54f74b69-host-run-netns\") pod \"multus-tt257\" (UID: \"09299baf-b713-431a-9b8b-dfbe54f74b69\") " pod="openshift-multus/multus-tt257"
Sep 10 19:15:30.387449 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.387331    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d896f9fb-21e2-4845-ab2d-a855a43b0a17-etc-systemd\") pod \"tuned-g4hld\" (UID: \"d896f9fb-21e2-4845-ab2d-a855a43b0a17\") " pod="openshift-cluster-node-tuning-operator/tuned-g4hld"
Sep 10 19:15:30.387449 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.387368    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/09299baf-b713-431a-9b8b-dfbe54f74b69-host-run-netns\") pod \"multus-tt257\" (UID: \"09299baf-b713-431a-9b8b-dfbe54f74b69\") " pod="openshift-multus/multus-tt257"
Sep 10 19:15:30.387449 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.387372    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/09299baf-b713-431a-9b8b-dfbe54f74b69-multus-conf-dir\") pod \"multus-tt257\" (UID: \"09299baf-b713-431a-9b8b-dfbe54f74b69\") " pod="openshift-multus/multus-tt257"
Sep 10 19:15:30.387449 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.387399    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b4070c77-f931-4973-b076-035a9e33e600-kubelet-dir\") pod \"aws-ebs-csi-driver-node-tx2mc\" (UID: \"b4070c77-f931-4973-b076-035a9e33e600\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tx2mc"
Sep 10 19:15:30.387449 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.387419    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c7511dfe-c588-4be1-bd16-1dce34740809-host-kubelet\") pod \"ovnkube-node-rrxfj\" (UID: \"c7511dfe-c588-4be1-bd16-1dce34740809\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrxfj"
Sep 10 19:15:30.387449 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.387432    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c7511dfe-c588-4be1-bd16-1dce34740809-log-socket\") pod \"ovnkube-node-rrxfj\" (UID: \"c7511dfe-c588-4be1-bd16-1dce34740809\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrxfj"
Sep 10 19:15:30.387449 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.387453    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d896f9fb-21e2-4845-ab2d-a855a43b0a17-sys\") pod \"tuned-g4hld\" (UID: \"d896f9fb-21e2-4845-ab2d-a855a43b0a17\") " pod="openshift-cluster-node-tuning-operator/tuned-g4hld"
Sep 10 19:15:30.388062 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.387468    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d896f9fb-21e2-4845-ab2d-a855a43b0a17-etc-tuned\") pod \"tuned-g4hld\" (UID: \"d896f9fb-21e2-4845-ab2d-a855a43b0a17\") " pod="openshift-cluster-node-tuning-operator/tuned-g4hld"
Sep 10 19:15:30.388062 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.387490    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/09299baf-b713-431a-9b8b-dfbe54f74b69-cnibin\") pod \"multus-tt257\" (UID: \"09299baf-b713-431a-9b8b-dfbe54f74b69\") " pod="openshift-multus/multus-tt257"
Sep 10 19:15:30.388062 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.387491    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/09299baf-b713-431a-9b8b-dfbe54f74b69-multus-conf-dir\") pod \"multus-tt257\" (UID: \"09299baf-b713-431a-9b8b-dfbe54f74b69\") " pod="openshift-multus/multus-tt257"
Sep 10 19:15:30.388062 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.387513    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/09299baf-b713-431a-9b8b-dfbe54f74b69-host-var-lib-kubelet\") pod \"multus-tt257\" (UID: \"09299baf-b713-431a-9b8b-dfbe54f74b69\") " pod="openshift-multus/multus-tt257"
Sep 10 19:15:30.388062 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.387528    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/09299baf-b713-431a-9b8b-dfbe54f74b69-cni-binary-copy\") pod \"multus-tt257\" (UID: \"09299baf-b713-431a-9b8b-dfbe54f74b69\") " pod="openshift-multus/multus-tt257"
Sep 10 19:15:30.388062 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.387536    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/cf162519-8424-43b0-8de5-a19bc237a9d0-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wt6fs\" (UID: \"cf162519-8424-43b0-8de5-a19bc237a9d0\") " pod="openshift-multus/multus-additional-cni-plugins-wt6fs"
Sep 10 19:15:30.388062 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.387529    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d896f9fb-21e2-4845-ab2d-a855a43b0a17-sys\") pod \"tuned-g4hld\" (UID: \"d896f9fb-21e2-4845-ab2d-a855a43b0a17\") " pod="openshift-cluster-node-tuning-operator/tuned-g4hld"
Sep 10 19:15:30.388062 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.387562    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z5zkc\" (UniqueName: \"kubernetes.io/projected/cf162519-8424-43b0-8de5-a19bc237a9d0-kube-api-access-z5zkc\") pod \"multus-additional-cni-plugins-wt6fs\" (UID: \"cf162519-8424-43b0-8de5-a19bc237a9d0\") " pod="openshift-multus/multus-additional-cni-plugins-wt6fs"
Sep 10 19:15:30.388062 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.387568    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/09299baf-b713-431a-9b8b-dfbe54f74b69-cnibin\") pod \"multus-tt257\" (UID: \"09299baf-b713-431a-9b8b-dfbe54f74b69\") " pod="openshift-multus/multus-tt257"
Sep 10 19:15:30.388062 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.387569    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/09299baf-b713-431a-9b8b-dfbe54f74b69-host-var-lib-kubelet\") pod \"multus-tt257\" (UID: \"09299baf-b713-431a-9b8b-dfbe54f74b69\") " pod="openshift-multus/multus-tt257"
Sep 10 19:15:30.388062 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.387594    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c7511dfe-c588-4be1-bd16-1dce34740809-ovnkube-config\") pod \"ovnkube-node-rrxfj\" (UID: \"c7511dfe-c588-4be1-bd16-1dce34740809\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrxfj"
Sep 10 19:15:30.388062 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.387626    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d093d956-9c62-4201-a4ae-60187edb6fbf-tmp-dir\") pod \"node-resolver-vmvhn\" (UID: \"d093d956-9c62-4201-a4ae-60187edb6fbf\") " pod="openshift-dns/node-resolver-vmvhn"
Sep 10 19:15:30.388062 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.387673    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a76b5c99-a97f-461d-bce3-1fc84ef52bae-konnectivity-ca\") pod \"konnectivity-agent-bbmp5\" (UID: \"a76b5c99-a97f-461d-bce3-1fc84ef52bae\") " pod="kube-system/konnectivity-agent-bbmp5"
Sep 10 19:15:30.388062 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.387697    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c7511dfe-c588-4be1-bd16-1dce34740809-host-slash\") pod \"ovnkube-node-rrxfj\" (UID: \"c7511dfe-c588-4be1-bd16-1dce34740809\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrxfj"
Sep 10 19:15:30.388062 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.387719    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c7511dfe-c588-4be1-bd16-1dce34740809-run-openvswitch\") pod \"ovnkube-node-rrxfj\" (UID: \"c7511dfe-c588-4be1-bd16-1dce34740809\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrxfj"
Sep 10 19:15:30.388062 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.387739    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c7511dfe-c588-4be1-bd16-1dce34740809-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-rrxfj\" (UID: \"c7511dfe-c588-4be1-bd16-1dce34740809\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrxfj"
Sep 10 19:15:30.388062 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.387761    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d896f9fb-21e2-4845-ab2d-a855a43b0a17-var-lib-kubelet\") pod \"tuned-g4hld\" (UID: \"d896f9fb-21e2-4845-ab2d-a855a43b0a17\") " pod="openshift-cluster-node-tuning-operator/tuned-g4hld"
Sep 10 19:15:30.388807 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.387799    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/09299baf-b713-431a-9b8b-dfbe54f74b69-multus-daemon-config\") pod \"multus-tt257\" (UID: \"09299baf-b713-431a-9b8b-dfbe54f74b69\") " pod="openshift-multus/multus-tt257"
Sep 10 19:15:30.388807 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.387819    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d896f9fb-21e2-4845-ab2d-a855a43b0a17-var-lib-kubelet\") pod \"tuned-g4hld\" (UID: \"d896f9fb-21e2-4845-ab2d-a855a43b0a17\") " pod="openshift-cluster-node-tuning-operator/tuned-g4hld"
Sep 10 19:15:30.388807 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.387831    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b4070c77-f931-4973-b076-035a9e33e600-sys-fs\") pod \"aws-ebs-csi-driver-node-tx2mc\" (UID: \"b4070c77-f931-4973-b076-035a9e33e600\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tx2mc"
Sep 10 19:15:30.388807 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.387875    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d093d956-9c62-4201-a4ae-60187edb6fbf-tmp-dir\") pod \"node-resolver-vmvhn\" (UID: \"d093d956-9c62-4201-a4ae-60187edb6fbf\") " pod="openshift-dns/node-resolver-vmvhn"
Sep 10 19:15:30.388807 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.387887    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c7511dfe-c588-4be1-bd16-1dce34740809-host-cni-bin\") pod \"ovnkube-node-rrxfj\" (UID: \"c7511dfe-c588-4be1-bd16-1dce34740809\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrxfj"
Sep 10 19:15:30.388807 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.387904    2575 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 10 19:15:30.388807 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.387911    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c7511dfe-c588-4be1-bd16-1dce34740809-host-cni-netd\") pod \"ovnkube-node-rrxfj\" (UID: \"c7511dfe-c588-4be1-bd16-1dce34740809\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrxfj"
Sep 10 19:15:30.388807 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.387955    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c7511dfe-c588-4be1-bd16-1dce34740809-ovnkube-script-lib\") pod \"ovnkube-node-rrxfj\" (UID: \"c7511dfe-c588-4be1-bd16-1dce34740809\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrxfj"
Sep 10 19:15:30.388807 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.387989    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d093d956-9c62-4201-a4ae-60187edb6fbf-hosts-file\") pod \"node-resolver-vmvhn\" (UID: \"d093d956-9c62-4201-a4ae-60187edb6fbf\") " pod="openshift-dns/node-resolver-vmvhn"
Sep 10 19:15:30.388807 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.388013    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/446e8624-4aaf-471d-9653-2451db8c6889-host\") pod \"node-ca-nh8r2\" (UID: \"446e8624-4aaf-471d-9653-2451db8c6889\") " pod="openshift-image-registry/node-ca-nh8r2"
Sep 10 19:15:30.388807 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.388037    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ac571b4d-dbf7-4e62-8d4d-c8c2a527deb3-kubelet-config\") pod \"global-pull-secret-syncer-24lb9\" (UID: \"ac571b4d-dbf7-4e62-8d4d-c8c2a527deb3\") " pod="kube-system/global-pull-secret-syncer-24lb9"
Sep 10 19:15:30.388807 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.388066    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d093d956-9c62-4201-a4ae-60187edb6fbf-hosts-file\") pod \"node-resolver-vmvhn\" (UID: \"d093d956-9c62-4201-a4ae-60187edb6fbf\") " pod="openshift-dns/node-resolver-vmvhn"
Sep 10 19:15:30.388807 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.388079    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/09299baf-b713-431a-9b8b-dfbe54f74b69-multus-socket-dir-parent\") pod \"multus-tt257\" (UID: \"09299baf-b713-431a-9b8b-dfbe54f74b69\") " pod="openshift-multus/multus-tt257"
Sep 10 19:15:30.388807 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.388103    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b4070c77-f931-4973-b076-035a9e33e600-registration-dir\") pod \"aws-ebs-csi-driver-node-tx2mc\" (UID: \"b4070c77-f931-4973-b076-035a9e33e600\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tx2mc"
Sep 10 19:15:30.388807 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.388127    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a76b5c99-a97f-461d-bce3-1fc84ef52bae-agent-certs\") pod \"konnectivity-agent-bbmp5\" (UID: \"a76b5c99-a97f-461d-bce3-1fc84ef52bae\") " pod="kube-system/konnectivity-agent-bbmp5"
Sep 10 19:15:30.388807 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.388148    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c7511dfe-c588-4be1-bd16-1dce34740809-systemd-units\") pod \"ovnkube-node-rrxfj\" (UID: \"c7511dfe-c588-4be1-bd16-1dce34740809\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrxfj"
Sep 10 19:15:30.388807 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.388166    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/09299baf-b713-431a-9b8b-dfbe54f74b69-multus-socket-dir-parent\") pod \"multus-tt257\" (UID: \"09299baf-b713-431a-9b8b-dfbe54f74b69\") " pod="openshift-multus/multus-tt257"
Sep 10 19:15:30.389518 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.388171    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c7511dfe-c588-4be1-bd16-1dce34740809-host-run-ovn-kubernetes\") pod \"ovnkube-node-rrxfj\" (UID: \"c7511dfe-c588-4be1-bd16-1dce34740809\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrxfj"
Sep 10 19:15:30.389518 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.388210    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/83c3166f-596e-4033-b92b-02f89683ccda-iptables-alerter-script\") pod \"iptables-alerter-qs9mw\" (UID: \"83c3166f-596e-4033-b92b-02f89683ccda\") " pod="openshift-network-operator/iptables-alerter-qs9mw"
Sep 10 19:15:30.389518 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.388235    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wtxj6\" (UniqueName: \"kubernetes.io/projected/83c3166f-596e-4033-b92b-02f89683ccda-kube-api-access-wtxj6\") pod \"iptables-alerter-qs9mw\" (UID: \"83c3166f-596e-4033-b92b-02f89683ccda\") " pod="openshift-network-operator/iptables-alerter-qs9mw"
Sep 10 19:15:30.389518 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.388240    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/09299baf-b713-431a-9b8b-dfbe54f74b69-multus-daemon-config\") pod \"multus-tt257\" (UID: \"09299baf-b713-431a-9b8b-dfbe54f74b69\") " pod="openshift-multus/multus-tt257"
Sep 10 19:15:30.389518 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.388267    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d896f9fb-21e2-4845-ab2d-a855a43b0a17-lib-modules\") pod \"tuned-g4hld\" (UID: \"d896f9fb-21e2-4845-ab2d-a855a43b0a17\") " pod="openshift-cluster-node-tuning-operator/tuned-g4hld"
Sep 10 19:15:30.389518 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.388300    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/09299baf-b713-431a-9b8b-dfbe54f74b69-system-cni-dir\") pod \"multus-tt257\" (UID: \"09299baf-b713-431a-9b8b-dfbe54f74b69\") " pod="openshift-multus/multus-tt257"
Sep 10 19:15:30.389518 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.388324    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/09299baf-b713-431a-9b8b-dfbe54f74b69-host-var-lib-cni-multus\") pod \"multus-tt257\" (UID: \"09299baf-b713-431a-9b8b-dfbe54f74b69\") " pod="openshift-multus/multus-tt257"
Sep 10 19:15:30.389518 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.388361    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b4070c77-f931-4973-b076-035a9e33e600-etc-selinux\") pod \"aws-ebs-csi-driver-node-tx2mc\" (UID: \"b4070c77-f931-4973-b076-035a9e33e600\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tx2mc"
Sep 10 19:15:30.389518 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.388382    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cf162519-8424-43b0-8de5-a19bc237a9d0-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wt6fs\" (UID: \"cf162519-8424-43b0-8de5-a19bc237a9d0\") " pod="openshift-multus/multus-additional-cni-plugins-wt6fs"
Sep 10 19:15:30.389518 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.388401    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d896f9fb-21e2-4845-ab2d-a855a43b0a17-etc-sysconfig\") pod \"tuned-g4hld\" (UID: \"d896f9fb-21e2-4845-ab2d-a855a43b0a17\") " pod="openshift-cluster-node-tuning-operator/tuned-g4hld"
Sep 10 19:15:30.389518 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.388413    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/09299baf-b713-431a-9b8b-dfbe54f74b69-system-cni-dir\") pod \"multus-tt257\" (UID: \"09299baf-b713-431a-9b8b-dfbe54f74b69\") " pod="openshift-multus/multus-tt257"
Sep 10 19:15:30.389518 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.388419    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xcw5d\" (UniqueName: \"kubernetes.io/projected/d896f9fb-21e2-4845-ab2d-a855a43b0a17-kube-api-access-xcw5d\") pod \"tuned-g4hld\" (UID: \"d896f9fb-21e2-4845-ab2d-a855a43b0a17\") " pod="openshift-cluster-node-tuning-operator/tuned-g4hld"
Sep 10 19:15:30.389518 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.388443    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d896f9fb-21e2-4845-ab2d-a855a43b0a17-lib-modules\") pod \"tuned-g4hld\" (UID: \"d896f9fb-21e2-4845-ab2d-a855a43b0a17\") " pod="openshift-cluster-node-tuning-operator/tuned-g4hld"
Sep 10 19:15:30.389518 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.388461    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b4070c77-f931-4973-b076-035a9e33e600-device-dir\") pod \"aws-ebs-csi-driver-node-tx2mc\" (UID: \"b4070c77-f931-4973-b076-035a9e33e600\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tx2mc"
Sep 10 19:15:30.389518 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.388494    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/446e8624-4aaf-471d-9653-2451db8c6889-serviceca\") pod \"node-ca-nh8r2\" (UID: \"446e8624-4aaf-471d-9653-2451db8c6889\") " pod="openshift-image-registry/node-ca-nh8r2"
Sep 10 19:15:30.389518 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.388494    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/09299baf-b713-431a-9b8b-dfbe54f74b69-host-var-lib-cni-multus\") pod \"multus-tt257\" (UID: \"09299baf-b713-431a-9b8b-dfbe54f74b69\") " pod="openshift-multus/multus-tt257"
Sep 10 19:15:30.389518 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.388528    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d896f9fb-21e2-4845-ab2d-a855a43b0a17-etc-sysconfig\") pod \"tuned-g4hld\" (UID: \"d896f9fb-21e2-4845-ab2d-a855a43b0a17\") " pod="openshift-cluster-node-tuning-operator/tuned-g4hld"
Sep 10 19:15:30.390157 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.388581    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c7511dfe-c588-4be1-bd16-1dce34740809-run-ovn\") pod \"ovnkube-node-rrxfj\" (UID: \"c7511dfe-c588-4be1-bd16-1dce34740809\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrxfj"
Sep 10 19:15:30.390157 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.388613    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c7511dfe-c588-4be1-bd16-1dce34740809-node-log\") pod \"ovnkube-node-rrxfj\" (UID: \"c7511dfe-c588-4be1-bd16-1dce34740809\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrxfj"
Sep 10 19:15:30.390157 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.388649    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/09299baf-b713-431a-9b8b-dfbe54f74b69-hostroot\") pod \"multus-tt257\" (UID: \"09299baf-b713-431a-9b8b-dfbe54f74b69\") " pod="openshift-multus/multus-tt257"
Sep 10 19:15:30.390157 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.388683    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/09299baf-b713-431a-9b8b-dfbe54f74b69-hostroot\") pod \"multus-tt257\" (UID: \"09299baf-b713-431a-9b8b-dfbe54f74b69\") " pod="openshift-multus/multus-tt257"
Sep 10 19:15:30.390157 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.388681    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g7skd\" (UniqueName: \"kubernetes.io/projected/d093d956-9c62-4201-a4ae-60187edb6fbf-kube-api-access-g7skd\") pod \"node-resolver-vmvhn\" (UID: \"d093d956-9c62-4201-a4ae-60187edb6fbf\") " pod="openshift-dns/node-resolver-vmvhn"
Sep 10 19:15:30.390157 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.388733    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n2p5q\" (UniqueName: \"kubernetes.io/projected/cbd42c54-3649-4d2d-8f94-d2789f7f66c7-kube-api-access-n2p5q\") pod \"network-check-target-9rkjw\" (UID: \"cbd42c54-3649-4d2d-8f94-d2789f7f66c7\") " pod="openshift-network-diagnostics/network-check-target-9rkjw"
Sep 10 19:15:30.390157 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.388755    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d896f9fb-21e2-4845-ab2d-a855a43b0a17-etc-sysctl-d\") pod \"tuned-g4hld\" (UID: \"d896f9fb-21e2-4845-ab2d-a855a43b0a17\") " pod="openshift-cluster-node-tuning-operator/tuned-g4hld"
Sep 10 19:15:30.390157 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.388785    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/09299baf-b713-431a-9b8b-dfbe54f74b69-host-run-multus-certs\") pod \"multus-tt257\" (UID: \"09299baf-b713-431a-9b8b-dfbe54f74b69\") " pod="openshift-multus/multus-tt257"
Sep 10 19:15:30.390157 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.388805    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c7511dfe-c588-4be1-bd16-1dce34740809-env-overrides\") pod \"ovnkube-node-rrxfj\" (UID: \"c7511dfe-c588-4be1-bd16-1dce34740809\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrxfj"
Sep 10 19:15:30.390157 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.388823    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d896f9fb-21e2-4845-ab2d-a855a43b0a17-run\") pod \"tuned-g4hld\" (UID: \"d896f9fb-21e2-4845-ab2d-a855a43b0a17\") " pod="openshift-cluster-node-tuning-operator/tuned-g4hld"
Sep 10 19:15:30.390157 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.388842    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d896f9fb-21e2-4845-ab2d-a855a43b0a17-tmp\") pod \"tuned-g4hld\" (UID: \"d896f9fb-21e2-4845-ab2d-a855a43b0a17\") " pod="openshift-cluster-node-tuning-operator/tuned-g4hld"
Sep 10 19:15:30.390157 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.388861    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nqnn5\" (UniqueName: \"kubernetes.io/projected/09299baf-b713-431a-9b8b-dfbe54f74b69-kube-api-access-nqnn5\") pod \"multus-tt257\" (UID: \"09299baf-b713-431a-9b8b-dfbe54f74b69\") " pod="openshift-multus/multus-tt257"
Sep 10 19:15:30.390157 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.388882    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d896f9fb-21e2-4845-ab2d-a855a43b0a17-etc-sysctl-d\") pod \"tuned-g4hld\" (UID: \"d896f9fb-21e2-4845-ab2d-a855a43b0a17\") " pod="openshift-cluster-node-tuning-operator/tuned-g4hld"
Sep 10 19:15:30.390157 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.388883    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/09299baf-b713-431a-9b8b-dfbe54f74b69-host-run-multus-certs\") pod \"multus-tt257\" (UID: \"09299baf-b713-431a-9b8b-dfbe54f74b69\") " pod="openshift-multus/multus-tt257"
Sep 10 19:15:30.390157 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.388901    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d896f9fb-21e2-4845-ab2d-a855a43b0a17-run\") pod \"tuned-g4hld\" (UID: \"d896f9fb-21e2-4845-ab2d-a855a43b0a17\") " pod="openshift-cluster-node-tuning-operator/tuned-g4hld"
Sep 10 19:15:30.390157 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.388914    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rs7nz\" (UniqueName: \"kubernetes.io/projected/b4070c77-f931-4973-b076-035a9e33e600-kube-api-access-rs7nz\") pod \"aws-ebs-csi-driver-node-tx2mc\" (UID: \"b4070c77-f931-4973-b076-035a9e33e600\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tx2mc"
Sep 10 19:15:30.390157 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.388945    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/cf162519-8424-43b0-8de5-a19bc237a9d0-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wt6fs\" (UID: \"cf162519-8424-43b0-8de5-a19bc237a9d0\") " pod="openshift-multus/multus-additional-cni-plugins-wt6fs"
Sep 10 19:15:30.390773 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.388961    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ac571b4d-dbf7-4e62-8d4d-c8c2a527deb3-dbus\") pod \"global-pull-secret-syncer-24lb9\" (UID: \"ac571b4d-dbf7-4e62-8d4d-c8c2a527deb3\") " pod="kube-system/global-pull-secret-syncer-24lb9"
Sep 10 19:15:30.390773 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.388975    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c7511dfe-c588-4be1-bd16-1dce34740809-run-systemd\") pod \"ovnkube-node-rrxfj\" (UID: \"c7511dfe-c588-4be1-bd16-1dce34740809\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrxfj"
Sep 10 19:15:30.390773 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.388989    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c7511dfe-c588-4be1-bd16-1dce34740809-var-lib-openvswitch\") pod \"ovnkube-node-rrxfj\" (UID: \"c7511dfe-c588-4be1-bd16-1dce34740809\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrxfj"
Sep 10 19:15:30.390773 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.389004    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d896f9fb-21e2-4845-ab2d-a855a43b0a17-etc-sysctl-conf\") pod \"tuned-g4hld\" (UID: \"d896f9fb-21e2-4845-ab2d-a855a43b0a17\") " pod="openshift-cluster-node-tuning-operator/tuned-g4hld"
Sep 10 19:15:30.390773 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.389046    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/09299baf-b713-431a-9b8b-dfbe54f74b69-multus-cni-dir\") pod \"multus-tt257\" (UID: \"09299baf-b713-431a-9b8b-dfbe54f74b69\") " pod="openshift-multus/multus-tt257"
Sep 10 19:15:30.390773 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.389067    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/09299baf-b713-431a-9b8b-dfbe54f74b69-host-run-k8s-cni-cncf-io\") pod \"multus-tt257\" (UID: \"09299baf-b713-431a-9b8b-dfbe54f74b69\") " pod="openshift-multus/multus-tt257"
Sep 10 19:15:30.390773 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.389078    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d896f9fb-21e2-4845-ab2d-a855a43b0a17-etc-sysctl-conf\") pod \"tuned-g4hld\" (UID: \"d896f9fb-21e2-4845-ab2d-a855a43b0a17\") " pod="openshift-cluster-node-tuning-operator/tuned-g4hld"
Sep 10 19:15:30.390773 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.389087    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bfed5912-60ab-48fc-91c2-10d325fd6428-metrics-certs\") pod \"network-metrics-daemon-wx2q6\" (UID: \"bfed5912-60ab-48fc-91c2-10d325fd6428\") " pod="openshift-multus/network-metrics-daemon-wx2q6"
Sep 10 19:15:30.390773 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.389122    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/83c3166f-596e-4033-b92b-02f89683ccda-host-slash\") pod \"iptables-alerter-qs9mw\" (UID: \"83c3166f-596e-4033-b92b-02f89683ccda\") " pod="openshift-network-operator/iptables-alerter-qs9mw"
Sep 10 19:15:30.390773 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.389134    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/09299baf-b713-431a-9b8b-dfbe54f74b69-multus-cni-dir\") pod \"multus-tt257\" (UID: \"09299baf-b713-431a-9b8b-dfbe54f74b69\") " pod="openshift-multus/multus-tt257"
Sep 10 19:15:30.390773 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.389142    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/09299baf-b713-431a-9b8b-dfbe54f74b69-etc-kubernetes\") pod \"multus-tt257\" (UID: \"09299baf-b713-431a-9b8b-dfbe54f74b69\") " pod="openshift-multus/multus-tt257"
Sep 10 19:15:30.390773 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.389151    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/09299baf-b713-431a-9b8b-dfbe54f74b69-host-run-k8s-cni-cncf-io\") pod \"multus-tt257\" (UID: \"09299baf-b713-431a-9b8b-dfbe54f74b69\") " pod="openshift-multus/multus-tt257"
Sep 10 19:15:30.390773 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.389163    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-66rjd\" (UniqueName: \"kubernetes.io/projected/446e8624-4aaf-471d-9653-2451db8c6889-kube-api-access-66rjd\") pod \"node-ca-nh8r2\" (UID: \"446e8624-4aaf-471d-9653-2451db8c6889\") " pod="openshift-image-registry/node-ca-nh8r2"
Sep 10 19:15:30.390773 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.389184    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ac571b4d-dbf7-4e62-8d4d-c8c2a527deb3-original-pull-secret\") pod \"global-pull-secret-syncer-24lb9\" (UID: \"ac571b4d-dbf7-4e62-8d4d-c8c2a527deb3\") " pod="kube-system/global-pull-secret-syncer-24lb9"
Sep 10 19:15:30.390773 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.389206    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/09299baf-b713-431a-9b8b-dfbe54f74b69-etc-kubernetes\") pod \"multus-tt257\" (UID: \"09299baf-b713-431a-9b8b-dfbe54f74b69\") " pod="openshift-multus/multus-tt257"
Sep 10 19:15:30.390773 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.389224    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c7511dfe-c588-4be1-bd16-1dce34740809-ovn-node-metrics-cert\") pod \"ovnkube-node-rrxfj\" (UID: \"c7511dfe-c588-4be1-bd16-1dce34740809\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrxfj"
Sep 10 19:15:30.390773 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.389253    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-spmdl\" (UniqueName: \"kubernetes.io/projected/c7511dfe-c588-4be1-bd16-1dce34740809-kube-api-access-spmdl\") pod \"ovnkube-node-rrxfj\" (UID: \"c7511dfe-c588-4be1-bd16-1dce34740809\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrxfj"
Sep 10 19:15:30.391427 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.390811    2575 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 19:15:30.391427 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.391124    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d896f9fb-21e2-4845-ab2d-a855a43b0a17-etc-tuned\") pod \"tuned-g4hld\" (UID: \"d896f9fb-21e2-4845-ab2d-a855a43b0a17\") " pod="openshift-cluster-node-tuning-operator/tuned-g4hld"
Sep 10 19:15:30.391427 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.391255    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d896f9fb-21e2-4845-ab2d-a855a43b0a17-tmp\") pod \"tuned-g4hld\" (UID: \"d896f9fb-21e2-4845-ab2d-a855a43b0a17\") " pod="openshift-cluster-node-tuning-operator/tuned-g4hld"
Sep 10 19:15:30.395768 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.395745    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xcw5d\" (UniqueName: \"kubernetes.io/projected/d896f9fb-21e2-4845-ab2d-a855a43b0a17-kube-api-access-xcw5d\") pod \"tuned-g4hld\" (UID: \"d896f9fb-21e2-4845-ab2d-a855a43b0a17\") " pod="openshift-cluster-node-tuning-operator/tuned-g4hld"
Sep 10 19:15:30.396382 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.396364    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g7skd\" (UniqueName: \"kubernetes.io/projected/d093d956-9c62-4201-a4ae-60187edb6fbf-kube-api-access-g7skd\") pod \"node-resolver-vmvhn\" (UID: \"d093d956-9c62-4201-a4ae-60187edb6fbf\") " pod="openshift-dns/node-resolver-vmvhn"
Sep 10 19:15:30.397118 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.397102    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nqnn5\" (UniqueName: \"kubernetes.io/projected/09299baf-b713-431a-9b8b-dfbe54f74b69-kube-api-access-nqnn5\") pod \"multus-tt257\" (UID: \"09299baf-b713-431a-9b8b-dfbe54f74b69\") " pod="openshift-multus/multus-tt257"
Sep 10 19:15:30.427729 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:30.427706    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5967616edfbe6b806f38063e88e3534.slice/crio-1fa25cdb26ba66686906c961df48025bded93f5eb08a1ac36ae5dc6f46cc9186 WatchSource:0}: Error finding container 1fa25cdb26ba66686906c961df48025bded93f5eb08a1ac36ae5dc6f46cc9186: Status 404 returned error can't find the container with id 1fa25cdb26ba66686906c961df48025bded93f5eb08a1ac36ae5dc6f46cc9186
Sep 10 19:15:30.428673 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:30.428656    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3eeea337b91a216332a11d902b8fa0ef.slice/crio-e859312411f93e58c849f6750fc398da1f33ec3a6364921ddec36793e8fb5b22 WatchSource:0}: Error finding container e859312411f93e58c849f6750fc398da1f33ec3a6364921ddec36793e8fb5b22: Status 404 returned error can't find the container with id e859312411f93e58c849f6750fc398da1f33ec3a6364921ddec36793e8fb5b22
Sep 10 19:15:30.432307 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.432293    2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 19:15:30.490334 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.490313    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ac571b4d-dbf7-4e62-8d4d-c8c2a527deb3-original-pull-secret\") pod \"global-pull-secret-syncer-24lb9\" (UID: \"ac571b4d-dbf7-4e62-8d4d-c8c2a527deb3\") " pod="kube-system/global-pull-secret-syncer-24lb9"
Sep 10 19:15:30.490431 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.490354    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c7511dfe-c588-4be1-bd16-1dce34740809-ovn-node-metrics-cert\") pod \"ovnkube-node-rrxfj\" (UID: \"c7511dfe-c588-4be1-bd16-1dce34740809\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrxfj"
Sep 10 19:15:30.490431 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.490374    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-spmdl\" (UniqueName: \"kubernetes.io/projected/c7511dfe-c588-4be1-bd16-1dce34740809-kube-api-access-spmdl\") pod \"ovnkube-node-rrxfj\" (UID: \"c7511dfe-c588-4be1-bd16-1dce34740809\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrxfj"
Sep 10 19:15:30.490431 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.490390    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b4070c77-f931-4973-b076-035a9e33e600-socket-dir\") pod \"aws-ebs-csi-driver-node-tx2mc\" (UID: \"b4070c77-f931-4973-b076-035a9e33e600\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tx2mc"
Sep 10 19:15:30.490431 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.490404    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cf162519-8424-43b0-8de5-a19bc237a9d0-cni-binary-copy\") pod \"multus-additional-cni-plugins-wt6fs\" (UID: \"cf162519-8424-43b0-8de5-a19bc237a9d0\") " pod="openshift-multus/multus-additional-cni-plugins-wt6fs"
Sep 10 19:15:30.490431 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:15:30.490412    2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 19:15:30.490431 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.490418    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c7511dfe-c588-4be1-bd16-1dce34740809-etc-openvswitch\") pod \"ovnkube-node-rrxfj\" (UID: \"c7511dfe-c588-4be1-bd16-1dce34740809\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrxfj"
Sep 10 19:15:30.490662 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:15:30.490469    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac571b4d-dbf7-4e62-8d4d-c8c2a527deb3-original-pull-secret podName:ac571b4d-dbf7-4e62-8d4d-c8c2a527deb3 nodeName:}" failed. No retries permitted until 2026-09-10 19:15:30.990448401 +0000 UTC m=+2.050276066 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ac571b4d-dbf7-4e62-8d4d-c8c2a527deb3-original-pull-secret") pod "global-pull-secret-syncer-24lb9" (UID: "ac571b4d-dbf7-4e62-8d4d-c8c2a527deb3") : object "kube-system"/"original-pull-secret" not registered
Sep 10 19:15:30.490662 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.490514    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c7511dfe-c588-4be1-bd16-1dce34740809-etc-openvswitch\") pod \"ovnkube-node-rrxfj\" (UID: \"c7511dfe-c588-4be1-bd16-1dce34740809\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrxfj"
Sep 10 19:15:30.490662 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.490537    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b4070c77-f931-4973-b076-035a9e33e600-socket-dir\") pod \"aws-ebs-csi-driver-node-tx2mc\" (UID: \"b4070c77-f931-4973-b076-035a9e33e600\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tx2mc"
Sep 10 19:15:30.490662 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.490571    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-24wkn\" (UniqueName: \"kubernetes.io/projected/bfed5912-60ab-48fc-91c2-10d325fd6428-kube-api-access-24wkn\") pod \"network-metrics-daemon-wx2q6\" (UID: \"bfed5912-60ab-48fc-91c2-10d325fd6428\") " pod="openshift-multus/network-metrics-daemon-wx2q6"
Sep 10 19:15:30.490662 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.490599    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cf162519-8424-43b0-8de5-a19bc237a9d0-system-cni-dir\") pod \"multus-additional-cni-plugins-wt6fs\" (UID: \"cf162519-8424-43b0-8de5-a19bc237a9d0\") " pod="openshift-multus/multus-additional-cni-plugins-wt6fs"
Sep 10 19:15:30.490662 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.490622    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cf162519-8424-43b0-8de5-a19bc237a9d0-cnibin\") pod \"multus-additional-cni-plugins-wt6fs\" (UID: \"cf162519-8424-43b0-8de5-a19bc237a9d0\") " pod="openshift-multus/multus-additional-cni-plugins-wt6fs"
Sep 10 19:15:30.490662 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.490649    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c7511dfe-c588-4be1-bd16-1dce34740809-host-run-netns\") pod \"ovnkube-node-rrxfj\" (UID: \"c7511dfe-c588-4be1-bd16-1dce34740809\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrxfj"
Sep 10 19:15:30.491006 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.490676    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b4070c77-f931-4973-b076-035a9e33e600-kubelet-dir\") pod \"aws-ebs-csi-driver-node-tx2mc\" (UID: \"b4070c77-f931-4973-b076-035a9e33e600\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tx2mc"
Sep 10 19:15:30.491006 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.490699    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c7511dfe-c588-4be1-bd16-1dce34740809-host-kubelet\") pod \"ovnkube-node-rrxfj\" (UID: \"c7511dfe-c588-4be1-bd16-1dce34740809\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrxfj"
Sep 10 19:15:30.491006 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.490722    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c7511dfe-c588-4be1-bd16-1dce34740809-log-socket\") pod \"ovnkube-node-rrxfj\" (UID: \"c7511dfe-c588-4be1-bd16-1dce34740809\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrxfj"
Sep 10 19:15:30.491006 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.490733    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cf162519-8424-43b0-8de5-a19bc237a9d0-cnibin\") pod \"multus-additional-cni-plugins-wt6fs\" (UID: \"cf162519-8424-43b0-8de5-a19bc237a9d0\") " pod="openshift-multus/multus-additional-cni-plugins-wt6fs"
Sep 10 19:15:30.491006 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.490751    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/cf162519-8424-43b0-8de5-a19bc237a9d0-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wt6fs\" (UID: \"cf162519-8424-43b0-8de5-a19bc237a9d0\") " pod="openshift-multus/multus-additional-cni-plugins-wt6fs"
Sep 10 19:15:30.491006 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.490694    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cf162519-8424-43b0-8de5-a19bc237a9d0-system-cni-dir\") pod \"multus-additional-cni-plugins-wt6fs\" (UID: \"cf162519-8424-43b0-8de5-a19bc237a9d0\") " pod="openshift-multus/multus-additional-cni-plugins-wt6fs"
Sep 10 19:15:30.491006 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.490759    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c7511dfe-c588-4be1-bd16-1dce34740809-host-run-netns\") pod \"ovnkube-node-rrxfj\" (UID: \"c7511dfe-c588-4be1-bd16-1dce34740809\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrxfj"
Sep 10 19:15:30.491006 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.490777    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z5zkc\" (UniqueName: \"kubernetes.io/projected/cf162519-8424-43b0-8de5-a19bc237a9d0-kube-api-access-z5zkc\") pod \"multus-additional-cni-plugins-wt6fs\" (UID: \"cf162519-8424-43b0-8de5-a19bc237a9d0\") " pod="openshift-multus/multus-additional-cni-plugins-wt6fs"
Sep 10 19:15:30.491006 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.490785    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b4070c77-f931-4973-b076-035a9e33e600-kubelet-dir\") pod \"aws-ebs-csi-driver-node-tx2mc\" (UID: \"b4070c77-f931-4973-b076-035a9e33e600\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tx2mc"
Sep 10 19:15:30.491006 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.490800    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c7511dfe-c588-4be1-bd16-1dce34740809-ovnkube-config\") pod \"ovnkube-node-rrxfj\" (UID: \"c7511dfe-c588-4be1-bd16-1dce34740809\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrxfj"
Sep 10 19:15:30.491006 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.490806    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c7511dfe-c588-4be1-bd16-1dce34740809-log-socket\") pod \"ovnkube-node-rrxfj\" (UID: \"c7511dfe-c588-4be1-bd16-1dce34740809\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrxfj"
Sep 10 19:15:30.491006 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.490823    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c7511dfe-c588-4be1-bd16-1dce34740809-host-kubelet\") pod \"ovnkube-node-rrxfj\" (UID: \"c7511dfe-c588-4be1-bd16-1dce34740809\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrxfj"
Sep 10 19:15:30.491006 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.490833    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a76b5c99-a97f-461d-bce3-1fc84ef52bae-konnectivity-ca\") pod \"konnectivity-agent-bbmp5\" (UID: \"a76b5c99-a97f-461d-bce3-1fc84ef52bae\") " pod="kube-system/konnectivity-agent-bbmp5"
Sep 10 19:15:30.491006 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.490869    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c7511dfe-c588-4be1-bd16-1dce34740809-host-slash\") pod \"ovnkube-node-rrxfj\" (UID: \"c7511dfe-c588-4be1-bd16-1dce34740809\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrxfj"
Sep 10 19:15:30.491006 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.490895    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c7511dfe-c588-4be1-bd16-1dce34740809-run-openvswitch\") pod \"ovnkube-node-rrxfj\" (UID: \"c7511dfe-c588-4be1-bd16-1dce34740809\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrxfj"
Sep 10 19:15:30.491006 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.490921    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c7511dfe-c588-4be1-bd16-1dce34740809-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-rrxfj\" (UID: \"c7511dfe-c588-4be1-bd16-1dce34740809\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrxfj"
Sep 10 19:15:30.491006 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.490946    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b4070c77-f931-4973-b076-035a9e33e600-sys-fs\") pod \"aws-ebs-csi-driver-node-tx2mc\" (UID: \"b4070c77-f931-4973-b076-035a9e33e600\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tx2mc"
Sep 10 19:15:30.491881 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.490968    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c7511dfe-c588-4be1-bd16-1dce34740809-host-cni-bin\") pod \"ovnkube-node-rrxfj\" (UID: \"c7511dfe-c588-4be1-bd16-1dce34740809\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrxfj"
Sep 10 19:15:30.491881 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.490997    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c7511dfe-c588-4be1-bd16-1dce34740809-host-cni-netd\") pod \"ovnkube-node-rrxfj\" (UID: \"c7511dfe-c588-4be1-bd16-1dce34740809\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrxfj"
Sep 10 19:15:30.491881 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.491010    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c7511dfe-c588-4be1-bd16-1dce34740809-run-openvswitch\") pod \"ovnkube-node-rrxfj\" (UID: \"c7511dfe-c588-4be1-bd16-1dce34740809\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrxfj"
Sep 10 19:15:30.491881 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.491019    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c7511dfe-c588-4be1-bd16-1dce34740809-ovnkube-script-lib\") pod \"ovnkube-node-rrxfj\" (UID: \"c7511dfe-c588-4be1-bd16-1dce34740809\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrxfj"
Sep 10 19:15:30.491881 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.491023    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c7511dfe-c588-4be1-bd16-1dce34740809-host-slash\") pod \"ovnkube-node-rrxfj\" (UID: \"c7511dfe-c588-4be1-bd16-1dce34740809\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrxfj"
Sep 10 19:15:30.491881 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.491026    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cf162519-8424-43b0-8de5-a19bc237a9d0-cni-binary-copy\") pod \"multus-additional-cni-plugins-wt6fs\" (UID: \"cf162519-8424-43b0-8de5-a19bc237a9d0\") " pod="openshift-multus/multus-additional-cni-plugins-wt6fs"
Sep 10 19:15:30.491881 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.491049    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/446e8624-4aaf-471d-9653-2451db8c6889-host\") pod \"node-ca-nh8r2\" (UID: \"446e8624-4aaf-471d-9653-2451db8c6889\") " pod="openshift-image-registry/node-ca-nh8r2"
Sep 10 19:15:30.491881 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.491075    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b4070c77-f931-4973-b076-035a9e33e600-sys-fs\") pod \"aws-ebs-csi-driver-node-tx2mc\" (UID: \"b4070c77-f931-4973-b076-035a9e33e600\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tx2mc"
Sep 10 19:15:30.491881 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.491074    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c7511dfe-c588-4be1-bd16-1dce34740809-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-rrxfj\" (UID: \"c7511dfe-c588-4be1-bd16-1dce34740809\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrxfj"
Sep 10 19:15:30.491881 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.491097    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/446e8624-4aaf-471d-9653-2451db8c6889-host\") pod \"node-ca-nh8r2\" (UID: \"446e8624-4aaf-471d-9653-2451db8c6889\") " pod="openshift-image-registry/node-ca-nh8r2"
Sep 10 19:15:30.491881 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.491098    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c7511dfe-c588-4be1-bd16-1dce34740809-host-cni-bin\") pod \"ovnkube-node-rrxfj\" (UID: \"c7511dfe-c588-4be1-bd16-1dce34740809\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrxfj"
Sep 10 19:15:30.491881 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.491148    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c7511dfe-c588-4be1-bd16-1dce34740809-host-cni-netd\") pod \"ovnkube-node-rrxfj\" (UID: \"c7511dfe-c588-4be1-bd16-1dce34740809\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrxfj"
Sep 10 19:15:30.491881 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.491173    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ac571b4d-dbf7-4e62-8d4d-c8c2a527deb3-kubelet-config\") pod \"global-pull-secret-syncer-24lb9\" (UID: \"ac571b4d-dbf7-4e62-8d4d-c8c2a527deb3\") " pod="kube-system/global-pull-secret-syncer-24lb9"
Sep 10 19:15:30.491881 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.491208    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b4070c77-f931-4973-b076-035a9e33e600-registration-dir\") pod \"aws-ebs-csi-driver-node-tx2mc\" (UID: \"b4070c77-f931-4973-b076-035a9e33e600\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tx2mc"
Sep 10 19:15:30.491881 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.491231    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a76b5c99-a97f-461d-bce3-1fc84ef52bae-agent-certs\") pod \"konnectivity-agent-bbmp5\" (UID: \"a76b5c99-a97f-461d-bce3-1fc84ef52bae\") " pod="kube-system/konnectivity-agent-bbmp5"
Sep 10 19:15:30.491881 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.491253    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c7511dfe-c588-4be1-bd16-1dce34740809-systemd-units\") pod \"ovnkube-node-rrxfj\" (UID: \"c7511dfe-c588-4be1-bd16-1dce34740809\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrxfj"
Sep 10 19:15:30.491881 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.491267    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b4070c77-f931-4973-b076-035a9e33e600-registration-dir\") pod \"aws-ebs-csi-driver-node-tx2mc\" (UID: \"b4070c77-f931-4973-b076-035a9e33e600\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tx2mc"
Sep 10 19:15:30.492641 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.491273    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ac571b4d-dbf7-4e62-8d4d-c8c2a527deb3-kubelet-config\") pod \"global-pull-secret-syncer-24lb9\" (UID: \"ac571b4d-dbf7-4e62-8d4d-c8c2a527deb3\") " pod="kube-system/global-pull-secret-syncer-24lb9"
Sep 10 19:15:30.492641 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.491283    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c7511dfe-c588-4be1-bd16-1dce34740809-host-run-ovn-kubernetes\") pod \"ovnkube-node-rrxfj\" (UID: \"c7511dfe-c588-4be1-bd16-1dce34740809\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrxfj"
Sep 10 19:15:30.492641 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.491309    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/83c3166f-596e-4033-b92b-02f89683ccda-iptables-alerter-script\") pod \"iptables-alerter-qs9mw\" (UID: \"83c3166f-596e-4033-b92b-02f89683ccda\") " pod="openshift-network-operator/iptables-alerter-qs9mw"
Sep 10 19:15:30.492641 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.491324    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c7511dfe-c588-4be1-bd16-1dce34740809-systemd-units\") pod \"ovnkube-node-rrxfj\" (UID: \"c7511dfe-c588-4be1-bd16-1dce34740809\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrxfj"
Sep 10 19:15:30.492641 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.491333    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wtxj6\" (UniqueName: \"kubernetes.io/projected/83c3166f-596e-4033-b92b-02f89683ccda-kube-api-access-wtxj6\") pod \"iptables-alerter-qs9mw\" (UID: \"83c3166f-596e-4033-b92b-02f89683ccda\") " pod="openshift-network-operator/iptables-alerter-qs9mw"
Sep 10 19:15:30.492641 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.491389    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b4070c77-f931-4973-b076-035a9e33e600-etc-selinux\") pod \"aws-ebs-csi-driver-node-tx2mc\" (UID: \"b4070c77-f931-4973-b076-035a9e33e600\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tx2mc"
Sep 10 19:15:30.492641 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.491414    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cf162519-8424-43b0-8de5-a19bc237a9d0-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wt6fs\" (UID: \"cf162519-8424-43b0-8de5-a19bc237a9d0\") " pod="openshift-multus/multus-additional-cni-plugins-wt6fs"
Sep 10 19:15:30.492641 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.491431    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/cf162519-8424-43b0-8de5-a19bc237a9d0-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wt6fs\" (UID: \"cf162519-8424-43b0-8de5-a19bc237a9d0\") " pod="openshift-multus/multus-additional-cni-plugins-wt6fs"
Sep 10 19:15:30.492641 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.491440    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b4070c77-f931-4973-b076-035a9e33e600-device-dir\") pod \"aws-ebs-csi-driver-node-tx2mc\" (UID: \"b4070c77-f931-4973-b076-035a9e33e600\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tx2mc"
Sep 10 19:15:30.492641 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.491386    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c7511dfe-c588-4be1-bd16-1dce34740809-host-run-ovn-kubernetes\") pod \"ovnkube-node-rrxfj\" (UID: \"c7511dfe-c588-4be1-bd16-1dce34740809\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrxfj"
Sep 10 19:15:30.492641 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.491462    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/446e8624-4aaf-471d-9653-2451db8c6889-serviceca\") pod \"node-ca-nh8r2\" (UID: \"446e8624-4aaf-471d-9653-2451db8c6889\") " pod="openshift-image-registry/node-ca-nh8r2"
Sep 10 19:15:30.492641 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.491487    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c7511dfe-c588-4be1-bd16-1dce34740809-run-ovn\") pod \"ovnkube-node-rrxfj\" (UID: \"c7511dfe-c588-4be1-bd16-1dce34740809\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrxfj"
Sep 10 19:15:30.492641 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.491511    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c7511dfe-c588-4be1-bd16-1dce34740809-node-log\") pod \"ovnkube-node-rrxfj\" (UID: \"c7511dfe-c588-4be1-bd16-1dce34740809\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrxfj"
Sep 10 19:15:30.492641 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.491534    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a76b5c99-a97f-461d-bce3-1fc84ef52bae-konnectivity-ca\") pod \"konnectivity-agent-bbmp5\" (UID: \"a76b5c99-a97f-461d-bce3-1fc84ef52bae\") " pod="kube-system/konnectivity-agent-bbmp5"
Sep 10 19:15:30.492641 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.491563    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n2p5q\" (UniqueName: \"kubernetes.io/projected/cbd42c54-3649-4d2d-8f94-d2789f7f66c7-kube-api-access-n2p5q\") pod \"network-check-target-9rkjw\" (UID: \"cbd42c54-3649-4d2d-8f94-d2789f7f66c7\") " pod="openshift-network-diagnostics/network-check-target-9rkjw"
Sep 10 19:15:30.492641 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.491589    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c7511dfe-c588-4be1-bd16-1dce34740809-env-overrides\") pod \"ovnkube-node-rrxfj\" (UID: \"c7511dfe-c588-4be1-bd16-1dce34740809\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrxfj"
Sep 10 19:15:30.492641 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.491607    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b4070c77-f931-4973-b076-035a9e33e600-etc-selinux\") pod \"aws-ebs-csi-driver-node-tx2mc\" (UID: \"b4070c77-f931-4973-b076-035a9e33e600\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tx2mc"
Sep 10 19:15:30.493126 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.491616    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rs7nz\" (UniqueName: \"kubernetes.io/projected/b4070c77-f931-4973-b076-035a9e33e600-kube-api-access-rs7nz\") pod \"aws-ebs-csi-driver-node-tx2mc\" (UID: \"b4070c77-f931-4973-b076-035a9e33e600\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tx2mc"
Sep 10 19:15:30.493126 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.491651    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/cf162519-8424-43b0-8de5-a19bc237a9d0-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wt6fs\" (UID: \"cf162519-8424-43b0-8de5-a19bc237a9d0\") " pod="openshift-multus/multus-additional-cni-plugins-wt6fs"
Sep 10 19:15:30.493126 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.491675    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ac571b4d-dbf7-4e62-8d4d-c8c2a527deb3-dbus\") pod \"global-pull-secret-syncer-24lb9\" (UID: \"ac571b4d-dbf7-4e62-8d4d-c8c2a527deb3\") " pod="kube-system/global-pull-secret-syncer-24lb9"
Sep 10 19:15:30.493126 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.491697    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c7511dfe-c588-4be1-bd16-1dce34740809-run-systemd\") pod \"ovnkube-node-rrxfj\" (UID: \"c7511dfe-c588-4be1-bd16-1dce34740809\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrxfj"
Sep 10 19:15:30.493126 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.491699    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b4070c77-f931-4973-b076-035a9e33e600-device-dir\") pod \"aws-ebs-csi-driver-node-tx2mc\" (UID: \"b4070c77-f931-4973-b076-035a9e33e600\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tx2mc"
Sep 10 19:15:30.493126 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.491762    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c7511dfe-c588-4be1-bd16-1dce34740809-node-log\") pod \"ovnkube-node-rrxfj\" (UID: \"c7511dfe-c588-4be1-bd16-1dce34740809\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrxfj"
Sep 10 19:15:30.493126 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.491888    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/83c3166f-596e-4033-b92b-02f89683ccda-iptables-alerter-script\") pod \"iptables-alerter-qs9mw\" (UID: \"83c3166f-596e-4033-b92b-02f89683ccda\") " pod="openshift-network-operator/iptables-alerter-qs9mw"
Sep 10 19:15:30.493126 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.491944    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c7511dfe-c588-4be1-bd16-1dce34740809-run-ovn\") pod \"ovnkube-node-rrxfj\" (UID: \"c7511dfe-c588-4be1-bd16-1dce34740809\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrxfj"
Sep 10 19:15:30.493126 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.491975    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c7511dfe-c588-4be1-bd16-1dce34740809-ovnkube-config\") pod \"ovnkube-node-rrxfj\" (UID: \"c7511dfe-c588-4be1-bd16-1dce34740809\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrxfj"
Sep 10 19:15:30.493126 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.492059    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ac571b4d-dbf7-4e62-8d4d-c8c2a527deb3-dbus\") pod \"global-pull-secret-syncer-24lb9\" (UID: \"ac571b4d-dbf7-4e62-8d4d-c8c2a527deb3\") " pod="kube-system/global-pull-secret-syncer-24lb9"
Sep 10 19:15:30.493126 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.492107    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c7511dfe-c588-4be1-bd16-1dce34740809-run-systemd\") pod \"ovnkube-node-rrxfj\" (UID: \"c7511dfe-c588-4be1-bd16-1dce34740809\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrxfj"
Sep 10 19:15:30.493126 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.492119    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/446e8624-4aaf-471d-9653-2451db8c6889-serviceca\") pod \"node-ca-nh8r2\" (UID: \"446e8624-4aaf-471d-9653-2451db8c6889\") " pod="openshift-image-registry/node-ca-nh8r2"
Sep 10 19:15:30.493126 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.492142    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c7511dfe-c588-4be1-bd16-1dce34740809-ovnkube-script-lib\") pod \"ovnkube-node-rrxfj\" (UID: \"c7511dfe-c588-4be1-bd16-1dce34740809\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrxfj"
Sep 10 19:15:30.493126 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.492160    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c7511dfe-c588-4be1-bd16-1dce34740809-var-lib-openvswitch\") pod \"ovnkube-node-rrxfj\" (UID: \"c7511dfe-c588-4be1-bd16-1dce34740809\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrxfj"
Sep 10 19:15:30.493126 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.492192    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c7511dfe-c588-4be1-bd16-1dce34740809-var-lib-openvswitch\") pod \"ovnkube-node-rrxfj\" (UID: \"c7511dfe-c588-4be1-bd16-1dce34740809\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrxfj"
Sep 10 19:15:30.493126 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.492192    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bfed5912-60ab-48fc-91c2-10d325fd6428-metrics-certs\") pod \"network-metrics-daemon-wx2q6\" (UID: \"bfed5912-60ab-48fc-91c2-10d325fd6428\") " pod="openshift-multus/network-metrics-daemon-wx2q6"
Sep 10 19:15:30.493126 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.492229    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/83c3166f-596e-4033-b92b-02f89683ccda-host-slash\") pod \"iptables-alerter-qs9mw\" (UID: \"83c3166f-596e-4033-b92b-02f89683ccda\") " pod="openshift-network-operator/iptables-alerter-qs9mw"
Sep 10 19:15:30.493606 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.492254    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-66rjd\" (UniqueName: \"kubernetes.io/projected/446e8624-4aaf-471d-9653-2451db8c6889-kube-api-access-66rjd\") pod \"node-ca-nh8r2\" (UID: \"446e8624-4aaf-471d-9653-2451db8c6889\") " pod="openshift-image-registry/node-ca-nh8r2"
Sep 10 19:15:30.493606 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:15:30.492257    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 19:15:30.493606 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.492285    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/83c3166f-596e-4033-b92b-02f89683ccda-host-slash\") pod \"iptables-alerter-qs9mw\" (UID: \"83c3166f-596e-4033-b92b-02f89683ccda\") " pod="openshift-network-operator/iptables-alerter-qs9mw"
Sep 10 19:15:30.493606 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.492300    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c7511dfe-c588-4be1-bd16-1dce34740809-env-overrides\") pod \"ovnkube-node-rrxfj\" (UID: \"c7511dfe-c588-4be1-bd16-1dce34740809\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrxfj"
Sep 10 19:15:30.493606 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:15:30.492332    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bfed5912-60ab-48fc-91c2-10d325fd6428-metrics-certs podName:bfed5912-60ab-48fc-91c2-10d325fd6428 nodeName:}" failed. No retries permitted until 2026-09-10 19:15:30.992315754 +0000 UTC m=+2.052143403 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bfed5912-60ab-48fc-91c2-10d325fd6428-metrics-certs") pod "network-metrics-daemon-wx2q6" (UID: "bfed5912-60ab-48fc-91c2-10d325fd6428") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 19:15:30.493606 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.492441    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cf162519-8424-43b0-8de5-a19bc237a9d0-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wt6fs\" (UID: \"cf162519-8424-43b0-8de5-a19bc237a9d0\") " pod="openshift-multus/multus-additional-cni-plugins-wt6fs"
Sep 10 19:15:30.493606 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.492452    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/cf162519-8424-43b0-8de5-a19bc237a9d0-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wt6fs\" (UID: \"cf162519-8424-43b0-8de5-a19bc237a9d0\") " pod="openshift-multus/multus-additional-cni-plugins-wt6fs"
Sep 10 19:15:30.493606 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.492981    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c7511dfe-c588-4be1-bd16-1dce34740809-ovn-node-metrics-cert\") pod \"ovnkube-node-rrxfj\" (UID: \"c7511dfe-c588-4be1-bd16-1dce34740809\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrxfj"
Sep 10 19:15:30.493606 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.493524    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a76b5c99-a97f-461d-bce3-1fc84ef52bae-agent-certs\") pod \"konnectivity-agent-bbmp5\" (UID: \"a76b5c99-a97f-461d-bce3-1fc84ef52bae\") " pod="kube-system/konnectivity-agent-bbmp5"
Sep 10 19:15:30.498169 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:15:30.498149    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 19:15:30.498271 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:15:30.498177    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 19:15:30.498271 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:15:30.498193    2575 projected.go:196] Error preparing data for projected volume kube-api-access-n2p5q for pod openshift-network-diagnostics/network-check-target-9rkjw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 19:15:30.498271 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:15:30.498265    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cbd42c54-3649-4d2d-8f94-d2789f7f66c7-kube-api-access-n2p5q podName:cbd42c54-3649-4d2d-8f94-d2789f7f66c7 nodeName:}" failed. No retries permitted until 2026-09-10 19:15:30.998248589 +0000 UTC m=+2.058076239 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-n2p5q" (UniqueName: "kubernetes.io/projected/cbd42c54-3649-4d2d-8f94-d2789f7f66c7-kube-api-access-n2p5q") pod "network-check-target-9rkjw" (UID: "cbd42c54-3649-4d2d-8f94-d2789f7f66c7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 19:15:30.498844 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.498820    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-24wkn\" (UniqueName: \"kubernetes.io/projected/bfed5912-60ab-48fc-91c2-10d325fd6428-kube-api-access-24wkn\") pod \"network-metrics-daemon-wx2q6\" (UID: \"bfed5912-60ab-48fc-91c2-10d325fd6428\") " pod="openshift-multus/network-metrics-daemon-wx2q6"
Sep 10 19:15:30.499137 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.499113    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-spmdl\" (UniqueName: \"kubernetes.io/projected/c7511dfe-c588-4be1-bd16-1dce34740809-kube-api-access-spmdl\") pod \"ovnkube-node-rrxfj\" (UID: \"c7511dfe-c588-4be1-bd16-1dce34740809\") " pod="openshift-ovn-kubernetes/ovnkube-node-rrxfj"
Sep 10 19:15:30.499463 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.499448    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z5zkc\" (UniqueName: \"kubernetes.io/projected/cf162519-8424-43b0-8de5-a19bc237a9d0-kube-api-access-z5zkc\") pod \"multus-additional-cni-plugins-wt6fs\" (UID: \"cf162519-8424-43b0-8de5-a19bc237a9d0\") " pod="openshift-multus/multus-additional-cni-plugins-wt6fs"
Sep 10 19:15:30.499894 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.499875    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wtxj6\" (UniqueName: \"kubernetes.io/projected/83c3166f-596e-4033-b92b-02f89683ccda-kube-api-access-wtxj6\") pod \"iptables-alerter-qs9mw\" (UID: \"83c3166f-596e-4033-b92b-02f89683ccda\") " pod="openshift-network-operator/iptables-alerter-qs9mw"
Sep 10 19:15:30.500221 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.500206    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-66rjd\" (UniqueName: \"kubernetes.io/projected/446e8624-4aaf-471d-9653-2451db8c6889-kube-api-access-66rjd\") pod \"node-ca-nh8r2\" (UID: \"446e8624-4aaf-471d-9653-2451db8c6889\") " pod="openshift-image-registry/node-ca-nh8r2"
Sep 10 19:15:30.500264 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.500242    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rs7nz\" (UniqueName: \"kubernetes.io/projected/b4070c77-f931-4973-b076-035a9e33e600-kube-api-access-rs7nz\") pod \"aws-ebs-csi-driver-node-tx2mc\" (UID: \"b4070c77-f931-4973-b076-035a9e33e600\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tx2mc"
Sep 10 19:15:30.609436 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.609385    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-tt257"
Sep 10 19:15:30.609534 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.609516    2575 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 19:15:30.618976 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:30.618946    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09299baf_b713_431a_9b8b_dfbe54f74b69.slice/crio-2387519df02b6b5734e9f0f304d6658732d9d95235572ed328f03dd22822a861 WatchSource:0}: Error finding container 2387519df02b6b5734e9f0f304d6658732d9d95235572ed328f03dd22822a861: Status 404 returned error can't find the container with id 2387519df02b6b5734e9f0f304d6658732d9d95235572ed328f03dd22822a861
Sep 10 19:15:30.623189 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.623175    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-vmvhn"
Sep 10 19:15:30.630880 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:30.630861    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd093d956_9c62_4201_a4ae_60187edb6fbf.slice/crio-78f1c8b48eaf458f31540450aeb4ae58d7ec0e791ef310e219bd932280e80da5 WatchSource:0}: Error finding container 78f1c8b48eaf458f31540450aeb4ae58d7ec0e791ef310e219bd932280e80da5: Status 404 returned error can't find the container with id 78f1c8b48eaf458f31540450aeb4ae58d7ec0e791ef310e219bd932280e80da5
Sep 10 19:15:30.634636 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.634620    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-g4hld"
Sep 10 19:15:30.642569 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:30.642550    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd896f9fb_21e2_4845_ab2d_a855a43b0a17.slice/crio-b800647f783695e6019508fdeacedec12365f7e020160f6796a8156b8a396acf WatchSource:0}: Error finding container b800647f783695e6019508fdeacedec12365f7e020160f6796a8156b8a396acf: Status 404 returned error can't find the container with id b800647f783695e6019508fdeacedec12365f7e020160f6796a8156b8a396acf
Sep 10 19:15:30.650183 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.650166    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-nh8r2"
Sep 10 19:15:30.655941 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.655924    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-rrxfj"
Sep 10 19:15:30.657317 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:30.657295    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod446e8624_4aaf_471d_9653_2451db8c6889.slice/crio-da1afeaf7ab07d474cdfa565f2d80ed4420c74c5597503b8b448a7198111dece WatchSource:0}: Error finding container da1afeaf7ab07d474cdfa565f2d80ed4420c74c5597503b8b448a7198111dece: Status 404 returned error can't find the container with id da1afeaf7ab07d474cdfa565f2d80ed4420c74c5597503b8b448a7198111dece
Sep 10 19:15:30.663128 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:30.663111    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7511dfe_c588_4be1_bd16_1dce34740809.slice/crio-f2649a254e0cb5a49370c1a37efe9a870c8fb373b3ab2873ca1a7ea1bf5831ec WatchSource:0}: Error finding container f2649a254e0cb5a49370c1a37efe9a870c8fb373b3ab2873ca1a7ea1bf5831ec: Status 404 returned error can't find the container with id f2649a254e0cb5a49370c1a37efe9a870c8fb373b3ab2873ca1a7ea1bf5831ec
Sep 10 19:15:30.682559 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.682540    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-wt6fs"
Sep 10 19:15:30.689221 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.689206    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-qs9mw"
Sep 10 19:15:30.689890 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:30.689872    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf162519_8424_43b0_8de5_a19bc237a9d0.slice/crio-3644cbc0cffbe3a7b921d43d5b866bf9b16254c4ec1d7d18e25edc4977e5e3c4 WatchSource:0}: Error finding container 3644cbc0cffbe3a7b921d43d5b866bf9b16254c4ec1d7d18e25edc4977e5e3c4: Status 404 returned error can't find the container with id 3644cbc0cffbe3a7b921d43d5b866bf9b16254c4ec1d7d18e25edc4977e5e3c4
Sep 10 19:15:30.696640 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:30.696620    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83c3166f_596e_4033_b92b_02f89683ccda.slice/crio-40c21859d75cfc4038e0c842a18164f1e6e18ef73c1387befdff3d124126567d WatchSource:0}: Error finding container 40c21859d75cfc4038e0c842a18164f1e6e18ef73c1387befdff3d124126567d: Status 404 returned error can't find the container with id 40c21859d75cfc4038e0c842a18164f1e6e18ef73c1387befdff3d124126567d
Sep 10 19:15:30.717474 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.717453    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tx2mc"
Sep 10 19:15:30.724032 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.724017    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-bbmp5"
Sep 10 19:15:30.726114 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:30.726097    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4070c77_f931_4973_b076_035a9e33e600.slice/crio-8a6575a9565ec4361a0d35675df2cac6d858e61eb87735aadb45f987a60a8f5d WatchSource:0}: Error finding container 8a6575a9565ec4361a0d35675df2cac6d858e61eb87735aadb45f987a60a8f5d: Status 404 returned error can't find the container with id 8a6575a9565ec4361a0d35675df2cac6d858e61eb87735aadb45f987a60a8f5d
Sep 10 19:15:30.731803 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:15:30.731784    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda76b5c99_a97f_461d_bce3_1fc84ef52bae.slice/crio-07f98618cfe2260b49a1683382a0eac245485f4a77da392a8148cc16a985fa5f WatchSource:0}: Error finding container 07f98618cfe2260b49a1683382a0eac245485f4a77da392a8148cc16a985fa5f: Status 404 returned error can't find the container with id 07f98618cfe2260b49a1683382a0eac245485f4a77da392a8148cc16a985fa5f
Sep 10 19:15:30.996106 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.995825    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bfed5912-60ab-48fc-91c2-10d325fd6428-metrics-certs\") pod \"network-metrics-daemon-wx2q6\" (UID: \"bfed5912-60ab-48fc-91c2-10d325fd6428\") " pod="openshift-multus/network-metrics-daemon-wx2q6"
Sep 10 19:15:30.996106 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:30.995867    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ac571b4d-dbf7-4e62-8d4d-c8c2a527deb3-original-pull-secret\") pod \"global-pull-secret-syncer-24lb9\" (UID: \"ac571b4d-dbf7-4e62-8d4d-c8c2a527deb3\") " pod="kube-system/global-pull-secret-syncer-24lb9"
Sep 10 19:15:30.996106 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:15:30.996036    2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 19:15:30.996106 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:15:30.996096    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac571b4d-dbf7-4e62-8d4d-c8c2a527deb3-original-pull-secret podName:ac571b4d-dbf7-4e62-8d4d-c8c2a527deb3 nodeName:}" failed. No retries permitted until 2026-09-10 19:15:31.996076724 +0000 UTC m=+3.055904388 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ac571b4d-dbf7-4e62-8d4d-c8c2a527deb3-original-pull-secret") pod "global-pull-secret-syncer-24lb9" (UID: "ac571b4d-dbf7-4e62-8d4d-c8c2a527deb3") : object "kube-system"/"original-pull-secret" not registered
Sep 10 19:15:30.996469 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:15:30.996161    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 19:15:30.996469 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:15:30.996215    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bfed5912-60ab-48fc-91c2-10d325fd6428-metrics-certs podName:bfed5912-60ab-48fc-91c2-10d325fd6428 nodeName:}" failed. No retries permitted until 2026-09-10 19:15:31.996199534 +0000 UTC m=+3.056027182 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bfed5912-60ab-48fc-91c2-10d325fd6428-metrics-certs") pod "network-metrics-daemon-wx2q6" (UID: "bfed5912-60ab-48fc-91c2-10d325fd6428") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 19:15:31.097060 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:31.097031    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n2p5q\" (UniqueName: \"kubernetes.io/projected/cbd42c54-3649-4d2d-8f94-d2789f7f66c7-kube-api-access-n2p5q\") pod \"network-check-target-9rkjw\" (UID: \"cbd42c54-3649-4d2d-8f94-d2789f7f66c7\") " pod="openshift-network-diagnostics/network-check-target-9rkjw"
Sep 10 19:15:31.097224 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:15:31.097146    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 19:15:31.097224 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:15:31.097163    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 19:15:31.097224 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:15:31.097171    2575 projected.go:196] Error preparing data for projected volume kube-api-access-n2p5q for pod openshift-network-diagnostics/network-check-target-9rkjw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 19:15:31.097224 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:15:31.097216    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cbd42c54-3649-4d2d-8f94-d2789f7f66c7-kube-api-access-n2p5q podName:cbd42c54-3649-4d2d-8f94-d2789f7f66c7 nodeName:}" failed. No retries permitted until 2026-09-10 19:15:32.097200298 +0000 UTC m=+3.157027962 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-n2p5q" (UniqueName: "kubernetes.io/projected/cbd42c54-3649-4d2d-8f94-d2789f7f66c7-kube-api-access-n2p5q") pod "network-check-target-9rkjw" (UID: "cbd42c54-3649-4d2d-8f94-d2789f7f66c7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 19:15:31.331174 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:31.331075    2575 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-09 19:10:30 +0000 UTC" deadline="2028-06-11 10:08:02.351399744 +0000 UTC"
Sep 10 19:15:31.331174 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:31.331133    2575 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15350h52m31.020270372s"
Sep 10 19:15:31.400760 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:31.400702    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rrxfj" event={"ID":"c7511dfe-c588-4be1-bd16-1dce34740809","Type":"ContainerStarted","Data":"f2649a254e0cb5a49370c1a37efe9a870c8fb373b3ab2873ca1a7ea1bf5831ec"}
Sep 10 19:15:31.408578 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:31.408550    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-nh8r2" event={"ID":"446e8624-4aaf-471d-9653-2451db8c6889","Type":"ContainerStarted","Data":"da1afeaf7ab07d474cdfa565f2d80ed4420c74c5597503b8b448a7198111dece"}
Sep 10 19:15:31.416902 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:31.416872    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-g4hld" event={"ID":"d896f9fb-21e2-4845-ab2d-a855a43b0a17","Type":"ContainerStarted","Data":"b800647f783695e6019508fdeacedec12365f7e020160f6796a8156b8a396acf"}
Sep 10 19:15:31.419744 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:31.419687    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-vmvhn" event={"ID":"d093d956-9c62-4201-a4ae-60187edb6fbf","Type":"ContainerStarted","Data":"78f1c8b48eaf458f31540450aeb4ae58d7ec0e791ef310e219bd932280e80da5"}
Sep 10 19:15:31.421685 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:31.421657    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-tt257" event={"ID":"09299baf-b713-431a-9b8b-dfbe54f74b69","Type":"ContainerStarted","Data":"2387519df02b6b5734e9f0f304d6658732d9d95235572ed328f03dd22822a861"}
Sep 10 19:15:31.423546 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:31.423517    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-114.ec2.internal" event={"ID":"3eeea337b91a216332a11d902b8fa0ef","Type":"ContainerStarted","Data":"e859312411f93e58c849f6750fc398da1f33ec3a6364921ddec36793e8fb5b22"}
Sep 10 19:15:31.426118 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:31.426094    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tx2mc" event={"ID":"b4070c77-f931-4973-b076-035a9e33e600","Type":"ContainerStarted","Data":"8a6575a9565ec4361a0d35675df2cac6d858e61eb87735aadb45f987a60a8f5d"}
Sep 10 19:15:31.428148 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:31.428121    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-qs9mw" event={"ID":"83c3166f-596e-4033-b92b-02f89683ccda","Type":"ContainerStarted","Data":"40c21859d75cfc4038e0c842a18164f1e6e18ef73c1387befdff3d124126567d"}
Sep 10 19:15:31.432006 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:31.431981    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-114.ec2.internal" event={"ID":"a5967616edfbe6b806f38063e88e3534","Type":"ContainerStarted","Data":"1fa25cdb26ba66686906c961df48025bded93f5eb08a1ac36ae5dc6f46cc9186"}
Sep 10 19:15:31.436888 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:31.436864    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-bbmp5" event={"ID":"a76b5c99-a97f-461d-bce3-1fc84ef52bae","Type":"ContainerStarted","Data":"07f98618cfe2260b49a1683382a0eac245485f4a77da392a8148cc16a985fa5f"}
Sep 10 19:15:31.443896 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:31.443873    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wt6fs" event={"ID":"cf162519-8424-43b0-8de5-a19bc237a9d0","Type":"ContainerStarted","Data":"3644cbc0cffbe3a7b921d43d5b866bf9b16254c4ec1d7d18e25edc4977e5e3c4"}
Sep 10 19:15:31.743863 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:31.743820    2575 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 19:15:32.005222 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:32.005147    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bfed5912-60ab-48fc-91c2-10d325fd6428-metrics-certs\") pod \"network-metrics-daemon-wx2q6\" (UID: \"bfed5912-60ab-48fc-91c2-10d325fd6428\") " pod="openshift-multus/network-metrics-daemon-wx2q6"
Sep 10 19:15:32.005222 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:32.005205    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ac571b4d-dbf7-4e62-8d4d-c8c2a527deb3-original-pull-secret\") pod \"global-pull-secret-syncer-24lb9\" (UID: \"ac571b4d-dbf7-4e62-8d4d-c8c2a527deb3\") " pod="kube-system/global-pull-secret-syncer-24lb9"
Sep 10 19:15:32.005476 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:15:32.005433    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 19:15:32.005537 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:15:32.005500    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bfed5912-60ab-48fc-91c2-10d325fd6428-metrics-certs podName:bfed5912-60ab-48fc-91c2-10d325fd6428 nodeName:}" failed. No retries permitted until 2026-09-10 19:15:34.005480512 +0000 UTC m=+5.065308163 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bfed5912-60ab-48fc-91c2-10d325fd6428-metrics-certs") pod "network-metrics-daemon-wx2q6" (UID: "bfed5912-60ab-48fc-91c2-10d325fd6428") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 19:15:32.006084 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:15:32.006059    2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 19:15:32.006170 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:15:32.006133    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac571b4d-dbf7-4e62-8d4d-c8c2a527deb3-original-pull-secret podName:ac571b4d-dbf7-4e62-8d4d-c8c2a527deb3 nodeName:}" failed. No retries permitted until 2026-09-10 19:15:34.00611601 +0000 UTC m=+5.065943660 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ac571b4d-dbf7-4e62-8d4d-c8c2a527deb3-original-pull-secret") pod "global-pull-secret-syncer-24lb9" (UID: "ac571b4d-dbf7-4e62-8d4d-c8c2a527deb3") : object "kube-system"/"original-pull-secret" not registered
Sep 10 19:15:32.107059 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:32.106420    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n2p5q\" (UniqueName: \"kubernetes.io/projected/cbd42c54-3649-4d2d-8f94-d2789f7f66c7-kube-api-access-n2p5q\") pod \"network-check-target-9rkjw\" (UID: \"cbd42c54-3649-4d2d-8f94-d2789f7f66c7\") " pod="openshift-network-diagnostics/network-check-target-9rkjw"
Sep 10 19:15:32.107059 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:15:32.106656    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 19:15:32.107059 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:15:32.106679    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 19:15:32.107059 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:15:32.106691    2575 projected.go:196] Error preparing data for projected volume kube-api-access-n2p5q for pod openshift-network-diagnostics/network-check-target-9rkjw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 19:15:32.107059 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:15:32.106744    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cbd42c54-3649-4d2d-8f94-d2789f7f66c7-kube-api-access-n2p5q podName:cbd42c54-3649-4d2d-8f94-d2789f7f66c7 nodeName:}" failed. No retries permitted until 2026-09-10 19:15:34.106726382 +0000 UTC m=+5.166554047 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-n2p5q" (UniqueName: "kubernetes.io/projected/cbd42c54-3649-4d2d-8f94-d2789f7f66c7-kube-api-access-n2p5q") pod "network-check-target-9rkjw" (UID: "cbd42c54-3649-4d2d-8f94-d2789f7f66c7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 19:15:32.331368 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:32.331268    2575 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-09 19:10:30 +0000 UTC" deadline="2028-03-29 05:57:05.445748348 +0000 UTC"
Sep 10 19:15:32.331368 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:32.331306    2575 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13570h41m33.114446456s"
Sep 10 19:15:32.378834 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:32.378422    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wx2q6"
Sep 10 19:15:32.378834 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:15:32.378557    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wx2q6" podUID="bfed5912-60ab-48fc-91c2-10d325fd6428"
Sep 10 19:15:32.379067 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:32.378880    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-24lb9"
Sep 10 19:15:32.379067 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:15:32.378969    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-24lb9" podUID="ac571b4d-dbf7-4e62-8d4d-c8c2a527deb3"
Sep 10 19:15:32.379173 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:32.379119    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9rkjw"
Sep 10 19:15:32.379225 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:15:32.379198    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9rkjw" podUID="cbd42c54-3649-4d2d-8f94-d2789f7f66c7"
Sep 10 19:15:32.490013 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:32.489982    2575 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 19:15:34.023204 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:34.022980    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bfed5912-60ab-48fc-91c2-10d325fd6428-metrics-certs\") pod \"network-metrics-daemon-wx2q6\" (UID: \"bfed5912-60ab-48fc-91c2-10d325fd6428\") " pod="openshift-multus/network-metrics-daemon-wx2q6"
Sep 10 19:15:34.023582 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:34.023230    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ac571b4d-dbf7-4e62-8d4d-c8c2a527deb3-original-pull-secret\") pod \"global-pull-secret-syncer-24lb9\" (UID: \"ac571b4d-dbf7-4e62-8d4d-c8c2a527deb3\") " pod="kube-system/global-pull-secret-syncer-24lb9"
Sep 10 19:15:34.023582 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:15:34.023120    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 19:15:34.023582 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:15:34.023450    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bfed5912-60ab-48fc-91c2-10d325fd6428-metrics-certs podName:bfed5912-60ab-48fc-91c2-10d325fd6428 nodeName:}" failed. No retries permitted until 2026-09-10 19:15:38.023432617 +0000 UTC m=+9.083260278 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bfed5912-60ab-48fc-91c2-10d325fd6428-metrics-certs") pod "network-metrics-daemon-wx2q6" (UID: "bfed5912-60ab-48fc-91c2-10d325fd6428") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 19:15:34.023761 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:15:34.023392    2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 19:15:34.023815 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:15:34.023803    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac571b4d-dbf7-4e62-8d4d-c8c2a527deb3-original-pull-secret podName:ac571b4d-dbf7-4e62-8d4d-c8c2a527deb3 nodeName:}" failed. No retries permitted until 2026-09-10 19:15:38.023786199 +0000 UTC m=+9.083613851 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ac571b4d-dbf7-4e62-8d4d-c8c2a527deb3-original-pull-secret") pod "global-pull-secret-syncer-24lb9" (UID: "ac571b4d-dbf7-4e62-8d4d-c8c2a527deb3") : object "kube-system"/"original-pull-secret" not registered
Sep 10 19:15:34.124305 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:34.124273    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n2p5q\" (UniqueName: \"kubernetes.io/projected/cbd42c54-3649-4d2d-8f94-d2789f7f66c7-kube-api-access-n2p5q\") pod \"network-check-target-9rkjw\" (UID: \"cbd42c54-3649-4d2d-8f94-d2789f7f66c7\") " pod="openshift-network-diagnostics/network-check-target-9rkjw"
Sep 10 19:15:34.124485 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:15:34.124451    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 19:15:34.124485 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:15:34.124474    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 19:15:34.124485 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:15:34.124485    2575 projected.go:196] Error preparing data for projected volume kube-api-access-n2p5q for pod openshift-network-diagnostics/network-check-target-9rkjw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 19:15:34.124656 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:15:34.124534    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cbd42c54-3649-4d2d-8f94-d2789f7f66c7-kube-api-access-n2p5q podName:cbd42c54-3649-4d2d-8f94-d2789f7f66c7 nodeName:}" failed. No retries permitted until 2026-09-10 19:15:38.124520373 +0000 UTC m=+9.184348023 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-n2p5q" (UniqueName: "kubernetes.io/projected/cbd42c54-3649-4d2d-8f94-d2789f7f66c7-kube-api-access-n2p5q") pod "network-check-target-9rkjw" (UID: "cbd42c54-3649-4d2d-8f94-d2789f7f66c7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 19:15:34.379107 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:34.379028    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9rkjw"
Sep 10 19:15:34.379254 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:15:34.379151    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9rkjw" podUID="cbd42c54-3649-4d2d-8f94-d2789f7f66c7"
Sep 10 19:15:34.379537 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:34.379517    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wx2q6"
Sep 10 19:15:34.379658 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:15:34.379629    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wx2q6" podUID="bfed5912-60ab-48fc-91c2-10d325fd6428"
Sep 10 19:15:34.379721 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:34.379673    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-24lb9"
Sep 10 19:15:34.379773 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:15:34.379742    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-24lb9" podUID="ac571b4d-dbf7-4e62-8d4d-c8c2a527deb3"
Sep 10 19:15:36.379266 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:36.379227    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9rkjw"
Sep 10 19:15:36.379738 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:36.379233    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wx2q6"
Sep 10 19:15:36.379738 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:15:36.379351    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9rkjw" podUID="cbd42c54-3649-4d2d-8f94-d2789f7f66c7"
Sep 10 19:15:36.379738 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:36.379232    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-24lb9"
Sep 10 19:15:36.379738 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:15:36.379425    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wx2q6" podUID="bfed5912-60ab-48fc-91c2-10d325fd6428"
Sep 10 19:15:36.379738 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:15:36.379530    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-24lb9" podUID="ac571b4d-dbf7-4e62-8d4d-c8c2a527deb3"
Sep 10 19:15:38.058432 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:38.058229    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bfed5912-60ab-48fc-91c2-10d325fd6428-metrics-certs\") pod \"network-metrics-daemon-wx2q6\" (UID: \"bfed5912-60ab-48fc-91c2-10d325fd6428\") " pod="openshift-multus/network-metrics-daemon-wx2q6"
Sep 10 19:15:38.058432 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:38.058274    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ac571b4d-dbf7-4e62-8d4d-c8c2a527deb3-original-pull-secret\") pod \"global-pull-secret-syncer-24lb9\" (UID: \"ac571b4d-dbf7-4e62-8d4d-c8c2a527deb3\") " pod="kube-system/global-pull-secret-syncer-24lb9"
Sep 10 19:15:38.058871 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:15:38.058449    2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 19:15:38.058871 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:15:38.058510    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac571b4d-dbf7-4e62-8d4d-c8c2a527deb3-original-pull-secret podName:ac571b4d-dbf7-4e62-8d4d-c8c2a527deb3 nodeName:}" failed. No retries permitted until 2026-09-10 19:15:46.058492926 +0000 UTC m=+17.118320592 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ac571b4d-dbf7-4e62-8d4d-c8c2a527deb3-original-pull-secret") pod "global-pull-secret-syncer-24lb9" (UID: "ac571b4d-dbf7-4e62-8d4d-c8c2a527deb3") : object "kube-system"/"original-pull-secret" not registered
Sep 10 19:15:38.058871 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:15:38.058631    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 19:15:38.058871 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:15:38.058678    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bfed5912-60ab-48fc-91c2-10d325fd6428-metrics-certs podName:bfed5912-60ab-48fc-91c2-10d325fd6428 nodeName:}" failed. No retries permitted until 2026-09-10 19:15:46.058665572 +0000 UTC m=+17.118493223 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bfed5912-60ab-48fc-91c2-10d325fd6428-metrics-certs") pod "network-metrics-daemon-wx2q6" (UID: "bfed5912-60ab-48fc-91c2-10d325fd6428") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 19:15:38.158878 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:38.158806    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n2p5q\" (UniqueName: \"kubernetes.io/projected/cbd42c54-3649-4d2d-8f94-d2789f7f66c7-kube-api-access-n2p5q\") pod \"network-check-target-9rkjw\" (UID: \"cbd42c54-3649-4d2d-8f94-d2789f7f66c7\") " pod="openshift-network-diagnostics/network-check-target-9rkjw"
Sep 10 19:15:38.159041 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:15:38.159025    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 19:15:38.159103 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:15:38.159087    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 19:15:38.159167 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:15:38.159117    2575 projected.go:196] Error preparing data for projected volume kube-api-access-n2p5q for pod openshift-network-diagnostics/network-check-target-9rkjw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 19:15:38.159215 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:15:38.159188    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cbd42c54-3649-4d2d-8f94-d2789f7f66c7-kube-api-access-n2p5q podName:cbd42c54-3649-4d2d-8f94-d2789f7f66c7 nodeName:}" failed. No retries permitted until 2026-09-10 19:15:46.159169458 +0000 UTC m=+17.218997106 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-n2p5q" (UniqueName: "kubernetes.io/projected/cbd42c54-3649-4d2d-8f94-d2789f7f66c7-kube-api-access-n2p5q") pod "network-check-target-9rkjw" (UID: "cbd42c54-3649-4d2d-8f94-d2789f7f66c7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 19:15:38.378376 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:38.378290    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9rkjw"
Sep 10 19:15:38.378515 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:38.378423    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wx2q6"
Sep 10 19:15:38.378515 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:15:38.378428    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9rkjw" podUID="cbd42c54-3649-4d2d-8f94-d2789f7f66c7"
Sep 10 19:15:38.378632 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:15:38.378540    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wx2q6" podUID="bfed5912-60ab-48fc-91c2-10d325fd6428"
Sep 10 19:15:38.378632 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:38.378580    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-24lb9"
Sep 10 19:15:38.378723 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:15:38.378655    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-24lb9" podUID="ac571b4d-dbf7-4e62-8d4d-c8c2a527deb3"
Sep 10 19:15:40.378311 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:40.378278    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-24lb9"
Sep 10 19:15:40.378311 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:40.378321    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9rkjw"
Sep 10 19:15:40.378906 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:40.378326    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wx2q6"
Sep 10 19:15:40.378906 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:15:40.378430    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-24lb9" podUID="ac571b4d-dbf7-4e62-8d4d-c8c2a527deb3"
Sep 10 19:15:40.378906 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:15:40.378492    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9rkjw" podUID="cbd42c54-3649-4d2d-8f94-d2789f7f66c7"
Sep 10 19:15:40.378906 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:15:40.378561    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wx2q6" podUID="bfed5912-60ab-48fc-91c2-10d325fd6428"
Sep 10 19:15:42.379136 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:42.379106    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9rkjw"
Sep 10 19:15:42.379565 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:42.379106    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-24lb9"
Sep 10 19:15:42.379565 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:15:42.379214    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9rkjw" podUID="cbd42c54-3649-4d2d-8f94-d2789f7f66c7"
Sep 10 19:15:42.379565 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:15:42.379264    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-24lb9" podUID="ac571b4d-dbf7-4e62-8d4d-c8c2a527deb3"
Sep 10 19:15:42.379565 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:42.379113    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wx2q6"
Sep 10 19:15:42.379565 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:15:42.379391    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wx2q6" podUID="bfed5912-60ab-48fc-91c2-10d325fd6428"
Sep 10 19:15:44.378151 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:44.378118    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wx2q6"
Sep 10 19:15:44.378700 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:44.378243    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-24lb9"
Sep 10 19:15:44.378700 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:15:44.378250    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wx2q6" podUID="bfed5912-60ab-48fc-91c2-10d325fd6428"
Sep 10 19:15:44.378700 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:44.378264    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9rkjw"
Sep 10 19:15:44.378700 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:15:44.378358    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-24lb9" podUID="ac571b4d-dbf7-4e62-8d4d-c8c2a527deb3"
Sep 10 19:15:44.378700 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:15:44.378427    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9rkjw" podUID="cbd42c54-3649-4d2d-8f94-d2789f7f66c7"
Sep 10 19:15:46.116935 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:46.116897    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bfed5912-60ab-48fc-91c2-10d325fd6428-metrics-certs\") pod \"network-metrics-daemon-wx2q6\" (UID: \"bfed5912-60ab-48fc-91c2-10d325fd6428\") " pod="openshift-multus/network-metrics-daemon-wx2q6"
Sep 10 19:15:46.117447 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:46.116947    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ac571b4d-dbf7-4e62-8d4d-c8c2a527deb3-original-pull-secret\") pod \"global-pull-secret-syncer-24lb9\" (UID: \"ac571b4d-dbf7-4e62-8d4d-c8c2a527deb3\") " pod="kube-system/global-pull-secret-syncer-24lb9"
Sep 10 19:15:46.117447 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:15:46.117056    2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 19:15:46.117447 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:15:46.117059    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 19:15:46.117447 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:15:46.117118    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac571b4d-dbf7-4e62-8d4d-c8c2a527deb3-original-pull-secret podName:ac571b4d-dbf7-4e62-8d4d-c8c2a527deb3 nodeName:}" failed. No retries permitted until 2026-09-10 19:16:02.117100049 +0000 UTC m=+33.176927699 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ac571b4d-dbf7-4e62-8d4d-c8c2a527deb3-original-pull-secret") pod "global-pull-secret-syncer-24lb9" (UID: "ac571b4d-dbf7-4e62-8d4d-c8c2a527deb3") : object "kube-system"/"original-pull-secret" not registered
Sep 10 19:15:46.117447 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:15:46.117136    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bfed5912-60ab-48fc-91c2-10d325fd6428-metrics-certs podName:bfed5912-60ab-48fc-91c2-10d325fd6428 nodeName:}" failed. No retries permitted until 2026-09-10 19:16:02.117126026 +0000 UTC m=+33.176953676 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bfed5912-60ab-48fc-91c2-10d325fd6428-metrics-certs") pod "network-metrics-daemon-wx2q6" (UID: "bfed5912-60ab-48fc-91c2-10d325fd6428") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 19:15:46.217579 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:46.217546    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n2p5q\" (UniqueName: \"kubernetes.io/projected/cbd42c54-3649-4d2d-8f94-d2789f7f66c7-kube-api-access-n2p5q\") pod \"network-check-target-9rkjw\" (UID: \"cbd42c54-3649-4d2d-8f94-d2789f7f66c7\") " pod="openshift-network-diagnostics/network-check-target-9rkjw"
Sep 10 19:15:46.217730 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:15:46.217714    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 19:15:46.217806 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:15:46.217741    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 19:15:46.217806 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:15:46.217752    2575 projected.go:196] Error preparing data for projected volume kube-api-access-n2p5q for pod openshift-network-diagnostics/network-check-target-9rkjw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 19:15:46.217806 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:15:46.217802    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cbd42c54-3649-4d2d-8f94-d2789f7f66c7-kube-api-access-n2p5q podName:cbd42c54-3649-4d2d-8f94-d2789f7f66c7 nodeName:}" failed. No retries permitted until 2026-09-10 19:16:02.21778766 +0000 UTC m=+33.277615323 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-n2p5q" (UniqueName: "kubernetes.io/projected/cbd42c54-3649-4d2d-8f94-d2789f7f66c7-kube-api-access-n2p5q") pod "network-check-target-9rkjw" (UID: "cbd42c54-3649-4d2d-8f94-d2789f7f66c7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 19:15:46.378326 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:46.378251    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9rkjw"
Sep 10 19:15:46.378326 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:46.378290    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-24lb9"
Sep 10 19:15:46.378546 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:46.378382    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wx2q6"
Sep 10 19:15:46.378546 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:15:46.378384    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9rkjw" podUID="cbd42c54-3649-4d2d-8f94-d2789f7f66c7"
Sep 10 19:15:46.378546 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:15:46.378477    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-24lb9" podUID="ac571b4d-dbf7-4e62-8d4d-c8c2a527deb3"
Sep 10 19:15:46.378659 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:15:46.378567    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wx2q6" podUID="bfed5912-60ab-48fc-91c2-10d325fd6428"
Sep 10 19:15:48.378595 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:48.378563    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9rkjw"
Sep 10 19:15:48.379026 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:48.378563    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wx2q6"
Sep 10 19:15:48.379026 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:15:48.378670    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9rkjw" podUID="cbd42c54-3649-4d2d-8f94-d2789f7f66c7"
Sep 10 19:15:48.379026 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:15:48.378783    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wx2q6" podUID="bfed5912-60ab-48fc-91c2-10d325fd6428"
Sep 10 19:15:48.379026 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:48.378563    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-24lb9"
Sep 10 19:15:48.379026 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:15:48.378885    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-24lb9" podUID="ac571b4d-dbf7-4e62-8d4d-c8c2a527deb3"
Sep 10 19:15:49.492373 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:49.492205    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rrxfj_c7511dfe-c588-4be1-bd16-1dce34740809/ovn-acl-logging/0.log"
Sep 10 19:15:49.492899 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:49.492687    2575 generic.go:356] "Generic (PLEG): container finished" podID="c7511dfe-c588-4be1-bd16-1dce34740809" containerID="aead309b462cebbc8883425da42d619f1ebd6969c5214b182a1c56e6275bcc32" exitCode=1
Sep 10 19:15:49.492899 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:49.492750    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rrxfj" event={"ID":"c7511dfe-c588-4be1-bd16-1dce34740809","Type":"ContainerStarted","Data":"63e321bb94faed49db5c84692ea1dd56b9a956dcab0c7cb399d4625d78c5db84"}
Sep 10 19:15:49.492899 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:49.492822    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rrxfj" event={"ID":"c7511dfe-c588-4be1-bd16-1dce34740809","Type":"ContainerDied","Data":"aead309b462cebbc8883425da42d619f1ebd6969c5214b182a1c56e6275bcc32"}
Sep 10 19:15:49.492899 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:49.492863    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rrxfj" event={"ID":"c7511dfe-c588-4be1-bd16-1dce34740809","Type":"ContainerStarted","Data":"c0a25eeafd01ee871c402d4be7fb8d1a22976b4a7def7e30ae2d14a7346e69e2"}
Sep 10 19:15:49.508959 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:49.508934    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-g4hld" event={"ID":"d896f9fb-21e2-4845-ab2d-a855a43b0a17","Type":"ContainerStarted","Data":"3806879afccb2bd54d5c86aa3759be3612eae6680d9d5e2636b3d12fe9b8a6f6"}
Sep 10 19:15:49.510215 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:49.510197    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-114.ec2.internal" event={"ID":"3eeea337b91a216332a11d902b8fa0ef","Type":"ContainerStarted","Data":"ba035ffbcf92a56af8a655cfb1d23beae806ce3b5a167fdb04e3a213bda75294"}
Sep 10 19:15:49.528218 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:49.528179    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-g4hld" podStartSLOduration=2.363083749 podStartE2EDuration="20.528167773s" totalImagesPullingTime="18.165084024s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 19:15:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:15:49.528020649 +0000 UTC m=+20.587848316" watchObservedRunningTime="2026-09-10 19:15:49.528167773 +0000 UTC m=+20.587995468"
Sep 10 19:15:50.378259 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:50.378105    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wx2q6"
Sep 10 19:15:50.378386 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:50.378105    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9rkjw"
Sep 10 19:15:50.378386 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:15:50.378329    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wx2q6" podUID="bfed5912-60ab-48fc-91c2-10d325fd6428"
Sep 10 19:15:50.378467 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:15:50.378402    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9rkjw" podUID="cbd42c54-3649-4d2d-8f94-d2789f7f66c7"
Sep 10 19:15:50.378467 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:50.378108    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-24lb9"
Sep 10 19:15:50.378532 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:15:50.378467    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-24lb9" podUID="ac571b4d-dbf7-4e62-8d4d-c8c2a527deb3"
Sep 10 19:15:50.514019 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:50.513989    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-qs9mw" event={"ID":"83c3166f-596e-4033-b92b-02f89683ccda","Type":"ContainerStarted","Data":"aacb3fd962d318b9e5ea60976472a2ed7e9e18a7356b8d33abfdbf52c9ac3b5c"}
Sep 10 19:15:50.515814 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:50.515783    2575 generic.go:356] "Generic (PLEG): container finished" podID="a5967616edfbe6b806f38063e88e3534" containerID="f7afcea85677883428e7a24218e5b938bac52b1bcfa28d324c1d42921ce42918" exitCode=0
Sep 10 19:15:50.515912 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:50.515851    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-114.ec2.internal" event={"ID":"a5967616edfbe6b806f38063e88e3534","Type":"ContainerDied","Data":"f7afcea85677883428e7a24218e5b938bac52b1bcfa28d324c1d42921ce42918"}
Sep 10 19:15:50.517700 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:50.517679    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-bbmp5" event={"ID":"a76b5c99-a97f-461d-bce3-1fc84ef52bae","Type":"ContainerStarted","Data":"f111c8df9c6d319c5a97429f5669a593b10bd06b45c840aa3d173e693889718a"}
Sep 10 19:15:50.519411 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:50.519388    2575 generic.go:356] "Generic (PLEG): container finished" podID="cf162519-8424-43b0-8de5-a19bc237a9d0" containerID="579cf24372186d6b0067907598d7b4c63892e96a2c7d00f2a004c6c1178dbe1d" exitCode=0
Sep 10 19:15:50.519478 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:50.519451    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wt6fs" event={"ID":"cf162519-8424-43b0-8de5-a19bc237a9d0","Type":"ContainerDied","Data":"579cf24372186d6b0067907598d7b4c63892e96a2c7d00f2a004c6c1178dbe1d"}
Sep 10 19:15:50.522808 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:50.522788    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rrxfj_c7511dfe-c588-4be1-bd16-1dce34740809/ovn-acl-logging/0.log"
Sep 10 19:15:50.523454 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:50.523431    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rrxfj" event={"ID":"c7511dfe-c588-4be1-bd16-1dce34740809","Type":"ContainerStarted","Data":"e17b2eccac438d8d34ada5aabe9adea6234eb6dac51827176419a178edc1cefb"}
Sep 10 19:15:50.523546 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:50.523513    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rrxfj" event={"ID":"c7511dfe-c588-4be1-bd16-1dce34740809","Type":"ContainerStarted","Data":"dd55a0679e69af9c71d48d07bcf12bfe39166905b5a0c1a88724156c08abe75c"}
Sep 10 19:15:50.523591 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:50.523568    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rrxfj" event={"ID":"c7511dfe-c588-4be1-bd16-1dce34740809","Type":"ContainerStarted","Data":"fe802f8b144d6f1bcf0ee69c413e079877b3fb02260c55969ce1a7d176df20f4"}
Sep 10 19:15:50.524994 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:50.524973    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-nh8r2" event={"ID":"446e8624-4aaf-471d-9653-2451db8c6889","Type":"ContainerStarted","Data":"0ed9744ed7b427fa838ae3f3e491e37945134d2d3f716bfe07112157764d2de7"}
Sep 10 19:15:50.526617 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:50.526597    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-vmvhn" event={"ID":"d093d956-9c62-4201-a4ae-60187edb6fbf","Type":"ContainerStarted","Data":"e2b7bba81b66eafffe454fc354f2dd919ff920e8247c3ca2b49b9460047a0c20"}
Sep 10 19:15:50.528256 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:50.528232    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-tt257" event={"ID":"09299baf-b713-431a-9b8b-dfbe54f74b69","Type":"ContainerStarted","Data":"378cbbf5240d22443948c041cefb994b80fdfde79ba5858637f24ae62e4513c1"}
Sep 10 19:15:50.529431 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:50.529390    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-114.ec2.internal" podStartSLOduration=20.529377837 podStartE2EDuration="20.529377837s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 19:15:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:15:49.540702949 +0000 UTC m=+20.600530617" watchObservedRunningTime="2026-09-10 19:15:50.529377837 +0000 UTC m=+21.589205505"
Sep 10 19:15:50.530003 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:50.529985    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tx2mc" event={"ID":"b4070c77-f931-4973-b076-035a9e33e600","Type":"ContainerStarted","Data":"0c0ac22643f8306c367dd5a1cb0c76ee23c85e9af6de46f3941371e3a603d52f"}
Sep 10 19:15:50.530165 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:50.530133    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-qs9mw" podStartSLOduration=7.521470636 podStartE2EDuration="21.530125247s" totalImagesPullingTime="14.008654611s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 19:15:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:15:50.529138024 +0000 UTC m=+21.588965692" watchObservedRunningTime="2026-09-10 19:15:50.530125247 +0000 UTC m=+21.589952928"
Sep 10 19:15:50.575682 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:50.575644    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-bbmp5" podStartSLOduration=3.520356795 podStartE2EDuration="21.575631258s" totalImagesPullingTime="18.055274463s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:15:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:15:50.575257542 +0000 UTC m=+21.635085211" watchObservedRunningTime="2026-09-10 19:15:50.575631258 +0000 UTC m=+21.635458927"
Sep 10 19:15:50.588714 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:50.588676    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-nh8r2" podStartSLOduration=3.440113321 podStartE2EDuration="21.588659954s" totalImagesPullingTime="18.148546633s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 19:15:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:15:50.587870755 +0000 UTC m=+21.647698422" watchObservedRunningTime="2026-09-10 19:15:50.588659954 +0000 UTC m=+21.648487628"
Sep 10 19:15:50.603996 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:50.603956    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-tt257" podStartSLOduration=2.5696064830000003 podStartE2EDuration="21.603940426s" totalImagesPullingTime="19.034333943s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 19:15:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:15:50.603511803 +0000 UTC m=+21.663339497" watchObservedRunningTime="2026-09-10 19:15:50.603940426 +0000 UTC m=+21.663768094"
Sep 10 19:15:50.619819 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:50.619784    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-vmvhn" podStartSLOduration=3.443819162 podStartE2EDuration="21.619766634s" totalImagesPullingTime="18.175947472s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 19:15:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:15:50.619570955 +0000 UTC m=+21.679398623" watchObservedRunningTime="2026-09-10 19:15:50.619766634 +0000 UTC m=+21.679594301"
Sep 10 19:15:51.024445 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:51.024425    2575 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 10 19:15:51.355788 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:51.355691    2575 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-10T19:15:51.024441271Z","UUID":"4f27e63c-a2af-4ae3-96d1-aa23acf335cb","Handler":null,"Name":"","Endpoint":""}
Sep 10 19:15:51.357555 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:51.357534    2575 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 10 19:15:51.357676 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:51.357567    2575 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 10 19:15:51.534919 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:51.534877    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tx2mc" event={"ID":"b4070c77-f931-4973-b076-035a9e33e600","Type":"ContainerStarted","Data":"f182cf417ca556cce54e1619e35876b438d4f32b4a67d60c2d9ed8ed6bfb0775"}
Sep 10 19:15:51.537217 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:51.537171    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-114.ec2.internal" event={"ID":"a5967616edfbe6b806f38063e88e3534","Type":"ContainerStarted","Data":"602a86d98a7ec959e511fd794d97b47e9aaa4582b4e8f269ba93c05cb40a3424"}
Sep 10 19:15:51.559276 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:51.559223    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-114.ec2.internal" podStartSLOduration=21.559205045 podStartE2EDuration="21.559205045s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 19:15:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:15:51.558421285 +0000 UTC m=+22.618248953" watchObservedRunningTime="2026-09-10 19:15:51.559205045 +0000 UTC m=+22.619032714"
Sep 10 19:15:52.378613 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:52.378420    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wx2q6"
Sep 10 19:15:52.378783 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:52.378434    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9rkjw"
Sep 10 19:15:52.378783 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:15:52.378706    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wx2q6" podUID="bfed5912-60ab-48fc-91c2-10d325fd6428"
Sep 10 19:15:52.378890 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:15:52.378790    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9rkjw" podUID="cbd42c54-3649-4d2d-8f94-d2789f7f66c7"
Sep 10 19:15:52.378890 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:52.378476    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-24lb9"
Sep 10 19:15:52.378978 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:15:52.378910    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-24lb9" podUID="ac571b4d-dbf7-4e62-8d4d-c8c2a527deb3"
Sep 10 19:15:52.542905 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:52.542875    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rrxfj_c7511dfe-c588-4be1-bd16-1dce34740809/ovn-acl-logging/0.log"
Sep 10 19:15:52.543381 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:52.543332    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rrxfj" event={"ID":"c7511dfe-c588-4be1-bd16-1dce34740809","Type":"ContainerStarted","Data":"69f8e186328afa4110ba5c5d8aa40c5b23b5294be9c2f222e34a905797ca0fff"}
Sep 10 19:15:52.545505 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:52.545478    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tx2mc" event={"ID":"b4070c77-f931-4973-b076-035a9e33e600","Type":"ContainerStarted","Data":"8d03f22129f8e8dde06a7622434f7225b99dd27cbb958776d9372296703c0899"}
Sep 10 19:15:52.563877 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:52.563834    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tx2mc" podStartSLOduration=3.598917967 podStartE2EDuration="23.563822591s" totalImagesPullingTime="19.964904624s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 19:15:29 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:15:52.563738891 +0000 UTC m=+23.623566558" watchObservedRunningTime="2026-09-10 19:15:52.563822591 +0000 UTC m=+23.623650259"
Sep 10 19:15:53.554086 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:53.554055    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-bbmp5"
Sep 10 19:15:53.554757 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:53.554738    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-bbmp5"
Sep 10 19:15:54.378630 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:54.378600    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-24lb9"
Sep 10 19:15:54.378808 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:54.378709    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wx2q6"
Sep 10 19:15:54.378808 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:54.378723    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9rkjw"
Sep 10 19:15:54.378917 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:15:54.378710    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-24lb9" podUID="ac571b4d-dbf7-4e62-8d4d-c8c2a527deb3"
Sep 10 19:15:54.378917 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:15:54.378813    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wx2q6" podUID="bfed5912-60ab-48fc-91c2-10d325fd6428"
Sep 10 19:15:54.378917 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:15:54.378864    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9rkjw" podUID="cbd42c54-3649-4d2d-8f94-d2789f7f66c7"
Sep 10 19:15:54.549769 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:54.549735    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-bbmp5"
Sep 10 19:15:54.550260 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:54.550233    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-bbmp5"
Sep 10 19:15:55.555668 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:55.555635    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rrxfj_c7511dfe-c588-4be1-bd16-1dce34740809/ovn-acl-logging/0.log"
Sep 10 19:15:55.556127 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:55.556097    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rrxfj" event={"ID":"c7511dfe-c588-4be1-bd16-1dce34740809","Type":"ContainerStarted","Data":"0c08341eca925d37b623c59f1578cddfc055ecb6343ce790a2e00960375f9c41"}
Sep 10 19:15:55.556473 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:55.556458    2575 scope.go:117] "RemoveContainer" containerID="aead309b462cebbc8883425da42d619f1ebd6969c5214b182a1c56e6275bcc32"
Sep 10 19:15:56.379097 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:56.378927    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wx2q6"
Sep 10 19:15:56.379224 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:56.378929    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-24lb9"
Sep 10 19:15:56.379224 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:15:56.379169    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wx2q6" podUID="bfed5912-60ab-48fc-91c2-10d325fd6428"
Sep 10 19:15:56.379359 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:56.378929    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9rkjw"
Sep 10 19:15:56.379359 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:15:56.379244    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-24lb9" podUID="ac571b4d-dbf7-4e62-8d4d-c8c2a527deb3"
Sep 10 19:15:56.379453 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:15:56.379397    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9rkjw" podUID="cbd42c54-3649-4d2d-8f94-d2789f7f66c7"
Sep 10 19:15:56.559726 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:56.559702    2575 generic.go:356] "Generic (PLEG): container finished" podID="cf162519-8424-43b0-8de5-a19bc237a9d0" containerID="e68f4686055e65d9d5da862b460d66a97fdb4ea9a130b200bc66fc5d1721737f" exitCode=0
Sep 10 19:15:56.560148 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:56.559773    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wt6fs" event={"ID":"cf162519-8424-43b0-8de5-a19bc237a9d0","Type":"ContainerDied","Data":"e68f4686055e65d9d5da862b460d66a97fdb4ea9a130b200bc66fc5d1721737f"}
Sep 10 19:15:56.563637 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:56.563620    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rrxfj_c7511dfe-c588-4be1-bd16-1dce34740809/ovn-acl-logging/0.log"
Sep 10 19:15:56.564108 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:56.564083    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rrxfj" event={"ID":"c7511dfe-c588-4be1-bd16-1dce34740809","Type":"ContainerStarted","Data":"795717ff457cb05255decf59c43a0d05e58d6da23895f6f7491585ade415b702"}
Sep 10 19:15:56.564202 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:56.564186    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-rrxfj"
Sep 10 19:15:56.564270 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:56.564238    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-rrxfj"
Sep 10 19:15:56.564319 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:56.564289    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-rrxfj"
Sep 10 19:15:56.578962 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:56.578910    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-rrxfj"
Sep 10 19:15:56.579068 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:56.579054    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-rrxfj"
Sep 10 19:15:56.607779 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:56.607736    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-rrxfj" podStartSLOduration=9.12704717 podStartE2EDuration="27.60772419s" totalImagesPullingTime="18.48067702s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 19:15:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:15:56.607268257 +0000 UTC m=+27.667095925" watchObservedRunningTime="2026-09-10 19:15:56.60772419 +0000 UTC m=+27.667551857"
Sep 10 19:15:57.333044 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:57.333017    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-24lb9"]
Sep 10 19:15:57.333186 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:57.333129    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-24lb9"
Sep 10 19:15:57.333250 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:15:57.333218    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-24lb9" podUID="ac571b4d-dbf7-4e62-8d4d-c8c2a527deb3"
Sep 10 19:15:57.336035 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:57.335793    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-9rkjw"]
Sep 10 19:15:57.336311 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:57.336180    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9rkjw"
Sep 10 19:15:57.336311 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:15:57.336272    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9rkjw" podUID="cbd42c54-3649-4d2d-8f94-d2789f7f66c7"
Sep 10 19:15:57.336519 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:57.336499    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-wx2q6"]
Sep 10 19:15:57.336595 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:57.336582    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wx2q6"
Sep 10 19:15:57.336673 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:15:57.336654    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wx2q6" podUID="bfed5912-60ab-48fc-91c2-10d325fd6428"
Sep 10 19:15:57.568206 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:57.568133    2575 generic.go:356] "Generic (PLEG): container finished" podID="cf162519-8424-43b0-8de5-a19bc237a9d0" containerID="86e9e13a8cb50dadd674bdf14ea8fbf1a93195a8153999a7bdd0d6aa85a1074c" exitCode=0
Sep 10 19:15:57.568754 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:57.568218    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wt6fs" event={"ID":"cf162519-8424-43b0-8de5-a19bc237a9d0","Type":"ContainerDied","Data":"86e9e13a8cb50dadd674bdf14ea8fbf1a93195a8153999a7bdd0d6aa85a1074c"}
Sep 10 19:15:58.378840 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:58.378767    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9rkjw"
Sep 10 19:15:58.378973 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:15:58.378865    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9rkjw" podUID="cbd42c54-3649-4d2d-8f94-d2789f7f66c7"
Sep 10 19:15:58.573319 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:58.573161    2575 generic.go:356] "Generic (PLEG): container finished" podID="cf162519-8424-43b0-8de5-a19bc237a9d0" containerID="304e97bd3a25d1889ead8ff1a147fc251af7b15fe852ac8733c08ab6b04348c5" exitCode=0
Sep 10 19:15:58.573759 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:58.573230    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wt6fs" event={"ID":"cf162519-8424-43b0-8de5-a19bc237a9d0","Type":"ContainerDied","Data":"304e97bd3a25d1889ead8ff1a147fc251af7b15fe852ac8733c08ab6b04348c5"}
Sep 10 19:15:59.379461 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:59.379433    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wx2q6"
Sep 10 19:15:59.379673 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:15:59.379547    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wx2q6" podUID="bfed5912-60ab-48fc-91c2-10d325fd6428"
Sep 10 19:15:59.379673 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:15:59.379570    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-24lb9"
Sep 10 19:15:59.379763 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:15:59.379714    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-24lb9" podUID="ac571b4d-dbf7-4e62-8d4d-c8c2a527deb3"
Sep 10 19:16:00.378657 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:00.378629    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9rkjw"
Sep 10 19:16:00.379107 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:16:00.378743    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9rkjw" podUID="cbd42c54-3649-4d2d-8f94-d2789f7f66c7"
Sep 10 19:16:01.382470 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:01.382432    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wx2q6"
Sep 10 19:16:01.382884 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:16:01.382564    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wx2q6" podUID="bfed5912-60ab-48fc-91c2-10d325fd6428"
Sep 10 19:16:01.382884 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:01.382598    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-24lb9"
Sep 10 19:16:01.382884 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:16:01.382674    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-24lb9" podUID="ac571b4d-dbf7-4e62-8d4d-c8c2a527deb3"
Sep 10 19:16:02.142873 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:02.142844    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bfed5912-60ab-48fc-91c2-10d325fd6428-metrics-certs\") pod \"network-metrics-daemon-wx2q6\" (UID: \"bfed5912-60ab-48fc-91c2-10d325fd6428\") " pod="openshift-multus/network-metrics-daemon-wx2q6"
Sep 10 19:16:02.143024 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:02.142886    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ac571b4d-dbf7-4e62-8d4d-c8c2a527deb3-original-pull-secret\") pod \"global-pull-secret-syncer-24lb9\" (UID: \"ac571b4d-dbf7-4e62-8d4d-c8c2a527deb3\") " pod="kube-system/global-pull-secret-syncer-24lb9"
Sep 10 19:16:02.143024 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:16:02.143013    2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 19:16:02.143106 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:16:02.143016    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 19:16:02.143106 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:16:02.143065    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac571b4d-dbf7-4e62-8d4d-c8c2a527deb3-original-pull-secret podName:ac571b4d-dbf7-4e62-8d4d-c8c2a527deb3 nodeName:}" failed. No retries permitted until 2026-09-10 19:16:34.14305111 +0000 UTC m=+65.202878756 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ac571b4d-dbf7-4e62-8d4d-c8c2a527deb3-original-pull-secret") pod "global-pull-secret-syncer-24lb9" (UID: "ac571b4d-dbf7-4e62-8d4d-c8c2a527deb3") : object "kube-system"/"original-pull-secret" not registered
Sep 10 19:16:02.143106 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:16:02.143078    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bfed5912-60ab-48fc-91c2-10d325fd6428-metrics-certs podName:bfed5912-60ab-48fc-91c2-10d325fd6428 nodeName:}" failed. No retries permitted until 2026-09-10 19:16:34.143071884 +0000 UTC m=+65.202899529 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bfed5912-60ab-48fc-91c2-10d325fd6428-metrics-certs") pod "network-metrics-daemon-wx2q6" (UID: "bfed5912-60ab-48fc-91c2-10d325fd6428") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 19:16:02.215488 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:02.215466    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-114.ec2.internal" event="NodeReady"
Sep 10 19:16:02.215609 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:02.215582    2575 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 10 19:16:02.243463 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:02.243431    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n2p5q\" (UniqueName: \"kubernetes.io/projected/cbd42c54-3649-4d2d-8f94-d2789f7f66c7-kube-api-access-n2p5q\") pod \"network-check-target-9rkjw\" (UID: \"cbd42c54-3649-4d2d-8f94-d2789f7f66c7\") " pod="openshift-network-diagnostics/network-check-target-9rkjw"
Sep 10 19:16:02.243614 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:16:02.243596    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 19:16:02.243687 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:16:02.243621    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 19:16:02.243687 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:16:02.243632    2575 projected.go:196] Error preparing data for projected volume kube-api-access-n2p5q for pod openshift-network-diagnostics/network-check-target-9rkjw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 19:16:02.243794 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:16:02.243689    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cbd42c54-3649-4d2d-8f94-d2789f7f66c7-kube-api-access-n2p5q podName:cbd42c54-3649-4d2d-8f94-d2789f7f66c7 nodeName:}" failed. No retries permitted until 2026-09-10 19:16:34.243672025 +0000 UTC m=+65.303499685 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-n2p5q" (UniqueName: "kubernetes.io/projected/cbd42c54-3649-4d2d-8f94-d2789f7f66c7-kube-api-access-n2p5q") pod "network-check-target-9rkjw" (UID: "cbd42c54-3649-4d2d-8f94-d2789f7f66c7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 19:16:02.251326 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:02.251301    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-8cdf54dd5-22kjj"]
Sep 10 19:16:02.270605 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:02.270544    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-rjdmw"]
Sep 10 19:16:02.270703 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:02.270681    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-8cdf54dd5-22kjj"
Sep 10 19:16:02.273389 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:02.273329    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 10 19:16:02.273518 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:02.273431    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 10 19:16:02.273747 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:02.273728    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 10 19:16:02.273837 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:02.273789    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-n7kcr\""
Sep 10 19:16:02.279319 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:02.279299    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 10 19:16:02.286507 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:02.286487    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-8cdf54dd5-22kjj"]
Sep 10 19:16:02.286602 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:02.286527    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-gbtv9"]
Sep 10 19:16:02.286660 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:02.286600    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-rjdmw"
Sep 10 19:16:02.288499 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:02.288478    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 10 19:16:02.288499 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:02.288494    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 10 19:16:02.288648 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:02.288558    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-wks57\""
Sep 10 19:16:02.300544 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:02.300526    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-rjdmw"]
Sep 10 19:16:02.300619 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:02.300564    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-gbtv9"]
Sep 10 19:16:02.300663 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:02.300624    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-gbtv9"
Sep 10 19:16:02.302913 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:02.302884    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-rxhx6\""
Sep 10 19:16:02.303005 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:02.302941    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 10 19:16:02.303490 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:02.303383    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 10 19:16:02.303490 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:02.303434    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 10 19:16:02.343752 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:02.343728    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7-installation-pull-secrets\") pod \"image-registry-8cdf54dd5-22kjj\" (UID: \"7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7\") " pod="openshift-image-registry/image-registry-8cdf54dd5-22kjj"
Sep 10 19:16:02.343858 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:02.343767    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7-trusted-ca\") pod \"image-registry-8cdf54dd5-22kjj\" (UID: \"7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7\") " pod="openshift-image-registry/image-registry-8cdf54dd5-22kjj"
Sep 10 19:16:02.343858 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:02.343798    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7-ca-trust-extracted\") pod \"image-registry-8cdf54dd5-22kjj\" (UID: \"7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7\") " pod="openshift-image-registry/image-registry-8cdf54dd5-22kjj"
Sep 10 19:16:02.343858 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:02.343825    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jlj2m\" (UniqueName: \"kubernetes.io/projected/7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7-kube-api-access-jlj2m\") pod \"image-registry-8cdf54dd5-22kjj\" (UID: \"7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7\") " pod="openshift-image-registry/image-registry-8cdf54dd5-22kjj"
Sep 10 19:16:02.343982 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:02.343866    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7-registry-tls\") pod \"image-registry-8cdf54dd5-22kjj\" (UID: \"7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7\") " pod="openshift-image-registry/image-registry-8cdf54dd5-22kjj"
Sep 10 19:16:02.343982 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:02.343921    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7-image-registry-private-configuration\") pod \"image-registry-8cdf54dd5-22kjj\" (UID: \"7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7\") " pod="openshift-image-registry/image-registry-8cdf54dd5-22kjj"
Sep 10 19:16:02.343982 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:02.343951    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7-bound-sa-token\") pod \"image-registry-8cdf54dd5-22kjj\" (UID: \"7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7\") " pod="openshift-image-registry/image-registry-8cdf54dd5-22kjj"
Sep 10 19:16:02.344095 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:02.344004    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7-registry-certificates\") pod \"image-registry-8cdf54dd5-22kjj\" (UID: \"7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7\") " pod="openshift-image-registry/image-registry-8cdf54dd5-22kjj"
Sep 10 19:16:02.378607 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:02.378584    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9rkjw"
Sep 10 19:16:02.380832 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:02.380815    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-tsrfs\""
Sep 10 19:16:02.380932 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:02.380855    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 10 19:16:02.380932 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:02.380879    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 10 19:16:02.430862 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:02.430820    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:02.444560 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:02.444533    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7-installation-pull-secrets\") pod \"image-registry-8cdf54dd5-22kjj\" (UID: \"7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7\") " pod="openshift-image-registry/image-registry-8cdf54dd5-22kjj"
Sep 10 19:16:02.444677 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:02.444567    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7-trusted-ca\") pod \"image-registry-8cdf54dd5-22kjj\" (UID: \"7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7\") " pod="openshift-image-registry/image-registry-8cdf54dd5-22kjj"
Sep 10 19:16:02.444677 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:02.444586    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7-ca-trust-extracted\") pod \"image-registry-8cdf54dd5-22kjj\" (UID: \"7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7\") " pod="openshift-image-registry/image-registry-8cdf54dd5-22kjj"
Sep 10 19:16:02.444677 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:02.444601    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jlj2m\" (UniqueName: \"kubernetes.io/projected/7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7-kube-api-access-jlj2m\") pod \"image-registry-8cdf54dd5-22kjj\" (UID: \"7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7\") " pod="openshift-image-registry/image-registry-8cdf54dd5-22kjj"
Sep 10 19:16:02.444836 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:02.444718    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7-registry-tls\") pod \"image-registry-8cdf54dd5-22kjj\" (UID: \"7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7\") " pod="openshift-image-registry/image-registry-8cdf54dd5-22kjj"
Sep 10 19:16:02.444836 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:02.444751    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7bcc0525-ded3-4f3b-896e-73c566a68e34-tmp-dir\") pod \"dns-default-rjdmw\" (UID: \"7bcc0525-ded3-4f3b-896e-73c566a68e34\") " pod="openshift-dns/dns-default-rjdmw"
Sep 10 19:16:02.444836 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:02.444811    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7-image-registry-private-configuration\") pod \"image-registry-8cdf54dd5-22kjj\" (UID: \"7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7\") " pod="openshift-image-registry/image-registry-8cdf54dd5-22kjj"
Sep 10 19:16:02.444988 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:02.444838    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4cb26dfc-5d83-4cec-b4f2-52f3ff4c47d6-cert\") pod \"ingress-canary-gbtv9\" (UID: \"4cb26dfc-5d83-4cec-b4f2-52f3ff4c47d6\") " pod="openshift-ingress-canary/ingress-canary-gbtv9"
Sep 10 19:16:02.444988 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:02.444868    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7-bound-sa-token\") pod \"image-registry-8cdf54dd5-22kjj\" (UID: \"7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7\") " pod="openshift-image-registry/image-registry-8cdf54dd5-22kjj"
Sep 10 19:16:02.444988 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:02.444897    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7-registry-certificates\") pod \"image-registry-8cdf54dd5-22kjj\" (UID: \"7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7\") " pod="openshift-image-registry/image-registry-8cdf54dd5-22kjj"
Sep 10 19:16:02.444988 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:16:02.444908    2575 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 19:16:02.444988 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:16:02.444921    2575 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-8cdf54dd5-22kjj: secret "image-registry-tls" not found
Sep 10 19:16:02.444988 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:02.444936    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7bcc0525-ded3-4f3b-896e-73c566a68e34-metrics-tls\") pod \"dns-default-rjdmw\" (UID: \"7bcc0525-ded3-4f3b-896e-73c566a68e34\") " pod="openshift-dns/dns-default-rjdmw"
Sep 10 19:16:02.444988 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:16:02.444968    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7-registry-tls podName:7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7 nodeName:}" failed. No retries permitted until 2026-09-10 19:16:02.944952286 +0000 UTC m=+34.004779935 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7-registry-tls") pod "image-registry-8cdf54dd5-22kjj" (UID: "7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7") : secret "image-registry-tls" not found
Sep 10 19:16:02.444988 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:02.444986    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2924b\" (UniqueName: \"kubernetes.io/projected/7bcc0525-ded3-4f3b-896e-73c566a68e34-kube-api-access-2924b\") pod \"dns-default-rjdmw\" (UID: \"7bcc0525-ded3-4f3b-896e-73c566a68e34\") " pod="openshift-dns/dns-default-rjdmw"
Sep 10 19:16:02.445444 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:02.445016    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8ph7k\" (UniqueName: \"kubernetes.io/projected/4cb26dfc-5d83-4cec-b4f2-52f3ff4c47d6-kube-api-access-8ph7k\") pod \"ingress-canary-gbtv9\" (UID: \"4cb26dfc-5d83-4cec-b4f2-52f3ff4c47d6\") " pod="openshift-ingress-canary/ingress-canary-gbtv9"
Sep 10 19:16:02.445444 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:02.445003    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7-ca-trust-extracted\") pod \"image-registry-8cdf54dd5-22kjj\" (UID: \"7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7\") " pod="openshift-image-registry/image-registry-8cdf54dd5-22kjj"
Sep 10 19:16:02.445444 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:02.445046    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/7bcc0525-ded3-4f3b-896e-73c566a68e34-config-volume\") pod \"dns-default-rjdmw\" (UID: \"7bcc0525-ded3-4f3b-896e-73c566a68e34\") " pod="openshift-dns/dns-default-rjdmw"
Sep 10 19:16:02.445609 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:02.445587    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7-trusted-ca\") pod \"image-registry-8cdf54dd5-22kjj\" (UID: \"7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7\") " pod="openshift-image-registry/image-registry-8cdf54dd5-22kjj"
Sep 10 19:16:02.445837 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:02.445814    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7-registry-certificates\") pod \"image-registry-8cdf54dd5-22kjj\" (UID: \"7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7\") " pod="openshift-image-registry/image-registry-8cdf54dd5-22kjj"
Sep 10 19:16:02.449062 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:02.449040    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7-installation-pull-secrets\") pod \"image-registry-8cdf54dd5-22kjj\" (UID: \"7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7\") " pod="openshift-image-registry/image-registry-8cdf54dd5-22kjj"
Sep 10 19:16:02.449167 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:02.449040    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7-image-registry-private-configuration\") pod \"image-registry-8cdf54dd5-22kjj\" (UID: \"7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7\") " pod="openshift-image-registry/image-registry-8cdf54dd5-22kjj"
Sep 10 19:16:02.454143 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:02.454114    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jlj2m\" (UniqueName: \"kubernetes.io/projected/7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7-kube-api-access-jlj2m\") pod \"image-registry-8cdf54dd5-22kjj\" (UID: \"7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7\") " pod="openshift-image-registry/image-registry-8cdf54dd5-22kjj"
Sep 10 19:16:02.454226 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:02.454168    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7-bound-sa-token\") pod \"image-registry-8cdf54dd5-22kjj\" (UID: \"7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7\") " pod="openshift-image-registry/image-registry-8cdf54dd5-22kjj"
Sep 10 19:16:02.546160 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:02.546096    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4cb26dfc-5d83-4cec-b4f2-52f3ff4c47d6-cert\") pod \"ingress-canary-gbtv9\" (UID: \"4cb26dfc-5d83-4cec-b4f2-52f3ff4c47d6\") " pod="openshift-ingress-canary/ingress-canary-gbtv9"
Sep 10 19:16:02.546160 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:02.546147    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7bcc0525-ded3-4f3b-896e-73c566a68e34-metrics-tls\") pod \"dns-default-rjdmw\" (UID: \"7bcc0525-ded3-4f3b-896e-73c566a68e34\") " pod="openshift-dns/dns-default-rjdmw"
Sep 10 19:16:02.546336 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:02.546167    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2924b\" (UniqueName: \"kubernetes.io/projected/7bcc0525-ded3-4f3b-896e-73c566a68e34-kube-api-access-2924b\") pod \"dns-default-rjdmw\" (UID: \"7bcc0525-ded3-4f3b-896e-73c566a68e34\") " pod="openshift-dns/dns-default-rjdmw"
Sep 10 19:16:02.546336 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:02.546192    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8ph7k\" (UniqueName: \"kubernetes.io/projected/4cb26dfc-5d83-4cec-b4f2-52f3ff4c47d6-kube-api-access-8ph7k\") pod \"ingress-canary-gbtv9\" (UID: \"4cb26dfc-5d83-4cec-b4f2-52f3ff4c47d6\") " pod="openshift-ingress-canary/ingress-canary-gbtv9"
Sep 10 19:16:02.546336 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:02.546218    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/7bcc0525-ded3-4f3b-896e-73c566a68e34-config-volume\") pod \"dns-default-rjdmw\" (UID: \"7bcc0525-ded3-4f3b-896e-73c566a68e34\") " pod="openshift-dns/dns-default-rjdmw"
Sep 10 19:16:02.546336 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:16:02.546261    2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 19:16:02.546336 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:02.546281    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7bcc0525-ded3-4f3b-896e-73c566a68e34-tmp-dir\") pod \"dns-default-rjdmw\" (UID: \"7bcc0525-ded3-4f3b-896e-73c566a68e34\") " pod="openshift-dns/dns-default-rjdmw"
Sep 10 19:16:02.546336 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:16:02.546324    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7bcc0525-ded3-4f3b-896e-73c566a68e34-metrics-tls podName:7bcc0525-ded3-4f3b-896e-73c566a68e34 nodeName:}" failed. No retries permitted until 2026-09-10 19:16:03.046304925 +0000 UTC m=+34.106132575 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7bcc0525-ded3-4f3b-896e-73c566a68e34-metrics-tls") pod "dns-default-rjdmw" (UID: "7bcc0525-ded3-4f3b-896e-73c566a68e34") : secret "dns-default-metrics-tls" not found
Sep 10 19:16:02.546743 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:02.546723    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7bcc0525-ded3-4f3b-896e-73c566a68e34-tmp-dir\") pod \"dns-default-rjdmw\" (UID: \"7bcc0525-ded3-4f3b-896e-73c566a68e34\") " pod="openshift-dns/dns-default-rjdmw"
Sep 10 19:16:02.546915 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:02.546894    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/7bcc0525-ded3-4f3b-896e-73c566a68e34-config-volume\") pod \"dns-default-rjdmw\" (UID: \"7bcc0525-ded3-4f3b-896e-73c566a68e34\") " pod="openshift-dns/dns-default-rjdmw"
Sep 10 19:16:02.548629 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:02.548606    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4cb26dfc-5d83-4cec-b4f2-52f3ff4c47d6-cert\") pod \"ingress-canary-gbtv9\" (UID: \"4cb26dfc-5d83-4cec-b4f2-52f3ff4c47d6\") " pod="openshift-ingress-canary/ingress-canary-gbtv9"
Sep 10 19:16:02.554739 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:02.554719    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2924b\" (UniqueName: \"kubernetes.io/projected/7bcc0525-ded3-4f3b-896e-73c566a68e34-kube-api-access-2924b\") pod \"dns-default-rjdmw\" (UID: \"7bcc0525-ded3-4f3b-896e-73c566a68e34\") " pod="openshift-dns/dns-default-rjdmw"
Sep 10 19:16:02.554848 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:02.554827    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8ph7k\" (UniqueName: \"kubernetes.io/projected/4cb26dfc-5d83-4cec-b4f2-52f3ff4c47d6-kube-api-access-8ph7k\") pod \"ingress-canary-gbtv9\" (UID: \"4cb26dfc-5d83-4cec-b4f2-52f3ff4c47d6\") " pod="openshift-ingress-canary/ingress-canary-gbtv9"
Sep 10 19:16:02.609863 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:02.609834    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-gbtv9"
Sep 10 19:16:02.663736 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:02.663708    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:02.949398 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:02.949359    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7-registry-tls\") pod \"image-registry-8cdf54dd5-22kjj\" (UID: \"7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7\") " pod="openshift-image-registry/image-registry-8cdf54dd5-22kjj"
Sep 10 19:16:02.949566 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:16:02.949471    2575 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 19:16:02.949566 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:16:02.949493    2575 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-8cdf54dd5-22kjj: secret "image-registry-tls" not found
Sep 10 19:16:02.949566 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:16:02.949561    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7-registry-tls podName:7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7 nodeName:}" failed. No retries permitted until 2026-09-10 19:16:03.949538128 +0000 UTC m=+35.009365779 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7-registry-tls") pod "image-registry-8cdf54dd5-22kjj" (UID: "7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7") : secret "image-registry-tls" not found
Sep 10 19:16:03.050042 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:03.050007    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7bcc0525-ded3-4f3b-896e-73c566a68e34-metrics-tls\") pod \"dns-default-rjdmw\" (UID: \"7bcc0525-ded3-4f3b-896e-73c566a68e34\") " pod="openshift-dns/dns-default-rjdmw"
Sep 10 19:16:03.050218 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:16:03.050187    2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 19:16:03.050287 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:16:03.050274    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7bcc0525-ded3-4f3b-896e-73c566a68e34-metrics-tls podName:7bcc0525-ded3-4f3b-896e-73c566a68e34 nodeName:}" failed. No retries permitted until 2026-09-10 19:16:04.050254148 +0000 UTC m=+35.110081794 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7bcc0525-ded3-4f3b-896e-73c566a68e34-metrics-tls") pod "dns-default-rjdmw" (UID: "7bcc0525-ded3-4f3b-896e-73c566a68e34") : secret "dns-default-metrics-tls" not found
Sep 10 19:16:03.378586 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:03.378502    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wx2q6"
Sep 10 19:16:03.378736 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:03.378694    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-24lb9"
Sep 10 19:16:03.380774 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:03.380750    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-q8jbb\""
Sep 10 19:16:03.380890 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:03.380757    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 10 19:16:03.380890 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:03.380870    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 10 19:16:03.764319 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:03.764173    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-gbtv9"]
Sep 10 19:16:03.770384 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:16:03.770356    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4cb26dfc_5d83_4cec_b4f2_52f3ff4c47d6.slice/crio-504d73c5345ab01daf5fd16594e5d2b5f04ba8c9101d53e06691ea36d8a563a7 WatchSource:0}: Error finding container 504d73c5345ab01daf5fd16594e5d2b5f04ba8c9101d53e06691ea36d8a563a7: Status 404 returned error can't find the container with id 504d73c5345ab01daf5fd16594e5d2b5f04ba8c9101d53e06691ea36d8a563a7
Sep 10 19:16:03.956293 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:03.956270    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7-registry-tls\") pod \"image-registry-8cdf54dd5-22kjj\" (UID: \"7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7\") " pod="openshift-image-registry/image-registry-8cdf54dd5-22kjj"
Sep 10 19:16:03.956415 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:16:03.956401    2575 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 19:16:03.956456 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:16:03.956417    2575 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-8cdf54dd5-22kjj: secret "image-registry-tls" not found
Sep 10 19:16:03.956492 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:16:03.956470    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7-registry-tls podName:7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7 nodeName:}" failed. No retries permitted until 2026-09-10 19:16:05.956453871 +0000 UTC m=+37.016281516 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7-registry-tls") pod "image-registry-8cdf54dd5-22kjj" (UID: "7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7") : secret "image-registry-tls" not found
Sep 10 19:16:04.057130 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:04.057105    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7bcc0525-ded3-4f3b-896e-73c566a68e34-metrics-tls\") pod \"dns-default-rjdmw\" (UID: \"7bcc0525-ded3-4f3b-896e-73c566a68e34\") " pod="openshift-dns/dns-default-rjdmw"
Sep 10 19:16:04.057233 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:16:04.057200    2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 19:16:04.057288 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:16:04.057247    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7bcc0525-ded3-4f3b-896e-73c566a68e34-metrics-tls podName:7bcc0525-ded3-4f3b-896e-73c566a68e34 nodeName:}" failed. No retries permitted until 2026-09-10 19:16:06.057232818 +0000 UTC m=+37.117060478 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7bcc0525-ded3-4f3b-896e-73c566a68e34-metrics-tls") pod "dns-default-rjdmw" (UID: "7bcc0525-ded3-4f3b-896e-73c566a68e34") : secret "dns-default-metrics-tls" not found
Sep 10 19:16:04.589357 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:04.589299    2575 generic.go:356] "Generic (PLEG): container finished" podID="cf162519-8424-43b0-8de5-a19bc237a9d0" containerID="0cbc52c8890d3a793521a97c0c00f16a0947a0563e97a1156254ef18f90a18cb" exitCode=0
Sep 10 19:16:04.589514 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:04.589437    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wt6fs" event={"ID":"cf162519-8424-43b0-8de5-a19bc237a9d0","Type":"ContainerDied","Data":"0cbc52c8890d3a793521a97c0c00f16a0947a0563e97a1156254ef18f90a18cb"}
Sep 10 19:16:04.590692 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:04.590673    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-gbtv9" event={"ID":"4cb26dfc-5d83-4cec-b4f2-52f3ff4c47d6","Type":"ContainerStarted","Data":"504d73c5345ab01daf5fd16594e5d2b5f04ba8c9101d53e06691ea36d8a563a7"}
Sep 10 19:16:05.596970 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:05.596938    2575 generic.go:356] "Generic (PLEG): container finished" podID="cf162519-8424-43b0-8de5-a19bc237a9d0" containerID="89368ce1800dfe61ac1be08f57d0d64525f9341670da57cbdafac94f7ac4d38d" exitCode=0
Sep 10 19:16:05.597522 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:05.596990    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wt6fs" event={"ID":"cf162519-8424-43b0-8de5-a19bc237a9d0","Type":"ContainerDied","Data":"89368ce1800dfe61ac1be08f57d0d64525f9341670da57cbdafac94f7ac4d38d"}
Sep 10 19:16:05.970940 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:05.970906    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7-registry-tls\") pod \"image-registry-8cdf54dd5-22kjj\" (UID: \"7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7\") " pod="openshift-image-registry/image-registry-8cdf54dd5-22kjj"
Sep 10 19:16:05.971085 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:16:05.971052    2575 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 19:16:05.971085 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:16:05.971071    2575 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-8cdf54dd5-22kjj: secret "image-registry-tls" not found
Sep 10 19:16:05.971159 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:16:05.971124    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7-registry-tls podName:7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7 nodeName:}" failed. No retries permitted until 2026-09-10 19:16:09.971109154 +0000 UTC m=+41.030936799 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7-registry-tls") pod "image-registry-8cdf54dd5-22kjj" (UID: "7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7") : secret "image-registry-tls" not found
Sep 10 19:16:06.072177 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:06.072153    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7bcc0525-ded3-4f3b-896e-73c566a68e34-metrics-tls\") pod \"dns-default-rjdmw\" (UID: \"7bcc0525-ded3-4f3b-896e-73c566a68e34\") " pod="openshift-dns/dns-default-rjdmw"
Sep 10 19:16:06.072323 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:16:06.072306    2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 19:16:06.072401 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:16:06.072391    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7bcc0525-ded3-4f3b-896e-73c566a68e34-metrics-tls podName:7bcc0525-ded3-4f3b-896e-73c566a68e34 nodeName:}" failed. No retries permitted until 2026-09-10 19:16:10.072375025 +0000 UTC m=+41.132202681 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7bcc0525-ded3-4f3b-896e-73c566a68e34-metrics-tls") pod "dns-default-rjdmw" (UID: "7bcc0525-ded3-4f3b-896e-73c566a68e34") : secret "dns-default-metrics-tls" not found
Sep 10 19:16:06.600603 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:06.600520    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-gbtv9" event={"ID":"4cb26dfc-5d83-4cec-b4f2-52f3ff4c47d6","Type":"ContainerStarted","Data":"511bc432c80c1dd72cbe24e34a84daff1e289dbf05f8801600cf09bcf359af66"}
Sep 10 19:16:06.603834 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:06.603807    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wt6fs" event={"ID":"cf162519-8424-43b0-8de5-a19bc237a9d0","Type":"ContainerStarted","Data":"7540d305f1a89f3c5372f53dd905e771e4b466d4cc95c271d9be53b8ad3c4403"}
Sep 10 19:16:06.614986 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:06.614935    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-gbtv9" podStartSLOduration=2.238203815 podStartE2EDuration="4.614916692s" totalImagesPullingTime="2.376712877s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:16:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:16:06.61472737 +0000 UTC m=+37.674555037" watchObservedRunningTime="2026-09-10 19:16:06.614916692 +0000 UTC m=+37.674744360"
Sep 10 19:16:06.634963 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:06.634925    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-wt6fs" podStartSLOduration=7.768322078 podStartE2EDuration="37.634909013s" totalImagesPullingTime="29.866586935s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 19:15:29 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:16:06.633867432 +0000 UTC m=+37.693695091" watchObservedRunningTime="2026-09-10 19:16:06.634909013 +0000 UTC m=+37.694736681"
Sep 10 19:16:06.756909 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:06.756879    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:06.858199 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:06.858145    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:07.314294 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:07.314263    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:07.416057 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:07.416026    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:08.471595 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:08.471556    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:08.562839 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:08.562805    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:08.633091 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:08.633066    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:08.731384 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:08.731316    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:08.832134 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:08.832100    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:09.997564 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:09.997523    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7-registry-tls\") pod \"image-registry-8cdf54dd5-22kjj\" (UID: \"7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7\") " pod="openshift-image-registry/image-registry-8cdf54dd5-22kjj"
Sep 10 19:16:09.997950 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:16:09.997664    2575 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 19:16:09.997950 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:16:09.997682    2575 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-8cdf54dd5-22kjj: secret "image-registry-tls" not found
Sep 10 19:16:09.997950 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:16:09.997734    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7-registry-tls podName:7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7 nodeName:}" failed. No retries permitted until 2026-09-10 19:16:17.997719644 +0000 UTC m=+49.057547290 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7-registry-tls") pod "image-registry-8cdf54dd5-22kjj" (UID: "7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7") : secret "image-registry-tls" not found
Sep 10 19:16:10.098235 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:10.098205    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7bcc0525-ded3-4f3b-896e-73c566a68e34-metrics-tls\") pod \"dns-default-rjdmw\" (UID: \"7bcc0525-ded3-4f3b-896e-73c566a68e34\") " pod="openshift-dns/dns-default-rjdmw"
Sep 10 19:16:10.098386 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:16:10.098361    2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 19:16:10.098455 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:16:10.098426    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7bcc0525-ded3-4f3b-896e-73c566a68e34-metrics-tls podName:7bcc0525-ded3-4f3b-896e-73c566a68e34 nodeName:}" failed. No retries permitted until 2026-09-10 19:16:18.098407737 +0000 UTC m=+49.158235399 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7bcc0525-ded3-4f3b-896e-73c566a68e34-metrics-tls") pod "dns-default-rjdmw" (UID: "7bcc0525-ded3-4f3b-896e-73c566a68e34") : secret "dns-default-metrics-tls" not found
Sep 10 19:16:11.511097 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:11.511058    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:13.566674 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:13.566640    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:13.682433 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:13.682406    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:13.783323 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:13.783292    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:14.599870 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:14.599841    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:14.700383 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:14.700354    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:16.580116 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:16.580078    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:18.051027 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:18.050994    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7-registry-tls\") pod \"image-registry-8cdf54dd5-22kjj\" (UID: \"7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7\") " pod="openshift-image-registry/image-registry-8cdf54dd5-22kjj"
Sep 10 19:16:18.051483 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:16:18.051160    2575 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 19:16:18.051483 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:16:18.051180    2575 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-8cdf54dd5-22kjj: secret "image-registry-tls" not found
Sep 10 19:16:18.051483 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:16:18.051262    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7-registry-tls podName:7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7 nodeName:}" failed. No retries permitted until 2026-09-10 19:16:34.051241078 +0000 UTC m=+65.111068748 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7-registry-tls") pod "image-registry-8cdf54dd5-22kjj" (UID: "7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7") : secret "image-registry-tls" not found
Sep 10 19:16:18.151385 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:18.151352    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7bcc0525-ded3-4f3b-896e-73c566a68e34-metrics-tls\") pod \"dns-default-rjdmw\" (UID: \"7bcc0525-ded3-4f3b-896e-73c566a68e34\") " pod="openshift-dns/dns-default-rjdmw"
Sep 10 19:16:18.151560 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:16:18.151506    2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 19:16:18.151611 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:16:18.151575    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7bcc0525-ded3-4f3b-896e-73c566a68e34-metrics-tls podName:7bcc0525-ded3-4f3b-896e-73c566a68e34 nodeName:}" failed. No retries permitted until 2026-09-10 19:16:34.151559714 +0000 UTC m=+65.211387384 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7bcc0525-ded3-4f3b-896e-73c566a68e34-metrics-tls") pod "dns-default-rjdmw" (UID: "7bcc0525-ded3-4f3b-896e-73c566a68e34") : secret "dns-default-metrics-tls" not found
Sep 10 19:16:20.367161 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:20.367126    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:20.468435 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:20.468404    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:24.691001 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:24.690960    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:24.776816 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:24.776784    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:25.429540 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:25.429507    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:25.530951 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:25.530916    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:25.630214 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:25.630186    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:27.192858 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:27.192829    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-rvczm"]
Sep 10 19:16:27.195158 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:27.195141    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-xzxvx"]
Sep 10 19:16:27.195265 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:27.195250    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-rvczm"
Sep 10 19:16:27.196858 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:27.196839    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-xzxvx"
Sep 10 19:16:27.197272 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:27.197257    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Sep 10 19:16:27.197356 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:27.197279    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-c5z7q\""
Sep 10 19:16:27.197736 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:27.197721    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Sep 10 19:16:27.197838 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:27.197821    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Sep 10 19:16:27.199064 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:27.199043    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Sep 10 19:16:27.199189 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:27.199043    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Sep 10 19:16:27.199680 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:27.199666    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-pfbp6\""
Sep 10 19:16:27.209948 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:27.209584    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-rvczm"]
Sep 10 19:16:27.210034 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:27.209999    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-xzxvx"]
Sep 10 19:16:27.287076 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:27.287051    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-685456655f-n4jkz"]
Sep 10 19:16:27.288716 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:27.288694    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-685456655f-n4jkz"
Sep 10 19:16:27.290626 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:27.290609    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-dxjr7\""
Sep 10 19:16:27.300463 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:27.300447    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-685456655f-n4jkz"]
Sep 10 19:16:27.314336 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:27.314316    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b4f4c54c-042a-42ea-8b53-81426d7f8f48-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-rvczm\" (UID: \"b4f4c54c-042a-42ea-8b53-81426d7f8f48\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-rvczm"
Sep 10 19:16:27.314452 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:27.314377    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kc7rk\" (UniqueName: \"kubernetes.io/projected/b4f4c54c-042a-42ea-8b53-81426d7f8f48-kube-api-access-kc7rk\") pod \"cluster-samples-operator-7cdcfcf76c-rvczm\" (UID: \"b4f4c54c-042a-42ea-8b53-81426d7f8f48\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-rvczm"
Sep 10 19:16:27.314452 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:27.314445    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-slq8p\" (UniqueName: \"kubernetes.io/projected/977b75ac-232a-46e1-92d4-5496efa09faa-kube-api-access-slq8p\") pod \"volume-data-source-validator-695c895bd6-xzxvx\" (UID: \"977b75ac-232a-46e1-92d4-5496efa09faa\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-xzxvx"
Sep 10 19:16:27.337498 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:27.337477    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:27.382399 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:27.382376    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-9gl2b"]
Sep 10 19:16:27.384214 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:27.384196    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-9gl2b"
Sep 10 19:16:27.389978 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:27.389960    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Sep 10 19:16:27.390060 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:27.390013    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Sep 10 19:16:27.390116 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:27.390104    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-94jfq\""
Sep 10 19:16:27.390413 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:27.390399    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Sep 10 19:16:27.391680 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:27.391664    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Sep 10 19:16:27.399989 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:27.399973    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-9gl2b"]
Sep 10 19:16:27.415561 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:27.415536    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wg257\" (UniqueName: \"kubernetes.io/projected/4b6c65c7-f7b9-4585-9010-977e12e4a63f-kube-api-access-wg257\") pod \"network-check-source-685456655f-n4jkz\" (UID: \"4b6c65c7-f7b9-4585-9010-977e12e4a63f\") " pod="openshift-network-diagnostics/network-check-source-685456655f-n4jkz"
Sep 10 19:16:27.415653 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:27.415582    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b4f4c54c-042a-42ea-8b53-81426d7f8f48-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-rvczm\" (UID: \"b4f4c54c-042a-42ea-8b53-81426d7f8f48\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-rvczm"
Sep 10 19:16:27.415653 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:27.415598    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kc7rk\" (UniqueName: \"kubernetes.io/projected/b4f4c54c-042a-42ea-8b53-81426d7f8f48-kube-api-access-kc7rk\") pod \"cluster-samples-operator-7cdcfcf76c-rvczm\" (UID: \"b4f4c54c-042a-42ea-8b53-81426d7f8f48\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-rvczm"
Sep 10 19:16:27.415763 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:27.415683    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-slq8p\" (UniqueName: \"kubernetes.io/projected/977b75ac-232a-46e1-92d4-5496efa09faa-kube-api-access-slq8p\") pod \"volume-data-source-validator-695c895bd6-xzxvx\" (UID: \"977b75ac-232a-46e1-92d4-5496efa09faa\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-xzxvx"
Sep 10 19:16:27.415763 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:16:27.415689    2575 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 10 19:16:27.415856 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:16:27.415788    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b4f4c54c-042a-42ea-8b53-81426d7f8f48-samples-operator-tls podName:b4f4c54c-042a-42ea-8b53-81426d7f8f48 nodeName:}" failed. No retries permitted until 2026-09-10 19:16:27.915769454 +0000 UTC m=+58.975597106 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b4f4c54c-042a-42ea-8b53-81426d7f8f48-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-rvczm" (UID: "b4f4c54c-042a-42ea-8b53-81426d7f8f48") : secret "samples-operator-tls" not found
Sep 10 19:16:27.434205 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:27.434181    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kc7rk\" (UniqueName: \"kubernetes.io/projected/b4f4c54c-042a-42ea-8b53-81426d7f8f48-kube-api-access-kc7rk\") pod \"cluster-samples-operator-7cdcfcf76c-rvczm\" (UID: \"b4f4c54c-042a-42ea-8b53-81426d7f8f48\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-rvczm"
Sep 10 19:16:27.434282 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:27.434264    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-slq8p\" (UniqueName: \"kubernetes.io/projected/977b75ac-232a-46e1-92d4-5496efa09faa-kube-api-access-slq8p\") pod \"volume-data-source-validator-695c895bd6-xzxvx\" (UID: \"977b75ac-232a-46e1-92d4-5496efa09faa\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-xzxvx"
Sep 10 19:16:27.438176 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:27.438112    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:27.512173 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:27.512154    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-xzxvx"
Sep 10 19:16:27.515939 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:27.515922    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/38dcdfc7-edac-4ddb-b51a-bbf5857139a8-serving-cert\") pod \"service-ca-operator-859f94c4b-9gl2b\" (UID: \"38dcdfc7-edac-4ddb-b51a-bbf5857139a8\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-9gl2b"
Sep 10 19:16:27.516008 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:27.515956    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/38dcdfc7-edac-4ddb-b51a-bbf5857139a8-config\") pod \"service-ca-operator-859f94c4b-9gl2b\" (UID: \"38dcdfc7-edac-4ddb-b51a-bbf5857139a8\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-9gl2b"
Sep 10 19:16:27.516049 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:27.516037    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wg257\" (UniqueName: \"kubernetes.io/projected/4b6c65c7-f7b9-4585-9010-977e12e4a63f-kube-api-access-wg257\") pod \"network-check-source-685456655f-n4jkz\" (UID: \"4b6c65c7-f7b9-4585-9010-977e12e4a63f\") " pod="openshift-network-diagnostics/network-check-source-685456655f-n4jkz"
Sep 10 19:16:27.516091 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:27.516056    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qq5kd\" (UniqueName: \"kubernetes.io/projected/38dcdfc7-edac-4ddb-b51a-bbf5857139a8-kube-api-access-qq5kd\") pod \"service-ca-operator-859f94c4b-9gl2b\" (UID: \"38dcdfc7-edac-4ddb-b51a-bbf5857139a8\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-9gl2b"
Sep 10 19:16:27.524134 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:27.524111    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wg257\" (UniqueName: \"kubernetes.io/projected/4b6c65c7-f7b9-4585-9010-977e12e4a63f-kube-api-access-wg257\") pod \"network-check-source-685456655f-n4jkz\" (UID: \"4b6c65c7-f7b9-4585-9010-977e12e4a63f\") " pod="openshift-network-diagnostics/network-check-source-685456655f-n4jkz"
Sep 10 19:16:27.595810 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:27.595778    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-685456655f-n4jkz"
Sep 10 19:16:27.616883 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:27.616850    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qq5kd\" (UniqueName: \"kubernetes.io/projected/38dcdfc7-edac-4ddb-b51a-bbf5857139a8-kube-api-access-qq5kd\") pod \"service-ca-operator-859f94c4b-9gl2b\" (UID: \"38dcdfc7-edac-4ddb-b51a-bbf5857139a8\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-9gl2b"
Sep 10 19:16:27.617024 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:27.616898    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/38dcdfc7-edac-4ddb-b51a-bbf5857139a8-serving-cert\") pod \"service-ca-operator-859f94c4b-9gl2b\" (UID: \"38dcdfc7-edac-4ddb-b51a-bbf5857139a8\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-9gl2b"
Sep 10 19:16:27.617024 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:27.616921    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/38dcdfc7-edac-4ddb-b51a-bbf5857139a8-config\") pod \"service-ca-operator-859f94c4b-9gl2b\" (UID: \"38dcdfc7-edac-4ddb-b51a-bbf5857139a8\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-9gl2b"
Sep 10 19:16:27.617545 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:27.617524    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/38dcdfc7-edac-4ddb-b51a-bbf5857139a8-config\") pod \"service-ca-operator-859f94c4b-9gl2b\" (UID: \"38dcdfc7-edac-4ddb-b51a-bbf5857139a8\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-9gl2b"
Sep 10 19:16:27.619135 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:27.619104    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/38dcdfc7-edac-4ddb-b51a-bbf5857139a8-serving-cert\") pod \"service-ca-operator-859f94c4b-9gl2b\" (UID: \"38dcdfc7-edac-4ddb-b51a-bbf5857139a8\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-9gl2b"
Sep 10 19:16:27.623019 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:27.622998    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:27.624897 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:27.624880    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qq5kd\" (UniqueName: \"kubernetes.io/projected/38dcdfc7-edac-4ddb-b51a-bbf5857139a8-kube-api-access-qq5kd\") pod \"service-ca-operator-859f94c4b-9gl2b\" (UID: \"38dcdfc7-edac-4ddb-b51a-bbf5857139a8\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-9gl2b"
Sep 10 19:16:27.692248 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:27.692226    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-9gl2b"
Sep 10 19:16:27.713147 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:27.713116    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:27.777039 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:27.776985    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-xzxvx"]
Sep 10 19:16:27.779997 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:16:27.779972    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod977b75ac_232a_46e1_92d4_5496efa09faa.slice/crio-f926e8c644ebf5197dc379526f6849911a185a16bbc3e27b82ad35b422abae47 WatchSource:0}: Error finding container f926e8c644ebf5197dc379526f6849911a185a16bbc3e27b82ad35b422abae47: Status 404 returned error can't find the container with id f926e8c644ebf5197dc379526f6849911a185a16bbc3e27b82ad35b422abae47
Sep 10 19:16:27.861321 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:27.861289    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-685456655f-n4jkz"]
Sep 10 19:16:27.864601 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:16:27.864579    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b6c65c7_f7b9_4585_9010_977e12e4a63f.slice/crio-a36407dc3b6d52a8d0ca7117c7ee8ea813cf8d07bf2531a606a67355c4aeb0c3 WatchSource:0}: Error finding container a36407dc3b6d52a8d0ca7117c7ee8ea813cf8d07bf2531a606a67355c4aeb0c3: Status 404 returned error can't find the container with id a36407dc3b6d52a8d0ca7117c7ee8ea813cf8d07bf2531a606a67355c4aeb0c3
Sep 10 19:16:27.920352 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:27.920314    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b4f4c54c-042a-42ea-8b53-81426d7f8f48-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-rvczm\" (UID: \"b4f4c54c-042a-42ea-8b53-81426d7f8f48\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-rvczm"
Sep 10 19:16:27.920476 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:16:27.920458    2575 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 10 19:16:27.920527 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:16:27.920517    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b4f4c54c-042a-42ea-8b53-81426d7f8f48-samples-operator-tls podName:b4f4c54c-042a-42ea-8b53-81426d7f8f48 nodeName:}" failed. No retries permitted until 2026-09-10 19:16:28.920503833 +0000 UTC m=+59.980331479 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b4f4c54c-042a-42ea-8b53-81426d7f8f48-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-rvczm" (UID: "b4f4c54c-042a-42ea-8b53-81426d7f8f48") : secret "samples-operator-tls" not found
Sep 10 19:16:27.955249 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:27.955225    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-9gl2b"]
Sep 10 19:16:27.957991 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:16:27.957969    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38dcdfc7_edac_4ddb_b51a_bbf5857139a8.slice/crio-2dbd096a028015270a6d362cf036c3f34de2ce4544793ee7ff871a1587d84106 WatchSource:0}: Error finding container 2dbd096a028015270a6d362cf036c3f34de2ce4544793ee7ff871a1587d84106: Status 404 returned error can't find the container with id 2dbd096a028015270a6d362cf036c3f34de2ce4544793ee7ff871a1587d84106
Sep 10 19:16:28.586811 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:28.586771    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-rrxfj"
Sep 10 19:16:28.654578 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:28.654512    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-685456655f-n4jkz" event={"ID":"4b6c65c7-f7b9-4585-9010-977e12e4a63f","Type":"ContainerStarted","Data":"a36407dc3b6d52a8d0ca7117c7ee8ea813cf8d07bf2531a606a67355c4aeb0c3"}
Sep 10 19:16:28.656441 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:28.656398    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-9gl2b" event={"ID":"38dcdfc7-edac-4ddb-b51a-bbf5857139a8","Type":"ContainerStarted","Data":"2dbd096a028015270a6d362cf036c3f34de2ce4544793ee7ff871a1587d84106"}
Sep 10 19:16:28.658108 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:28.658083    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-xzxvx" event={"ID":"977b75ac-232a-46e1-92d4-5496efa09faa","Type":"ContainerStarted","Data":"f926e8c644ebf5197dc379526f6849911a185a16bbc3e27b82ad35b422abae47"}
Sep 10 19:16:28.767535 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:28.767504    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:28.869295 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:28.869224    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:28.928657 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:28.928625    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b4f4c54c-042a-42ea-8b53-81426d7f8f48-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-rvczm\" (UID: \"b4f4c54c-042a-42ea-8b53-81426d7f8f48\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-rvczm"
Sep 10 19:16:28.928798 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:16:28.928781    2575 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 10 19:16:28.928851 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:16:28.928845    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b4f4c54c-042a-42ea-8b53-81426d7f8f48-samples-operator-tls podName:b4f4c54c-042a-42ea-8b53-81426d7f8f48 nodeName:}" failed. No retries permitted until 2026-09-10 19:16:30.928825931 +0000 UTC m=+61.988653578 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b4f4c54c-042a-42ea-8b53-81426d7f8f48-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-rvczm" (UID: "b4f4c54c-042a-42ea-8b53-81426d7f8f48") : secret "samples-operator-tls" not found
Sep 10 19:16:29.662993 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:29.662958    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-xzxvx" event={"ID":"977b75ac-232a-46e1-92d4-5496efa09faa","Type":"ContainerStarted","Data":"ac620e5ee74c855c342baf7fe1874fe8387cf7cee83928061d57db8b5b44f5e4"}
Sep 10 19:16:29.679981 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:29.679930    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-xzxvx" podStartSLOduration=1.676511978 podStartE2EDuration="2.679914257s" totalImagesPullingTime="1.003402279s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:16:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:16:29.678180266 +0000 UTC m=+60.738007935" watchObservedRunningTime="2026-09-10 19:16:29.679914257 +0000 UTC m=+60.739741926"
Sep 10 19:16:29.680845 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:29.680802    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:29.780155 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:29.780118    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:29.884223 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:29.884163    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:30.453707 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:30.453670    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:30.525632 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:30.525603    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:30.946138 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:30.946105    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b4f4c54c-042a-42ea-8b53-81426d7f8f48-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-rvczm\" (UID: \"b4f4c54c-042a-42ea-8b53-81426d7f8f48\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-rvczm"
Sep 10 19:16:30.946546 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:16:30.946247    2575 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 10 19:16:30.946546 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:16:30.946316    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b4f4c54c-042a-42ea-8b53-81426d7f8f48-samples-operator-tls podName:b4f4c54c-042a-42ea-8b53-81426d7f8f48 nodeName:}" failed. No retries permitted until 2026-09-10 19:16:34.946297496 +0000 UTC m=+66.006125148 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b4f4c54c-042a-42ea-8b53-81426d7f8f48-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-rvczm" (UID: "b4f4c54c-042a-42ea-8b53-81426d7f8f48") : secret "samples-operator-tls" not found
Sep 10 19:16:31.670367 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:31.670263    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-9gl2b" event={"ID":"38dcdfc7-edac-4ddb-b51a-bbf5857139a8","Type":"ContainerStarted","Data":"e92c3f84cfc1ef0a1d8e4e0258dbe1eb239ce3e920337aeb4a33cb9c575ba7c4"}
Sep 10 19:16:31.671873 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:31.671848    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-685456655f-n4jkz" event={"ID":"4b6c65c7-f7b9-4585-9010-977e12e4a63f","Type":"ContainerStarted","Data":"f105190d10c94d0223ba173fa3f2fdb664cb356d0a7f3750f5b1e8552b10fb4e"}
Sep 10 19:16:31.685511 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:31.685458    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-9gl2b" podStartSLOduration=1.272481939 podStartE2EDuration="4.685440266s" totalImagesPullingTime="3.412958327s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:16:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:16:31.684770024 +0000 UTC m=+62.744597693" watchObservedRunningTime="2026-09-10 19:16:31.685440266 +0000 UTC m=+62.745267935"
Sep 10 19:16:31.699595 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:31.699550    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-685456655f-n4jkz" podStartSLOduration=1.185749762 podStartE2EDuration="4.699532574s" totalImagesPullingTime="3.513782812s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:16:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:16:31.699510825 +0000 UTC m=+62.759338504" watchObservedRunningTime="2026-09-10 19:16:31.699532574 +0000 UTC m=+62.759360243"
Sep 10 19:16:32.412567 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:32.412532    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:32.476703 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:32.476673    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:32.737329 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:32.737299    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-ktbdm"]
Sep 10 19:16:32.742929 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:32.742914    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-ktbdm"
Sep 10 19:16:32.745440 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:32.745412    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 10 19:16:32.745545 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:32.745450    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-q7pt8\""
Sep 10 19:16:32.745697 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:32.745678    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 10 19:16:32.745800 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:32.745757    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 10 19:16:32.745800 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:32.745758    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 10 19:16:32.755478 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:32.755459    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-ktbdm"]
Sep 10 19:16:32.760032 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:32.760005    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:32.829377 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:32.829327    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:32.862578 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:32.862559    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e06bba47-fb00-41eb-a0d4-34c98888d69a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ktbdm\" (UID: \"e06bba47-fb00-41eb-a0d4-34c98888d69a\") " pod="openshift-insights/insights-runtime-extractor-ktbdm"
Sep 10 19:16:32.862664 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:32.862617    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-spw8k\" (UniqueName: \"kubernetes.io/projected/e06bba47-fb00-41eb-a0d4-34c98888d69a-kube-api-access-spw8k\") pod \"insights-runtime-extractor-ktbdm\" (UID: \"e06bba47-fb00-41eb-a0d4-34c98888d69a\") " pod="openshift-insights/insights-runtime-extractor-ktbdm"
Sep 10 19:16:32.862664 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:32.862659    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e06bba47-fb00-41eb-a0d4-34c98888d69a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ktbdm\" (UID: \"e06bba47-fb00-41eb-a0d4-34c98888d69a\") " pod="openshift-insights/insights-runtime-extractor-ktbdm"
Sep 10 19:16:32.862751 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:32.862695    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e06bba47-fb00-41eb-a0d4-34c98888d69a-data-volume\") pod \"insights-runtime-extractor-ktbdm\" (UID: \"e06bba47-fb00-41eb-a0d4-34c98888d69a\") " pod="openshift-insights/insights-runtime-extractor-ktbdm"
Sep 10 19:16:32.862812 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:32.862797    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e06bba47-fb00-41eb-a0d4-34c98888d69a-crio-socket\") pod \"insights-runtime-extractor-ktbdm\" (UID: \"e06bba47-fb00-41eb-a0d4-34c98888d69a\") " pod="openshift-insights/insights-runtime-extractor-ktbdm"
Sep 10 19:16:32.891933 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:32.891906    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:32.963070 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:32.963050    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-spw8k\" (UniqueName: \"kubernetes.io/projected/e06bba47-fb00-41eb-a0d4-34c98888d69a-kube-api-access-spw8k\") pod \"insights-runtime-extractor-ktbdm\" (UID: \"e06bba47-fb00-41eb-a0d4-34c98888d69a\") " pod="openshift-insights/insights-runtime-extractor-ktbdm"
Sep 10 19:16:32.963175 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:32.963091    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e06bba47-fb00-41eb-a0d4-34c98888d69a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ktbdm\" (UID: \"e06bba47-fb00-41eb-a0d4-34c98888d69a\") " pod="openshift-insights/insights-runtime-extractor-ktbdm"
Sep 10 19:16:32.963175 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:32.963122    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e06bba47-fb00-41eb-a0d4-34c98888d69a-data-volume\") pod \"insights-runtime-extractor-ktbdm\" (UID: \"e06bba47-fb00-41eb-a0d4-34c98888d69a\") " pod="openshift-insights/insights-runtime-extractor-ktbdm"
Sep 10 19:16:32.963292 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:16:32.963182    2575 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 19:16:32.963292 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:32.963222    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e06bba47-fb00-41eb-a0d4-34c98888d69a-crio-socket\") pod \"insights-runtime-extractor-ktbdm\" (UID: \"e06bba47-fb00-41eb-a0d4-34c98888d69a\") " pod="openshift-insights/insights-runtime-extractor-ktbdm"
Sep 10 19:16:32.963292 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:16:32.963254    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e06bba47-fb00-41eb-a0d4-34c98888d69a-kube-rbac-proxy-cm podName:e06bba47-fb00-41eb-a0d4-34c98888d69a nodeName:}" failed. No retries permitted until 2026-09-10 19:16:33.463234692 +0000 UTC m=+64.523062351 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e06bba47-fb00-41eb-a0d4-34c98888d69a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ktbdm" (UID: "e06bba47-fb00-41eb-a0d4-34c98888d69a") : configmap "kube-rbac-proxy" not found
Sep 10 19:16:32.963292 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:32.963277    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e06bba47-fb00-41eb-a0d4-34c98888d69a-crio-socket\") pod \"insights-runtime-extractor-ktbdm\" (UID: \"e06bba47-fb00-41eb-a0d4-34c98888d69a\") " pod="openshift-insights/insights-runtime-extractor-ktbdm"
Sep 10 19:16:32.963464 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:32.963358    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e06bba47-fb00-41eb-a0d4-34c98888d69a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ktbdm\" (UID: \"e06bba47-fb00-41eb-a0d4-34c98888d69a\") " pod="openshift-insights/insights-runtime-extractor-ktbdm"
Sep 10 19:16:32.963464 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:32.963379    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e06bba47-fb00-41eb-a0d4-34c98888d69a-data-volume\") pod \"insights-runtime-extractor-ktbdm\" (UID: \"e06bba47-fb00-41eb-a0d4-34c98888d69a\") " pod="openshift-insights/insights-runtime-extractor-ktbdm"
Sep 10 19:16:32.963464 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:16:32.963450    2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 19:16:32.963556 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:16:32.963497    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e06bba47-fb00-41eb-a0d4-34c98888d69a-insights-runtime-extractor-tls podName:e06bba47-fb00-41eb-a0d4-34c98888d69a nodeName:}" failed. No retries permitted until 2026-09-10 19:16:33.463486499 +0000 UTC m=+64.523314144 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e06bba47-fb00-41eb-a0d4-34c98888d69a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ktbdm" (UID: "e06bba47-fb00-41eb-a0d4-34c98888d69a") : secret "insights-runtime-extractor-tls" not found
Sep 10 19:16:32.977649 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:32.977622    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-spw8k\" (UniqueName: \"kubernetes.io/projected/e06bba47-fb00-41eb-a0d4-34c98888d69a-kube-api-access-spw8k\") pod \"insights-runtime-extractor-ktbdm\" (UID: \"e06bba47-fb00-41eb-a0d4-34c98888d69a\") " pod="openshift-insights/insights-runtime-extractor-ktbdm"
Sep 10 19:16:33.249486 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:33.249460    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:33.466871 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:33.466847    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e06bba47-fb00-41eb-a0d4-34c98888d69a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ktbdm\" (UID: \"e06bba47-fb00-41eb-a0d4-34c98888d69a\") " pod="openshift-insights/insights-runtime-extractor-ktbdm"
Sep 10 19:16:33.467214 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:33.466906    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e06bba47-fb00-41eb-a0d4-34c98888d69a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ktbdm\" (UID: \"e06bba47-fb00-41eb-a0d4-34c98888d69a\") " pod="openshift-insights/insights-runtime-extractor-ktbdm"
Sep 10 19:16:33.467214 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:16:33.466978    2575 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 19:16:33.467214 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:16:33.466996    2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 19:16:33.467214 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:16:33.467039    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e06bba47-fb00-41eb-a0d4-34c98888d69a-insights-runtime-extractor-tls podName:e06bba47-fb00-41eb-a0d4-34c98888d69a nodeName:}" failed. No retries permitted until 2026-09-10 19:16:34.467026996 +0000 UTC m=+65.526854642 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e06bba47-fb00-41eb-a0d4-34c98888d69a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ktbdm" (UID: "e06bba47-fb00-41eb-a0d4-34c98888d69a") : secret "insights-runtime-extractor-tls" not found
Sep 10 19:16:33.467214 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:16:33.467053    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e06bba47-fb00-41eb-a0d4-34c98888d69a-kube-rbac-proxy-cm podName:e06bba47-fb00-41eb-a0d4-34c98888d69a nodeName:}" failed. No retries permitted until 2026-09-10 19:16:34.467046755 +0000 UTC m=+65.526874401 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e06bba47-fb00-41eb-a0d4-34c98888d69a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ktbdm" (UID: "e06bba47-fb00-41eb-a0d4-34c98888d69a") : configmap "kube-rbac-proxy" not found
Sep 10 19:16:33.569711 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:33.569658    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:34.034699 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:34.034670    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:34.071416 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:34.071396    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7-registry-tls\") pod \"image-registry-8cdf54dd5-22kjj\" (UID: \"7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7\") " pod="openshift-image-registry/image-registry-8cdf54dd5-22kjj"
Sep 10 19:16:34.071552 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:16:34.071538    2575 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 19:16:34.071618 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:16:34.071555    2575 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-8cdf54dd5-22kjj: secret "image-registry-tls" not found
Sep 10 19:16:34.071618 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:16:34.071614    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7-registry-tls podName:7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7 nodeName:}" failed. No retries permitted until 2026-09-10 19:17:06.071597865 +0000 UTC m=+97.131425528 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7-registry-tls") pod "image-registry-8cdf54dd5-22kjj" (UID: "7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7") : secret "image-registry-tls" not found
Sep 10 19:16:34.091653 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:34.091627    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:34.171809 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:34.171793    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7bcc0525-ded3-4f3b-896e-73c566a68e34-metrics-tls\") pod \"dns-default-rjdmw\" (UID: \"7bcc0525-ded3-4f3b-896e-73c566a68e34\") " pod="openshift-dns/dns-default-rjdmw"
Sep 10 19:16:34.171892 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:34.171856    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bfed5912-60ab-48fc-91c2-10d325fd6428-metrics-certs\") pod \"network-metrics-daemon-wx2q6\" (UID: \"bfed5912-60ab-48fc-91c2-10d325fd6428\") " pod="openshift-multus/network-metrics-daemon-wx2q6"
Sep 10 19:16:34.171892 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:34.171882    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ac571b4d-dbf7-4e62-8d4d-c8c2a527deb3-original-pull-secret\") pod \"global-pull-secret-syncer-24lb9\" (UID: \"ac571b4d-dbf7-4e62-8d4d-c8c2a527deb3\") " pod="kube-system/global-pull-secret-syncer-24lb9"
Sep 10 19:16:34.171975 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:16:34.171933    2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 19:16:34.172059 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:16:34.171993    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7bcc0525-ded3-4f3b-896e-73c566a68e34-metrics-tls podName:7bcc0525-ded3-4f3b-896e-73c566a68e34 nodeName:}" failed. No retries permitted until 2026-09-10 19:17:06.171976645 +0000 UTC m=+97.231804290 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7bcc0525-ded3-4f3b-896e-73c566a68e34-metrics-tls") pod "dns-default-rjdmw" (UID: "7bcc0525-ded3-4f3b-896e-73c566a68e34") : secret "dns-default-metrics-tls" not found
Sep 10 19:16:34.173835 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:34.173819    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 10 19:16:34.173910 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:34.173867    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 10 19:16:34.182563 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:16:34.182543    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 10 19:16:34.182659 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:16:34.182595    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bfed5912-60ab-48fc-91c2-10d325fd6428-metrics-certs podName:bfed5912-60ab-48fc-91c2-10d325fd6428 nodeName:}" failed. No retries permitted until 2026-09-10 19:17:38.182579247 +0000 UTC m=+129.242406893 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bfed5912-60ab-48fc-91c2-10d325fd6428-metrics-certs") pod "network-metrics-daemon-wx2q6" (UID: "bfed5912-60ab-48fc-91c2-10d325fd6428") : secret "metrics-daemon-secret" not found
Sep 10 19:16:34.184598 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:34.184582    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ac571b4d-dbf7-4e62-8d4d-c8c2a527deb3-original-pull-secret\") pod \"global-pull-secret-syncer-24lb9\" (UID: \"ac571b4d-dbf7-4e62-8d4d-c8c2a527deb3\") " pod="kube-system/global-pull-secret-syncer-24lb9"
Sep 10 19:16:34.272706 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:34.272684    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n2p5q\" (UniqueName: \"kubernetes.io/projected/cbd42c54-3649-4d2d-8f94-d2789f7f66c7-kube-api-access-n2p5q\") pod \"network-check-target-9rkjw\" (UID: \"cbd42c54-3649-4d2d-8f94-d2789f7f66c7\") " pod="openshift-network-diagnostics/network-check-target-9rkjw"
Sep 10 19:16:34.274806 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:34.274791    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n2p5q\" (UniqueName: \"kubernetes.io/projected/cbd42c54-3649-4d2d-8f94-d2789f7f66c7-kube-api-access-n2p5q\") pod \"network-check-target-9rkjw\" (UID: \"cbd42c54-3649-4d2d-8f94-d2789f7f66c7\") " pod="openshift-network-diagnostics/network-check-target-9rkjw"
Sep 10 19:16:34.297842 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:34.297794    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-24lb9"
Sep 10 19:16:34.474445 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:34.474420    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e06bba47-fb00-41eb-a0d4-34c98888d69a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ktbdm\" (UID: \"e06bba47-fb00-41eb-a0d4-34c98888d69a\") " pod="openshift-insights/insights-runtime-extractor-ktbdm"
Sep 10 19:16:34.474822 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:34.474509    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e06bba47-fb00-41eb-a0d4-34c98888d69a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ktbdm\" (UID: \"e06bba47-fb00-41eb-a0d4-34c98888d69a\") " pod="openshift-insights/insights-runtime-extractor-ktbdm"
Sep 10 19:16:34.474822 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:16:34.474546    2575 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 19:16:34.474822 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:16:34.474609    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e06bba47-fb00-41eb-a0d4-34c98888d69a-kube-rbac-proxy-cm podName:e06bba47-fb00-41eb-a0d4-34c98888d69a nodeName:}" failed. No retries permitted until 2026-09-10 19:16:36.474594856 +0000 UTC m=+67.534422518 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e06bba47-fb00-41eb-a0d4-34c98888d69a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ktbdm" (UID: "e06bba47-fb00-41eb-a0d4-34c98888d69a") : configmap "kube-rbac-proxy" not found
Sep 10 19:16:34.474822 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:16:34.474638    2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 19:16:34.474822 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:16:34.474698    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e06bba47-fb00-41eb-a0d4-34c98888d69a-insights-runtime-extractor-tls podName:e06bba47-fb00-41eb-a0d4-34c98888d69a nodeName:}" failed. No retries permitted until 2026-09-10 19:16:36.474680286 +0000 UTC m=+67.534507933 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e06bba47-fb00-41eb-a0d4-34c98888d69a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ktbdm" (UID: "e06bba47-fb00-41eb-a0d4-34c98888d69a") : secret "insights-runtime-extractor-tls" not found
Sep 10 19:16:34.492947 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:34.492931    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-tsrfs\""
Sep 10 19:16:34.501495 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:34.501477    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9rkjw"
Sep 10 19:16:34.573256 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:34.573210    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-24lb9"]
Sep 10 19:16:34.576588 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:16:34.576561    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac571b4d_dbf7_4e62_8d4d_c8c2a527deb3.slice/crio-8c21dfd98196277ad6e41a4e41a26a9c02dae0711ccdb2d23d3e042baaddac6d WatchSource:0}: Error finding container 8c21dfd98196277ad6e41a4e41a26a9c02dae0711ccdb2d23d3e042baaddac6d: Status 404 returned error can't find the container with id 8c21dfd98196277ad6e41a4e41a26a9c02dae0711ccdb2d23d3e042baaddac6d
Sep 10 19:16:34.680414 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:34.680390    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-24lb9" event={"ID":"ac571b4d-dbf7-4e62-8d4d-c8c2a527deb3","Type":"ContainerStarted","Data":"8c21dfd98196277ad6e41a4e41a26a9c02dae0711ccdb2d23d3e042baaddac6d"}
Sep 10 19:16:34.772394 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:34.772373    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-9rkjw"]
Sep 10 19:16:34.776082 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:16:34.776059    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcbd42c54_3649_4d2d_8f94_d2789f7f66c7.slice/crio-133dc9f846cbee2de3f2a5ee33ab9c89dd8f0e59056b372996ba056e0aadbe6b WatchSource:0}: Error finding container 133dc9f846cbee2de3f2a5ee33ab9c89dd8f0e59056b372996ba056e0aadbe6b: Status 404 returned error can't find the container with id 133dc9f846cbee2de3f2a5ee33ab9c89dd8f0e59056b372996ba056e0aadbe6b
Sep 10 19:16:34.802528 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:34.802493    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:34.897808 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:34.897791    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-vmvhn_d093d956-9c62-4201-a4ae-60187edb6fbf/dns-node-resolver/0.log"
Sep 10 19:16:34.899158 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:34.899137    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:34.978720 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:34.978698    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b4f4c54c-042a-42ea-8b53-81426d7f8f48-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-rvczm\" (UID: \"b4f4c54c-042a-42ea-8b53-81426d7f8f48\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-rvczm"
Sep 10 19:16:34.978856 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:16:34.978839    2575 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 10 19:16:34.978906 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:16:34.978897    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b4f4c54c-042a-42ea-8b53-81426d7f8f48-samples-operator-tls podName:b4f4c54c-042a-42ea-8b53-81426d7f8f48 nodeName:}" failed. No retries permitted until 2026-09-10 19:16:42.978879412 +0000 UTC m=+74.038707062 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b4f4c54c-042a-42ea-8b53-81426d7f8f48-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-rvczm" (UID: "b4f4c54c-042a-42ea-8b53-81426d7f8f48") : secret "samples-operator-tls" not found
Sep 10 19:16:35.685335 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:35.685299    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-9rkjw" event={"ID":"cbd42c54-3649-4d2d-8f94-d2789f7f66c7","Type":"ContainerStarted","Data":"0485c80efcc9851b761bbd7c7b5d9952465663bb138e1318ce5c19511da29c68"}
Sep 10 19:16:35.685754 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:35.685361    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-9rkjw" event={"ID":"cbd42c54-3649-4d2d-8f94-d2789f7f66c7","Type":"ContainerStarted","Data":"133dc9f846cbee2de3f2a5ee33ab9c89dd8f0e59056b372996ba056e0aadbe6b"}
Sep 10 19:16:35.685754 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:35.685460    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-9rkjw"
Sep 10 19:16:35.699890 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:35.699870    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-nh8r2_446e8624-4aaf-471d-9653-2451db8c6889/node-ca/0.log"
Sep 10 19:16:35.706937 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:35.706897    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-9rkjw" podStartSLOduration=66.706885335 podStartE2EDuration="1m6.706885335s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:15:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:16:35.70574701 +0000 UTC m=+66.765574679" watchObservedRunningTime="2026-09-10 19:16:35.706885335 +0000 UTC m=+66.766713063"
Sep 10 19:16:35.729246 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:35.729221    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:36.301631 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:36.301598    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-gbtv9_4cb26dfc-5d83-4cec-b4f2-52f3ff4c47d6/serve-healthcheck-canary/0.log"
Sep 10 19:16:36.489756 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:36.489721    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e06bba47-fb00-41eb-a0d4-34c98888d69a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ktbdm\" (UID: \"e06bba47-fb00-41eb-a0d4-34c98888d69a\") " pod="openshift-insights/insights-runtime-extractor-ktbdm"
Sep 10 19:16:36.489915 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:36.489779    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e06bba47-fb00-41eb-a0d4-34c98888d69a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ktbdm\" (UID: \"e06bba47-fb00-41eb-a0d4-34c98888d69a\") " pod="openshift-insights/insights-runtime-extractor-ktbdm"
Sep 10 19:16:36.489915 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:16:36.489861    2575 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 19:16:36.489915 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:16:36.489865    2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 19:16:36.489915 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:16:36.489908    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e06bba47-fb00-41eb-a0d4-34c98888d69a-kube-rbac-proxy-cm podName:e06bba47-fb00-41eb-a0d4-34c98888d69a nodeName:}" failed. No retries permitted until 2026-09-10 19:16:40.489896377 +0000 UTC m=+71.549724023 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e06bba47-fb00-41eb-a0d4-34c98888d69a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ktbdm" (UID: "e06bba47-fb00-41eb-a0d4-34c98888d69a") : configmap "kube-rbac-proxy" not found
Sep 10 19:16:36.490065 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:16:36.489920    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e06bba47-fb00-41eb-a0d4-34c98888d69a-insights-runtime-extractor-tls podName:e06bba47-fb00-41eb-a0d4-34c98888d69a nodeName:}" failed. No retries permitted until 2026-09-10 19:16:40.489914705 +0000 UTC m=+71.549742351 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e06bba47-fb00-41eb-a0d4-34c98888d69a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ktbdm" (UID: "e06bba47-fb00-41eb-a0d4-34c98888d69a") : secret "insights-runtime-extractor-tls" not found
Sep 10 19:16:38.477349 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:38.477308    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:39.585918 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:39.585873    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:39.782153 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:39.782126    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:39.883586 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:39.883496    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:40.001729 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:40.001692    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:40.143152 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:40.143073    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:40.286806 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:40.286778    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:40.519251 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:40.519223    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e06bba47-fb00-41eb-a0d4-34c98888d69a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ktbdm\" (UID: \"e06bba47-fb00-41eb-a0d4-34c98888d69a\") " pod="openshift-insights/insights-runtime-extractor-ktbdm"
Sep 10 19:16:40.519412 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:40.519280    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e06bba47-fb00-41eb-a0d4-34c98888d69a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ktbdm\" (UID: \"e06bba47-fb00-41eb-a0d4-34c98888d69a\") " pod="openshift-insights/insights-runtime-extractor-ktbdm"
Sep 10 19:16:40.519412 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:16:40.519376    2575 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 19:16:40.519412 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:16:40.519389    2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 19:16:40.519564 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:16:40.519439    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e06bba47-fb00-41eb-a0d4-34c98888d69a-kube-rbac-proxy-cm podName:e06bba47-fb00-41eb-a0d4-34c98888d69a nodeName:}" failed. No retries permitted until 2026-09-10 19:16:48.519424448 +0000 UTC m=+79.579252107 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e06bba47-fb00-41eb-a0d4-34c98888d69a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ktbdm" (UID: "e06bba47-fb00-41eb-a0d4-34c98888d69a") : configmap "kube-rbac-proxy" not found
Sep 10 19:16:40.519564 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:16:40.519452    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e06bba47-fb00-41eb-a0d4-34c98888d69a-insights-runtime-extractor-tls podName:e06bba47-fb00-41eb-a0d4-34c98888d69a nodeName:}" failed. No retries permitted until 2026-09-10 19:16:48.519447051 +0000 UTC m=+79.579274697 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e06bba47-fb00-41eb-a0d4-34c98888d69a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ktbdm" (UID: "e06bba47-fb00-41eb-a0d4-34c98888d69a") : secret "insights-runtime-extractor-tls" not found
Sep 10 19:16:40.702618 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:40.702585    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-24lb9" event={"ID":"ac571b4d-dbf7-4e62-8d4d-c8c2a527deb3","Type":"ContainerStarted","Data":"745673598db7fd08bf693832e03a0daf24c3eb135cae86dd08d58f3f0e76f342"}
Sep 10 19:16:40.718572 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:40.718523    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-24lb9" podStartSLOduration=66.623740408 podStartE2EDuration="1m11.71850935s" totalImagesPullingTime="5.094768942s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 19:15:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:16:40.717634916 +0000 UTC m=+71.777462588" watchObservedRunningTime="2026-09-10 19:16:40.71850935 +0000 UTC m=+71.778337057"
Sep 10 19:16:41.393874 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:41.393840    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:41.494672 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:41.494641    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:41.858561 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:41.858536    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:41.961034 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:41.961002    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:42.083569 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:42.083540    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:42.282191 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:42.282117    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:43.035509 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:43.035481    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b4f4c54c-042a-42ea-8b53-81426d7f8f48-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-rvczm\" (UID: \"b4f4c54c-042a-42ea-8b53-81426d7f8f48\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-rvczm"
Sep 10 19:16:43.038615 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:43.038593    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b4f4c54c-042a-42ea-8b53-81426d7f8f48-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-rvczm\" (UID: \"b4f4c54c-042a-42ea-8b53-81426d7f8f48\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-rvczm"
Sep 10 19:16:43.107985 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:43.107964    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-c5z7q\""
Sep 10 19:16:43.116606 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:43.116591    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-rvczm"
Sep 10 19:16:43.281163 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:43.281133    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:43.337431 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:43.337369    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:43.384446 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:43.384415    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-rvczm"]
Sep 10 19:16:43.486995 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:43.486965    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:43.681983 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:43.681920    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:43.711637 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:43.711614    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-rvczm" event={"ID":"b4f4c54c-042a-42ea-8b53-81426d7f8f48","Type":"ContainerStarted","Data":"ef6d03fb406aad0fa61047cce6fdcd9aca1782d0db9426036766a67bdad09deb"}
Sep 10 19:16:43.891187 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:43.891162    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:44.080829 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:44.080794    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:44.135541 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:44.135515    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:44.287665 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:44.287630    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:45.208519 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:45.208484    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:45.515664 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:45.515570    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:45.719561 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:45.719534    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-rvczm" event={"ID":"b4f4c54c-042a-42ea-8b53-81426d7f8f48","Type":"ContainerStarted","Data":"07ce557560e641cd0968141e884cf201ecf14aa5b0aeccfad45802e202ddcc8e"}
Sep 10 19:16:45.719721 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:45.719582    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-rvczm" event={"ID":"b4f4c54c-042a-42ea-8b53-81426d7f8f48","Type":"ContainerStarted","Data":"f708825d771c87efd82aef15e978be14a3e1e9bfc5a9f6abc0347d8db74d2db0"}
Sep 10 19:16:45.736503 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:45.736462    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-rvczm" podStartSLOduration=16.930898749 podStartE2EDuration="18.736447704s" totalImagesPullingTime="1.805548955s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:16:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:16:45.735237537 +0000 UTC m=+76.795065205" watchObservedRunningTime="2026-09-10 19:16:45.736447704 +0000 UTC m=+76.796275402"
Sep 10 19:16:45.804679 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:45.804614    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:45.906019 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:45.905989    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:46.007289 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:46.007266    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:46.106787 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:46.106732    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:46.684823 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:46.684779    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:47.090028 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:47.089974    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:47.489588 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:47.489558    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:47.686310 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:47.686263    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:48.097271 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:48.097243    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:48.577628 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:48.577604    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e06bba47-fb00-41eb-a0d4-34c98888d69a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ktbdm\" (UID: \"e06bba47-fb00-41eb-a0d4-34c98888d69a\") " pod="openshift-insights/insights-runtime-extractor-ktbdm"
Sep 10 19:16:48.577766 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:48.577656    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e06bba47-fb00-41eb-a0d4-34c98888d69a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ktbdm\" (UID: \"e06bba47-fb00-41eb-a0d4-34c98888d69a\") " pod="openshift-insights/insights-runtime-extractor-ktbdm"
Sep 10 19:16:48.577766 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:16:48.577761    2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 19:16:48.577838 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:16:48.577826    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e06bba47-fb00-41eb-a0d4-34c98888d69a-insights-runtime-extractor-tls podName:e06bba47-fb00-41eb-a0d4-34c98888d69a nodeName:}" failed. No retries permitted until 2026-09-10 19:17:04.577809381 +0000 UTC m=+95.637637043 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e06bba47-fb00-41eb-a0d4-34c98888d69a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ktbdm" (UID: "e06bba47-fb00-41eb-a0d4-34c98888d69a") : secret "insights-runtime-extractor-tls" not found
Sep 10 19:16:48.578240 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:48.578222    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e06bba47-fb00-41eb-a0d4-34c98888d69a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ktbdm\" (UID: \"e06bba47-fb00-41eb-a0d4-34c98888d69a\") " pod="openshift-insights/insights-runtime-extractor-ktbdm"
Sep 10 19:16:48.889378 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:48.889281    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:49.352198 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:49.352172    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:49.488565 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:49.488532    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:49.696055 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:49.695972    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:49.885492 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:49.885460    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:50.292356 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:50.292300    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:51.084113 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:51.084079    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:51.186262 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:51.186228    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:51.245653 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:51.245620    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:51.388148 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:51.388081    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:51.758553 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:51.758522    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:51.816058 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:51.816035    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:51.965177 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:51.965073    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:52.034688 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:52.034618    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:52.194158 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:52.194136    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:52.772959 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:52.772928    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:52.970010 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:52.969984    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:53.403319 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:53.403286    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:53.503478 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:53.503448    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:54.001396 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:54.001332    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:54.073893 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:54.073865    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:57.526880 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:57.526842    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:58.281947 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:58.281913    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-85744dd87c-6c4nx"]
Sep 10 19:16:58.285198 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:58.285182    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-85744dd87c-6c4nx"
Sep 10 19:16:58.287787 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:58.287763    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Sep 10 19:16:58.287910 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:58.287834    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Sep 10 19:16:58.287910 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:58.287834    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Sep 10 19:16:58.289264 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:58.289250    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Sep 10 19:16:58.310407 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:58.310388    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-85744dd87c-6c4nx"]
Sep 10 19:16:58.333890 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:58.333867    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/ce07a792-7d43-43e8-9301-8ddb129856f9-klusterlet-config\") pod \"klusterlet-addon-workmgr-85744dd87c-6c4nx\" (UID: \"ce07a792-7d43-43e8-9301-8ddb129856f9\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-85744dd87c-6c4nx"
Sep 10 19:16:58.334066 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:58.333908    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ce07a792-7d43-43e8-9301-8ddb129856f9-tmp\") pod \"klusterlet-addon-workmgr-85744dd87c-6c4nx\" (UID: \"ce07a792-7d43-43e8-9301-8ddb129856f9\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-85744dd87c-6c4nx"
Sep 10 19:16:58.334066 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:58.333995    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jmfnj\" (UniqueName: \"kubernetes.io/projected/ce07a792-7d43-43e8-9301-8ddb129856f9-kube-api-access-jmfnj\") pod \"klusterlet-addon-workmgr-85744dd87c-6c4nx\" (UID: \"ce07a792-7d43-43e8-9301-8ddb129856f9\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-85744dd87c-6c4nx"
Sep 10 19:16:58.344435 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:58.344411    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-85d98c998f-dlmbt"]
Sep 10 19:16:58.347148 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:58.347135    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-85d98c998f-dlmbt"
Sep 10 19:16:58.349361 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:58.349329    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 10 19:16:58.349474 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:58.349378    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 10 19:16:58.349614 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:58.349600    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-mcdfz\""
Sep 10 19:16:58.361459 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:58.361435    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-85d98c998f-dlmbt"]
Sep 10 19:16:58.435052 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:58.435021    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-88k82\" (UniqueName: \"kubernetes.io/projected/8b439918-46b0-436f-923e-f765d793a7fd-kube-api-access-88k82\") pod \"downloads-85d98c998f-dlmbt\" (UID: \"8b439918-46b0-436f-923e-f765d793a7fd\") " pod="openshift-console/downloads-85d98c998f-dlmbt"
Sep 10 19:16:58.435206 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:58.435066    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ce07a792-7d43-43e8-9301-8ddb129856f9-tmp\") pod \"klusterlet-addon-workmgr-85744dd87c-6c4nx\" (UID: \"ce07a792-7d43-43e8-9301-8ddb129856f9\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-85744dd87c-6c4nx"
Sep 10 19:16:58.435206 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:58.435189    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jmfnj\" (UniqueName: \"kubernetes.io/projected/ce07a792-7d43-43e8-9301-8ddb129856f9-kube-api-access-jmfnj\") pod \"klusterlet-addon-workmgr-85744dd87c-6c4nx\" (UID: \"ce07a792-7d43-43e8-9301-8ddb129856f9\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-85744dd87c-6c4nx"
Sep 10 19:16:58.435278 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:58.435218    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/ce07a792-7d43-43e8-9301-8ddb129856f9-klusterlet-config\") pod \"klusterlet-addon-workmgr-85744dd87c-6c4nx\" (UID: \"ce07a792-7d43-43e8-9301-8ddb129856f9\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-85744dd87c-6c4nx"
Sep 10 19:16:58.435592 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:58.435571    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ce07a792-7d43-43e8-9301-8ddb129856f9-tmp\") pod \"klusterlet-addon-workmgr-85744dd87c-6c4nx\" (UID: \"ce07a792-7d43-43e8-9301-8ddb129856f9\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-85744dd87c-6c4nx"
Sep 10 19:16:58.437812 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:58.437792    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/ce07a792-7d43-43e8-9301-8ddb129856f9-klusterlet-config\") pod \"klusterlet-addon-workmgr-85744dd87c-6c4nx\" (UID: \"ce07a792-7d43-43e8-9301-8ddb129856f9\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-85744dd87c-6c4nx"
Sep 10 19:16:58.446713 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:58.446694    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jmfnj\" (UniqueName: \"kubernetes.io/projected/ce07a792-7d43-43e8-9301-8ddb129856f9-kube-api-access-jmfnj\") pod \"klusterlet-addon-workmgr-85744dd87c-6c4nx\" (UID: \"ce07a792-7d43-43e8-9301-8ddb129856f9\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-85744dd87c-6c4nx"
Sep 10 19:16:58.535814 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:58.535766    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-88k82\" (UniqueName: \"kubernetes.io/projected/8b439918-46b0-436f-923e-f765d793a7fd-kube-api-access-88k82\") pod \"downloads-85d98c998f-dlmbt\" (UID: \"8b439918-46b0-436f-923e-f765d793a7fd\") " pod="openshift-console/downloads-85d98c998f-dlmbt"
Sep 10 19:16:58.544658 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:58.544640    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-88k82\" (UniqueName: \"kubernetes.io/projected/8b439918-46b0-436f-923e-f765d793a7fd-kube-api-access-88k82\") pod \"downloads-85d98c998f-dlmbt\" (UID: \"8b439918-46b0-436f-923e-f765d793a7fd\") " pod="openshift-console/downloads-85d98c998f-dlmbt"
Sep 10 19:16:58.594452 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:58.594429    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-85744dd87c-6c4nx"
Sep 10 19:16:58.655656 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:58.655596    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-85d98c998f-dlmbt"
Sep 10 19:16:58.729520 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:58.729458    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:58.832066 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:58.831999    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:16:58.863488 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:58.863374    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-85744dd87c-6c4nx"]
Sep 10 19:16:58.865876 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:16:58.865847    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce07a792_7d43_43e8_9301_8ddb129856f9.slice/crio-c5d1686f103cc80f94ae12faf4f6c0f86cc860560d2d886aa9d6464a00be7209 WatchSource:0}: Error finding container c5d1686f103cc80f94ae12faf4f6c0f86cc860560d2d886aa9d6464a00be7209: Status 404 returned error can't find the container with id c5d1686f103cc80f94ae12faf4f6c0f86cc860560d2d886aa9d6464a00be7209
Sep 10 19:16:58.935112 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:58.935087    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-85d98c998f-dlmbt"]
Sep 10 19:16:58.937589 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:16:58.937564    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b439918_46b0_436f_923e_f765d793a7fd.slice/crio-fbeb36aaac97646a97d8047ce554210d94873a44527e2193f47612797cc1b98d WatchSource:0}: Error finding container fbeb36aaac97646a97d8047ce554210d94873a44527e2193f47612797cc1b98d: Status 404 returned error can't find the container with id fbeb36aaac97646a97d8047ce554210d94873a44527e2193f47612797cc1b98d
Sep 10 19:16:59.762074 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:59.762001    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-85744dd87c-6c4nx" event={"ID":"ce07a792-7d43-43e8-9301-8ddb129856f9","Type":"ContainerStarted","Data":"c5d1686f103cc80f94ae12faf4f6c0f86cc860560d2d886aa9d6464a00be7209"}
Sep 10 19:16:59.763939 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:59.763873    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-85d98c998f-dlmbt" event={"ID":"8b439918-46b0-436f-923e-f765d793a7fd","Type":"ContainerStarted","Data":"fbeb36aaac97646a97d8047ce554210d94873a44527e2193f47612797cc1b98d"}
Sep 10 19:16:59.779756 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:16:59.779727    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:17:00.053606 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:00.053508    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:17:00.149640 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:00.149587    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:17:03.780557 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:03.780518    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-85744dd87c-6c4nx" event={"ID":"ce07a792-7d43-43e8-9301-8ddb129856f9","Type":"ContainerStarted","Data":"c3b6f2f5f0510110640b628a89d272821b13279d86eccd96289d6dd708887339"}
Sep 10 19:17:03.780982 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:03.780825    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-85744dd87c-6c4nx"
Sep 10 19:17:03.782583 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:03.782559    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-85744dd87c-6c4nx"
Sep 10 19:17:03.802674 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:03.802635    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-85744dd87c-6c4nx" podStartSLOduration=1.3445736799999999 podStartE2EDuration="5.802622222s" totalImagesPullingTime="4.458048542s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:16:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:17:03.801188762 +0000 UTC m=+94.861016635" watchObservedRunningTime="2026-09-10 19:17:03.802622222 +0000 UTC m=+94.862449889"
Sep 10 19:17:03.809850 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:03.809826    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5bd779b86c-rqxkh"]
Sep 10 19:17:03.811862 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:03.811843    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5bd779b86c-rqxkh"
Sep 10 19:17:03.813812 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:03.813674    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-kgnp2\""
Sep 10 19:17:03.813812 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:03.813716    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 10 19:17:03.813812 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:03.813743    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 10 19:17:03.813992 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:03.813974    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 10 19:17:03.814132 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:03.814109    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 10 19:17:03.814236 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:03.814138    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 10 19:17:03.822179 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:03.822156    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5bd779b86c-rqxkh"]
Sep 10 19:17:03.877832 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:03.877807    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7594800a-6546-4425-8eab-3a1cda053b6e-console-oauth-config\") pod \"console-5bd779b86c-rqxkh\" (UID: \"7594800a-6546-4425-8eab-3a1cda053b6e\") " pod="openshift-console/console-5bd779b86c-rqxkh"
Sep 10 19:17:03.877992 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:03.877856    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7594800a-6546-4425-8eab-3a1cda053b6e-oauth-serving-cert\") pod \"console-5bd779b86c-rqxkh\" (UID: \"7594800a-6546-4425-8eab-3a1cda053b6e\") " pod="openshift-console/console-5bd779b86c-rqxkh"
Sep 10 19:17:03.877992 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:03.877895    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7594800a-6546-4425-8eab-3a1cda053b6e-service-ca\") pod \"console-5bd779b86c-rqxkh\" (UID: \"7594800a-6546-4425-8eab-3a1cda053b6e\") " pod="openshift-console/console-5bd779b86c-rqxkh"
Sep 10 19:17:03.877992 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:03.877980    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7594800a-6546-4425-8eab-3a1cda053b6e-console-config\") pod \"console-5bd779b86c-rqxkh\" (UID: \"7594800a-6546-4425-8eab-3a1cda053b6e\") " pod="openshift-console/console-5bd779b86c-rqxkh"
Sep 10 19:17:03.878142 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:03.878017    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7594800a-6546-4425-8eab-3a1cda053b6e-console-serving-cert\") pod \"console-5bd779b86c-rqxkh\" (UID: \"7594800a-6546-4425-8eab-3a1cda053b6e\") " pod="openshift-console/console-5bd779b86c-rqxkh"
Sep 10 19:17:03.878142 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:03.878036    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mp8p7\" (UniqueName: \"kubernetes.io/projected/7594800a-6546-4425-8eab-3a1cda053b6e-kube-api-access-mp8p7\") pod \"console-5bd779b86c-rqxkh\" (UID: \"7594800a-6546-4425-8eab-3a1cda053b6e\") " pod="openshift-console/console-5bd779b86c-rqxkh"
Sep 10 19:17:03.978920 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:03.978887    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7594800a-6546-4425-8eab-3a1cda053b6e-console-oauth-config\") pod \"console-5bd779b86c-rqxkh\" (UID: \"7594800a-6546-4425-8eab-3a1cda053b6e\") " pod="openshift-console/console-5bd779b86c-rqxkh"
Sep 10 19:17:03.979088 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:03.978935    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7594800a-6546-4425-8eab-3a1cda053b6e-oauth-serving-cert\") pod \"console-5bd779b86c-rqxkh\" (UID: \"7594800a-6546-4425-8eab-3a1cda053b6e\") " pod="openshift-console/console-5bd779b86c-rqxkh"
Sep 10 19:17:03.979088 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:03.978979    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7594800a-6546-4425-8eab-3a1cda053b6e-service-ca\") pod \"console-5bd779b86c-rqxkh\" (UID: \"7594800a-6546-4425-8eab-3a1cda053b6e\") " pod="openshift-console/console-5bd779b86c-rqxkh"
Sep 10 19:17:03.979088 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:03.979021    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7594800a-6546-4425-8eab-3a1cda053b6e-console-config\") pod \"console-5bd779b86c-rqxkh\" (UID: \"7594800a-6546-4425-8eab-3a1cda053b6e\") " pod="openshift-console/console-5bd779b86c-rqxkh"
Sep 10 19:17:03.979088 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:03.979046    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7594800a-6546-4425-8eab-3a1cda053b6e-console-serving-cert\") pod \"console-5bd779b86c-rqxkh\" (UID: \"7594800a-6546-4425-8eab-3a1cda053b6e\") " pod="openshift-console/console-5bd779b86c-rqxkh"
Sep 10 19:17:03.979088 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:03.979069    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mp8p7\" (UniqueName: \"kubernetes.io/projected/7594800a-6546-4425-8eab-3a1cda053b6e-kube-api-access-mp8p7\") pod \"console-5bd779b86c-rqxkh\" (UID: \"7594800a-6546-4425-8eab-3a1cda053b6e\") " pod="openshift-console/console-5bd779b86c-rqxkh"
Sep 10 19:17:03.979850 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:03.979815    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7594800a-6546-4425-8eab-3a1cda053b6e-service-ca\") pod \"console-5bd779b86c-rqxkh\" (UID: \"7594800a-6546-4425-8eab-3a1cda053b6e\") " pod="openshift-console/console-5bd779b86c-rqxkh"
Sep 10 19:17:03.979850 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:03.979842    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7594800a-6546-4425-8eab-3a1cda053b6e-oauth-serving-cert\") pod \"console-5bd779b86c-rqxkh\" (UID: \"7594800a-6546-4425-8eab-3a1cda053b6e\") " pod="openshift-console/console-5bd779b86c-rqxkh"
Sep 10 19:17:03.980032 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:03.979842    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7594800a-6546-4425-8eab-3a1cda053b6e-console-config\") pod \"console-5bd779b86c-rqxkh\" (UID: \"7594800a-6546-4425-8eab-3a1cda053b6e\") " pod="openshift-console/console-5bd779b86c-rqxkh"
Sep 10 19:17:03.981678 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:03.981660    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7594800a-6546-4425-8eab-3a1cda053b6e-console-oauth-config\") pod \"console-5bd779b86c-rqxkh\" (UID: \"7594800a-6546-4425-8eab-3a1cda053b6e\") " pod="openshift-console/console-5bd779b86c-rqxkh"
Sep 10 19:17:03.981780 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:03.981675    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7594800a-6546-4425-8eab-3a1cda053b6e-console-serving-cert\") pod \"console-5bd779b86c-rqxkh\" (UID: \"7594800a-6546-4425-8eab-3a1cda053b6e\") " pod="openshift-console/console-5bd779b86c-rqxkh"
Sep 10 19:17:03.987291 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:03.987269    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mp8p7\" (UniqueName: \"kubernetes.io/projected/7594800a-6546-4425-8eab-3a1cda053b6e-kube-api-access-mp8p7\") pod \"console-5bd779b86c-rqxkh\" (UID: \"7594800a-6546-4425-8eab-3a1cda053b6e\") " pod="openshift-console/console-5bd779b86c-rqxkh"
Sep 10 19:17:04.121982 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:04.121914    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5bd779b86c-rqxkh"
Sep 10 19:17:04.398978 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:04.398908    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5bd779b86c-rqxkh"]
Sep 10 19:17:04.402603 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:17:04.402577    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7594800a_6546_4425_8eab_3a1cda053b6e.slice/crio-c54a88f2e01741404583617b88323c041d7e184c3889519b739737595094f60b WatchSource:0}: Error finding container c54a88f2e01741404583617b88323c041d7e184c3889519b739737595094f60b: Status 404 returned error can't find the container with id c54a88f2e01741404583617b88323c041d7e184c3889519b739737595094f60b
Sep 10 19:17:04.584026 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:04.583988    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e06bba47-fb00-41eb-a0d4-34c98888d69a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ktbdm\" (UID: \"e06bba47-fb00-41eb-a0d4-34c98888d69a\") " pod="openshift-insights/insights-runtime-extractor-ktbdm"
Sep 10 19:17:04.586587 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:04.586557    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e06bba47-fb00-41eb-a0d4-34c98888d69a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ktbdm\" (UID: \"e06bba47-fb00-41eb-a0d4-34c98888d69a\") " pod="openshift-insights/insights-runtime-extractor-ktbdm"
Sep 10 19:17:04.785510 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:04.785474    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5bd779b86c-rqxkh" event={"ID":"7594800a-6546-4425-8eab-3a1cda053b6e","Type":"ContainerStarted","Data":"c54a88f2e01741404583617b88323c041d7e184c3889519b739737595094f60b"}
Sep 10 19:17:04.851913 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:04.851884    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-ktbdm"
Sep 10 19:17:05.126768 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:05.126642    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-ktbdm"]
Sep 10 19:17:05.130262 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:17:05.130227    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode06bba47_fb00_41eb_a0d4_34c98888d69a.slice/crio-a15d2920ba7991f6037bb6f7e470e8e178644221a5b121a40509e73b1882fc9a WatchSource:0}: Error finding container a15d2920ba7991f6037bb6f7e470e8e178644221a5b121a40509e73b1882fc9a: Status 404 returned error can't find the container with id a15d2920ba7991f6037bb6f7e470e8e178644221a5b121a40509e73b1882fc9a
Sep 10 19:17:05.791837 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:05.791803    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ktbdm" event={"ID":"e06bba47-fb00-41eb-a0d4-34c98888d69a","Type":"ContainerStarted","Data":"6defbc9906d22b8b614edefcf026b1f74c3fd748a0b581209925cc6b4e8171cb"}
Sep 10 19:17:05.792265 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:05.791868    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ktbdm" event={"ID":"e06bba47-fb00-41eb-a0d4-34c98888d69a","Type":"ContainerStarted","Data":"a15d2920ba7991f6037bb6f7e470e8e178644221a5b121a40509e73b1882fc9a"}
Sep 10 19:17:06.096991 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:06.096873    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7-registry-tls\") pod \"image-registry-8cdf54dd5-22kjj\" (UID: \"7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7\") " pod="openshift-image-registry/image-registry-8cdf54dd5-22kjj"
Sep 10 19:17:06.099681 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:06.099632    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7-registry-tls\") pod \"image-registry-8cdf54dd5-22kjj\" (UID: \"7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7\") " pod="openshift-image-registry/image-registry-8cdf54dd5-22kjj"
Sep 10 19:17:06.187933 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:06.187900    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-n7kcr\""
Sep 10 19:17:06.194887 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:06.194854    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-8cdf54dd5-22kjj"
Sep 10 19:17:06.197388 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:06.197360    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7bcc0525-ded3-4f3b-896e-73c566a68e34-metrics-tls\") pod \"dns-default-rjdmw\" (UID: \"7bcc0525-ded3-4f3b-896e-73c566a68e34\") " pod="openshift-dns/dns-default-rjdmw"
Sep 10 19:17:06.200328 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:06.200309    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7bcc0525-ded3-4f3b-896e-73c566a68e34-metrics-tls\") pod \"dns-default-rjdmw\" (UID: \"7bcc0525-ded3-4f3b-896e-73c566a68e34\") " pod="openshift-dns/dns-default-rjdmw"
Sep 10 19:17:06.481935 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:06.481897    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-8cdf54dd5-22kjj"]
Sep 10 19:17:06.497565 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:06.497533    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-wks57\""
Sep 10 19:17:06.506304 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:06.506275    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-rjdmw"
Sep 10 19:17:06.575698 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:06.575650    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:17:06.677505 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:06.677469    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:17:06.691256 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:06.691231    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-9rkjw"
Sep 10 19:17:06.758221 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:06.758126    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-298zr"]
Sep 10 19:17:06.761285 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:06.761187    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-298zr"
Sep 10 19:17:06.762055 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:06.761971    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:17:06.763286 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:06.763103    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-6dlpb\""
Sep 10 19:17:06.763798 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:06.763776    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Sep 10 19:17:06.774371 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:06.774135    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-298zr"]
Sep 10 19:17:06.798519 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:06.798445    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ktbdm" event={"ID":"e06bba47-fb00-41eb-a0d4-34c98888d69a","Type":"ContainerStarted","Data":"b29ec202979c1dc3c9b335ca2983388e2ce3dcf82842498164267e632677fdab"}
Sep 10 19:17:06.803259 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:06.803235    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/e2b13e6d-fc2c-47a5-a584-1ed70c44b8e7-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-298zr\" (UID: \"e2b13e6d-fc2c-47a5-a584-1ed70c44b8e7\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-298zr"
Sep 10 19:17:06.904314 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:06.904251    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/e2b13e6d-fc2c-47a5-a584-1ed70c44b8e7-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-298zr\" (UID: \"e2b13e6d-fc2c-47a5-a584-1ed70c44b8e7\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-298zr"
Sep 10 19:17:06.907537 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:06.907512    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/e2b13e6d-fc2c-47a5-a584-1ed70c44b8e7-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-298zr\" (UID: \"e2b13e6d-fc2c-47a5-a584-1ed70c44b8e7\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-298zr"
Sep 10 19:17:06.949857 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:06.949801    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:17:07.062241 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:07.062156    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:17:07.075007 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:07.074979    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-298zr"
Sep 10 19:17:07.127102 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:07.127073    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:17:07.483304 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:17:07.483264    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7224f0a5_f7de_4d50_bdec_7ebd9a0b5bf7.slice/crio-b725465d0316bab0360780b45a9491058039b5570774d06bb6997894c4f5f451 WatchSource:0}: Error finding container b725465d0316bab0360780b45a9491058039b5570774d06bb6997894c4f5f451: Status 404 returned error can't find the container with id b725465d0316bab0360780b45a9491058039b5570774d06bb6997894c4f5f451
Sep 10 19:17:07.767122 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:07.766990    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-rjdmw"]
Sep 10 19:17:07.776400 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:07.775939    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-298zr"]
Sep 10 19:17:07.779014 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:17:07.778986    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2b13e6d_fc2c_47a5_a584_1ed70c44b8e7.slice/crio-2b0a03c042d3c3a48b4d4d0baee61208c5c70316ef6bfe226a4018f43567f663 WatchSource:0}: Error finding container 2b0a03c042d3c3a48b4d4d0baee61208c5c70316ef6bfe226a4018f43567f663: Status 404 returned error can't find the container with id 2b0a03c042d3c3a48b4d4d0baee61208c5c70316ef6bfe226a4018f43567f663
Sep 10 19:17:07.804410 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:07.804379    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-298zr" event={"ID":"e2b13e6d-fc2c-47a5-a584-1ed70c44b8e7","Type":"ContainerStarted","Data":"2b0a03c042d3c3a48b4d4d0baee61208c5c70316ef6bfe226a4018f43567f663"}
Sep 10 19:17:07.806444 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:07.806417    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-8cdf54dd5-22kjj" event={"ID":"7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7","Type":"ContainerStarted","Data":"e6a780c4b09f82ce1534920759aefbf2fd7abae4a70d1998dd4073d5ee41f5be"}
Sep 10 19:17:07.806529 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:07.806469    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-8cdf54dd5-22kjj"
Sep 10 19:17:07.806529 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:07.806491    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-8cdf54dd5-22kjj" event={"ID":"7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7","Type":"ContainerStarted","Data":"b725465d0316bab0360780b45a9491058039b5570774d06bb6997894c4f5f451"}
Sep 10 19:17:07.808730 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:07.808704    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5bd779b86c-rqxkh" event={"ID":"7594800a-6546-4425-8eab-3a1cda053b6e","Type":"ContainerStarted","Data":"9b9f47e1705c2bbc6588130cc938f7c9223fe38752162f672475931ba81ad1a8"}
Sep 10 19:17:07.810142 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:07.810106    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-rjdmw" event={"ID":"7bcc0525-ded3-4f3b-896e-73c566a68e34","Type":"ContainerStarted","Data":"836690b06997776ad14ea0627d2e3d635aeae07a63a1ed34f176a4741bae1559"}
Sep 10 19:17:07.828282 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:07.828243    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-8cdf54dd5-22kjj" podStartSLOduration=98.8282236 podStartE2EDuration="1m38.8282236s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:15:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:17:07.827173087 +0000 UTC m=+98.887000757" watchObservedRunningTime="2026-09-10 19:17:07.8282236 +0000 UTC m=+98.888051269"
Sep 10 19:17:07.846410 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:07.846368    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5bd779b86c-rqxkh" podStartSLOduration=1.7052123049999999 podStartE2EDuration="4.846357369s" totalImagesPullingTime="3.141145064s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:17:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:17:07.845236713 +0000 UTC m=+98.905064382" watchObservedRunningTime="2026-09-10 19:17:07.846357369 +0000 UTC m=+98.906185034"
Sep 10 19:17:08.819852 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:08.819649    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:17:08.819852 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:08.819831    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ktbdm" event={"ID":"e06bba47-fb00-41eb-a0d4-34c98888d69a","Type":"ContainerStarted","Data":"f160d30ad568e09595de844b43557fd506ae7f739bcf150de7152e28bc0e51e9"}
Sep 10 19:17:08.923649 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:08.923515    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:17:10.816141 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:10.816104    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:17:10.875046 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:10.875014    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:17:14.122373 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:14.122321    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5bd779b86c-rqxkh"
Sep 10 19:17:14.122927 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:14.122411    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5bd779b86c-rqxkh"
Sep 10 19:17:14.128009 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:14.127982    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5bd779b86c-rqxkh"
Sep 10 19:17:14.144518 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:14.144465    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-ktbdm" podStartSLOduration=38.837585175 podStartE2EDuration="42.144447007s" totalImagesPullingTime="3.306861832s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 19:16:32 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:17:08.838932425 +0000 UTC m=+99.898760095" watchObservedRunningTime="2026-09-10 19:17:14.144447007 +0000 UTC m=+105.204274675"
Sep 10 19:17:14.846802 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:14.846768    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5bd779b86c-rqxkh"
Sep 10 19:17:15.273778 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:15.273732    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:17:15.336956 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:15.336923    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:17:15.602128 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:15.602004    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:17:15.665637 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:15.665601    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:17:16.803056 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:16.803019    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:17:17.786356 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:17.786315    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5bd779b86c-rqxkh"]
Sep 10 19:17:17.855846 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:17.855805    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-rjdmw" event={"ID":"7bcc0525-ded3-4f3b-896e-73c566a68e34","Type":"ContainerStarted","Data":"de47a5b693f5f1182bfde65a6fcb7e807bb0f1799f82b15087e19cb611221cf9"}
Sep 10 19:17:17.856301 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:17.855910    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-rjdmw" event={"ID":"7bcc0525-ded3-4f3b-896e-73c566a68e34","Type":"ContainerStarted","Data":"66b98e86be2070a17815dfdec3dda52d7580ae67f63ebdcaf20533cb6b8e2378"}
Sep 10 19:17:17.856301 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:17.855947    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-rjdmw"
Sep 10 19:17:17.857812 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:17.857776    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-298zr" event={"ID":"e2b13e6d-fc2c-47a5-a584-1ed70c44b8e7","Type":"ContainerStarted","Data":"67ee333b7b00529794b2840a77f634ba4c9a4c75d795ba51ae5d0a62d31622bd"}
Sep 10 19:17:17.857972 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:17.857951    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-298zr"
Sep 10 19:17:17.860117 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:17.860087    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-85d98c998f-dlmbt" event={"ID":"8b439918-46b0-436f-923e-f765d793a7fd","Type":"ContainerStarted","Data":"c8c476070f721ef79705afa455027183c4381097008aa726b5cfffa6aa59f71c"}
Sep 10 19:17:17.860335 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:17.860312    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-85d98c998f-dlmbt"
Sep 10 19:17:17.863999 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:17.863980    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-298zr"
Sep 10 19:17:17.872931 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:17.872881    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-rjdmw" podStartSLOduration=66.407943842 podStartE2EDuration="1m15.872863761s" totalImagesPullingTime="9.464919919s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:16:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:17:17.871828327 +0000 UTC m=+108.931655994" watchObservedRunningTime="2026-09-10 19:17:17.872863761 +0000 UTC m=+108.932691431"
Sep 10 19:17:17.875439 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:17.875419    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-85d98c998f-dlmbt"
Sep 10 19:17:17.889747 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:17.889697    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-85d98c998f-dlmbt" podStartSLOduration=1.535998342 podStartE2EDuration="19.889680743s" totalImagesPullingTime="18.353682401s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:16:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:17:17.887772073 +0000 UTC m=+108.947599742" watchObservedRunningTime="2026-09-10 19:17:17.889680743 +0000 UTC m=+108.949508413"
Sep 10 19:17:17.903951 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:17.903910    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-298zr" podStartSLOduration=2.443754036 podStartE2EDuration="11.903890586s" totalImagesPullingTime="9.46013655s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:17:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:17:17.903161259 +0000 UTC m=+108.962988929" watchObservedRunningTime="2026-09-10 19:17:17.903890586 +0000 UTC m=+108.963718259"
Sep 10 19:17:18.039309 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:18.039222    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:17:18.142217 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:18.142180    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:17:18.802857 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:18.802815    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:17:18.815615 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:18.815582    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-ztmr6"]
Sep 10 19:17:18.845462 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:18.845425    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-ztmr6"]
Sep 10 19:17:18.845631 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:18.845560    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-6f4946db56-ztmr6"
Sep 10 19:17:18.847719 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:18.847617    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Sep 10 19:17:18.847853 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:18.847768    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 10 19:17:18.847853 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:18.847778    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 10 19:17:18.848401 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:18.848377    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-2zhnm\""
Sep 10 19:17:18.848534 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:18.848417    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 10 19:17:18.848534 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:18.848508    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Sep 10 19:17:18.873562 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:18.873503    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:17:18.902136 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:18.902103    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4fc69fb7-6f7c-4239-b7ef-7b8379f01e30-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-ztmr6\" (UID: \"4fc69fb7-6f7c-4239-b7ef-7b8379f01e30\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-ztmr6"
Sep 10 19:17:18.902274 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:18.902147    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/4fc69fb7-6f7c-4239-b7ef-7b8379f01e30-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-ztmr6\" (UID: \"4fc69fb7-6f7c-4239-b7ef-7b8379f01e30\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-ztmr6"
Sep 10 19:17:18.902353 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:18.902281    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4fc69fb7-6f7c-4239-b7ef-7b8379f01e30-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-ztmr6\" (UID: \"4fc69fb7-6f7c-4239-b7ef-7b8379f01e30\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-ztmr6"
Sep 10 19:17:18.902353 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:18.902315    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2r75d\" (UniqueName: \"kubernetes.io/projected/4fc69fb7-6f7c-4239-b7ef-7b8379f01e30-kube-api-access-2r75d\") pod \"prometheus-operator-6f4946db56-ztmr6\" (UID: \"4fc69fb7-6f7c-4239-b7ef-7b8379f01e30\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-ztmr6"
Sep 10 19:17:19.003665 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:19.003623    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/4fc69fb7-6f7c-4239-b7ef-7b8379f01e30-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-ztmr6\" (UID: \"4fc69fb7-6f7c-4239-b7ef-7b8379f01e30\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-ztmr6"
Sep 10 19:17:19.003850 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:19.003740    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4fc69fb7-6f7c-4239-b7ef-7b8379f01e30-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-ztmr6\" (UID: \"4fc69fb7-6f7c-4239-b7ef-7b8379f01e30\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-ztmr6"
Sep 10 19:17:19.003850 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:19.003763    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2r75d\" (UniqueName: \"kubernetes.io/projected/4fc69fb7-6f7c-4239-b7ef-7b8379f01e30-kube-api-access-2r75d\") pod \"prometheus-operator-6f4946db56-ztmr6\" (UID: \"4fc69fb7-6f7c-4239-b7ef-7b8379f01e30\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-ztmr6"
Sep 10 19:17:19.003850 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:17:19.003802    2575 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found
Sep 10 19:17:19.004039 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:19.003853    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4fc69fb7-6f7c-4239-b7ef-7b8379f01e30-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-ztmr6\" (UID: \"4fc69fb7-6f7c-4239-b7ef-7b8379f01e30\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-ztmr6"
Sep 10 19:17:19.004039 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:17:19.003893    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4fc69fb7-6f7c-4239-b7ef-7b8379f01e30-prometheus-operator-tls podName:4fc69fb7-6f7c-4239-b7ef-7b8379f01e30 nodeName:}" failed. No retries permitted until 2026-09-10 19:17:19.503872041 +0000 UTC m=+110.563699694 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/4fc69fb7-6f7c-4239-b7ef-7b8379f01e30-prometheus-operator-tls") pod "prometheus-operator-6f4946db56-ztmr6" (UID: "4fc69fb7-6f7c-4239-b7ef-7b8379f01e30") : secret "prometheus-operator-tls" not found
Sep 10 19:17:19.004815 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:19.004789    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4fc69fb7-6f7c-4239-b7ef-7b8379f01e30-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-ztmr6\" (UID: \"4fc69fb7-6f7c-4239-b7ef-7b8379f01e30\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-ztmr6"
Sep 10 19:17:19.006807 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:19.006784    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4fc69fb7-6f7c-4239-b7ef-7b8379f01e30-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-ztmr6\" (UID: \"4fc69fb7-6f7c-4239-b7ef-7b8379f01e30\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-ztmr6"
Sep 10 19:17:19.012292 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:19.012267    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2r75d\" (UniqueName: \"kubernetes.io/projected/4fc69fb7-6f7c-4239-b7ef-7b8379f01e30-kube-api-access-2r75d\") pod \"prometheus-operator-6f4946db56-ztmr6\" (UID: \"4fc69fb7-6f7c-4239-b7ef-7b8379f01e30\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-ztmr6"
Sep 10 19:17:19.093660 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:19.093581    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:17:19.149946 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:19.149882    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:17:19.450059 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:19.450024    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:17:19.506976 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:19.506948    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/4fc69fb7-6f7c-4239-b7ef-7b8379f01e30-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-ztmr6\" (UID: \"4fc69fb7-6f7c-4239-b7ef-7b8379f01e30\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-ztmr6"
Sep 10 19:17:19.509821 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:19.509793    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/4fc69fb7-6f7c-4239-b7ef-7b8379f01e30-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-ztmr6\" (UID: \"4fc69fb7-6f7c-4239-b7ef-7b8379f01e30\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-ztmr6"
Sep 10 19:17:19.756762 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:19.756684    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-6f4946db56-ztmr6"
Sep 10 19:17:20.059486 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:20.059416    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-ztmr6"]
Sep 10 19:17:20.062886 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:17:20.062854    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4fc69fb7_6f7c_4239_b7ef_7b8379f01e30.slice/crio-ae2e174eafb735d000bbbdf4ced4ac8a8b41589e9600d3cf79246e5343df0f57 WatchSource:0}: Error finding container ae2e174eafb735d000bbbdf4ced4ac8a8b41589e9600d3cf79246e5343df0f57: Status 404 returned error can't find the container with id ae2e174eafb735d000bbbdf4ced4ac8a8b41589e9600d3cf79246e5343df0f57
Sep 10 19:17:20.725583 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:20.725515    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-8cdf54dd5-22kjj"]
Sep 10 19:17:20.818870 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:20.818821    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:17:20.879740 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:20.879704    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-ztmr6" event={"ID":"4fc69fb7-6f7c-4239-b7ef-7b8379f01e30","Type":"ContainerStarted","Data":"ae2e174eafb735d000bbbdf4ced4ac8a8b41589e9600d3cf79246e5343df0f57"}
Sep 10 19:17:20.879913 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:20.879769    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:17:20.940267 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:20.940143    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:17:22.240353 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:22.240300    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:17:22.890075 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:22.890031    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-ztmr6" event={"ID":"4fc69fb7-6f7c-4239-b7ef-7b8379f01e30","Type":"ContainerStarted","Data":"fc1327face899605884d25eb540e084dee303a266aa447b5419b1691344402ac"}
Sep 10 19:17:22.890265 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:22.890105    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-ztmr6" event={"ID":"4fc69fb7-6f7c-4239-b7ef-7b8379f01e30","Type":"ContainerStarted","Data":"d0d623d6788911061a2eb9f81adfa55ab1f796c4c2aaea704fa96a226fb6110d"}
Sep 10 19:17:22.906881 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:22.906827    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-6f4946db56-ztmr6" podStartSLOduration=3.16333119 podStartE2EDuration="4.906808346s" totalImagesPullingTime="1.743477156s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:17:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:17:22.905823566 +0000 UTC m=+113.965651237" watchObservedRunningTime="2026-09-10 19:17:22.906808346 +0000 UTC m=+113.966636014"
Sep 10 19:17:23.342502 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:23.342460    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:17:23.415727 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:23.415692    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:17:24.942852 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:24.942815    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:17:25.170027 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:25.169992    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:17:25.187835 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:25.187806    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-lxjbh"]
Sep 10 19:17:25.213432 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:25.213400    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-lxjbh"]
Sep 10 19:17:25.213565 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:25.213518    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-lxjbh"
Sep 10 19:17:25.215677 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:25.215652    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-nvrb9\""
Sep 10 19:17:25.216098 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:25.216076    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Sep 10 19:17:25.216295 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:25.216273    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Sep 10 19:17:25.225312 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:25.225293    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-6tcld"]
Sep 10 19:17:25.237368 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:25.237332    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-6tcld"
Sep 10 19:17:25.239273 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:25.239251    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-gfqql\""
Sep 10 19:17:25.239273 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:25.239262    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 10 19:17:25.239469 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:25.239380    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 10 19:17:25.239469 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:25.239263    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 10 19:17:25.270980 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:25.270951    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:17:25.343348 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:25.343306    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:17:25.355302 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:25.355277    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d970bc06-018c-41e4-86c6-f41f489a21c6-node-exporter-wtmp\") pod \"node-exporter-6tcld\" (UID: \"d970bc06-018c-41e4-86c6-f41f489a21c6\") " pod="openshift-monitoring/node-exporter-6tcld"
Sep 10 19:17:25.355447 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:25.355315    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d970bc06-018c-41e4-86c6-f41f489a21c6-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-6tcld\" (UID: \"d970bc06-018c-41e4-86c6-f41f489a21c6\") " pod="openshift-monitoring/node-exporter-6tcld"
Sep 10 19:17:25.355447 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:25.355373    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/152c0c8b-18bd-4977-9e5d-05d4a494cff9-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-lxjbh\" (UID: \"152c0c8b-18bd-4977-9e5d-05d4a494cff9\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-lxjbh"
Sep 10 19:17:25.355571 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:25.355467    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2c52s\" (UniqueName: \"kubernetes.io/projected/d970bc06-018c-41e4-86c6-f41f489a21c6-kube-api-access-2c52s\") pod \"node-exporter-6tcld\" (UID: \"d970bc06-018c-41e4-86c6-f41f489a21c6\") " pod="openshift-monitoring/node-exporter-6tcld"
Sep 10 19:17:25.355571 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:25.355512    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/152c0c8b-18bd-4977-9e5d-05d4a494cff9-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-lxjbh\" (UID: \"152c0c8b-18bd-4977-9e5d-05d4a494cff9\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-lxjbh"
Sep 10 19:17:25.355571 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:25.355539    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d970bc06-018c-41e4-86c6-f41f489a21c6-sys\") pod \"node-exporter-6tcld\" (UID: \"d970bc06-018c-41e4-86c6-f41f489a21c6\") " pod="openshift-monitoring/node-exporter-6tcld"
Sep 10 19:17:25.355571 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:25.355564    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d970bc06-018c-41e4-86c6-f41f489a21c6-node-exporter-accelerators-collector-config\") pod \"node-exporter-6tcld\" (UID: \"d970bc06-018c-41e4-86c6-f41f489a21c6\") " pod="openshift-monitoring/node-exporter-6tcld"
Sep 10 19:17:25.355769 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:25.355591    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d970bc06-018c-41e4-86c6-f41f489a21c6-node-exporter-tls\") pod \"node-exporter-6tcld\" (UID: \"d970bc06-018c-41e4-86c6-f41f489a21c6\") " pod="openshift-monitoring/node-exporter-6tcld"
Sep 10 19:17:25.355769 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:25.355618    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gtx88\" (UniqueName: \"kubernetes.io/projected/152c0c8b-18bd-4977-9e5d-05d4a494cff9-kube-api-access-gtx88\") pod \"openshift-state-metrics-657fd6c54-lxjbh\" (UID: \"152c0c8b-18bd-4977-9e5d-05d4a494cff9\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-lxjbh"
Sep 10 19:17:25.355769 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:25.355649    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/152c0c8b-18bd-4977-9e5d-05d4a494cff9-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-lxjbh\" (UID: \"152c0c8b-18bd-4977-9e5d-05d4a494cff9\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-lxjbh"
Sep 10 19:17:25.355769 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:25.355695    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d970bc06-018c-41e4-86c6-f41f489a21c6-node-exporter-textfile\") pod \"node-exporter-6tcld\" (UID: \"d970bc06-018c-41e4-86c6-f41f489a21c6\") " pod="openshift-monitoring/node-exporter-6tcld"
Sep 10 19:17:25.355769 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:25.355724    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d970bc06-018c-41e4-86c6-f41f489a21c6-metrics-client-ca\") pod \"node-exporter-6tcld\" (UID: \"d970bc06-018c-41e4-86c6-f41f489a21c6\") " pod="openshift-monitoring/node-exporter-6tcld"
Sep 10 19:17:25.355769 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:25.355749    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d970bc06-018c-41e4-86c6-f41f489a21c6-root\") pod \"node-exporter-6tcld\" (UID: \"d970bc06-018c-41e4-86c6-f41f489a21c6\") " pod="openshift-monitoring/node-exporter-6tcld"
Sep 10 19:17:25.445881 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:25.445843    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:17:25.456479 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:25.456450    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2c52s\" (UniqueName: \"kubernetes.io/projected/d970bc06-018c-41e4-86c6-f41f489a21c6-kube-api-access-2c52s\") pod \"node-exporter-6tcld\" (UID: \"d970bc06-018c-41e4-86c6-f41f489a21c6\") " pod="openshift-monitoring/node-exporter-6tcld"
Sep 10 19:17:25.456604 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:25.456485    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/152c0c8b-18bd-4977-9e5d-05d4a494cff9-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-lxjbh\" (UID: \"152c0c8b-18bd-4977-9e5d-05d4a494cff9\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-lxjbh"
Sep 10 19:17:25.456604 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:25.456510    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d970bc06-018c-41e4-86c6-f41f489a21c6-sys\") pod \"node-exporter-6tcld\" (UID: \"d970bc06-018c-41e4-86c6-f41f489a21c6\") " pod="openshift-monitoring/node-exporter-6tcld"
Sep 10 19:17:25.456604 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:25.456535    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d970bc06-018c-41e4-86c6-f41f489a21c6-node-exporter-accelerators-collector-config\") pod \"node-exporter-6tcld\" (UID: \"d970bc06-018c-41e4-86c6-f41f489a21c6\") " pod="openshift-monitoring/node-exporter-6tcld"
Sep 10 19:17:25.456604 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:25.456563    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d970bc06-018c-41e4-86c6-f41f489a21c6-node-exporter-tls\") pod \"node-exporter-6tcld\" (UID: \"d970bc06-018c-41e4-86c6-f41f489a21c6\") " pod="openshift-monitoring/node-exporter-6tcld"
Sep 10 19:17:25.456604 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:25.456589    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gtx88\" (UniqueName: \"kubernetes.io/projected/152c0c8b-18bd-4977-9e5d-05d4a494cff9-kube-api-access-gtx88\") pod \"openshift-state-metrics-657fd6c54-lxjbh\" (UID: \"152c0c8b-18bd-4977-9e5d-05d4a494cff9\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-lxjbh"
Sep 10 19:17:25.456848 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:25.456619    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d970bc06-018c-41e4-86c6-f41f489a21c6-sys\") pod \"node-exporter-6tcld\" (UID: \"d970bc06-018c-41e4-86c6-f41f489a21c6\") " pod="openshift-monitoring/node-exporter-6tcld"
Sep 10 19:17:25.456848 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:25.456623    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/152c0c8b-18bd-4977-9e5d-05d4a494cff9-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-lxjbh\" (UID: \"152c0c8b-18bd-4977-9e5d-05d4a494cff9\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-lxjbh"
Sep 10 19:17:25.456848 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:25.456691    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d970bc06-018c-41e4-86c6-f41f489a21c6-node-exporter-textfile\") pod \"node-exporter-6tcld\" (UID: \"d970bc06-018c-41e4-86c6-f41f489a21c6\") " pod="openshift-monitoring/node-exporter-6tcld"
Sep 10 19:17:25.456848 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:25.456729    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d970bc06-018c-41e4-86c6-f41f489a21c6-metrics-client-ca\") pod \"node-exporter-6tcld\" (UID: \"d970bc06-018c-41e4-86c6-f41f489a21c6\") " pod="openshift-monitoring/node-exporter-6tcld"
Sep 10 19:17:25.456848 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:25.456768    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d970bc06-018c-41e4-86c6-f41f489a21c6-root\") pod \"node-exporter-6tcld\" (UID: \"d970bc06-018c-41e4-86c6-f41f489a21c6\") " pod="openshift-monitoring/node-exporter-6tcld"
Sep 10 19:17:25.456848 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:25.456807    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d970bc06-018c-41e4-86c6-f41f489a21c6-node-exporter-wtmp\") pod \"node-exporter-6tcld\" (UID: \"d970bc06-018c-41e4-86c6-f41f489a21c6\") " pod="openshift-monitoring/node-exporter-6tcld"
Sep 10 19:17:25.456848 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:25.456840    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d970bc06-018c-41e4-86c6-f41f489a21c6-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-6tcld\" (UID: \"d970bc06-018c-41e4-86c6-f41f489a21c6\") " pod="openshift-monitoring/node-exporter-6tcld"
Sep 10 19:17:25.457182 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:25.456877    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/152c0c8b-18bd-4977-9e5d-05d4a494cff9-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-lxjbh\" (UID: \"152c0c8b-18bd-4977-9e5d-05d4a494cff9\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-lxjbh"
Sep 10 19:17:25.457182 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:25.457025    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d970bc06-018c-41e4-86c6-f41f489a21c6-node-exporter-wtmp\") pod \"node-exporter-6tcld\" (UID: \"d970bc06-018c-41e4-86c6-f41f489a21c6\") " pod="openshift-monitoring/node-exporter-6tcld"
Sep 10 19:17:25.457182 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:25.457093    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d970bc06-018c-41e4-86c6-f41f489a21c6-root\") pod \"node-exporter-6tcld\" (UID: \"d970bc06-018c-41e4-86c6-f41f489a21c6\") " pod="openshift-monitoring/node-exporter-6tcld"
Sep 10 19:17:25.457330 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:25.457190    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d970bc06-018c-41e4-86c6-f41f489a21c6-node-exporter-accelerators-collector-config\") pod \"node-exporter-6tcld\" (UID: \"d970bc06-018c-41e4-86c6-f41f489a21c6\") " pod="openshift-monitoring/node-exporter-6tcld"
Sep 10 19:17:25.457669 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:25.457644    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/152c0c8b-18bd-4977-9e5d-05d4a494cff9-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-lxjbh\" (UID: \"152c0c8b-18bd-4977-9e5d-05d4a494cff9\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-lxjbh"
Sep 10 19:17:25.457855 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:25.457714    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d970bc06-018c-41e4-86c6-f41f489a21c6-node-exporter-textfile\") pod \"node-exporter-6tcld\" (UID: \"d970bc06-018c-41e4-86c6-f41f489a21c6\") " pod="openshift-monitoring/node-exporter-6tcld"
Sep 10 19:17:25.458321 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:25.458254    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d970bc06-018c-41e4-86c6-f41f489a21c6-metrics-client-ca\") pod \"node-exporter-6tcld\" (UID: \"d970bc06-018c-41e4-86c6-f41f489a21c6\") " pod="openshift-monitoring/node-exporter-6tcld"
Sep 10 19:17:25.459485 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:25.459462    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d970bc06-018c-41e4-86c6-f41f489a21c6-node-exporter-tls\") pod \"node-exporter-6tcld\" (UID: \"d970bc06-018c-41e4-86c6-f41f489a21c6\") " pod="openshift-monitoring/node-exporter-6tcld"
Sep 10 19:17:25.459724 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:25.459701    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/152c0c8b-18bd-4977-9e5d-05d4a494cff9-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-lxjbh\" (UID: \"152c0c8b-18bd-4977-9e5d-05d4a494cff9\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-lxjbh"
Sep 10 19:17:25.459990 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:25.459973    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/152c0c8b-18bd-4977-9e5d-05d4a494cff9-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-lxjbh\" (UID: \"152c0c8b-18bd-4977-9e5d-05d4a494cff9\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-lxjbh"
Sep 10 19:17:25.460355 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:25.460317    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d970bc06-018c-41e4-86c6-f41f489a21c6-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-6tcld\" (UID: \"d970bc06-018c-41e4-86c6-f41f489a21c6\") " pod="openshift-monitoring/node-exporter-6tcld"
Sep 10 19:17:25.464997 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:25.464951    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2c52s\" (UniqueName: \"kubernetes.io/projected/d970bc06-018c-41e4-86c6-f41f489a21c6-kube-api-access-2c52s\") pod \"node-exporter-6tcld\" (UID: \"d970bc06-018c-41e4-86c6-f41f489a21c6\") " pod="openshift-monitoring/node-exporter-6tcld"
Sep 10 19:17:25.465287 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:25.465263    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gtx88\" (UniqueName: \"kubernetes.io/projected/152c0c8b-18bd-4977-9e5d-05d4a494cff9-kube-api-access-gtx88\") pod \"openshift-state-metrics-657fd6c54-lxjbh\" (UID: \"152c0c8b-18bd-4977-9e5d-05d4a494cff9\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-lxjbh"
Sep 10 19:17:25.524264 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:25.524239    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-lxjbh"
Sep 10 19:17:25.547087 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:25.547060    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-6tcld"
Sep 10 19:17:25.548119 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:25.548038    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:17:25.584333 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:17:25.584300    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd970bc06_018c_41e4_86c6_f41f489a21c6.slice/crio-e7ccc363aa7f344047ae26995a76b9332ec5e2ac6dc28bba9d8551cd4fc2ad0f WatchSource:0}: Error finding container e7ccc363aa7f344047ae26995a76b9332ec5e2ac6dc28bba9d8551cd4fc2ad0f: Status 404 returned error can't find the container with id e7ccc363aa7f344047ae26995a76b9332ec5e2ac6dc28bba9d8551cd4fc2ad0f
Sep 10 19:17:25.644781 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:25.644733    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:17:25.748060 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:25.747977    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:17:25.832045 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:25.832015    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-lxjbh"]
Sep 10 19:17:25.835331 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:17:25.835307    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod152c0c8b_18bd_4977_9e5d_05d4a494cff9.slice/crio-ec4ce5b9660bb0be2411542bbf46c214294cd1c41bcf1bb52990520e627d1f55 WatchSource:0}: Error finding container ec4ce5b9660bb0be2411542bbf46c214294cd1c41bcf1bb52990520e627d1f55: Status 404 returned error can't find the container with id ec4ce5b9660bb0be2411542bbf46c214294cd1c41bcf1bb52990520e627d1f55
Sep 10 19:17:25.871820 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:25.871788    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:17:25.904076 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:25.904045    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6tcld" event={"ID":"d970bc06-018c-41e4-86c6-f41f489a21c6","Type":"ContainerStarted","Data":"e7ccc363aa7f344047ae26995a76b9332ec5e2ac6dc28bba9d8551cd4fc2ad0f"}
Sep 10 19:17:25.905649 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:25.905614    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-lxjbh" event={"ID":"152c0c8b-18bd-4977-9e5d-05d4a494cff9","Type":"ContainerStarted","Data":"ec4ce5b9660bb0be2411542bbf46c214294cd1c41bcf1bb52990520e627d1f55"}
Sep 10 19:17:25.972334 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:25.972287    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:17:26.179703 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:26.179669    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:17:26.274675 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:26.274599    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:17:26.287123 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:26.287094    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 10 19:17:26.297447 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:26.297416    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 10 19:17:26.299715 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:26.299678    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Sep 10 19:17:26.299715 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:26.299692    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Sep 10 19:17:26.300063 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:26.299743    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Sep 10 19:17:26.300063 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:26.299696    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Sep 10 19:17:26.300063 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:26.299747    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Sep 10 19:17:26.300063 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:26.300042    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Sep 10 19:17:26.300291 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:26.300110    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Sep 10 19:17:26.300291 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:26.300112    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Sep 10 19:17:26.300291 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:26.300115    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Sep 10 19:17:26.300291 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:26.300244    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-mqmc7\""
Sep 10 19:17:26.309158 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:26.309133    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 10 19:17:26.365163 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:26.365131    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/93a950f6-8e9b-481f-9c1b-0eeb7a373c8d-config-volume\") pod \"alertmanager-main-0\" (UID: \"93a950f6-8e9b-481f-9c1b-0eeb7a373c8d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 19:17:26.365361 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:26.365169    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/93a950f6-8e9b-481f-9c1b-0eeb7a373c8d-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"93a950f6-8e9b-481f-9c1b-0eeb7a373c8d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 19:17:26.365361 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:26.365187    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/93a950f6-8e9b-481f-9c1b-0eeb7a373c8d-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"93a950f6-8e9b-481f-9c1b-0eeb7a373c8d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 19:17:26.365361 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:26.365255    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/93a950f6-8e9b-481f-9c1b-0eeb7a373c8d-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"93a950f6-8e9b-481f-9c1b-0eeb7a373c8d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 19:17:26.365361 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:26.365277    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/93a950f6-8e9b-481f-9c1b-0eeb7a373c8d-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"93a950f6-8e9b-481f-9c1b-0eeb7a373c8d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 19:17:26.365361 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:26.365313    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/93a950f6-8e9b-481f-9c1b-0eeb7a373c8d-tls-assets\") pod \"alertmanager-main-0\" (UID: \"93a950f6-8e9b-481f-9c1b-0eeb7a373c8d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 19:17:26.365639 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:26.365448    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-whmg7\" (UniqueName: \"kubernetes.io/projected/93a950f6-8e9b-481f-9c1b-0eeb7a373c8d-kube-api-access-whmg7\") pod \"alertmanager-main-0\" (UID: \"93a950f6-8e9b-481f-9c1b-0eeb7a373c8d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 19:17:26.365639 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:26.365489    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/93a950f6-8e9b-481f-9c1b-0eeb7a373c8d-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"93a950f6-8e9b-481f-9c1b-0eeb7a373c8d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 19:17:26.365639 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:26.365528    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/93a950f6-8e9b-481f-9c1b-0eeb7a373c8d-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"93a950f6-8e9b-481f-9c1b-0eeb7a373c8d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 19:17:26.365639 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:26.365561    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/93a950f6-8e9b-481f-9c1b-0eeb7a373c8d-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"93a950f6-8e9b-481f-9c1b-0eeb7a373c8d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 19:17:26.365639 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:26.365597    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/93a950f6-8e9b-481f-9c1b-0eeb7a373c8d-web-config\") pod \"alertmanager-main-0\" (UID: \"93a950f6-8e9b-481f-9c1b-0eeb7a373c8d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 19:17:26.365639 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:26.365626    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/93a950f6-8e9b-481f-9c1b-0eeb7a373c8d-config-out\") pod \"alertmanager-main-0\" (UID: \"93a950f6-8e9b-481f-9c1b-0eeb7a373c8d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 19:17:26.365892 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:26.365654    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/93a950f6-8e9b-481f-9c1b-0eeb7a373c8d-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"93a950f6-8e9b-481f-9c1b-0eeb7a373c8d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 19:17:26.467087 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:26.467038    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/93a950f6-8e9b-481f-9c1b-0eeb7a373c8d-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"93a950f6-8e9b-481f-9c1b-0eeb7a373c8d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 19:17:26.467087 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:26.467079    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/93a950f6-8e9b-481f-9c1b-0eeb7a373c8d-web-config\") pod \"alertmanager-main-0\" (UID: \"93a950f6-8e9b-481f-9c1b-0eeb7a373c8d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 19:17:26.467414 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:26.467107    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/93a950f6-8e9b-481f-9c1b-0eeb7a373c8d-config-out\") pod \"alertmanager-main-0\" (UID: \"93a950f6-8e9b-481f-9c1b-0eeb7a373c8d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 19:17:26.467414 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:26.467131    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/93a950f6-8e9b-481f-9c1b-0eeb7a373c8d-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"93a950f6-8e9b-481f-9c1b-0eeb7a373c8d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 19:17:26.467414 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:26.467164    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/93a950f6-8e9b-481f-9c1b-0eeb7a373c8d-config-volume\") pod \"alertmanager-main-0\" (UID: \"93a950f6-8e9b-481f-9c1b-0eeb7a373c8d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 19:17:26.467414 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:26.467188    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/93a950f6-8e9b-481f-9c1b-0eeb7a373c8d-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"93a950f6-8e9b-481f-9c1b-0eeb7a373c8d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 19:17:26.467414 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:26.467210    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/93a950f6-8e9b-481f-9c1b-0eeb7a373c8d-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"93a950f6-8e9b-481f-9c1b-0eeb7a373c8d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 19:17:26.467414 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:26.467265    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/93a950f6-8e9b-481f-9c1b-0eeb7a373c8d-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"93a950f6-8e9b-481f-9c1b-0eeb7a373c8d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 19:17:26.467414 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:26.467288    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/93a950f6-8e9b-481f-9c1b-0eeb7a373c8d-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"93a950f6-8e9b-481f-9c1b-0eeb7a373c8d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 19:17:26.467827 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:26.467806    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/93a950f6-8e9b-481f-9c1b-0eeb7a373c8d-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"93a950f6-8e9b-481f-9c1b-0eeb7a373c8d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 19:17:26.468183 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:17:26.468164    2575 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found
Sep 10 19:17:26.468280 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:17:26.468233    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/93a950f6-8e9b-481f-9c1b-0eeb7a373c8d-secret-alertmanager-main-tls podName:93a950f6-8e9b-481f-9c1b-0eeb7a373c8d nodeName:}" failed. No retries permitted until 2026-09-10 19:17:26.968212197 +0000 UTC m=+118.028039861 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/93a950f6-8e9b-481f-9c1b-0eeb7a373c8d-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "93a950f6-8e9b-481f-9c1b-0eeb7a373c8d") : secret "alertmanager-main-tls" not found
Sep 10 19:17:26.468497 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:26.468476    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/93a950f6-8e9b-481f-9c1b-0eeb7a373c8d-tls-assets\") pod \"alertmanager-main-0\" (UID: \"93a950f6-8e9b-481f-9c1b-0eeb7a373c8d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 19:17:26.468576 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:26.468536    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-whmg7\" (UniqueName: \"kubernetes.io/projected/93a950f6-8e9b-481f-9c1b-0eeb7a373c8d-kube-api-access-whmg7\") pod \"alertmanager-main-0\" (UID: \"93a950f6-8e9b-481f-9c1b-0eeb7a373c8d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 19:17:26.468671 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:26.468571    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/93a950f6-8e9b-481f-9c1b-0eeb7a373c8d-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"93a950f6-8e9b-481f-9c1b-0eeb7a373c8d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 19:17:26.468671 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:26.468571    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/93a950f6-8e9b-481f-9c1b-0eeb7a373c8d-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"93a950f6-8e9b-481f-9c1b-0eeb7a373c8d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 19:17:26.468671 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:26.468578    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/93a950f6-8e9b-481f-9c1b-0eeb7a373c8d-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"93a950f6-8e9b-481f-9c1b-0eeb7a373c8d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 19:17:26.468671 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:26.468659    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/93a950f6-8e9b-481f-9c1b-0eeb7a373c8d-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"93a950f6-8e9b-481f-9c1b-0eeb7a373c8d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 19:17:26.470522 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:26.470171    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/93a950f6-8e9b-481f-9c1b-0eeb7a373c8d-config-out\") pod \"alertmanager-main-0\" (UID: \"93a950f6-8e9b-481f-9c1b-0eeb7a373c8d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 19:17:26.470522 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:26.470481    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/93a950f6-8e9b-481f-9c1b-0eeb7a373c8d-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"93a950f6-8e9b-481f-9c1b-0eeb7a373c8d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 19:17:26.471426 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:26.471403    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/93a950f6-8e9b-481f-9c1b-0eeb7a373c8d-tls-assets\") pod \"alertmanager-main-0\" (UID: \"93a950f6-8e9b-481f-9c1b-0eeb7a373c8d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 19:17:26.471519 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:26.471481    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/93a950f6-8e9b-481f-9c1b-0eeb7a373c8d-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"93a950f6-8e9b-481f-9c1b-0eeb7a373c8d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 19:17:26.472124 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:26.472104    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/93a950f6-8e9b-481f-9c1b-0eeb7a373c8d-config-volume\") pod \"alertmanager-main-0\" (UID: \"93a950f6-8e9b-481f-9c1b-0eeb7a373c8d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 19:17:26.472533 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:26.472486    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/93a950f6-8e9b-481f-9c1b-0eeb7a373c8d-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"93a950f6-8e9b-481f-9c1b-0eeb7a373c8d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 19:17:26.472949 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:26.472916    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/93a950f6-8e9b-481f-9c1b-0eeb7a373c8d-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"93a950f6-8e9b-481f-9c1b-0eeb7a373c8d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 19:17:26.473331 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:26.473289    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/93a950f6-8e9b-481f-9c1b-0eeb7a373c8d-web-config\") pod \"alertmanager-main-0\" (UID: \"93a950f6-8e9b-481f-9c1b-0eeb7a373c8d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 19:17:26.479522 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:26.479498    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-whmg7\" (UniqueName: \"kubernetes.io/projected/93a950f6-8e9b-481f-9c1b-0eeb7a373c8d-kube-api-access-whmg7\") pod \"alertmanager-main-0\" (UID: \"93a950f6-8e9b-481f-9c1b-0eeb7a373c8d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 19:17:26.549883 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:26.549799    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:17:26.912432 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:26.912336    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-lxjbh" event={"ID":"152c0c8b-18bd-4977-9e5d-05d4a494cff9","Type":"ContainerStarted","Data":"320eaaa2066cfe0eda4477c8bdec264e6ce0aa49b920fa0dc33be516db043ead"}
Sep 10 19:17:26.912432 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:26.912425    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-lxjbh" event={"ID":"152c0c8b-18bd-4977-9e5d-05d4a494cff9","Type":"ContainerStarted","Data":"bffea2f0de6a091de1bdcdfe0699bb5ef67fc997600e9bcf8818334855d3c7e0"}
Sep 10 19:17:26.975460 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:26.975425    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/93a950f6-8e9b-481f-9c1b-0eeb7a373c8d-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"93a950f6-8e9b-481f-9c1b-0eeb7a373c8d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 19:17:26.977764 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:26.977732    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/93a950f6-8e9b-481f-9c1b-0eeb7a373c8d-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"93a950f6-8e9b-481f-9c1b-0eeb7a373c8d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 19:17:27.207217 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:27.207192    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 10 19:17:27.867041 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:27.866977    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-rjdmw"
Sep 10 19:17:27.875224 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:27.875204    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 10 19:17:27.918187 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:27.918155    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6tcld" event={"ID":"d970bc06-018c-41e4-86c6-f41f489a21c6","Type":"ContainerStarted","Data":"99b48dddccf706e099c67d67b0f22e72e7cb64bfb2be0be03831e5b741736eef"}
Sep 10 19:17:27.971447 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:17:27.971414    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93a950f6_8e9b_481f_9c1b_0eeb7a373c8d.slice/crio-189635f404594667faa75e95202aa75d5a06ff9e25a760628b461800b3dd5b70 WatchSource:0}: Error finding container 189635f404594667faa75e95202aa75d5a06ff9e25a760628b461800b3dd5b70: Status 404 returned error can't find the container with id 189635f404594667faa75e95202aa75d5a06ff9e25a760628b461800b3dd5b70
Sep 10 19:17:28.163758 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:28.163661    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:17:28.264951 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:28.264911    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:17:28.390162 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:28.390122    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:17:28.492511 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:28.492470    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:17:28.616707 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:28.616670    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:17:28.676812 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:28.676764    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:17:28.923257 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:28.923187    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"93a950f6-8e9b-481f-9c1b-0eeb7a373c8d","Type":"ContainerStarted","Data":"189635f404594667faa75e95202aa75d5a06ff9e25a760628b461800b3dd5b70"}
Sep 10 19:17:28.925414 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:28.925383    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-lxjbh" event={"ID":"152c0c8b-18bd-4977-9e5d-05d4a494cff9","Type":"ContainerStarted","Data":"ae98e6da3d2c8ddca7dda101f0ea8ada9d865ab66de192719f6f117cc69fdcc3"}
Sep 10 19:17:28.926812 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:28.926789    2575 generic.go:356] "Generic (PLEG): container finished" podID="d970bc06-018c-41e4-86c6-f41f489a21c6" containerID="99b48dddccf706e099c67d67b0f22e72e7cb64bfb2be0be03831e5b741736eef" exitCode=0
Sep 10 19:17:28.926884 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:28.926832    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6tcld" event={"ID":"d970bc06-018c-41e4-86c6-f41f489a21c6","Type":"ContainerDied","Data":"99b48dddccf706e099c67d67b0f22e72e7cb64bfb2be0be03831e5b741736eef"}
Sep 10 19:17:28.943958 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:28.943663    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-lxjbh" podStartSLOduration=2.007195522 podStartE2EDuration="3.943645152s" totalImagesPullingTime="1.93644963s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:17:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:17:28.942506307 +0000 UTC m=+120.002333975" watchObservedRunningTime="2026-09-10 19:17:28.943645152 +0000 UTC m=+120.003472821"
Sep 10 19:17:29.932516 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:29.932451    2575 generic.go:356] "Generic (PLEG): container finished" podID="93a950f6-8e9b-481f-9c1b-0eeb7a373c8d" containerID="beffef2b23f047bc23d8e1b304f4980b7da6e8f20355b9e21d55c78e308d7f6d" exitCode=0
Sep 10 19:17:29.932865 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:29.932541    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"93a950f6-8e9b-481f-9c1b-0eeb7a373c8d","Type":"ContainerDied","Data":"beffef2b23f047bc23d8e1b304f4980b7da6e8f20355b9e21d55c78e308d7f6d"}
Sep 10 19:17:29.935384 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:29.935335    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6tcld" event={"ID":"d970bc06-018c-41e4-86c6-f41f489a21c6","Type":"ContainerStarted","Data":"fe82b36322a42c5e4dfd8b4cbc489dc25c8084fa1b21adec115f1255b600d28f"}
Sep 10 19:17:29.935503 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:29.935418    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6tcld" event={"ID":"d970bc06-018c-41e4-86c6-f41f489a21c6","Type":"ContainerStarted","Data":"73991edaaa1e1a73b7240b503f1cd902028cfa4d183de2604ad75bd0d588c0d1"}
Sep 10 19:17:29.989391 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:29.989334    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-6tcld" podStartSLOduration=2.897265882 podStartE2EDuration="4.989320911s" totalImagesPullingTime="2.017649011s" totalInitContainerRuntime="74.406018ms" isStatefulPod=true podCreationTimestamp="2026-09-10 19:17:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:17:29.98850595 +0000 UTC m=+121.048333628" watchObservedRunningTime="2026-09-10 19:17:29.989320911 +0000 UTC m=+121.049148579"
Sep 10 19:17:29.989564 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:29.989542    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:17:30.062456 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:30.062433    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:17:30.164249 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:30.164219    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:17:30.221754 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:30.221724    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:17:30.313227 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:30.313204    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:17:30.733209 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:30.733178    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-8cdf54dd5-22kjj"
Sep 10 19:17:31.127017 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:31.126921    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:17:31.202896 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:31.202784    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:17:31.462677 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:31.462635    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:17:31.468802 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:31.468563    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 19:17:31.493646 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:31.493615    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:17:31.497860 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:31.497026    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Sep 10 19:17:31.497860 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:31.497264    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 10 19:17:31.497860 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:31.497592    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 10 19:17:31.497860 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:31.497821    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 10 19:17:31.498146 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:31.498004    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 10 19:17:31.498800 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:31.498436    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-7qkp5\""
Sep 10 19:17:31.498800 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:31.498457    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 10 19:17:31.498800 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:31.498642    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 10 19:17:31.499020 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:31.498995    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 10 19:17:31.499225 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:31.499205    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 10 19:17:31.499305 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:31.499264    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 10 19:17:31.499509 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:31.499491    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 10 19:17:31.499585 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:31.499518    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 10 19:17:31.499643 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:31.499603    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 10 19:17:31.499739 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:31.499721    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-18kbaadhbm0r1\""
Sep 10 19:17:31.501055 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:31.501033    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 10 19:17:31.508860 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:31.508840    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 19:17:31.509370 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:31.509293    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 10 19:17:31.615578 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:31.615537    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/64faa9bc-8055-4b10-9a86-046d6507ae4a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"64faa9bc-8055-4b10-9a86-046d6507ae4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:17:31.615578 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:31.615577    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/64faa9bc-8055-4b10-9a86-046d6507ae4a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"64faa9bc-8055-4b10-9a86-046d6507ae4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:17:31.615812 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:31.615606    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/64faa9bc-8055-4b10-9a86-046d6507ae4a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"64faa9bc-8055-4b10-9a86-046d6507ae4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:17:31.615812 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:31.615627    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/64faa9bc-8055-4b10-9a86-046d6507ae4a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"64faa9bc-8055-4b10-9a86-046d6507ae4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:17:31.615812 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:31.615651    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/64faa9bc-8055-4b10-9a86-046d6507ae4a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"64faa9bc-8055-4b10-9a86-046d6507ae4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:17:31.615812 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:31.615693    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/64faa9bc-8055-4b10-9a86-046d6507ae4a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"64faa9bc-8055-4b10-9a86-046d6507ae4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:17:31.615812 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:31.615735    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/64faa9bc-8055-4b10-9a86-046d6507ae4a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"64faa9bc-8055-4b10-9a86-046d6507ae4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:17:31.615812 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:31.615763    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/64faa9bc-8055-4b10-9a86-046d6507ae4a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"64faa9bc-8055-4b10-9a86-046d6507ae4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:17:31.616096 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:31.615861    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/64faa9bc-8055-4b10-9a86-046d6507ae4a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"64faa9bc-8055-4b10-9a86-046d6507ae4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:17:31.616096 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:31.615896    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/64faa9bc-8055-4b10-9a86-046d6507ae4a-web-config\") pod \"prometheus-k8s-0\" (UID: \"64faa9bc-8055-4b10-9a86-046d6507ae4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:17:31.616096 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:31.615922    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/64faa9bc-8055-4b10-9a86-046d6507ae4a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"64faa9bc-8055-4b10-9a86-046d6507ae4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:17:31.616096 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:31.615946    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/64faa9bc-8055-4b10-9a86-046d6507ae4a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"64faa9bc-8055-4b10-9a86-046d6507ae4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:17:31.616096 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:31.615979    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/64faa9bc-8055-4b10-9a86-046d6507ae4a-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"64faa9bc-8055-4b10-9a86-046d6507ae4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:17:31.616096 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:31.616053    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x6ttf\" (UniqueName: \"kubernetes.io/projected/64faa9bc-8055-4b10-9a86-046d6507ae4a-kube-api-access-x6ttf\") pod \"prometheus-k8s-0\" (UID: \"64faa9bc-8055-4b10-9a86-046d6507ae4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:17:31.616096 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:31.616094    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/64faa9bc-8055-4b10-9a86-046d6507ae4a-config-out\") pod \"prometheus-k8s-0\" (UID: \"64faa9bc-8055-4b10-9a86-046d6507ae4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:17:31.616436 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:31.616115    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/64faa9bc-8055-4b10-9a86-046d6507ae4a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"64faa9bc-8055-4b10-9a86-046d6507ae4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:17:31.616436 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:31.616136    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/64faa9bc-8055-4b10-9a86-046d6507ae4a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"64faa9bc-8055-4b10-9a86-046d6507ae4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:17:31.616436 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:31.616178    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/64faa9bc-8055-4b10-9a86-046d6507ae4a-config\") pod \"prometheus-k8s-0\" (UID: \"64faa9bc-8055-4b10-9a86-046d6507ae4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:17:31.616436 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:31.616201    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/64faa9bc-8055-4b10-9a86-046d6507ae4a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"64faa9bc-8055-4b10-9a86-046d6507ae4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:17:31.616436 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:31.616224    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/64faa9bc-8055-4b10-9a86-046d6507ae4a-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"64faa9bc-8055-4b10-9a86-046d6507ae4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:17:31.717593 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:31.717520    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/64faa9bc-8055-4b10-9a86-046d6507ae4a-config-out\") pod \"prometheus-k8s-0\" (UID: \"64faa9bc-8055-4b10-9a86-046d6507ae4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:17:31.717593 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:31.717556    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/64faa9bc-8055-4b10-9a86-046d6507ae4a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"64faa9bc-8055-4b10-9a86-046d6507ae4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:17:31.717593 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:31.717573    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/64faa9bc-8055-4b10-9a86-046d6507ae4a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"64faa9bc-8055-4b10-9a86-046d6507ae4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:17:31.717865 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:31.717598    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/64faa9bc-8055-4b10-9a86-046d6507ae4a-config\") pod \"prometheus-k8s-0\" (UID: \"64faa9bc-8055-4b10-9a86-046d6507ae4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:17:31.717865 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:31.717614    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/64faa9bc-8055-4b10-9a86-046d6507ae4a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"64faa9bc-8055-4b10-9a86-046d6507ae4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:17:31.717865 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:31.717680    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/64faa9bc-8055-4b10-9a86-046d6507ae4a-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"64faa9bc-8055-4b10-9a86-046d6507ae4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:17:31.717865 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:31.717727    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/64faa9bc-8055-4b10-9a86-046d6507ae4a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"64faa9bc-8055-4b10-9a86-046d6507ae4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:17:31.717865 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:31.717750    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/64faa9bc-8055-4b10-9a86-046d6507ae4a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"64faa9bc-8055-4b10-9a86-046d6507ae4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:17:31.717865 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:31.717779    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/64faa9bc-8055-4b10-9a86-046d6507ae4a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"64faa9bc-8055-4b10-9a86-046d6507ae4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:17:31.718177 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:31.718151    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/64faa9bc-8055-4b10-9a86-046d6507ae4a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"64faa9bc-8055-4b10-9a86-046d6507ae4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:17:31.718242 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:31.718197    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/64faa9bc-8055-4b10-9a86-046d6507ae4a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"64faa9bc-8055-4b10-9a86-046d6507ae4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:17:31.718242 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:31.718237    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/64faa9bc-8055-4b10-9a86-046d6507ae4a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"64faa9bc-8055-4b10-9a86-046d6507ae4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:17:31.718372 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:31.718267    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/64faa9bc-8055-4b10-9a86-046d6507ae4a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"64faa9bc-8055-4b10-9a86-046d6507ae4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:17:31.718372 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:31.718291    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/64faa9bc-8055-4b10-9a86-046d6507ae4a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"64faa9bc-8055-4b10-9a86-046d6507ae4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:17:31.718372 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:31.718332    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/64faa9bc-8055-4b10-9a86-046d6507ae4a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"64faa9bc-8055-4b10-9a86-046d6507ae4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:17:31.718525 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:31.718389    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/64faa9bc-8055-4b10-9a86-046d6507ae4a-web-config\") pod \"prometheus-k8s-0\" (UID: \"64faa9bc-8055-4b10-9a86-046d6507ae4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:17:31.718525 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:31.718407    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/64faa9bc-8055-4b10-9a86-046d6507ae4a-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"64faa9bc-8055-4b10-9a86-046d6507ae4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:17:31.718890 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:31.718702    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/64faa9bc-8055-4b10-9a86-046d6507ae4a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"64faa9bc-8055-4b10-9a86-046d6507ae4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:17:31.719013 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:31.718415    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/64faa9bc-8055-4b10-9a86-046d6507ae4a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"64faa9bc-8055-4b10-9a86-046d6507ae4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:17:31.719013 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:31.718976    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/64faa9bc-8055-4b10-9a86-046d6507ae4a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"64faa9bc-8055-4b10-9a86-046d6507ae4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:17:31.719124 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:31.719029    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/64faa9bc-8055-4b10-9a86-046d6507ae4a-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"64faa9bc-8055-4b10-9a86-046d6507ae4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:17:31.719124 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:31.719091    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x6ttf\" (UniqueName: \"kubernetes.io/projected/64faa9bc-8055-4b10-9a86-046d6507ae4a-kube-api-access-x6ttf\") pod \"prometheus-k8s-0\" (UID: \"64faa9bc-8055-4b10-9a86-046d6507ae4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:17:31.720385 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:31.719995    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/64faa9bc-8055-4b10-9a86-046d6507ae4a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"64faa9bc-8055-4b10-9a86-046d6507ae4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:17:31.720385 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:31.720236    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/64faa9bc-8055-4b10-9a86-046d6507ae4a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"64faa9bc-8055-4b10-9a86-046d6507ae4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:17:31.721228 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:31.721202    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/64faa9bc-8055-4b10-9a86-046d6507ae4a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"64faa9bc-8055-4b10-9a86-046d6507ae4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:17:31.723202 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:31.723181    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/64faa9bc-8055-4b10-9a86-046d6507ae4a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"64faa9bc-8055-4b10-9a86-046d6507ae4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:17:31.724327 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:31.724304    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/64faa9bc-8055-4b10-9a86-046d6507ae4a-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"64faa9bc-8055-4b10-9a86-046d6507ae4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:17:31.724599 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:31.724559    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/64faa9bc-8055-4b10-9a86-046d6507ae4a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"64faa9bc-8055-4b10-9a86-046d6507ae4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:17:31.725123 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:31.725079    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/64faa9bc-8055-4b10-9a86-046d6507ae4a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"64faa9bc-8055-4b10-9a86-046d6507ae4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:17:31.725123 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:31.725106    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/64faa9bc-8055-4b10-9a86-046d6507ae4a-config\") pod \"prometheus-k8s-0\" (UID: \"64faa9bc-8055-4b10-9a86-046d6507ae4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:17:31.725275 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:31.725131    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/64faa9bc-8055-4b10-9a86-046d6507ae4a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"64faa9bc-8055-4b10-9a86-046d6507ae4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:17:31.725275 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:31.725188    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/64faa9bc-8055-4b10-9a86-046d6507ae4a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"64faa9bc-8055-4b10-9a86-046d6507ae4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:17:31.725275 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:31.725238    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/64faa9bc-8055-4b10-9a86-046d6507ae4a-config-out\") pod \"prometheus-k8s-0\" (UID: \"64faa9bc-8055-4b10-9a86-046d6507ae4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:17:31.726094 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:31.726060    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/64faa9bc-8055-4b10-9a86-046d6507ae4a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"64faa9bc-8055-4b10-9a86-046d6507ae4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:17:31.726417 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:31.726394    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/64faa9bc-8055-4b10-9a86-046d6507ae4a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"64faa9bc-8055-4b10-9a86-046d6507ae4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:17:31.726842 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:31.726822    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/64faa9bc-8055-4b10-9a86-046d6507ae4a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"64faa9bc-8055-4b10-9a86-046d6507ae4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:17:31.727281 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:31.727253    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/64faa9bc-8055-4b10-9a86-046d6507ae4a-web-config\") pod \"prometheus-k8s-0\" (UID: \"64faa9bc-8055-4b10-9a86-046d6507ae4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:17:31.727281 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:31.727275    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/64faa9bc-8055-4b10-9a86-046d6507ae4a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"64faa9bc-8055-4b10-9a86-046d6507ae4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:17:31.727634 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:31.727615    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/64faa9bc-8055-4b10-9a86-046d6507ae4a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"64faa9bc-8055-4b10-9a86-046d6507ae4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:17:31.744676 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:31.744658    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x6ttf\" (UniqueName: \"kubernetes.io/projected/64faa9bc-8055-4b10-9a86-046d6507ae4a-kube-api-access-x6ttf\") pod \"prometheus-k8s-0\" (UID: \"64faa9bc-8055-4b10-9a86-046d6507ae4a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:17:31.810949 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:31.810866    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:17:32.109265 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:32.109237    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 19:17:32.120305 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:17:32.120275    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64faa9bc_8055_4b10_9a86_046d6507ae4a.slice/crio-d2abba133908c70eed99fec2ac87efd0108f9794b4ce9964602551dbbae82325 WatchSource:0}: Error finding container d2abba133908c70eed99fec2ac87efd0108f9794b4ce9964602551dbbae82325: Status 404 returned error can't find the container with id d2abba133908c70eed99fec2ac87efd0108f9794b4ce9964602551dbbae82325
Sep 10 19:17:32.949414 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:32.949379    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"93a950f6-8e9b-481f-9c1b-0eeb7a373c8d","Type":"ContainerStarted","Data":"3c73c5f11868a4793a615205ea583dc27b96785ddf8e517d129f85575e7725df"}
Sep 10 19:17:32.949848 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:32.949477    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"93a950f6-8e9b-481f-9c1b-0eeb7a373c8d","Type":"ContainerStarted","Data":"9ab3751a8228ff52f0f5a6189bdcb1e2c19863037c7d8116b62bbf59e0c87b4a"}
Sep 10 19:17:32.949848 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:32.949525    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"93a950f6-8e9b-481f-9c1b-0eeb7a373c8d","Type":"ContainerStarted","Data":"ce50988c24e8467d2e53d6655f74a833b47d34881f3dd158da63678aa71a3369"}
Sep 10 19:17:32.949848 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:32.949565    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"93a950f6-8e9b-481f-9c1b-0eeb7a373c8d","Type":"ContainerStarted","Data":"7e485bd5e4003246e13407f0429aa788a9af0472b00208eb4e2c15072684ad86"}
Sep 10 19:17:32.949848 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:32.949604    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"93a950f6-8e9b-481f-9c1b-0eeb7a373c8d","Type":"ContainerStarted","Data":"617b59673eaa71050b0e0051b4b691510fc45ab068ec5b97c73b1b18a73b1b06"}
Sep 10 19:17:32.951135 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:32.951105    2575 generic.go:356] "Generic (PLEG): container finished" podID="64faa9bc-8055-4b10-9a86-046d6507ae4a" containerID="64a92aaf66345394ea9c241ab00c434b109633e8ccb5eb4181033bbc4adccc14" exitCode=0
Sep 10 19:17:32.951264 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:32.951152    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"64faa9bc-8055-4b10-9a86-046d6507ae4a","Type":"ContainerDied","Data":"64a92aaf66345394ea9c241ab00c434b109633e8ccb5eb4181033bbc4adccc14"}
Sep 10 19:17:32.951264 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:32.951215    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"64faa9bc-8055-4b10-9a86-046d6507ae4a","Type":"ContainerStarted","Data":"d2abba133908c70eed99fec2ac87efd0108f9794b4ce9964602551dbbae82325"}
Sep 10 19:17:34.962367 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:34.962323    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"93a950f6-8e9b-481f-9c1b-0eeb7a373c8d","Type":"ContainerStarted","Data":"97981fd6aad704f963edb61c9eb3f4cd37aa4fe740ee0258db18b8fdd248ece7"}
Sep 10 19:17:34.990157 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:34.990110    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.507955542 podStartE2EDuration="8.990091465s" totalImagesPullingTime="5.469212686s" totalInitContainerRuntime="12.923237ms" isStatefulPod=false podCreationTimestamp="2026-09-10 19:17:26 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:17:34.988361629 +0000 UTC m=+126.048189290" watchObservedRunningTime="2026-09-10 19:17:34.990091465 +0000 UTC m=+126.049919132"
Sep 10 19:17:36.972477 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:36.972403    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"64faa9bc-8055-4b10-9a86-046d6507ae4a","Type":"ContainerStarted","Data":"dd0199936c02b69bc60ce76a8a1df3180f34f1546c023b253b522e7c5b49dea3"}
Sep 10 19:17:36.972477 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:36.972465    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"64faa9bc-8055-4b10-9a86-046d6507ae4a","Type":"ContainerStarted","Data":"fc8fab27e115cbe47e2d999763958f250fb05d0b39bc70a8badc46f55d49020a"}
Sep 10 19:17:37.008552 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:37.008508    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:17:37.108477 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:37.108436    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:17:37.751521 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:37.751487    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:17:37.816814 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:37.816778    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:17:38.282590 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:38.282560    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bfed5912-60ab-48fc-91c2-10d325fd6428-metrics-certs\") pod \"network-metrics-daemon-wx2q6\" (UID: \"bfed5912-60ab-48fc-91c2-10d325fd6428\") " pod="openshift-multus/network-metrics-daemon-wx2q6"
Sep 10 19:17:38.284996 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:38.284972    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bfed5912-60ab-48fc-91c2-10d325fd6428-metrics-certs\") pod \"network-metrics-daemon-wx2q6\" (UID: \"bfed5912-60ab-48fc-91c2-10d325fd6428\") " pod="openshift-multus/network-metrics-daemon-wx2q6"
Sep 10 19:17:38.342257 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:38.342219    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:17:38.442262 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:38.442234    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:17:38.494010 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:38.493980    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-q8jbb\""
Sep 10 19:17:38.502792 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:38.502769    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wx2q6"
Sep 10 19:17:39.246944 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:39.246918    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-wx2q6"]
Sep 10 19:17:39.351417 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:17:39.351384    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbfed5912_60ab_48fc_91c2_10d325fd6428.slice/crio-cc142fec2541e246cfac867430f3809352bf60151e2c32966455e5631385755c WatchSource:0}: Error finding container cc142fec2541e246cfac867430f3809352bf60151e2c32966455e5631385755c: Status 404 returned error can't find the container with id cc142fec2541e246cfac867430f3809352bf60151e2c32966455e5631385755c
Sep 10 19:17:39.989696 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:39.989651    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"64faa9bc-8055-4b10-9a86-046d6507ae4a","Type":"ContainerStarted","Data":"9e6deb581b517c68e3fe9dcc74524313bd5a3f38878b80fafdbf183fdcc1dd04"}
Sep 10 19:17:39.989873 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:39.989772    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"64faa9bc-8055-4b10-9a86-046d6507ae4a","Type":"ContainerStarted","Data":"5ca317ff2cea2b60884e0eab9dc7b3aaeaa1ad2b1705424c895b8a113b8d9198"}
Sep 10 19:17:39.989873 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:39.989807    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"64faa9bc-8055-4b10-9a86-046d6507ae4a","Type":"ContainerStarted","Data":"bdfdd24d8c162722302b5de864588bda1f7f1ccd00560bca2dd1cf7b4f2e6f93"}
Sep 10 19:17:39.989873 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:39.989836    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"64faa9bc-8055-4b10-9a86-046d6507ae4a","Type":"ContainerStarted","Data":"59e88bc6abac3eb83950b4513363cb3f8472080535b2660e328fe1b92b6818ff"}
Sep 10 19:17:39.991518 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:39.991488    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-wx2q6" event={"ID":"bfed5912-60ab-48fc-91c2-10d325fd6428","Type":"ContainerStarted","Data":"cc142fec2541e246cfac867430f3809352bf60151e2c32966455e5631385755c"}
Sep 10 19:17:40.020919 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:40.020845    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.883016814 podStartE2EDuration="9.020830221s" totalImagesPullingTime="6.116403584s" totalInitContainerRuntime="21.409823ms" isStatefulPod=false podCreationTimestamp="2026-09-10 19:17:31 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:17:40.019165618 +0000 UTC m=+131.078993287" watchObservedRunningTime="2026-09-10 19:17:40.020830221 +0000 UTC m=+131.080657888"
Sep 10 19:17:40.997060 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:40.997035    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-wx2q6" event={"ID":"bfed5912-60ab-48fc-91c2-10d325fd6428","Type":"ContainerStarted","Data":"b5b4adfae9b70de6ddc1638fdba7f076e2afd26307d9e09b4eb96f759ad70de6"}
Sep 10 19:17:41.811405 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:41.811372    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:17:42.002289 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:42.002254    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-wx2q6" event={"ID":"bfed5912-60ab-48fc-91c2-10d325fd6428","Type":"ContainerStarted","Data":"fb924e12719915c0091008f54ca07a73799bd5409ec979dd4487116d17efc9fa"}
Sep 10 19:17:42.018821 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:42.018776    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-wx2q6" podStartSLOduration=131.560258224 podStartE2EDuration="2m13.018755298s" totalImagesPullingTime="1.458497074s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:15:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:17:42.017866915 +0000 UTC m=+133.077694583" watchObservedRunningTime="2026-09-10 19:17:42.018755298 +0000 UTC m=+133.078582966"
Sep 10 19:17:42.112139 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:42.112075    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:17:42.213369 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:42.213322    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:17:42.813460 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:42.813403    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5bd779b86c-rqxkh" podUID="7594800a-6546-4425-8eab-3a1cda053b6e" containerName="console" containerID="cri-o://9b9f47e1705c2bbc6588130cc938f7c9223fe38752162f672475931ba81ad1a8" gracePeriod=15
Sep 10 19:17:43.012233 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:43.012197    2575 generic.go:356] "Generic (PLEG): container finished" podID="7594800a-6546-4425-8eab-3a1cda053b6e" containerID="9b9f47e1705c2bbc6588130cc938f7c9223fe38752162f672475931ba81ad1a8" exitCode=0
Sep 10 19:17:43.012627 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:43.012297    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5bd779b86c-rqxkh" event={"ID":"7594800a-6546-4425-8eab-3a1cda053b6e","Type":"ContainerDied","Data":"9b9f47e1705c2bbc6588130cc938f7c9223fe38752162f672475931ba81ad1a8"}
Sep 10 19:17:43.088501 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:43.088439    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5bd779b86c-rqxkh"
Sep 10 19:17:43.128486 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:43.128464    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7594800a-6546-4425-8eab-3a1cda053b6e-console-serving-cert\") pod \"7594800a-6546-4425-8eab-3a1cda053b6e\" (UID: \"7594800a-6546-4425-8eab-3a1cda053b6e\") "
Sep 10 19:17:43.128621 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:43.128496    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7594800a-6546-4425-8eab-3a1cda053b6e-service-ca\") pod \"7594800a-6546-4425-8eab-3a1cda053b6e\" (UID: \"7594800a-6546-4425-8eab-3a1cda053b6e\") "
Sep 10 19:17:43.128621 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:43.128533    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7594800a-6546-4425-8eab-3a1cda053b6e-console-config\") pod \"7594800a-6546-4425-8eab-3a1cda053b6e\" (UID: \"7594800a-6546-4425-8eab-3a1cda053b6e\") "
Sep 10 19:17:43.128621 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:43.128559    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mp8p7\" (UniqueName: \"kubernetes.io/projected/7594800a-6546-4425-8eab-3a1cda053b6e-kube-api-access-mp8p7\") pod \"7594800a-6546-4425-8eab-3a1cda053b6e\" (UID: \"7594800a-6546-4425-8eab-3a1cda053b6e\") "
Sep 10 19:17:43.128621 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:43.128577    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7594800a-6546-4425-8eab-3a1cda053b6e-oauth-serving-cert\") pod \"7594800a-6546-4425-8eab-3a1cda053b6e\" (UID: \"7594800a-6546-4425-8eab-3a1cda053b6e\") "
Sep 10 19:17:43.128621 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:43.128613    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7594800a-6546-4425-8eab-3a1cda053b6e-console-oauth-config\") pod \"7594800a-6546-4425-8eab-3a1cda053b6e\" (UID: \"7594800a-6546-4425-8eab-3a1cda053b6e\") "
Sep 10 19:17:43.128998 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:43.128973    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7594800a-6546-4425-8eab-3a1cda053b6e-console-config" (OuterVolumeSpecName: "console-config") pod "7594800a-6546-4425-8eab-3a1cda053b6e" (UID: "7594800a-6546-4425-8eab-3a1cda053b6e"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:17:43.129059 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:43.128976    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7594800a-6546-4425-8eab-3a1cda053b6e-service-ca" (OuterVolumeSpecName: "service-ca") pod "7594800a-6546-4425-8eab-3a1cda053b6e" (UID: "7594800a-6546-4425-8eab-3a1cda053b6e"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:17:43.129059 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:43.129040    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7594800a-6546-4425-8eab-3a1cda053b6e-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "7594800a-6546-4425-8eab-3a1cda053b6e" (UID: "7594800a-6546-4425-8eab-3a1cda053b6e"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:17:43.130798 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:43.130776    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7594800a-6546-4425-8eab-3a1cda053b6e-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "7594800a-6546-4425-8eab-3a1cda053b6e" (UID: "7594800a-6546-4425-8eab-3a1cda053b6e"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:17:43.130987 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:43.130961    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7594800a-6546-4425-8eab-3a1cda053b6e-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "7594800a-6546-4425-8eab-3a1cda053b6e" (UID: "7594800a-6546-4425-8eab-3a1cda053b6e"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:17:43.130987 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:43.130971    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7594800a-6546-4425-8eab-3a1cda053b6e-kube-api-access-mp8p7" (OuterVolumeSpecName: "kube-api-access-mp8p7") pod "7594800a-6546-4425-8eab-3a1cda053b6e" (UID: "7594800a-6546-4425-8eab-3a1cda053b6e"). InnerVolumeSpecName "kube-api-access-mp8p7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:17:43.230077 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:43.230052    2575 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7594800a-6546-4425-8eab-3a1cda053b6e-console-oauth-config\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:17:43.230077 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:43.230075    2575 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7594800a-6546-4425-8eab-3a1cda053b6e-console-serving-cert\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:17:43.230218 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:43.230090    2575 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7594800a-6546-4425-8eab-3a1cda053b6e-service-ca\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:17:43.230218 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:43.230102    2575 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7594800a-6546-4425-8eab-3a1cda053b6e-console-config\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:17:43.230218 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:43.230114    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mp8p7\" (UniqueName: \"kubernetes.io/projected/7594800a-6546-4425-8eab-3a1cda053b6e-kube-api-access-mp8p7\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:17:43.230218 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:43.230127    2575 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7594800a-6546-4425-8eab-3a1cda053b6e-oauth-serving-cert\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:17:43.706586 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:43.706561    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-gbtv9_4cb26dfc-5d83-4cec-b4f2-52f3ff4c47d6/serve-healthcheck-canary/0.log"
Sep 10 19:17:44.017848 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:44.017762    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5bd779b86c-rqxkh" event={"ID":"7594800a-6546-4425-8eab-3a1cda053b6e","Type":"ContainerDied","Data":"c54a88f2e01741404583617b88323c041d7e184c3889519b739737595094f60b"}
Sep 10 19:17:44.017848 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:44.017792    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5bd779b86c-rqxkh"
Sep 10 19:17:44.017848 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:44.017818    2575 scope.go:117] "RemoveContainer" containerID="9b9f47e1705c2bbc6588130cc938f7c9223fe38752162f672475931ba81ad1a8"
Sep 10 19:17:44.034565 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:44.034542    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5bd779b86c-rqxkh"]
Sep 10 19:17:44.039930 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:44.039909    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5bd779b86c-rqxkh"]
Sep 10 19:17:45.384459 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:45.384425    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7594800a-6546-4425-8eab-3a1cda053b6e" path="/var/lib/kubelet/pods/7594800a-6546-4425-8eab-3a1cda053b6e/volumes"
Sep 10 19:17:45.599477 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:45.599441    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:17:45.700712 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:45.700679    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:17:45.753234 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:45.753204    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-8cdf54dd5-22kjj" podUID="7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7" containerName="registry" containerID="cri-o://e6a780c4b09f82ce1534920759aefbf2fd7abae4a70d1998dd4073d5ee41f5be" gracePeriod=30
Sep 10 19:17:46.018104 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:46.018078    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-8cdf54dd5-22kjj"
Sep 10 19:17:46.027801 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:46.027772    2575 generic.go:356] "Generic (PLEG): container finished" podID="7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7" containerID="e6a780c4b09f82ce1534920759aefbf2fd7abae4a70d1998dd4073d5ee41f5be" exitCode=0
Sep 10 19:17:46.027897 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:46.027814    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-8cdf54dd5-22kjj" event={"ID":"7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7","Type":"ContainerDied","Data":"e6a780c4b09f82ce1534920759aefbf2fd7abae4a70d1998dd4073d5ee41f5be"}
Sep 10 19:17:46.027897 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:46.027824    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-8cdf54dd5-22kjj"
Sep 10 19:17:46.027897 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:46.027853    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-8cdf54dd5-22kjj" event={"ID":"7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7","Type":"ContainerDied","Data":"b725465d0316bab0360780b45a9491058039b5570774d06bb6997894c4f5f451"}
Sep 10 19:17:46.027897 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:46.027879    2575 scope.go:117] "RemoveContainer" containerID="e6a780c4b09f82ce1534920759aefbf2fd7abae4a70d1998dd4073d5ee41f5be"
Sep 10 19:17:46.035457 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:46.035436    2575 scope.go:117] "RemoveContainer" containerID="e6a780c4b09f82ce1534920759aefbf2fd7abae4a70d1998dd4073d5ee41f5be"
Sep 10 19:17:46.035727 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:17:46.035709    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e6a780c4b09f82ce1534920759aefbf2fd7abae4a70d1998dd4073d5ee41f5be\": container with ID starting with e6a780c4b09f82ce1534920759aefbf2fd7abae4a70d1998dd4073d5ee41f5be not found: ID does not exist" containerID="e6a780c4b09f82ce1534920759aefbf2fd7abae4a70d1998dd4073d5ee41f5be"
Sep 10 19:17:46.035798 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:46.035733    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e6a780c4b09f82ce1534920759aefbf2fd7abae4a70d1998dd4073d5ee41f5be"} err="failed to get container status \"e6a780c4b09f82ce1534920759aefbf2fd7abae4a70d1998dd4073d5ee41f5be\": rpc error: code = NotFound desc = could not find container \"e6a780c4b09f82ce1534920759aefbf2fd7abae4a70d1998dd4073d5ee41f5be\": container with ID starting with e6a780c4b09f82ce1534920759aefbf2fd7abae4a70d1998dd4073d5ee41f5be not found: ID does not exist"
Sep 10 19:17:46.052133 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:46.052117    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7-trusted-ca\") pod \"7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7\" (UID: \"7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7\") "
Sep 10 19:17:46.052204 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:46.052144    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jlj2m\" (UniqueName: \"kubernetes.io/projected/7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7-kube-api-access-jlj2m\") pod \"7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7\" (UID: \"7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7\") "
Sep 10 19:17:46.052240 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:46.052203    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7-registry-certificates\") pod \"7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7\" (UID: \"7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7\") "
Sep 10 19:17:46.052240 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:46.052233    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7-bound-sa-token\") pod \"7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7\" (UID: \"7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7\") "
Sep 10 19:17:46.052324 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:46.052265    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7-ca-trust-extracted\") pod \"7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7\" (UID: \"7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7\") "
Sep 10 19:17:46.052324 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:46.052307    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7-installation-pull-secrets\") pod \"7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7\" (UID: \"7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7\") "
Sep 10 19:17:46.052438 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:46.052355    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7-registry-tls\") pod \"7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7\" (UID: \"7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7\") "
Sep 10 19:17:46.052438 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:46.052381    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7-image-registry-private-configuration\") pod \"7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7\" (UID: \"7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7\") "
Sep 10 19:17:46.052591 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:46.052571    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7" (UID: "7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:17:46.052662 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:46.052636    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7" (UID: "7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:17:46.054854 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:46.054823    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7-kube-api-access-jlj2m" (OuterVolumeSpecName: "kube-api-access-jlj2m") pod "7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7" (UID: "7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7"). InnerVolumeSpecName "kube-api-access-jlj2m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:17:46.054945 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:46.054848    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7" (UID: "7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:17:46.054945 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:46.054872    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7" (UID: "7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:17:46.054945 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:46.054903    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7" (UID: "7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:17:46.055065 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:46.055012    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7" (UID: "7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:17:46.063621 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:46.063593    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7" (UID: "7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:17:46.103020 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:46.102988    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:17:46.153203 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:46.153179    2575 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7-trusted-ca\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:17:46.153203 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:46.153203    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jlj2m\" (UniqueName: \"kubernetes.io/projected/7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7-kube-api-access-jlj2m\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:17:46.153322 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:46.153213    2575 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7-registry-certificates\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:17:46.153322 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:46.153222    2575 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7-bound-sa-token\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:17:46.153322 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:46.153231    2575 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7-ca-trust-extracted\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:17:46.153322 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:46.153239    2575 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7-installation-pull-secrets\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:17:46.153322 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:46.153248    2575 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7-registry-tls\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:17:46.153322 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:46.153256    2575 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7-image-registry-private-configuration\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:17:46.351600 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:46.351570    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-8cdf54dd5-22kjj"]
Sep 10 19:17:46.357078 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:46.357057    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-8cdf54dd5-22kjj"]
Sep 10 19:17:47.069152 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:47.069111    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:17:47.173581 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:47.173555    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:17:47.239422 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:47.239393    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:17:47.383503 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:47.383425    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7" path="/var/lib/kubelet/pods/7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7/volumes"
Sep 10 19:17:53.342785 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:53.342744    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:17:53.402243 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:53.402209    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:17:57.079728 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:57.079638    2575 generic.go:356] "Generic (PLEG): container finished" podID="38dcdfc7-edac-4ddb-b51a-bbf5857139a8" containerID="e92c3f84cfc1ef0a1d8e4e0258dbe1eb239ce3e920337aeb4a33cb9c575ba7c4" exitCode=0
Sep 10 19:17:57.079728 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:57.079696    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-9gl2b" event={"ID":"38dcdfc7-edac-4ddb-b51a-bbf5857139a8","Type":"ContainerDied","Data":"e92c3f84cfc1ef0a1d8e4e0258dbe1eb239ce3e920337aeb4a33cb9c575ba7c4"}
Sep 10 19:17:57.080113 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:57.080032    2575 scope.go:117] "RemoveContainer" containerID="e92c3f84cfc1ef0a1d8e4e0258dbe1eb239ce3e920337aeb4a33cb9c575ba7c4"
Sep 10 19:17:58.085011 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:17:58.084974    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-9gl2b" event={"ID":"38dcdfc7-edac-4ddb-b51a-bbf5857139a8","Type":"ContainerStarted","Data":"167c00be58a0416e233f304d8b38f4c399b12fdd2db8d59025bb17edda959308"}
Sep 10 19:18:31.811277 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:18:31.811241    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:18:31.831462 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:18:31.831436    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:18:32.239175 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:18:32.239152    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:18:41.327286 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:18:41.327243    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:18:41.394541 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:18:41.394498    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:18:51.184041 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:18:51.183994    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:18:51.250763 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:18:51.250735    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:18:53.264781 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:18:53.264738    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:18:53.326298 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:18:53.326266    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:19:01.048191 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:19:01.048150    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:19:02.343070 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:19:02.343018    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:19:02.443877 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:19:02.443835    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:19:02.546562 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:19:02.546527    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:19:08.350156 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:19:08.350116    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:19:08.448142 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:19:08.448115    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:19:13.761801 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:19:13.761764    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:19:29.902357 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:19:29.902307    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:19:29.973929 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:19:29.973897    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:19:41.231763 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:19:41.231724    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:19:51.463494 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:19:51.463458    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:19:52.227740 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:19:52.227708    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:19:55.073707 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:19:55.073667    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:19:55.172109 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:19:55.172079    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:19:57.522281 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:19:57.522237    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:19:57.624621 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:19:57.624586    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:19:57.750295 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:19:57.750264    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:19:59.506437 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:19:59.506404    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:19:59.934266 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:19:59.934195    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:20:01.670148 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:20:01.670112    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:20:01.732606 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:20:01.732573    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:20:03.946906 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:20:03.946860    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:20:04.275874 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:20:04.275801    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:20:08.089445 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:20:08.089410    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:20:12.751860 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:20:12.751827    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:20:12.851161 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:20:12.851128    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:20:14.702732 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:20:14.702699    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hg59c"]
Sep 10 19:20:14.703099 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:20:14.702998    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7" containerName="registry"
Sep 10 19:20:14.703099 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:20:14.703008    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7" containerName="registry"
Sep 10 19:20:14.703099 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:20:14.703022    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7594800a-6546-4425-8eab-3a1cda053b6e" containerName="console"
Sep 10 19:20:14.703099 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:20:14.703027    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="7594800a-6546-4425-8eab-3a1cda053b6e" containerName="console"
Sep 10 19:20:14.703099 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:20:14.703075    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="7224f0a5-f7de-4d50-bdec-7ebd9a0b5bf7" containerName="registry"
Sep 10 19:20:14.703099 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:20:14.703081    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="7594800a-6546-4425-8eab-3a1cda053b6e" containerName="console"
Sep 10 19:20:14.705510 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:20:14.705494    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hg59c"
Sep 10 19:20:14.707592 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:20:14.707570    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 10 19:20:14.707778 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:20:14.707765    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-qtl55\""
Sep 10 19:20:14.708013 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:20:14.707997    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 10 19:20:14.712755 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:20:14.712733    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hg59c"]
Sep 10 19:20:14.809451 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:20:14.809425    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hrs7c\" (UniqueName: \"kubernetes.io/projected/91367cd9-4fe3-4775-90cd-3a5982b38057-kube-api-access-hrs7c\") pod \"openshift-lws-operator-fd8ccff4c-hg59c\" (UID: \"91367cd9-4fe3-4775-90cd-3a5982b38057\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hg59c"
Sep 10 19:20:14.809537 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:20:14.809468    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/91367cd9-4fe3-4775-90cd-3a5982b38057-tmp\") pod \"openshift-lws-operator-fd8ccff4c-hg59c\" (UID: \"91367cd9-4fe3-4775-90cd-3a5982b38057\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hg59c"
Sep 10 19:20:14.910694 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:20:14.910671    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hrs7c\" (UniqueName: \"kubernetes.io/projected/91367cd9-4fe3-4775-90cd-3a5982b38057-kube-api-access-hrs7c\") pod \"openshift-lws-operator-fd8ccff4c-hg59c\" (UID: \"91367cd9-4fe3-4775-90cd-3a5982b38057\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hg59c"
Sep 10 19:20:14.910786 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:20:14.910708    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/91367cd9-4fe3-4775-90cd-3a5982b38057-tmp\") pod \"openshift-lws-operator-fd8ccff4c-hg59c\" (UID: \"91367cd9-4fe3-4775-90cd-3a5982b38057\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hg59c"
Sep 10 19:20:14.911032 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:20:14.911016    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/91367cd9-4fe3-4775-90cd-3a5982b38057-tmp\") pod \"openshift-lws-operator-fd8ccff4c-hg59c\" (UID: \"91367cd9-4fe3-4775-90cd-3a5982b38057\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hg59c"
Sep 10 19:20:14.918077 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:20:14.918054    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hrs7c\" (UniqueName: \"kubernetes.io/projected/91367cd9-4fe3-4775-90cd-3a5982b38057-kube-api-access-hrs7c\") pod \"openshift-lws-operator-fd8ccff4c-hg59c\" (UID: \"91367cd9-4fe3-4775-90cd-3a5982b38057\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hg59c"
Sep 10 19:20:15.026173 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:20:15.026106    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hg59c"
Sep 10 19:20:15.294181 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:20:15.294150    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hg59c"]
Sep 10 19:20:15.296672 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:20:15.296643    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod91367cd9_4fe3_4775_90cd_3a5982b38057.slice/crio-c565068f3a8dd256c2698811113efb6417f22bd2c0443a8a511f04debb7563d5 WatchSource:0}: Error finding container c565068f3a8dd256c2698811113efb6417f22bd2c0443a8a511f04debb7563d5: Status 404 returned error can't find the container with id c565068f3a8dd256c2698811113efb6417f22bd2c0443a8a511f04debb7563d5
Sep 10 19:20:15.617870 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:20:15.617797    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hg59c" event={"ID":"91367cd9-4fe3-4775-90cd-3a5982b38057","Type":"ContainerStarted","Data":"c565068f3a8dd256c2698811113efb6417f22bd2c0443a8a511f04debb7563d5"}
Sep 10 19:20:18.632419 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:20:18.632380    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hg59c" event={"ID":"91367cd9-4fe3-4775-90cd-3a5982b38057","Type":"ContainerStarted","Data":"8d0605afa8d5e0f518781af224fa211fadacd6b726988f0862167d42eff13bcf"}
Sep 10 19:20:18.662417 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:20:18.662374    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hg59c" podStartSLOduration=2.083860653 podStartE2EDuration="4.662359838s" totalImagesPullingTime="2.578499185s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:20:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:20:18.661053214 +0000 UTC m=+289.720880881" watchObservedRunningTime="2026-09-10 19:20:18.662359838 +0000 UTC m=+289.722187498"
Sep 10 19:20:19.376573 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:20:19.376515    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:20:19.478579 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:20:19.478538    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:20:21.387317 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:20:21.387282    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:20:29.303662 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:20:29.303633    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rrxfj_c7511dfe-c588-4be1-bd16-1dce34740809/ovn-acl-logging/0.log"
Sep 10 19:20:29.308114 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:20:29.308095    2575 kubelet.go:1648] "Image garbage collection succeeded"
Sep 10 19:20:29.308400 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:20:29.308384    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rrxfj_c7511dfe-c588-4be1-bd16-1dce34740809/ovn-acl-logging/0.log"
Sep 10 19:20:33.798724 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:20:33.798693    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:20:33.895960 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:20:33.895913    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:20:34.413050 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:20:34.413015    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:20:34.862745 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:20:34.862710    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:20:34.964058 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:20:34.964031    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:20:35.022310 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:20:35.022281    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5d477566c-5h625"]
Sep 10 19:20:35.025547 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:20:35.025532    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5d477566c-5h625"
Sep 10 19:20:35.028304 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:20:35.028284    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 10 19:20:35.028577 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:20:35.028558    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 10 19:20:35.028656 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:20:35.028577    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-4qpwt\""
Sep 10 19:20:35.029175 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:20:35.029158    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 10 19:20:35.029500 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:20:35.029485    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 10 19:20:35.046809 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:20:35.046783    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5d477566c-5h625"]
Sep 10 19:20:35.153099 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:20:35.153050    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/d5588d80-e182-43c2-aecb-44038005f945-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5d477566c-5h625\" (UID: \"d5588d80-e182-43c2-aecb-44038005f945\") " pod="opendatahub/opendatahub-operator-controller-manager-5d477566c-5h625"
Sep 10 19:20:35.153099 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:20:35.153088    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-74vmz\" (UniqueName: \"kubernetes.io/projected/d5588d80-e182-43c2-aecb-44038005f945-kube-api-access-74vmz\") pod \"opendatahub-operator-controller-manager-5d477566c-5h625\" (UID: \"d5588d80-e182-43c2-aecb-44038005f945\") " pod="opendatahub/opendatahub-operator-controller-manager-5d477566c-5h625"
Sep 10 19:20:35.153234 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:20:35.153116    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d5588d80-e182-43c2-aecb-44038005f945-webhook-cert\") pod \"opendatahub-operator-controller-manager-5d477566c-5h625\" (UID: \"d5588d80-e182-43c2-aecb-44038005f945\") " pod="opendatahub/opendatahub-operator-controller-manager-5d477566c-5h625"
Sep 10 19:20:35.153234 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:20:35.153179    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/d5588d80-e182-43c2-aecb-44038005f945-manifests\") pod \"opendatahub-operator-controller-manager-5d477566c-5h625\" (UID: \"d5588d80-e182-43c2-aecb-44038005f945\") " pod="opendatahub/opendatahub-operator-controller-manager-5d477566c-5h625"
Sep 10 19:20:35.153314 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:20:35.153239    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d5588d80-e182-43c2-aecb-44038005f945-tmp\") pod \"opendatahub-operator-controller-manager-5d477566c-5h625\" (UID: \"d5588d80-e182-43c2-aecb-44038005f945\") " pod="opendatahub/opendatahub-operator-controller-manager-5d477566c-5h625"
Sep 10 19:20:35.163410 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:20:35.163356    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:20:35.253836 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:20:35.253815    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/d5588d80-e182-43c2-aecb-44038005f945-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5d477566c-5h625\" (UID: \"d5588d80-e182-43c2-aecb-44038005f945\") " pod="opendatahub/opendatahub-operator-controller-manager-5d477566c-5h625"
Sep 10 19:20:35.253916 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:20:35.253851    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-74vmz\" (UniqueName: \"kubernetes.io/projected/d5588d80-e182-43c2-aecb-44038005f945-kube-api-access-74vmz\") pod \"opendatahub-operator-controller-manager-5d477566c-5h625\" (UID: \"d5588d80-e182-43c2-aecb-44038005f945\") " pod="opendatahub/opendatahub-operator-controller-manager-5d477566c-5h625"
Sep 10 19:20:35.253916 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:20:35.253888    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d5588d80-e182-43c2-aecb-44038005f945-webhook-cert\") pod \"opendatahub-operator-controller-manager-5d477566c-5h625\" (UID: \"d5588d80-e182-43c2-aecb-44038005f945\") " pod="opendatahub/opendatahub-operator-controller-manager-5d477566c-5h625"
Sep 10 19:20:35.253916 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:20:35.253909    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/d5588d80-e182-43c2-aecb-44038005f945-manifests\") pod \"opendatahub-operator-controller-manager-5d477566c-5h625\" (UID: \"d5588d80-e182-43c2-aecb-44038005f945\") " pod="opendatahub/opendatahub-operator-controller-manager-5d477566c-5h625"
Sep 10 19:20:35.254013 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:20:35.253946    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d5588d80-e182-43c2-aecb-44038005f945-tmp\") pod \"opendatahub-operator-controller-manager-5d477566c-5h625\" (UID: \"d5588d80-e182-43c2-aecb-44038005f945\") " pod="opendatahub/opendatahub-operator-controller-manager-5d477566c-5h625"
Sep 10 19:20:35.254361 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:20:35.254315    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/d5588d80-e182-43c2-aecb-44038005f945-manifests\") pod \"opendatahub-operator-controller-manager-5d477566c-5h625\" (UID: \"d5588d80-e182-43c2-aecb-44038005f945\") " pod="opendatahub/opendatahub-operator-controller-manager-5d477566c-5h625"
Sep 10 19:20:35.256303 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:20:35.256281    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d5588d80-e182-43c2-aecb-44038005f945-tmp\") pod \"opendatahub-operator-controller-manager-5d477566c-5h625\" (UID: \"d5588d80-e182-43c2-aecb-44038005f945\") " pod="opendatahub/opendatahub-operator-controller-manager-5d477566c-5h625"
Sep 10 19:20:35.256481 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:20:35.256467    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d5588d80-e182-43c2-aecb-44038005f945-webhook-cert\") pod \"opendatahub-operator-controller-manager-5d477566c-5h625\" (UID: \"d5588d80-e182-43c2-aecb-44038005f945\") " pod="opendatahub/opendatahub-operator-controller-manager-5d477566c-5h625"
Sep 10 19:20:35.256524 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:20:35.256504    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/d5588d80-e182-43c2-aecb-44038005f945-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5d477566c-5h625\" (UID: \"d5588d80-e182-43c2-aecb-44038005f945\") " pod="opendatahub/opendatahub-operator-controller-manager-5d477566c-5h625"
Sep 10 19:20:35.277582 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:20:35.277558    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-74vmz\" (UniqueName: \"kubernetes.io/projected/d5588d80-e182-43c2-aecb-44038005f945-kube-api-access-74vmz\") pod \"opendatahub-operator-controller-manager-5d477566c-5h625\" (UID: \"d5588d80-e182-43c2-aecb-44038005f945\") " pod="opendatahub/opendatahub-operator-controller-manager-5d477566c-5h625"
Sep 10 19:20:35.278081 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:20:35.278055    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:20:35.335403 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:20:35.335380    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5d477566c-5h625"
Sep 10 19:20:35.370650 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:20:35.370604    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:20:35.471215 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:20:35.471182    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:20:35.566514 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:20:35.566484    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:20:35.618220 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:20:35.618192    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5d477566c-5h625"]
Sep 10 19:20:35.622239 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:20:35.622216    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5588d80_e182_43c2_aecb_44038005f945.slice/crio-6296737ac67658819a5976603e03ee567096c7455a9c5b6dacc800d899983963 WatchSource:0}: Error finding container 6296737ac67658819a5976603e03ee567096c7455a9c5b6dacc800d899983963: Status 404 returned error can't find the container with id 6296737ac67658819a5976603e03ee567096c7455a9c5b6dacc800d899983963
Sep 10 19:20:35.623972 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:20:35.623956    2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 19:20:35.696927 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:20:35.696905    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5d477566c-5h625" event={"ID":"d5588d80-e182-43c2-aecb-44038005f945","Type":"ContainerStarted","Data":"6296737ac67658819a5976603e03ee567096c7455a9c5b6dacc800d899983963"}
Sep 10 19:20:39.716907 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:20:39.716868    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5d477566c-5h625" event={"ID":"d5588d80-e182-43c2-aecb-44038005f945","Type":"ContainerStarted","Data":"37c85de5a873e854cd79a376fcc32419b18b89f47a9973af025c5bb8871be2dc"}
Sep 10 19:20:40.722403 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:20:40.722365    2575 generic.go:356] "Generic (PLEG): container finished" podID="d5588d80-e182-43c2-aecb-44038005f945" containerID="37c85de5a873e854cd79a376fcc32419b18b89f47a9973af025c5bb8871be2dc" exitCode=0
Sep 10 19:20:40.722757 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:20:40.722416    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5d477566c-5h625" event={"ID":"d5588d80-e182-43c2-aecb-44038005f945","Type":"ContainerDied","Data":"37c85de5a873e854cd79a376fcc32419b18b89f47a9973af025c5bb8871be2dc"}
Sep 10 19:20:41.728296 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:20:41.728257    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5d477566c-5h625" event={"ID":"d5588d80-e182-43c2-aecb-44038005f945","Type":"ContainerStarted","Data":"4e1bf7ee434ac37209aae021f7a6cc77373c557e0b178894e57c326cdbafe8d6"}
Sep 10 19:20:41.728773 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:20:41.728373    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-5d477566c-5h625"
Sep 10 19:20:41.751303 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:20:41.751260    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-5d477566c-5h625" podStartSLOduration=3.222919276 podStartE2EDuration="7.75124382s" totalImagesPullingTime="3.505601213s" totalInitContainerRuntime="1.022723331s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:20:34 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:20:41.749234114 +0000 UTC m=+312.809061782" watchObservedRunningTime="2026-09-10 19:20:41.75124382 +0000 UTC m=+312.811071490"
Sep 10 19:20:43.830518 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:20:43.830475    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:20:45.418007 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:20:45.417970    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:20:45.526542 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:20:45.526502    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:20:45.743652 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:20:45.743614    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:20:45.852629 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:20:45.852602    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:20:52.561451 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:20:52.561413    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:20:52.734508 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:20:52.734480    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-5d477566c-5h625"
Sep 10 19:20:54.006486 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:20:54.006448    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-b5db5bfb7-hxskb"]
Sep 10 19:20:54.011075 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:20:54.011052    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-b5db5bfb7-hxskb"
Sep 10 19:20:54.013620 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:20:54.013593    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-bmgfg\""
Sep 10 19:20:54.013753 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:20:54.013628    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Sep 10 19:20:54.013753 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:20:54.013601    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Sep 10 19:20:54.013753 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:20:54.013601    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Sep 10 19:20:54.027732 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:20:54.027713    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-b5db5bfb7-hxskb"]
Sep 10 19:20:54.089904 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:20:54.089881    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/6d168b87-3c20-4850-b276-d9fdeb0b53a8-metrics-cert\") pod \"lws-controller-manager-b5db5bfb7-hxskb\" (UID: \"6d168b87-3c20-4850-b276-d9fdeb0b53a8\") " pod="openshift-lws-operator/lws-controller-manager-b5db5bfb7-hxskb"
Sep 10 19:20:54.090026 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:20:54.089934    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6d168b87-3c20-4850-b276-d9fdeb0b53a8-cert\") pod \"lws-controller-manager-b5db5bfb7-hxskb\" (UID: \"6d168b87-3c20-4850-b276-d9fdeb0b53a8\") " pod="openshift-lws-operator/lws-controller-manager-b5db5bfb7-hxskb"
Sep 10 19:20:54.090026 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:20:54.089969    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cwktj\" (UniqueName: \"kubernetes.io/projected/6d168b87-3c20-4850-b276-d9fdeb0b53a8-kube-api-access-cwktj\") pod \"lws-controller-manager-b5db5bfb7-hxskb\" (UID: \"6d168b87-3c20-4850-b276-d9fdeb0b53a8\") " pod="openshift-lws-operator/lws-controller-manager-b5db5bfb7-hxskb"
Sep 10 19:20:54.090114 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:20:54.090022    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/6d168b87-3c20-4850-b276-d9fdeb0b53a8-manager-config\") pod \"lws-controller-manager-b5db5bfb7-hxskb\" (UID: \"6d168b87-3c20-4850-b276-d9fdeb0b53a8\") " pod="openshift-lws-operator/lws-controller-manager-b5db5bfb7-hxskb"
Sep 10 19:20:54.190849 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:20:54.190823    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/6d168b87-3c20-4850-b276-d9fdeb0b53a8-metrics-cert\") pod \"lws-controller-manager-b5db5bfb7-hxskb\" (UID: \"6d168b87-3c20-4850-b276-d9fdeb0b53a8\") " pod="openshift-lws-operator/lws-controller-manager-b5db5bfb7-hxskb"
Sep 10 19:20:54.190962 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:20:54.190880    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6d168b87-3c20-4850-b276-d9fdeb0b53a8-cert\") pod \"lws-controller-manager-b5db5bfb7-hxskb\" (UID: \"6d168b87-3c20-4850-b276-d9fdeb0b53a8\") " pod="openshift-lws-operator/lws-controller-manager-b5db5bfb7-hxskb"
Sep 10 19:20:54.190962 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:20:54.190906    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cwktj\" (UniqueName: \"kubernetes.io/projected/6d168b87-3c20-4850-b276-d9fdeb0b53a8-kube-api-access-cwktj\") pod \"lws-controller-manager-b5db5bfb7-hxskb\" (UID: \"6d168b87-3c20-4850-b276-d9fdeb0b53a8\") " pod="openshift-lws-operator/lws-controller-manager-b5db5bfb7-hxskb"
Sep 10 19:20:54.190962 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:20:54.190940    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/6d168b87-3c20-4850-b276-d9fdeb0b53a8-manager-config\") pod \"lws-controller-manager-b5db5bfb7-hxskb\" (UID: \"6d168b87-3c20-4850-b276-d9fdeb0b53a8\") " pod="openshift-lws-operator/lws-controller-manager-b5db5bfb7-hxskb"
Sep 10 19:20:54.191555 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:20:54.191535    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/6d168b87-3c20-4850-b276-d9fdeb0b53a8-manager-config\") pod \"lws-controller-manager-b5db5bfb7-hxskb\" (UID: \"6d168b87-3c20-4850-b276-d9fdeb0b53a8\") " pod="openshift-lws-operator/lws-controller-manager-b5db5bfb7-hxskb"
Sep 10 19:20:54.193208 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:20:54.193181    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/6d168b87-3c20-4850-b276-d9fdeb0b53a8-metrics-cert\") pod \"lws-controller-manager-b5db5bfb7-hxskb\" (UID: \"6d168b87-3c20-4850-b276-d9fdeb0b53a8\") " pod="openshift-lws-operator/lws-controller-manager-b5db5bfb7-hxskb"
Sep 10 19:20:54.193308 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:20:54.193291    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6d168b87-3c20-4850-b276-d9fdeb0b53a8-cert\") pod \"lws-controller-manager-b5db5bfb7-hxskb\" (UID: \"6d168b87-3c20-4850-b276-d9fdeb0b53a8\") " pod="openshift-lws-operator/lws-controller-manager-b5db5bfb7-hxskb"
Sep 10 19:20:54.219463 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:20:54.219434    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cwktj\" (UniqueName: \"kubernetes.io/projected/6d168b87-3c20-4850-b276-d9fdeb0b53a8-kube-api-access-cwktj\") pod \"lws-controller-manager-b5db5bfb7-hxskb\" (UID: \"6d168b87-3c20-4850-b276-d9fdeb0b53a8\") " pod="openshift-lws-operator/lws-controller-manager-b5db5bfb7-hxskb"
Sep 10 19:20:54.320301 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:20:54.320246    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-b5db5bfb7-hxskb"
Sep 10 19:20:54.371083 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:20:54.371002    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:20:54.474641 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:20:54.474612    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:20:54.591731 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:20:54.591697    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-b5db5bfb7-hxskb"]
Sep 10 19:20:54.594527 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:20:54.594500    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d168b87_3c20_4850_b276_d9fdeb0b53a8.slice/crio-aee02e27b340fd12c2b685f29c35aec6c0e9da99d15575d7be1a251fd465db1f WatchSource:0}: Error finding container aee02e27b340fd12c2b685f29c35aec6c0e9da99d15575d7be1a251fd465db1f: Status 404 returned error can't find the container with id aee02e27b340fd12c2b685f29c35aec6c0e9da99d15575d7be1a251fd465db1f
Sep 10 19:20:54.787396 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:20:54.787361    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-b5db5bfb7-hxskb" event={"ID":"6d168b87-3c20-4850-b276-d9fdeb0b53a8","Type":"ContainerStarted","Data":"aee02e27b340fd12c2b685f29c35aec6c0e9da99d15575d7be1a251fd465db1f"}
Sep 10 19:20:56.804212 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:20:56.804110    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-b5db5bfb7-hxskb" event={"ID":"6d168b87-3c20-4850-b276-d9fdeb0b53a8","Type":"ContainerStarted","Data":"fb81a9cfe82a673bb71e059d07a3f667de32accd907ebd95f4ec00bd0cb18cb5"}
Sep 10 19:20:56.804212 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:20:56.804210    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-b5db5bfb7-hxskb"
Sep 10 19:20:56.825599 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:20:56.825548    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-b5db5bfb7-hxskb" podStartSLOduration=1.927146576 podStartE2EDuration="3.825528511s" totalImagesPullingTime="1.898381935s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:20:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:20:56.823218674 +0000 UTC m=+327.883046341" watchObservedRunningTime="2026-09-10 19:20:56.825528511 +0000 UTC m=+327.885356180"
Sep 10 19:21:05.074498 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:21:05.074453    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:05.152700 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:21:05.152666    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-5dd7bdc797-q87x9"]
Sep 10 19:21:05.156223 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:21:05.156204    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5dd7bdc797-q87x9"
Sep 10 19:21:05.158023 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:21:05.158000    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 10 19:21:05.158256 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:21:05.158242    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Sep 10 19:21:05.158582 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:21:05.158559    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-bqx62\""
Sep 10 19:21:05.158690 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:21:05.158604    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 10 19:21:05.158690 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:21:05.158638    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 10 19:21:05.166314 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:21:05.166296    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5dd7bdc797-q87x9"]
Sep 10 19:21:05.172022 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:21:05.172005    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8lzv2\" (UniqueName: \"kubernetes.io/projected/6d6559e0-b731-4c66-aa5b-7dcbb9de04ad-kube-api-access-8lzv2\") pod \"kube-auth-proxy-5dd7bdc797-q87x9\" (UID: \"6d6559e0-b731-4c66-aa5b-7dcbb9de04ad\") " pod="openshift-ingress/kube-auth-proxy-5dd7bdc797-q87x9"
Sep 10 19:21:05.172116 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:21:05.172032    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6d6559e0-b731-4c66-aa5b-7dcbb9de04ad-tls-certs\") pod \"kube-auth-proxy-5dd7bdc797-q87x9\" (UID: \"6d6559e0-b731-4c66-aa5b-7dcbb9de04ad\") " pod="openshift-ingress/kube-auth-proxy-5dd7bdc797-q87x9"
Sep 10 19:21:05.172116 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:21:05.172048    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6d6559e0-b731-4c66-aa5b-7dcbb9de04ad-tmp\") pod \"kube-auth-proxy-5dd7bdc797-q87x9\" (UID: \"6d6559e0-b731-4c66-aa5b-7dcbb9de04ad\") " pod="openshift-ingress/kube-auth-proxy-5dd7bdc797-q87x9"
Sep 10 19:21:05.273500 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:21:05.273461    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8lzv2\" (UniqueName: \"kubernetes.io/projected/6d6559e0-b731-4c66-aa5b-7dcbb9de04ad-kube-api-access-8lzv2\") pod \"kube-auth-proxy-5dd7bdc797-q87x9\" (UID: \"6d6559e0-b731-4c66-aa5b-7dcbb9de04ad\") " pod="openshift-ingress/kube-auth-proxy-5dd7bdc797-q87x9"
Sep 10 19:21:05.273608 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:21:05.273524    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6d6559e0-b731-4c66-aa5b-7dcbb9de04ad-tls-certs\") pod \"kube-auth-proxy-5dd7bdc797-q87x9\" (UID: \"6d6559e0-b731-4c66-aa5b-7dcbb9de04ad\") " pod="openshift-ingress/kube-auth-proxy-5dd7bdc797-q87x9"
Sep 10 19:21:05.273608 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:21:05.273548    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6d6559e0-b731-4c66-aa5b-7dcbb9de04ad-tmp\") pod \"kube-auth-proxy-5dd7bdc797-q87x9\" (UID: \"6d6559e0-b731-4c66-aa5b-7dcbb9de04ad\") " pod="openshift-ingress/kube-auth-proxy-5dd7bdc797-q87x9"
Sep 10 19:21:05.273726 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:21:05.273677    2575 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found
Sep 10 19:21:05.273793 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:21:05.273759    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6d6559e0-b731-4c66-aa5b-7dcbb9de04ad-tls-certs podName:6d6559e0-b731-4c66-aa5b-7dcbb9de04ad nodeName:}" failed. No retries permitted until 2026-09-10 19:21:05.773739213 +0000 UTC m=+336.833566894 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/6d6559e0-b731-4c66-aa5b-7dcbb9de04ad-tls-certs") pod "kube-auth-proxy-5dd7bdc797-q87x9" (UID: "6d6559e0-b731-4c66-aa5b-7dcbb9de04ad") : secret "kube-auth-proxy-tls" not found
Sep 10 19:21:05.276074 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:21:05.276054    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6d6559e0-b731-4c66-aa5b-7dcbb9de04ad-tmp\") pod \"kube-auth-proxy-5dd7bdc797-q87x9\" (UID: \"6d6559e0-b731-4c66-aa5b-7dcbb9de04ad\") " pod="openshift-ingress/kube-auth-proxy-5dd7bdc797-q87x9"
Sep 10 19:21:05.286664 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:21:05.286641    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8lzv2\" (UniqueName: \"kubernetes.io/projected/6d6559e0-b731-4c66-aa5b-7dcbb9de04ad-kube-api-access-8lzv2\") pod \"kube-auth-proxy-5dd7bdc797-q87x9\" (UID: \"6d6559e0-b731-4c66-aa5b-7dcbb9de04ad\") " pod="openshift-ingress/kube-auth-proxy-5dd7bdc797-q87x9"
Sep 10 19:21:05.295878 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:21:05.295853    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:05.360189 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:21:05.360117    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:05.460355 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:21:05.460312    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:05.596978 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:21:05.596949    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:05.717526 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:21:05.717490    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:05.777697 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:21:05.777666    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6d6559e0-b731-4c66-aa5b-7dcbb9de04ad-tls-certs\") pod \"kube-auth-proxy-5dd7bdc797-q87x9\" (UID: \"6d6559e0-b731-4c66-aa5b-7dcbb9de04ad\") " pod="openshift-ingress/kube-auth-proxy-5dd7bdc797-q87x9"
Sep 10 19:21:05.780048 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:21:05.780030    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6d6559e0-b731-4c66-aa5b-7dcbb9de04ad-tls-certs\") pod \"kube-auth-proxy-5dd7bdc797-q87x9\" (UID: \"6d6559e0-b731-4c66-aa5b-7dcbb9de04ad\") " pod="openshift-ingress/kube-auth-proxy-5dd7bdc797-q87x9"
Sep 10 19:21:06.006321 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:21:06.006241    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:06.066193 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:21:06.066160    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5dd7bdc797-q87x9"
Sep 10 19:21:06.341172 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:21:06.341104    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5dd7bdc797-q87x9"]
Sep 10 19:21:06.345071 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:21:06.345045    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d6559e0_b731_4c66_aa5b_7dcbb9de04ad.slice/crio-dcdb9791bc6daaa601dc11303866039ec8cf6234abdc31e8362bea0a7156706d WatchSource:0}: Error finding container dcdb9791bc6daaa601dc11303866039ec8cf6234abdc31e8362bea0a7156706d: Status 404 returned error can't find the container with id dcdb9791bc6daaa601dc11303866039ec8cf6234abdc31e8362bea0a7156706d
Sep 10 19:21:06.402021 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:21:06.401996    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:06.501568 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:21:06.501543    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:06.847229 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:21:06.847192    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5dd7bdc797-q87x9" event={"ID":"6d6559e0-b731-4c66-aa5b-7dcbb9de04ad","Type":"ContainerStarted","Data":"dcdb9791bc6daaa601dc11303866039ec8cf6234abdc31e8362bea0a7156706d"}
Sep 10 19:21:07.494630 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:21:07.494596    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:07.810809 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:21:07.810733    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-b5db5bfb7-hxskb"
Sep 10 19:21:10.879328 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:21:10.879296    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5dd7bdc797-q87x9" event={"ID":"6d6559e0-b731-4c66-aa5b-7dcbb9de04ad","Type":"ContainerStarted","Data":"48534578bbaa3b5efb6beddf531cb31416f0d641eeb73f21354b902a6bc0fb6a"}
Sep 10 19:21:10.894714 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:21:10.894669    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-5dd7bdc797-q87x9" podStartSLOduration=2.300085665 podStartE2EDuration="5.894654198s" totalImagesPullingTime="3.594568533s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:21:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:21:10.893663185 +0000 UTC m=+341.953490853" watchObservedRunningTime="2026-09-10 19:21:10.894654198 +0000 UTC m=+341.954481935"
Sep 10 19:21:13.565535 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:21:13.565498    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:13.674075 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:21:13.674039    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:13.887395 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:21:13.887292    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:14.176715 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:21:14.176636    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:14.277615 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:21:14.277585    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:15.317799 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:21:15.317753    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:15.385278 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:21:15.385247    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-gdpds"]
Sep 10 19:21:15.388637 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:21:15.388617    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-gdpds"
Sep 10 19:21:15.390661 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:21:15.390643    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-tmksj\""
Sep 10 19:21:15.390933 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:21:15.390920    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Sep 10 19:21:15.400322 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:21:15.400301    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-gdpds"]
Sep 10 19:21:15.419563 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:21:15.419536    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:15.456653 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:21:15.456628    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/880bf6ca-872f-4005-b5cc-cd60bdf6caec-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-gdpds\" (UID: \"880bf6ca-872f-4005-b5cc-cd60bdf6caec\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-gdpds"
Sep 10 19:21:15.456808 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:21:15.456696    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sqrrh\" (UniqueName: \"kubernetes.io/projected/880bf6ca-872f-4005-b5cc-cd60bdf6caec-kube-api-access-sqrrh\") pod \"llmisvc-controller-manager-777c6bd7d5-gdpds\" (UID: \"880bf6ca-872f-4005-b5cc-cd60bdf6caec\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-gdpds"
Sep 10 19:21:15.557375 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:21:15.557330    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sqrrh\" (UniqueName: \"kubernetes.io/projected/880bf6ca-872f-4005-b5cc-cd60bdf6caec-kube-api-access-sqrrh\") pod \"llmisvc-controller-manager-777c6bd7d5-gdpds\" (UID: \"880bf6ca-872f-4005-b5cc-cd60bdf6caec\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-gdpds"
Sep 10 19:21:15.557525 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:21:15.557435    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/880bf6ca-872f-4005-b5cc-cd60bdf6caec-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-gdpds\" (UID: \"880bf6ca-872f-4005-b5cc-cd60bdf6caec\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-gdpds"
Sep 10 19:21:15.557574 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:21:15.557538    2575 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found
Sep 10 19:21:15.557617 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:21:15.557596    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/880bf6ca-872f-4005-b5cc-cd60bdf6caec-cert podName:880bf6ca-872f-4005-b5cc-cd60bdf6caec nodeName:}" failed. No retries permitted until 2026-09-10 19:21:16.057580667 +0000 UTC m=+347.117408313 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/880bf6ca-872f-4005-b5cc-cd60bdf6caec-cert") pod "llmisvc-controller-manager-777c6bd7d5-gdpds" (UID: "880bf6ca-872f-4005-b5cc-cd60bdf6caec") : secret "llmisvc-webhook-server-cert" not found
Sep 10 19:21:15.569779 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:21:15.569717    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sqrrh\" (UniqueName: \"kubernetes.io/projected/880bf6ca-872f-4005-b5cc-cd60bdf6caec-kube-api-access-sqrrh\") pod \"llmisvc-controller-manager-777c6bd7d5-gdpds\" (UID: \"880bf6ca-872f-4005-b5cc-cd60bdf6caec\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-gdpds"
Sep 10 19:21:15.716152 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:21:15.716110    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:15.900907 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:21:15.900829    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:16.062691 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:21:16.062656    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/880bf6ca-872f-4005-b5cc-cd60bdf6caec-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-gdpds\" (UID: \"880bf6ca-872f-4005-b5cc-cd60bdf6caec\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-gdpds"
Sep 10 19:21:16.065206 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:21:16.065181    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/880bf6ca-872f-4005-b5cc-cd60bdf6caec-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-gdpds\" (UID: \"880bf6ca-872f-4005-b5cc-cd60bdf6caec\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-gdpds"
Sep 10 19:21:16.301783 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:21:16.301754    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-gdpds"
Sep 10 19:21:16.592236 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:21:16.592165    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-gdpds"]
Sep 10 19:21:16.595405 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:21:16.595378    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod880bf6ca_872f_4005_b5cc_cd60bdf6caec.slice/crio-70044f60ef88a21a4a80fb923147b1c15826ed9f5dd07f0fcfbe9836441ef781 WatchSource:0}: Error finding container 70044f60ef88a21a4a80fb923147b1c15826ed9f5dd07f0fcfbe9836441ef781: Status 404 returned error can't find the container with id 70044f60ef88a21a4a80fb923147b1c15826ed9f5dd07f0fcfbe9836441ef781
Sep 10 19:21:16.600065 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:21:16.600043    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:16.713598 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:21:16.713514    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:16.832420 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:21:16.832385    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:16.906219 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:21:16.906146    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-gdpds" event={"ID":"880bf6ca-872f-4005-b5cc-cd60bdf6caec","Type":"ContainerStarted","Data":"70044f60ef88a21a4a80fb923147b1c15826ed9f5dd07f0fcfbe9836441ef781"}
Sep 10 19:21:19.924000 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:21:19.923910    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-gdpds" event={"ID":"880bf6ca-872f-4005-b5cc-cd60bdf6caec","Type":"ContainerStarted","Data":"b6c68688b83b2b2e30e0ba49e9701c99722c99260b09363c25b10122f0639982"}
Sep 10 19:21:19.924000 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:21:19.923976    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-gdpds"
Sep 10 19:21:19.940250 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:21:19.940203    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-gdpds" podStartSLOduration=2.063652685 podStartE2EDuration="4.940185367s" totalImagesPullingTime="2.876532682s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:21:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:21:19.939680061 +0000 UTC m=+350.999507731" watchObservedRunningTime="2026-09-10 19:21:19.940185367 +0000 UTC m=+351.000013036"
Sep 10 19:21:50.930775 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:21:50.930699    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-gdpds"
Sep 10 19:22:06.019560 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:22:06.019524    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:22:06.085161 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:22:06.085135    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:22:06.402455 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:22:06.402387    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:22:06.495907 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:22:06.495877    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:22:07.871923 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:22:07.871881    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:22:07.972353 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:22:07.972321    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:22:19.530462 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:22:19.530423    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:22:19.660334 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:22:19.660293    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:22:19.737179 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:22:19.737150    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:22:20.270705 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:22:20.270675    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:22:20.371532 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:22:20.371505    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:22:23.138309 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:22:23.138274    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:22:24.129720 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:22:24.129685    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:22:25.132591 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:22:25.132554    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:22:29.617193 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:22:29.617154    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:22:30.501410 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:22:30.501374    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:22:30.604643 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:22:30.604605    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:22:30.702032 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:22:30.701999    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:22:31.062126 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:22:31.062090    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:22:31.746213 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:22:31.746170    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:22:32.407023 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:22:32.406996    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:22:33.462994 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:22:33.462961    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:22:33.557783 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:22:33.557750    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:22:33.646030 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:22:33.645995    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:22:33.942836 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:22:33.942762    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:22:34.992056 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:22:34.992003    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:22:37.113202 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:22:37.113098    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:22:37.254415 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:22:37.254378    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:22:39.554875 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:22:39.554820    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:22:39.657045 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:22:39.657011    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:22:39.947897 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:22:39.947870    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:22:40.049316 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:22:40.049290    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:22:40.346363 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:22:40.346277    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:22:40.420149 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:22:40.420114    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:22:46.080044 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:22:46.080005    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:22:46.179784 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:22:46.179751    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:22:50.770658 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:22:50.770624    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:22:50.875792 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:22:50.875763    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:22:51.108116 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:22:51.108033    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:22:51.213899 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:22:51.213871    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:22:55.644032 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:22:55.643951    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:22:55.758273 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:22:55.758244    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:23:00.805068 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:23:00.805034    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:23:00.903395 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:23:00.903363    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:23:01.176651 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:23:01.176558    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:23:01.276153 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:23:01.276121    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:23:01.375087 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:23:01.375058    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:23:01.476404 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:23:01.476384    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:23:07.432389 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:23:07.432336    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:23:07.534388 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:23:07.534330    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:23:12.171864 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:23:12.171834    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:23:12.278506 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:23:12.278478    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:23:16.990027 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:23:16.989977    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:23:17.094107 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:23:17.094080    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:23:20.564006 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:23:20.563973    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:23:20.910140 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:23:20.910053    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:23:22.635388 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:23:22.635356    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:23:22.801796 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:23:22.801764    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:23:22.899619 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:23:22.899529    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:23:23.056921 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:23:23.056898    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qr7rl"]
Sep 10 19:23:23.060040 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:23:23.060025    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-84994f7ffc-qr7rl"
Sep 10 19:23:23.062716 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:23:23.062697    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 10 19:23:23.063261 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:23:23.063218    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 10 19:23:23.063384 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:23:23.063270    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-6txr8\""
Sep 10 19:23:23.063451 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:23:23.063409    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Sep 10 19:23:23.071228 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:23:23.071205    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qr7rl"]
Sep 10 19:23:23.099909 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:23:23.099880    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qr7rl"]
Sep 10 19:23:23.186550 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:23:23.186498    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dlmvg\" (UniqueName: \"kubernetes.io/projected/d522d914-4f36-4f67-ae2c-43fb14a69285-kube-api-access-dlmvg\") pod \"limitador-limitador-84994f7ffc-qr7rl\" (UID: \"d522d914-4f36-4f67-ae2c-43fb14a69285\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-qr7rl"
Sep 10 19:23:23.186550 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:23:23.186538    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/d522d914-4f36-4f67-ae2c-43fb14a69285-config-file\") pod \"limitador-limitador-84994f7ffc-qr7rl\" (UID: \"d522d914-4f36-4f67-ae2c-43fb14a69285\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-qr7rl"
Sep 10 19:23:23.287513 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:23:23.287490    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dlmvg\" (UniqueName: \"kubernetes.io/projected/d522d914-4f36-4f67-ae2c-43fb14a69285-kube-api-access-dlmvg\") pod \"limitador-limitador-84994f7ffc-qr7rl\" (UID: \"d522d914-4f36-4f67-ae2c-43fb14a69285\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-qr7rl"
Sep 10 19:23:23.287600 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:23:23.287528    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/d522d914-4f36-4f67-ae2c-43fb14a69285-config-file\") pod \"limitador-limitador-84994f7ffc-qr7rl\" (UID: \"d522d914-4f36-4f67-ae2c-43fb14a69285\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-qr7rl"
Sep 10 19:23:23.288055 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:23:23.288039    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/d522d914-4f36-4f67-ae2c-43fb14a69285-config-file\") pod \"limitador-limitador-84994f7ffc-qr7rl\" (UID: \"d522d914-4f36-4f67-ae2c-43fb14a69285\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-qr7rl"
Sep 10 19:23:23.296804 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:23:23.296776    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dlmvg\" (UniqueName: \"kubernetes.io/projected/d522d914-4f36-4f67-ae2c-43fb14a69285-kube-api-access-dlmvg\") pod \"limitador-limitador-84994f7ffc-qr7rl\" (UID: \"d522d914-4f36-4f67-ae2c-43fb14a69285\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-qr7rl"
Sep 10 19:23:23.375603 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:23:23.375582    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-84994f7ffc-qr7rl"
Sep 10 19:23:23.553829 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:23:23.553802    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:23:23.641768 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:23:23.641737    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:23:23.645255 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:23:23.645230    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-59c974bd48-mhwd7"]
Sep 10 19:23:23.649603 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:23:23.649584    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-mhwd7"
Sep 10 19:23:23.651774 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:23:23.651754    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-vj929\""
Sep 10 19:23:23.652105 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:23:23.652088    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qr7rl"]
Sep 10 19:23:23.655558 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:23:23.655529    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd522d914_4f36_4f67_ae2c_43fb14a69285.slice/crio-60aed9b81bda9e6be55423a25dfb6e95d9b099fd4edeb149b9d146e605821a2a WatchSource:0}: Error finding container 60aed9b81bda9e6be55423a25dfb6e95d9b099fd4edeb149b9d146e605821a2a: Status 404 returned error can't find the container with id 60aed9b81bda9e6be55423a25dfb6e95d9b099fd4edeb149b9d146e605821a2a
Sep 10 19:23:23.655558 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:23:23.655535    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-mhwd7"]
Sep 10 19:23:23.791318 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:23:23.791292    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6fpn5\" (UniqueName: \"kubernetes.io/projected/bd642f0e-bae0-4893-9269-c8f48187066e-kube-api-access-6fpn5\") pod \"authorino-59c974bd48-mhwd7\" (UID: \"bd642f0e-bae0-4893-9269-c8f48187066e\") " pod="kuadrant-system/authorino-59c974bd48-mhwd7"
Sep 10 19:23:23.891990 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:23:23.891924    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6fpn5\" (UniqueName: \"kubernetes.io/projected/bd642f0e-bae0-4893-9269-c8f48187066e-kube-api-access-6fpn5\") pod \"authorino-59c974bd48-mhwd7\" (UID: \"bd642f0e-bae0-4893-9269-c8f48187066e\") " pod="kuadrant-system/authorino-59c974bd48-mhwd7"
Sep 10 19:23:23.900841 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:23:23.900820    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6fpn5\" (UniqueName: \"kubernetes.io/projected/bd642f0e-bae0-4893-9269-c8f48187066e-kube-api-access-6fpn5\") pod \"authorino-59c974bd48-mhwd7\" (UID: \"bd642f0e-bae0-4893-9269-c8f48187066e\") " pod="kuadrant-system/authorino-59c974bd48-mhwd7"
Sep 10 19:23:23.967318 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:23:23.967294    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-mhwd7"
Sep 10 19:23:24.234472 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:23:24.234440    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-mhwd7"]
Sep 10 19:23:24.238441 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:23:24.238409    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbd642f0e_bae0_4893_9269_c8f48187066e.slice/crio-d40406e00c22f65ed837d11c04dfc0463962c8a24c6c92b6583b340219217abb WatchSource:0}: Error finding container d40406e00c22f65ed837d11c04dfc0463962c8a24c6c92b6583b340219217abb: Status 404 returned error can't find the container with id d40406e00c22f65ed837d11c04dfc0463962c8a24c6c92b6583b340219217abb
Sep 10 19:23:24.471629 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:23:24.471576    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-mhwd7" event={"ID":"bd642f0e-bae0-4893-9269-c8f48187066e","Type":"ContainerStarted","Data":"d40406e00c22f65ed837d11c04dfc0463962c8a24c6c92b6583b340219217abb"}
Sep 10 19:23:24.473012 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:23:24.472982    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-84994f7ffc-qr7rl" event={"ID":"d522d914-4f36-4f67-ae2c-43fb14a69285","Type":"ContainerStarted","Data":"60aed9b81bda9e6be55423a25dfb6e95d9b099fd4edeb149b9d146e605821a2a"}
Sep 10 19:23:26.483734 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:23:26.483699    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-84994f7ffc-qr7rl" event={"ID":"d522d914-4f36-4f67-ae2c-43fb14a69285","Type":"ContainerStarted","Data":"f1417aaa7dc49bb11d8c329e6875d3cfcca2b79d6992acd67d21d34e4b81cab0"}
Sep 10 19:23:26.484154 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:23:26.483816    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-84994f7ffc-qr7rl"
Sep 10 19:23:26.504570 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:23:26.504521    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-84994f7ffc-qr7rl" podStartSLOduration=1.296033049 podStartE2EDuration="3.504506834s" totalImagesPullingTime="2.208473785s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:23:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:23:26.501880285 +0000 UTC m=+477.561707954" watchObservedRunningTime="2026-09-10 19:23:26.504506834 +0000 UTC m=+477.564334501"
Sep 10 19:23:27.489395 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:23:27.489325    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-mhwd7" event={"ID":"bd642f0e-bae0-4893-9269-c8f48187066e","Type":"ContainerStarted","Data":"be948fb599d5f566268f351931dd914203aaa46aacc0c919a35fbaf2f0c1abd0"}
Sep 10 19:23:27.505027 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:23:27.504984    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-59c974bd48-mhwd7" podStartSLOduration=1.948463552 podStartE2EDuration="4.504965336s" totalImagesPullingTime="2.556501784s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:23:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:23:27.503549618 +0000 UTC m=+478.563377286" watchObservedRunningTime="2026-09-10 19:23:27.504965336 +0000 UTC m=+478.564793003"
Sep 10 19:23:37.490239 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:23:37.490209    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-84994f7ffc-qr7rl"
Sep 10 19:23:38.733663 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:23:38.733607    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:23:38.802870 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:23:38.802841    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:23:39.186697 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:23:39.186618    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:23:49.113590 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:23:49.113527    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:23:49.599985 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:23:49.599953    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:23:51.882693 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:23:51.882657    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-mhwd7"]
Sep 10 19:23:51.883050 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:23:51.882838    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-59c974bd48-mhwd7" podUID="bd642f0e-bae0-4893-9269-c8f48187066e" containerName="authorino" containerID="cri-o://be948fb599d5f566268f351931dd914203aaa46aacc0c919a35fbaf2f0c1abd0" gracePeriod=30
Sep 10 19:23:52.118029 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:23:52.118004    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-mhwd7"
Sep 10 19:23:52.202712 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:23:52.202689    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6fpn5\" (UniqueName: \"kubernetes.io/projected/bd642f0e-bae0-4893-9269-c8f48187066e-kube-api-access-6fpn5\") pod \"bd642f0e-bae0-4893-9269-c8f48187066e\" (UID: \"bd642f0e-bae0-4893-9269-c8f48187066e\") "
Sep 10 19:23:52.204775 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:23:52.204753    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd642f0e-bae0-4893-9269-c8f48187066e-kube-api-access-6fpn5" (OuterVolumeSpecName: "kube-api-access-6fpn5") pod "bd642f0e-bae0-4893-9269-c8f48187066e" (UID: "bd642f0e-bae0-4893-9269-c8f48187066e"). InnerVolumeSpecName "kube-api-access-6fpn5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:23:52.303323 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:23:52.303296    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6fpn5\" (UniqueName: \"kubernetes.io/projected/bd642f0e-bae0-4893-9269-c8f48187066e-kube-api-access-6fpn5\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:23:52.600083 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:23:52.600013    2575 generic.go:356] "Generic (PLEG): container finished" podID="bd642f0e-bae0-4893-9269-c8f48187066e" containerID="be948fb599d5f566268f351931dd914203aaa46aacc0c919a35fbaf2f0c1abd0" exitCode=0
Sep 10 19:23:52.600083 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:23:52.600056    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-mhwd7"
Sep 10 19:23:52.600253 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:23:52.600097    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-mhwd7" event={"ID":"bd642f0e-bae0-4893-9269-c8f48187066e","Type":"ContainerDied","Data":"be948fb599d5f566268f351931dd914203aaa46aacc0c919a35fbaf2f0c1abd0"}
Sep 10 19:23:52.600253 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:23:52.600135    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-mhwd7" event={"ID":"bd642f0e-bae0-4893-9269-c8f48187066e","Type":"ContainerDied","Data":"d40406e00c22f65ed837d11c04dfc0463962c8a24c6c92b6583b340219217abb"}
Sep 10 19:23:52.600253 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:23:52.600153    2575 scope.go:117] "RemoveContainer" containerID="be948fb599d5f566268f351931dd914203aaa46aacc0c919a35fbaf2f0c1abd0"
Sep 10 19:23:52.608608 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:23:52.608590    2575 scope.go:117] "RemoveContainer" containerID="be948fb599d5f566268f351931dd914203aaa46aacc0c919a35fbaf2f0c1abd0"
Sep 10 19:23:52.608868 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:23:52.608840    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"be948fb599d5f566268f351931dd914203aaa46aacc0c919a35fbaf2f0c1abd0\": container with ID starting with be948fb599d5f566268f351931dd914203aaa46aacc0c919a35fbaf2f0c1abd0 not found: ID does not exist" containerID="be948fb599d5f566268f351931dd914203aaa46aacc0c919a35fbaf2f0c1abd0"
Sep 10 19:23:52.608917 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:23:52.608875    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"be948fb599d5f566268f351931dd914203aaa46aacc0c919a35fbaf2f0c1abd0"} err="failed to get container status \"be948fb599d5f566268f351931dd914203aaa46aacc0c919a35fbaf2f0c1abd0\": rpc error: code = NotFound desc = could not find container \"be948fb599d5f566268f351931dd914203aaa46aacc0c919a35fbaf2f0c1abd0\": container with ID starting with be948fb599d5f566268f351931dd914203aaa46aacc0c919a35fbaf2f0c1abd0 not found: ID does not exist"
Sep 10 19:23:52.620245 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:23:52.620224    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-mhwd7"]
Sep 10 19:23:52.624319 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:23:52.624296    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-59c974bd48-mhwd7"]
Sep 10 19:23:53.384035 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:23:53.384001    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bd642f0e-bae0-4893-9269-c8f48187066e" path="/var/lib/kubelet/pods/bd642f0e-bae0-4893-9269-c8f48187066e/volumes"
Sep 10 19:23:56.553690 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:23:56.553636    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:23:56.636160 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:23:56.636130    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:23:56.709801 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:23:56.709768    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:23:56.888227 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:23:56.888143    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:23:57.190324 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:23:57.190248    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:24:00.671378 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:24:00.671331    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:24:00.831944 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:24:00.831913    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:24:12.661843 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:24:12.661806    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:24:32.500703 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:24:32.500660    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:24:32.591850 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:24:32.591822    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:24:34.000808 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:24:34.000770    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:24:34.093670 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:24:34.093638    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:24:34.151676 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:24:34.151642    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:24:34.352864 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:24:34.352787    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:24:34.783907 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:24:34.783868    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:24:34.880106 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:24:34.880081    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:24:34.890524 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:24:34.890500    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qr7rl"]
Sep 10 19:24:34.978517 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:24:34.978486    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-flfb7"]
Sep 10 19:24:34.978872 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:24:34.978856    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd642f0e-bae0-4893-9269-c8f48187066e" containerName="authorino"
Sep 10 19:24:34.978955 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:24:34.978875    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd642f0e-bae0-4893-9269-c8f48187066e" containerName="authorino"
Sep 10 19:24:34.979009 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:24:34.978989    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd642f0e-bae0-4893-9269-c8f48187066e" containerName="authorino"
Sep 10 19:24:34.983210 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:24:34.983191    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-flfb7"
Sep 10 19:24:34.985096 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:24:34.985068    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Sep 10 19:24:34.985096 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:24:34.985073    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-v6525\""
Sep 10 19:24:34.991172 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:24:34.991150    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-flfb7"]
Sep 10 19:24:35.128218 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:24:35.128150    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8924df89-f4a8-4d61-a636-066edf4dc4f0-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-flfb7\" (UID: \"8924df89-f4a8-4d61-a636-066edf4dc4f0\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-flfb7"
Sep 10 19:24:35.128218 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:24:35.128187    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jtrz9\" (UniqueName: \"kubernetes.io/projected/8924df89-f4a8-4d61-a636-066edf4dc4f0-kube-api-access-jtrz9\") pod \"payload-pre-processing-59fbc47bf8-flfb7\" (UID: \"8924df89-f4a8-4d61-a636-066edf4dc4f0\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-flfb7"
Sep 10 19:24:35.229136 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:24:35.229106    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8924df89-f4a8-4d61-a636-066edf4dc4f0-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-flfb7\" (UID: \"8924df89-f4a8-4d61-a636-066edf4dc4f0\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-flfb7"
Sep 10 19:24:35.229258 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:24:35.229141    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jtrz9\" (UniqueName: \"kubernetes.io/projected/8924df89-f4a8-4d61-a636-066edf4dc4f0-kube-api-access-jtrz9\") pod \"payload-pre-processing-59fbc47bf8-flfb7\" (UID: \"8924df89-f4a8-4d61-a636-066edf4dc4f0\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-flfb7"
Sep 10 19:24:35.229807 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:24:35.229790    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8924df89-f4a8-4d61-a636-066edf4dc4f0-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-flfb7\" (UID: \"8924df89-f4a8-4d61-a636-066edf4dc4f0\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-flfb7"
Sep 10 19:24:35.236907 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:24:35.236880    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jtrz9\" (UniqueName: \"kubernetes.io/projected/8924df89-f4a8-4d61-a636-066edf4dc4f0-kube-api-access-jtrz9\") pod \"payload-pre-processing-59fbc47bf8-flfb7\" (UID: \"8924df89-f4a8-4d61-a636-066edf4dc4f0\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-flfb7"
Sep 10 19:24:35.295011 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:24:35.294987    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-flfb7"
Sep 10 19:24:35.564940 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:24:35.564887    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-flfb7"]
Sep 10 19:24:35.568313 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:24:35.568286    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8924df89_f4a8_4d61_a636_066edf4dc4f0.slice/crio-67df01c59a3052b79ada806dbe0b6706bda199b435d9a001c36eeaa1fba7e86d WatchSource:0}: Error finding container 67df01c59a3052b79ada806dbe0b6706bda199b435d9a001c36eeaa1fba7e86d: Status 404 returned error can't find the container with id 67df01c59a3052b79ada806dbe0b6706bda199b435d9a001c36eeaa1fba7e86d
Sep 10 19:24:35.791180 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:24:35.791149    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-flfb7" event={"ID":"8924df89-f4a8-4d61-a636-066edf4dc4f0","Type":"ContainerStarted","Data":"67df01c59a3052b79ada806dbe0b6706bda199b435d9a001c36eeaa1fba7e86d"}
Sep 10 19:24:38.809098 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:24:38.809064    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-flfb7" event={"ID":"8924df89-f4a8-4d61-a636-066edf4dc4f0","Type":"ContainerStarted","Data":"c994c41266e8d2bc7c28d0fdb72cbbb7c69d5b315583f3416130e03ab054fe4a"}
Sep 10 19:24:38.809512 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:24:38.809200    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-flfb7"
Sep 10 19:24:38.825898 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:24:38.825855    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-flfb7" podStartSLOduration=2.336311783 podStartE2EDuration="4.825841365s" totalImagesPullingTime="2.489529582s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:24:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:24:38.824851515 +0000 UTC m=+549.884679183" watchObservedRunningTime="2026-09-10 19:24:38.825841365 +0000 UTC m=+549.885669032"
Sep 10 19:24:39.466984 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:24:39.466952    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:24:39.567736 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:24:39.567707    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:24:45.120685 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:24:45.120650    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:24:45.198025 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:24:45.197995    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:24:49.815510 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:24:49.815477    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-flfb7"
Sep 10 19:25:00.075981 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:25:00.075942    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:25:02.930131 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:25:02.930098    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:25:03.454635 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:25:03.454576    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qr7rl"]
Sep 10 19:25:16.005002 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:25:16.004965    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:25:16.649223 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:25:16.649186    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qr7rl"]
Sep 10 19:25:29.338187 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:25:29.338157    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rrxfj_c7511dfe-c588-4be1-bd16-1dce34740809/ovn-acl-logging/0.log"
Sep 10 19:25:29.339510 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:25:29.339488    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rrxfj_c7511dfe-c588-4be1-bd16-1dce34740809/ovn-acl-logging/0.log"
Sep 10 19:25:31.790385 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:25:31.790355    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-97zkh"]
Sep 10 19:25:31.793298 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:25:31.793279    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-97zkh"
Sep 10 19:25:31.795957 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:25:31.795937    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 10 19:25:31.796067 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:25:31.795970    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 10 19:25:31.796067 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:25:31.796011    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-embedding-simulated-kserve-self-signed-certs\""
Sep 10 19:25:31.796180 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:25:31.796154    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-6nnj8\""
Sep 10 19:25:31.805928 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:25:31.805905    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-97zkh"]
Sep 10 19:25:31.829745 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:25:31.829715    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:25:31.865685 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:25:31.865661    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3fb3804c-78bd-4040-b2d3-a2bff8640f5e-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-97zkh\" (UID: \"3fb3804c-78bd-4040-b2d3-a2bff8640f5e\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-97zkh"
Sep 10 19:25:31.865776 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:25:31.865698    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3fb3804c-78bd-4040-b2d3-a2bff8640f5e-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-97zkh\" (UID: \"3fb3804c-78bd-4040-b2d3-a2bff8640f5e\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-97zkh"
Sep 10 19:25:31.865776 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:25:31.865726    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3fb3804c-78bd-4040-b2d3-a2bff8640f5e-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-97zkh\" (UID: \"3fb3804c-78bd-4040-b2d3-a2bff8640f5e\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-97zkh"
Sep 10 19:25:31.865776 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:25:31.865760    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3fb3804c-78bd-4040-b2d3-a2bff8640f5e-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-97zkh\" (UID: \"3fb3804c-78bd-4040-b2d3-a2bff8640f5e\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-97zkh"
Sep 10 19:25:31.865932 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:25:31.865788    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3fb3804c-78bd-4040-b2d3-a2bff8640f5e-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-97zkh\" (UID: \"3fb3804c-78bd-4040-b2d3-a2bff8640f5e\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-97zkh"
Sep 10 19:25:31.865932 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:25:31.865811    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z5kd6\" (UniqueName: \"kubernetes.io/projected/3fb3804c-78bd-4040-b2d3-a2bff8640f5e-kube-api-access-z5kd6\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-97zkh\" (UID: \"3fb3804c-78bd-4040-b2d3-a2bff8640f5e\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-97zkh"
Sep 10 19:25:31.966963 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:25:31.966940    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3fb3804c-78bd-4040-b2d3-a2bff8640f5e-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-97zkh\" (UID: \"3fb3804c-78bd-4040-b2d3-a2bff8640f5e\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-97zkh"
Sep 10 19:25:31.967063 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:25:31.966970    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3fb3804c-78bd-4040-b2d3-a2bff8640f5e-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-97zkh\" (UID: \"3fb3804c-78bd-4040-b2d3-a2bff8640f5e\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-97zkh"
Sep 10 19:25:31.967063 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:25:31.966993    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3fb3804c-78bd-4040-b2d3-a2bff8640f5e-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-97zkh\" (UID: \"3fb3804c-78bd-4040-b2d3-a2bff8640f5e\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-97zkh"
Sep 10 19:25:31.967063 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:25:31.967011    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3fb3804c-78bd-4040-b2d3-a2bff8640f5e-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-97zkh\" (UID: \"3fb3804c-78bd-4040-b2d3-a2bff8640f5e\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-97zkh"
Sep 10 19:25:31.967063 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:25:31.967042    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3fb3804c-78bd-4040-b2d3-a2bff8640f5e-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-97zkh\" (UID: \"3fb3804c-78bd-4040-b2d3-a2bff8640f5e\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-97zkh"
Sep 10 19:25:31.967249 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:25:31.967063    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z5kd6\" (UniqueName: \"kubernetes.io/projected/3fb3804c-78bd-4040-b2d3-a2bff8640f5e-kube-api-access-z5kd6\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-97zkh\" (UID: \"3fb3804c-78bd-4040-b2d3-a2bff8640f5e\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-97zkh"
Sep 10 19:25:31.967432 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:25:31.967411    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3fb3804c-78bd-4040-b2d3-a2bff8640f5e-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-97zkh\" (UID: \"3fb3804c-78bd-4040-b2d3-a2bff8640f5e\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-97zkh"
Sep 10 19:25:31.967494 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:25:31.967435    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3fb3804c-78bd-4040-b2d3-a2bff8640f5e-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-97zkh\" (UID: \"3fb3804c-78bd-4040-b2d3-a2bff8640f5e\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-97zkh"
Sep 10 19:25:31.967540 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:25:31.967489    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3fb3804c-78bd-4040-b2d3-a2bff8640f5e-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-97zkh\" (UID: \"3fb3804c-78bd-4040-b2d3-a2bff8640f5e\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-97zkh"
Sep 10 19:25:31.969265 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:25:31.969244    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3fb3804c-78bd-4040-b2d3-a2bff8640f5e-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-97zkh\" (UID: \"3fb3804c-78bd-4040-b2d3-a2bff8640f5e\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-97zkh"
Sep 10 19:25:31.969553 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:25:31.969535    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3fb3804c-78bd-4040-b2d3-a2bff8640f5e-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-97zkh\" (UID: \"3fb3804c-78bd-4040-b2d3-a2bff8640f5e\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-97zkh"
Sep 10 19:25:31.975221 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:25:31.975203    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z5kd6\" (UniqueName: \"kubernetes.io/projected/3fb3804c-78bd-4040-b2d3-a2bff8640f5e-kube-api-access-z5kd6\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-97zkh\" (UID: \"3fb3804c-78bd-4040-b2d3-a2bff8640f5e\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-97zkh"
Sep 10 19:25:32.103492 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:25:32.103439    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-97zkh"
Sep 10 19:25:32.374883 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:25:32.374863    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-97zkh"]
Sep 10 19:25:32.377506 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:25:32.377476    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3fb3804c_78bd_4040_b2d3_a2bff8640f5e.slice/crio-9aef75f080716727e5ba8c2145e14f959cff35e099b810d14335d343d86dbaaf WatchSource:0}: Error finding container 9aef75f080716727e5ba8c2145e14f959cff35e099b810d14335d343d86dbaaf: Status 404 returned error can't find the container with id 9aef75f080716727e5ba8c2145e14f959cff35e099b810d14335d343d86dbaaf
Sep 10 19:25:32.753723 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:25:32.753667    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qr7rl"]
Sep 10 19:25:33.056253 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:25:33.056186    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-97zkh" event={"ID":"3fb3804c-78bd-4040-b2d3-a2bff8640f5e","Type":"ContainerStarted","Data":"9aef75f080716727e5ba8c2145e14f959cff35e099b810d14335d343d86dbaaf"}
Sep 10 19:25:34.062412 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:25:34.062378    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-97zkh" event={"ID":"3fb3804c-78bd-4040-b2d3-a2bff8640f5e","Type":"ContainerStarted","Data":"1053710e0a170706de70be3cb74031663207aaf6dd23cfb56744484b24db4b88"}
Sep 10 19:25:34.082186 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:25:34.082137    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-97zkh" podStartSLOduration=2.044484638 podStartE2EDuration="3.082118846s" totalImagesPullingTime="1.037634208s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:25:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:25:34.081479902 +0000 UTC m=+605.141307569" watchObservedRunningTime="2026-09-10 19:25:34.082118846 +0000 UTC m=+605.141946517"
Sep 10 19:25:40.416640 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:25:40.416588    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:25:41.651991 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:25:41.651944    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qr7rl"]
Sep 10 19:25:42.104479 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:25:42.104444    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-97zkh"
Sep 10 19:25:42.104642 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:25:42.104495    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-97zkh"
Sep 10 19:25:42.116943 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:25:42.116924    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-97zkh"
Sep 10 19:25:43.112681 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:25:43.112652    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-97zkh"
Sep 10 19:25:54.209290 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:25:54.209202    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:25:55.769748 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:25:55.769716    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qr7rl"]
Sep 10 19:26:11.036429 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:26:11.036388    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:26:12.849321 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:26:12.849290    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qr7rl"]
Sep 10 19:26:18.270037 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:26:18.270003    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-gfvhc"]
Sep 10 19:26:18.276141 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:26:18.276122    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gfvhc"
Sep 10 19:26:18.278438 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:26:18.278416    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Sep 10 19:26:18.285800 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:26:18.285773    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-gfvhc"]
Sep 10 19:26:18.321555 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:26:18.321527    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:26:18.331483 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:26:18.331461    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a3a1aee4-d4ce-498c-be27-a845c4ef18ac-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gfvhc\" (UID: \"a3a1aee4-d4ce-498c-be27-a845c4ef18ac\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gfvhc"
Sep 10 19:26:18.331585 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:26:18.331541    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a3a1aee4-d4ce-498c-be27-a845c4ef18ac-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gfvhc\" (UID: \"a3a1aee4-d4ce-498c-be27-a845c4ef18ac\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gfvhc"
Sep 10 19:26:18.331585 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:26:18.331578    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a3a1aee4-d4ce-498c-be27-a845c4ef18ac-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gfvhc\" (UID: \"a3a1aee4-d4ce-498c-be27-a845c4ef18ac\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gfvhc"
Sep 10 19:26:18.331692 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:26:18.331613    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a3a1aee4-d4ce-498c-be27-a845c4ef18ac-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gfvhc\" (UID: \"a3a1aee4-d4ce-498c-be27-a845c4ef18ac\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gfvhc"
Sep 10 19:26:18.331692 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:26:18.331662    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a3a1aee4-d4ce-498c-be27-a845c4ef18ac-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gfvhc\" (UID: \"a3a1aee4-d4ce-498c-be27-a845c4ef18ac\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gfvhc"
Sep 10 19:26:18.331757 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:26:18.331730    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vgm6n\" (UniqueName: \"kubernetes.io/projected/a3a1aee4-d4ce-498c-be27-a845c4ef18ac-kube-api-access-vgm6n\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gfvhc\" (UID: \"a3a1aee4-d4ce-498c-be27-a845c4ef18ac\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gfvhc"
Sep 10 19:26:18.432560 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:26:18.432535    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a3a1aee4-d4ce-498c-be27-a845c4ef18ac-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gfvhc\" (UID: \"a3a1aee4-d4ce-498c-be27-a845c4ef18ac\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gfvhc"
Sep 10 19:26:18.432679 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:26:18.432567    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vgm6n\" (UniqueName: \"kubernetes.io/projected/a3a1aee4-d4ce-498c-be27-a845c4ef18ac-kube-api-access-vgm6n\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gfvhc\" (UID: \"a3a1aee4-d4ce-498c-be27-a845c4ef18ac\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gfvhc"
Sep 10 19:26:18.432679 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:26:18.432590    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a3a1aee4-d4ce-498c-be27-a845c4ef18ac-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gfvhc\" (UID: \"a3a1aee4-d4ce-498c-be27-a845c4ef18ac\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gfvhc"
Sep 10 19:26:18.432679 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:26:18.432626    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a3a1aee4-d4ce-498c-be27-a845c4ef18ac-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gfvhc\" (UID: \"a3a1aee4-d4ce-498c-be27-a845c4ef18ac\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gfvhc"
Sep 10 19:26:18.432829 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:26:18.432736    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a3a1aee4-d4ce-498c-be27-a845c4ef18ac-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gfvhc\" (UID: \"a3a1aee4-d4ce-498c-be27-a845c4ef18ac\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gfvhc"
Sep 10 19:26:18.432829 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:26:18.432783    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a3a1aee4-d4ce-498c-be27-a845c4ef18ac-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gfvhc\" (UID: \"a3a1aee4-d4ce-498c-be27-a845c4ef18ac\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gfvhc"
Sep 10 19:26:18.432937 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:26:18.432910    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a3a1aee4-d4ce-498c-be27-a845c4ef18ac-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gfvhc\" (UID: \"a3a1aee4-d4ce-498c-be27-a845c4ef18ac\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gfvhc"
Sep 10 19:26:18.433064 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:26:18.433038    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a3a1aee4-d4ce-498c-be27-a845c4ef18ac-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gfvhc\" (UID: \"a3a1aee4-d4ce-498c-be27-a845c4ef18ac\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gfvhc"
Sep 10 19:26:18.433064 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:26:18.433051    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a3a1aee4-d4ce-498c-be27-a845c4ef18ac-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gfvhc\" (UID: \"a3a1aee4-d4ce-498c-be27-a845c4ef18ac\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gfvhc"
Sep 10 19:26:18.434942 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:26:18.434923    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a3a1aee4-d4ce-498c-be27-a845c4ef18ac-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gfvhc\" (UID: \"a3a1aee4-d4ce-498c-be27-a845c4ef18ac\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gfvhc"
Sep 10 19:26:18.435194 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:26:18.435176    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a3a1aee4-d4ce-498c-be27-a845c4ef18ac-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gfvhc\" (UID: \"a3a1aee4-d4ce-498c-be27-a845c4ef18ac\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gfvhc"
Sep 10 19:26:18.440441 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:26:18.440419    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vgm6n\" (UniqueName: \"kubernetes.io/projected/a3a1aee4-d4ce-498c-be27-a845c4ef18ac-kube-api-access-vgm6n\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gfvhc\" (UID: \"a3a1aee4-d4ce-498c-be27-a845c4ef18ac\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gfvhc"
Sep 10 19:26:18.585980 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:26:18.585912    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gfvhc"
Sep 10 19:26:18.860182 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:26:18.860160    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-gfvhc"]
Sep 10 19:26:18.862846 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:26:18.862808    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda3a1aee4_d4ce_498c_be27_a845c4ef18ac.slice/crio-9db533f8ad222cb1c990160e57409b0eace3e526756b548ec2bb1379288bb7cf WatchSource:0}: Error finding container 9db533f8ad222cb1c990160e57409b0eace3e526756b548ec2bb1379288bb7cf: Status 404 returned error can't find the container with id 9db533f8ad222cb1c990160e57409b0eace3e526756b548ec2bb1379288bb7cf
Sep 10 19:26:18.864153 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:26:18.864134    2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 19:26:19.260737 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:26:19.260701    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gfvhc" event={"ID":"a3a1aee4-d4ce-498c-be27-a845c4ef18ac","Type":"ContainerStarted","Data":"2f771b7d09aaaaf61dee10940e3c8d10de21fed3b7646728e1519e0f065049cc"}
Sep 10 19:26:19.261067 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:26:19.260750    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gfvhc" event={"ID":"a3a1aee4-d4ce-498c-be27-a845c4ef18ac","Type":"ContainerStarted","Data":"9db533f8ad222cb1c990160e57409b0eace3e526756b548ec2bb1379288bb7cf"}
Sep 10 19:26:19.278938 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:26:19.278888    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gfvhc" podStartSLOduration=1.07593595 podStartE2EDuration="1.278868489s" totalImagesPullingTime="202.932539ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:26:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:26:19.277237749 +0000 UTC m=+650.337065416" watchObservedRunningTime="2026-09-10 19:26:19.278868489 +0000 UTC m=+650.338696159"
Sep 10 19:26:20.190846 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:26:20.190811    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:26:20.256975 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:26:20.256948    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qr7rl"]
Sep 10 19:26:28.586455 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:26:28.586429    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gfvhc"
Sep 10 19:26:28.586947 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:26:28.586465    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gfvhc"
Sep 10 19:26:28.599283 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:26:28.599260    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gfvhc"
Sep 10 19:26:29.315905 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:26:29.315877    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gfvhc"
Sep 10 19:26:31.002571 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:26:31.002530    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:26:35.769326 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:26:35.769292    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:27:14.345647 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:27:14.345608    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:27:14.414923 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:27:14.414895    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:27:21.745460 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:27:21.745413    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-865f7bbd6c-psgbr"]
Sep 10 19:27:21.749086 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:27:21.749064    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-865f7bbd6c-psgbr"
Sep 10 19:27:21.751660 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:27:21.751637    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Sep 10 19:27:21.751773 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:27:21.751637    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-vj929\""
Sep 10 19:27:21.760150 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:27:21.760129    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-865f7bbd6c-psgbr"]
Sep 10 19:27:21.840485 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:27:21.840459    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t26pz\" (UniqueName: \"kubernetes.io/projected/1c780b1f-1a09-4477-a61f-5dc520974c3e-kube-api-access-t26pz\") pod \"authorino-865f7bbd6c-psgbr\" (UID: \"1c780b1f-1a09-4477-a61f-5dc520974c3e\") " pod="kuadrant-system/authorino-865f7bbd6c-psgbr"
Sep 10 19:27:21.840607 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:27:21.840504    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1c780b1f-1a09-4477-a61f-5dc520974c3e-tls-cert\") pod \"authorino-865f7bbd6c-psgbr\" (UID: \"1c780b1f-1a09-4477-a61f-5dc520974c3e\") " pod="kuadrant-system/authorino-865f7bbd6c-psgbr"
Sep 10 19:27:21.941392 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:27:21.941313    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1c780b1f-1a09-4477-a61f-5dc520974c3e-tls-cert\") pod \"authorino-865f7bbd6c-psgbr\" (UID: \"1c780b1f-1a09-4477-a61f-5dc520974c3e\") " pod="kuadrant-system/authorino-865f7bbd6c-psgbr"
Sep 10 19:27:21.941495 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:27:21.941437    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t26pz\" (UniqueName: \"kubernetes.io/projected/1c780b1f-1a09-4477-a61f-5dc520974c3e-kube-api-access-t26pz\") pod \"authorino-865f7bbd6c-psgbr\" (UID: \"1c780b1f-1a09-4477-a61f-5dc520974c3e\") " pod="kuadrant-system/authorino-865f7bbd6c-psgbr"
Sep 10 19:27:21.943705 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:27:21.943680    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1c780b1f-1a09-4477-a61f-5dc520974c3e-tls-cert\") pod \"authorino-865f7bbd6c-psgbr\" (UID: \"1c780b1f-1a09-4477-a61f-5dc520974c3e\") " pod="kuadrant-system/authorino-865f7bbd6c-psgbr"
Sep 10 19:27:21.950570 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:27:21.950546    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t26pz\" (UniqueName: \"kubernetes.io/projected/1c780b1f-1a09-4477-a61f-5dc520974c3e-kube-api-access-t26pz\") pod \"authorino-865f7bbd6c-psgbr\" (UID: \"1c780b1f-1a09-4477-a61f-5dc520974c3e\") " pod="kuadrant-system/authorino-865f7bbd6c-psgbr"
Sep 10 19:27:22.058393 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:27:22.058327    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-865f7bbd6c-psgbr"
Sep 10 19:27:22.329740 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:27:22.329670    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-865f7bbd6c-psgbr"]
Sep 10 19:27:22.332726 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:27:22.332702    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1c780b1f_1a09_4477_a61f_5dc520974c3e.slice/crio-7a568249f65f8c571fe854ca7760f1d3305d64abb9ef3bff400d8528279309c8 WatchSource:0}: Error finding container 7a568249f65f8c571fe854ca7760f1d3305d64abb9ef3bff400d8528279309c8: Status 404 returned error can't find the container with id 7a568249f65f8c571fe854ca7760f1d3305d64abb9ef3bff400d8528279309c8
Sep 10 19:27:22.550725 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:27:22.550683    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-865f7bbd6c-psgbr" event={"ID":"1c780b1f-1a09-4477-a61f-5dc520974c3e","Type":"ContainerStarted","Data":"7a568249f65f8c571fe854ca7760f1d3305d64abb9ef3bff400d8528279309c8"}
Sep 10 19:27:23.557807 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:27:23.557772    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-865f7bbd6c-psgbr" event={"ID":"1c780b1f-1a09-4477-a61f-5dc520974c3e","Type":"ContainerStarted","Data":"c32da406c6e89b619e765220aa866ee0dcb4e15aafd24f2907e6770cd6f94057"}
Sep 10 19:27:23.574428 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:27:23.574380    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-865f7bbd6c-psgbr" podStartSLOduration=1.917434061 podStartE2EDuration="2.5743627s" totalImagesPullingTime="656.928639ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:27:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:27:23.572949977 +0000 UTC m=+714.632777644" watchObservedRunningTime="2026-09-10 19:27:23.5743627 +0000 UTC m=+714.634190368"
Sep 10 19:27:24.176824 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:27:24.176775    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:27:24.265258 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:27:24.265222    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:27:25.300401 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:27:25.300333    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:27:25.397833 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:27:25.397802    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:27:42.559987 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:27:42.559947    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:27:42.664243 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:27:42.664203    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:27:42.892908 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:27:42.892830    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:27:43.193863 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:27:43.193792    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:27:44.086317 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:27:44.086281    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:27:44.195762 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:27:44.195724    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:27:44.642743 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:27:44.642705    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:27:45.369886 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:27:45.369856    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qr7rl"]
Sep 10 19:27:46.300808 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:27:46.300774    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:27:46.397722 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:27:46.397689    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:27:47.568633 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:27:47.568598    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:27:47.713913 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:27:47.713873    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:27:47.876791 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:27:47.876688    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:27:48.175918 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:27:48.175834    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:27:48.273096 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:27:48.273065    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:27:48.314259 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:27:48.314229    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-8300b9-855c8cfdc7-wb9qf"]
Sep 10 19:27:48.317706 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:27:48.317686    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-8300b9-855c8cfdc7-wb9qf"
Sep 10 19:27:48.321244 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:27:48.321215    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-8300b9-dockercfg-2bhzh\""
Sep 10 19:27:48.321404 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:27:48.321375    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-8300b9\""
Sep 10 19:27:48.334194 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:27:48.334171    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-8300b9-855c8cfdc7-wb9qf"]
Sep 10 19:27:48.376871 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:27:48.376849    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/913ee6b0-8326-493c-bb1f-db2ebda11283-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-8300b9-855c8cfdc7-wb9qf\" (UID: \"913ee6b0-8326-493c-bb1f-db2ebda11283\") " pod="openshift-ingress/payload-processing-e2e-inf-a-8300b9-855c8cfdc7-wb9qf"
Sep 10 19:27:48.376984 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:27:48.376928    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k58jw\" (UniqueName: \"kubernetes.io/projected/913ee6b0-8326-493c-bb1f-db2ebda11283-kube-api-access-k58jw\") pod \"payload-processing-e2e-inf-a-8300b9-855c8cfdc7-wb9qf\" (UID: \"913ee6b0-8326-493c-bb1f-db2ebda11283\") " pod="openshift-ingress/payload-processing-e2e-inf-a-8300b9-855c8cfdc7-wb9qf"
Sep 10 19:27:48.477955 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:27:48.477929    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/913ee6b0-8326-493c-bb1f-db2ebda11283-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-8300b9-855c8cfdc7-wb9qf\" (UID: \"913ee6b0-8326-493c-bb1f-db2ebda11283\") " pod="openshift-ingress/payload-processing-e2e-inf-a-8300b9-855c8cfdc7-wb9qf"
Sep 10 19:27:48.478084 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:27:48.477980    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k58jw\" (UniqueName: \"kubernetes.io/projected/913ee6b0-8326-493c-bb1f-db2ebda11283-kube-api-access-k58jw\") pod \"payload-processing-e2e-inf-a-8300b9-855c8cfdc7-wb9qf\" (UID: \"913ee6b0-8326-493c-bb1f-db2ebda11283\") " pod="openshift-ingress/payload-processing-e2e-inf-a-8300b9-855c8cfdc7-wb9qf"
Sep 10 19:27:48.478614 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:27:48.478594    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/913ee6b0-8326-493c-bb1f-db2ebda11283-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-8300b9-855c8cfdc7-wb9qf\" (UID: \"913ee6b0-8326-493c-bb1f-db2ebda11283\") " pod="openshift-ingress/payload-processing-e2e-inf-a-8300b9-855c8cfdc7-wb9qf"
Sep 10 19:27:48.487071 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:27:48.487051    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k58jw\" (UniqueName: \"kubernetes.io/projected/913ee6b0-8326-493c-bb1f-db2ebda11283-kube-api-access-k58jw\") pod \"payload-processing-e2e-inf-a-8300b9-855c8cfdc7-wb9qf\" (UID: \"913ee6b0-8326-493c-bb1f-db2ebda11283\") " pod="openshift-ingress/payload-processing-e2e-inf-a-8300b9-855c8cfdc7-wb9qf"
Sep 10 19:27:48.628064 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:27:48.628032    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-8300b9-855c8cfdc7-wb9qf"
Sep 10 19:27:48.897994 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:27:48.897969    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-8300b9-855c8cfdc7-wb9qf"]
Sep 10 19:27:48.900400 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:27:48.900375    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod913ee6b0_8326_493c_bb1f_db2ebda11283.slice/crio-46f9bd223f370ecb58af62def798a6692cf4bedcdf91568111c56988ec2bedbb WatchSource:0}: Error finding container 46f9bd223f370ecb58af62def798a6692cf4bedcdf91568111c56988ec2bedbb: Status 404 returned error can't find the container with id 46f9bd223f370ecb58af62def798a6692cf4bedcdf91568111c56988ec2bedbb
Sep 10 19:27:49.512236 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:27:49.512206    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:27:49.680255 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:27:49.680224    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-8300b9-855c8cfdc7-wb9qf" event={"ID":"913ee6b0-8326-493c-bb1f-db2ebda11283","Type":"ContainerStarted","Data":"6bd6ae959549747c2d7e0424bc578ac68675956717ca8542957f0430f8a4f180"}
Sep 10 19:27:49.680680 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:27:49.680310    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-8300b9-855c8cfdc7-wb9qf"
Sep 10 19:27:49.680680 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:27:49.680328    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-8300b9-855c8cfdc7-wb9qf" event={"ID":"913ee6b0-8326-493c-bb1f-db2ebda11283","Type":"ContainerStarted","Data":"46f9bd223f370ecb58af62def798a6692cf4bedcdf91568111c56988ec2bedbb"}
Sep 10 19:27:49.707481 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:27:49.707434    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-8300b9-855c8cfdc7-wb9qf" podStartSLOduration=1.7074200720000001 podStartE2EDuration="1.707420072s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:27:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:27:49.706332678 +0000 UTC m=+740.766160348" watchObservedRunningTime="2026-09-10 19:27:49.707420072 +0000 UTC m=+740.767247739"
Sep 10 19:27:49.953453 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:27:49.953410    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:27:50.591592 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:27:50.591559    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:27:51.874178 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:27:51.874134    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:27:51.973805 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:27:51.973771    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:27:52.115563 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:27:52.115526    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:27:52.480209 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:27:52.480174    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-85e90bf8-openshift-default-6cddcfb646-t6zzp"]
Sep 10 19:27:52.484130 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:27:52.484105    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-85e90bf8-openshift-default-6cddcfb646-t6zzp"
Sep 10 19:27:52.486881 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:27:52.486857    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 10 19:27:52.486992 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:27:52.486932    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-85e90bf8-openshift-default-dockercfg-dx6zz\""
Sep 10 19:27:52.506417 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:27:52.506392    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-85e90bf8-openshift-default-6cddcfb646-t6zzp"]
Sep 10 19:27:52.555860 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:27:52.555832    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:27:52.614041 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:27:52.614011    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/878f8b91-4648-4800-9031-576eb949ae08-istio-data\") pod \"e2e-ait-85e90bf8-openshift-default-6cddcfb646-t6zzp\" (UID: \"878f8b91-4648-4800-9031-576eb949ae08\") " pod="openshift-ingress/e2e-ait-85e90bf8-openshift-default-6cddcfb646-t6zzp"
Sep 10 19:27:52.614191 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:27:52.614049    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/878f8b91-4648-4800-9031-576eb949ae08-istio-podinfo\") pod \"e2e-ait-85e90bf8-openshift-default-6cddcfb646-t6zzp\" (UID: \"878f8b91-4648-4800-9031-576eb949ae08\") " pod="openshift-ingress/e2e-ait-85e90bf8-openshift-default-6cddcfb646-t6zzp"
Sep 10 19:27:52.614191 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:27:52.614075    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/878f8b91-4648-4800-9031-576eb949ae08-istiod-ca-cert\") pod \"e2e-ait-85e90bf8-openshift-default-6cddcfb646-t6zzp\" (UID: \"878f8b91-4648-4800-9031-576eb949ae08\") " pod="openshift-ingress/e2e-ait-85e90bf8-openshift-default-6cddcfb646-t6zzp"
Sep 10 19:27:52.614191 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:27:52.614138    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/878f8b91-4648-4800-9031-576eb949ae08-istio-envoy\") pod \"e2e-ait-85e90bf8-openshift-default-6cddcfb646-t6zzp\" (UID: \"878f8b91-4648-4800-9031-576eb949ae08\") " pod="openshift-ingress/e2e-ait-85e90bf8-openshift-default-6cddcfb646-t6zzp"
Sep 10 19:27:52.614191 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:27:52.614187    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zww2b\" (UniqueName: \"kubernetes.io/projected/878f8b91-4648-4800-9031-576eb949ae08-kube-api-access-zww2b\") pod \"e2e-ait-85e90bf8-openshift-default-6cddcfb646-t6zzp\" (UID: \"878f8b91-4648-4800-9031-576eb949ae08\") " pod="openshift-ingress/e2e-ait-85e90bf8-openshift-default-6cddcfb646-t6zzp"
Sep 10 19:27:52.614333 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:27:52.614252    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/878f8b91-4648-4800-9031-576eb949ae08-workload-socket\") pod \"e2e-ait-85e90bf8-openshift-default-6cddcfb646-t6zzp\" (UID: \"878f8b91-4648-4800-9031-576eb949ae08\") " pod="openshift-ingress/e2e-ait-85e90bf8-openshift-default-6cddcfb646-t6zzp"
Sep 10 19:27:52.614333 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:27:52.614275    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/878f8b91-4648-4800-9031-576eb949ae08-workload-certs\") pod \"e2e-ait-85e90bf8-openshift-default-6cddcfb646-t6zzp\" (UID: \"878f8b91-4648-4800-9031-576eb949ae08\") " pod="openshift-ingress/e2e-ait-85e90bf8-openshift-default-6cddcfb646-t6zzp"
Sep 10 19:27:52.614424 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:27:52.614329    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/878f8b91-4648-4800-9031-576eb949ae08-istio-token\") pod \"e2e-ait-85e90bf8-openshift-default-6cddcfb646-t6zzp\" (UID: \"878f8b91-4648-4800-9031-576eb949ae08\") " pod="openshift-ingress/e2e-ait-85e90bf8-openshift-default-6cddcfb646-t6zzp"
Sep 10 19:27:52.614424 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:27:52.614392    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/878f8b91-4648-4800-9031-576eb949ae08-credential-socket\") pod \"e2e-ait-85e90bf8-openshift-default-6cddcfb646-t6zzp\" (UID: \"878f8b91-4648-4800-9031-576eb949ae08\") " pod="openshift-ingress/e2e-ait-85e90bf8-openshift-default-6cddcfb646-t6zzp"
Sep 10 19:27:52.715011 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:27:52.714980    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/878f8b91-4648-4800-9031-576eb949ae08-istio-data\") pod \"e2e-ait-85e90bf8-openshift-default-6cddcfb646-t6zzp\" (UID: \"878f8b91-4648-4800-9031-576eb949ae08\") " pod="openshift-ingress/e2e-ait-85e90bf8-openshift-default-6cddcfb646-t6zzp"
Sep 10 19:27:52.715173 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:27:52.715023    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/878f8b91-4648-4800-9031-576eb949ae08-istio-podinfo\") pod \"e2e-ait-85e90bf8-openshift-default-6cddcfb646-t6zzp\" (UID: \"878f8b91-4648-4800-9031-576eb949ae08\") " pod="openshift-ingress/e2e-ait-85e90bf8-openshift-default-6cddcfb646-t6zzp"
Sep 10 19:27:52.715173 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:27:52.715149    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/878f8b91-4648-4800-9031-576eb949ae08-istiod-ca-cert\") pod \"e2e-ait-85e90bf8-openshift-default-6cddcfb646-t6zzp\" (UID: \"878f8b91-4648-4800-9031-576eb949ae08\") " pod="openshift-ingress/e2e-ait-85e90bf8-openshift-default-6cddcfb646-t6zzp"
Sep 10 19:27:52.715280 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:27:52.715208    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/878f8b91-4648-4800-9031-576eb949ae08-istio-envoy\") pod \"e2e-ait-85e90bf8-openshift-default-6cddcfb646-t6zzp\" (UID: \"878f8b91-4648-4800-9031-576eb949ae08\") " pod="openshift-ingress/e2e-ait-85e90bf8-openshift-default-6cddcfb646-t6zzp"
Sep 10 19:27:52.715280 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:27:52.715247    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zww2b\" (UniqueName: \"kubernetes.io/projected/878f8b91-4648-4800-9031-576eb949ae08-kube-api-access-zww2b\") pod \"e2e-ait-85e90bf8-openshift-default-6cddcfb646-t6zzp\" (UID: \"878f8b91-4648-4800-9031-576eb949ae08\") " pod="openshift-ingress/e2e-ait-85e90bf8-openshift-default-6cddcfb646-t6zzp"
Sep 10 19:27:52.715408 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:27:52.715317    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/878f8b91-4648-4800-9031-576eb949ae08-workload-socket\") pod \"e2e-ait-85e90bf8-openshift-default-6cddcfb646-t6zzp\" (UID: \"878f8b91-4648-4800-9031-576eb949ae08\") " pod="openshift-ingress/e2e-ait-85e90bf8-openshift-default-6cddcfb646-t6zzp"
Sep 10 19:27:52.715408 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:27:52.715373    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/878f8b91-4648-4800-9031-576eb949ae08-workload-certs\") pod \"e2e-ait-85e90bf8-openshift-default-6cddcfb646-t6zzp\" (UID: \"878f8b91-4648-4800-9031-576eb949ae08\") " pod="openshift-ingress/e2e-ait-85e90bf8-openshift-default-6cddcfb646-t6zzp"
Sep 10 19:27:52.715508 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:27:52.715414    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/878f8b91-4648-4800-9031-576eb949ae08-istio-data\") pod \"e2e-ait-85e90bf8-openshift-default-6cddcfb646-t6zzp\" (UID: \"878f8b91-4648-4800-9031-576eb949ae08\") " pod="openshift-ingress/e2e-ait-85e90bf8-openshift-default-6cddcfb646-t6zzp"
Sep 10 19:27:52.715508 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:27:52.715438    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/878f8b91-4648-4800-9031-576eb949ae08-istio-token\") pod \"e2e-ait-85e90bf8-openshift-default-6cddcfb646-t6zzp\" (UID: \"878f8b91-4648-4800-9031-576eb949ae08\") " pod="openshift-ingress/e2e-ait-85e90bf8-openshift-default-6cddcfb646-t6zzp"
Sep 10 19:27:52.715508 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:27:52.715488    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/878f8b91-4648-4800-9031-576eb949ae08-credential-socket\") pod \"e2e-ait-85e90bf8-openshift-default-6cddcfb646-t6zzp\" (UID: \"878f8b91-4648-4800-9031-576eb949ae08\") " pod="openshift-ingress/e2e-ait-85e90bf8-openshift-default-6cddcfb646-t6zzp"
Sep 10 19:27:52.715899 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:27:52.715844    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/878f8b91-4648-4800-9031-576eb949ae08-credential-socket\") pod \"e2e-ait-85e90bf8-openshift-default-6cddcfb646-t6zzp\" (UID: \"878f8b91-4648-4800-9031-576eb949ae08\") " pod="openshift-ingress/e2e-ait-85e90bf8-openshift-default-6cddcfb646-t6zzp"
Sep 10 19:27:52.716038 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:27:52.715907    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/878f8b91-4648-4800-9031-576eb949ae08-workload-socket\") pod \"e2e-ait-85e90bf8-openshift-default-6cddcfb646-t6zzp\" (UID: \"878f8b91-4648-4800-9031-576eb949ae08\") " pod="openshift-ingress/e2e-ait-85e90bf8-openshift-default-6cddcfb646-t6zzp"
Sep 10 19:27:52.716038 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:27:52.715983    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/878f8b91-4648-4800-9031-576eb949ae08-istiod-ca-cert\") pod \"e2e-ait-85e90bf8-openshift-default-6cddcfb646-t6zzp\" (UID: \"878f8b91-4648-4800-9031-576eb949ae08\") " pod="openshift-ingress/e2e-ait-85e90bf8-openshift-default-6cddcfb646-t6zzp"
Sep 10 19:27:52.716164 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:27:52.716060    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/878f8b91-4648-4800-9031-576eb949ae08-workload-certs\") pod \"e2e-ait-85e90bf8-openshift-default-6cddcfb646-t6zzp\" (UID: \"878f8b91-4648-4800-9031-576eb949ae08\") " pod="openshift-ingress/e2e-ait-85e90bf8-openshift-default-6cddcfb646-t6zzp"
Sep 10 19:27:52.717753 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:27:52.717730    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/878f8b91-4648-4800-9031-576eb949ae08-istio-envoy\") pod \"e2e-ait-85e90bf8-openshift-default-6cddcfb646-t6zzp\" (UID: \"878f8b91-4648-4800-9031-576eb949ae08\") " pod="openshift-ingress/e2e-ait-85e90bf8-openshift-default-6cddcfb646-t6zzp"
Sep 10 19:27:52.717920 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:27:52.717895    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/878f8b91-4648-4800-9031-576eb949ae08-istio-podinfo\") pod \"e2e-ait-85e90bf8-openshift-default-6cddcfb646-t6zzp\" (UID: \"878f8b91-4648-4800-9031-576eb949ae08\") " pod="openshift-ingress/e2e-ait-85e90bf8-openshift-default-6cddcfb646-t6zzp"
Sep 10 19:27:52.724209 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:27:52.724183    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zww2b\" (UniqueName: \"kubernetes.io/projected/878f8b91-4648-4800-9031-576eb949ae08-kube-api-access-zww2b\") pod \"e2e-ait-85e90bf8-openshift-default-6cddcfb646-t6zzp\" (UID: \"878f8b91-4648-4800-9031-576eb949ae08\") " pod="openshift-ingress/e2e-ait-85e90bf8-openshift-default-6cddcfb646-t6zzp"
Sep 10 19:27:52.724823 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:27:52.724802    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/878f8b91-4648-4800-9031-576eb949ae08-istio-token\") pod \"e2e-ait-85e90bf8-openshift-default-6cddcfb646-t6zzp\" (UID: \"878f8b91-4648-4800-9031-576eb949ae08\") " pod="openshift-ingress/e2e-ait-85e90bf8-openshift-default-6cddcfb646-t6zzp"
Sep 10 19:27:52.762543 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:27:52.762460    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:27:52.795600 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:27:52.795572    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-85e90bf8-openshift-default-6cddcfb646-t6zzp"
Sep 10 19:27:53.072879 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:27:53.072777    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-85e90bf8-openshift-default-6cddcfb646-t6zzp"]
Sep 10 19:27:53.077136 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:27:53.077102    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod878f8b91_4648_4800_9031_576eb949ae08.slice/crio-7ef0613c96269ab60a94f85dbf96493d40166496e574dc32a7a4e6f1e5a803a4 WatchSource:0}: Error finding container 7ef0613c96269ab60a94f85dbf96493d40166496e574dc32a7a4e6f1e5a803a4: Status 404 returned error can't find the container with id 7ef0613c96269ab60a94f85dbf96493d40166496e574dc32a7a4e6f1e5a803a4
Sep 10 19:27:53.483603 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:27:53.483561    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:27:53.576889 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:27:53.576854    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:27:53.703000 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:27:53.702969    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-85e90bf8-openshift-default-6cddcfb646-t6zzp" event={"ID":"878f8b91-4648-4800-9031-576eb949ae08","Type":"ContainerStarted","Data":"7ef0613c96269ab60a94f85dbf96493d40166496e574dc32a7a4e6f1e5a803a4"}
Sep 10 19:27:54.776229 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:27:54.776191    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:27:54.876262 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:27:54.876214    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:27:54.940260 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:27:54.940227    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:27:55.308906 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:27:55.308871    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:27:55.403309 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:27:55.403272    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:27:55.676065 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:27:55.676011    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 19:27:55.676140 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:27:55.676123    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 19:27:55.676182 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:27:55.676166    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 19:27:56.720276 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:27:56.720241    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-85e90bf8-openshift-default-6cddcfb646-t6zzp" event={"ID":"878f8b91-4648-4800-9031-576eb949ae08","Type":"ContainerStarted","Data":"1af16d615123b089010ca82da0a2a50f78aad9c0e41637e2aa48172b2af006d6"}
Sep 10 19:27:56.741647 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:27:56.741594    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-85e90bf8-openshift-default-6cddcfb646-t6zzp" podStartSLOduration=2.145495937 podStartE2EDuration="4.741577794s" totalImagesPullingTime="2.596081857s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:27:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:27:56.739172981 +0000 UTC m=+747.799000649" watchObservedRunningTime="2026-09-10 19:27:56.741577794 +0000 UTC m=+747.801405461"
Sep 10 19:27:56.796414 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:27:56.796380    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-85e90bf8-openshift-default-6cddcfb646-t6zzp"
Sep 10 19:27:56.801307 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:27:56.801277    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-85e90bf8-openshift-default-6cddcfb646-t6zzp"
Sep 10 19:27:57.725269 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:27:57.725239    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-85e90bf8-openshift-default-6cddcfb646-t6zzp"
Sep 10 19:27:57.726221 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:27:57.726204    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-85e90bf8-openshift-default-6cddcfb646-t6zzp"
Sep 10 19:27:58.106805 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:27:58.106726    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:27:58.195606 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:27:58.195571    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:27:58.368942 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:27:58.368866    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qr7rl"]
Sep 10 19:27:59.317257 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:27:59.317226    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:27:59.432952 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:27:59.432913    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:27:59.715198 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:27:59.715162    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:28:00.115232 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:00.115135    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:28:00.288175 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:00.288133    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:28:00.394931 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:00.394866    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-85e90bf8-7c44767c7c-qqvgc"]
Sep 10 19:28:00.398469 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:00.398453    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-85e90bf8-7c44767c7c-qqvgc"
Sep 10 19:28:00.400350 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:00.400318    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-85e90bf8-dockercfg-nnl6w\""
Sep 10 19:28:00.400552 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:00.400534    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-85e90bf8\""
Sep 10 19:28:00.422899 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:00.422873    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-85e90bf8-7c44767c7c-qqvgc"]
Sep 10 19:28:00.485686 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:00.485652    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a6a3b8b1-9656-46e4-97fc-bca77ec9d941-plugins-config-volume\") pod \"payload-processing-e2e-ait-85e90bf8-7c44767c7c-qqvgc\" (UID: \"a6a3b8b1-9656-46e4-97fc-bca77ec9d941\") " pod="openshift-ingress/payload-processing-e2e-ait-85e90bf8-7c44767c7c-qqvgc"
Sep 10 19:28:00.485832 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:00.485712    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bpj85\" (UniqueName: \"kubernetes.io/projected/a6a3b8b1-9656-46e4-97fc-bca77ec9d941-kube-api-access-bpj85\") pod \"payload-processing-e2e-ait-85e90bf8-7c44767c7c-qqvgc\" (UID: \"a6a3b8b1-9656-46e4-97fc-bca77ec9d941\") " pod="openshift-ingress/payload-processing-e2e-ait-85e90bf8-7c44767c7c-qqvgc"
Sep 10 19:28:00.586797 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:00.586765    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a6a3b8b1-9656-46e4-97fc-bca77ec9d941-plugins-config-volume\") pod \"payload-processing-e2e-ait-85e90bf8-7c44767c7c-qqvgc\" (UID: \"a6a3b8b1-9656-46e4-97fc-bca77ec9d941\") " pod="openshift-ingress/payload-processing-e2e-ait-85e90bf8-7c44767c7c-qqvgc"
Sep 10 19:28:00.586951 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:00.586820    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bpj85\" (UniqueName: \"kubernetes.io/projected/a6a3b8b1-9656-46e4-97fc-bca77ec9d941-kube-api-access-bpj85\") pod \"payload-processing-e2e-ait-85e90bf8-7c44767c7c-qqvgc\" (UID: \"a6a3b8b1-9656-46e4-97fc-bca77ec9d941\") " pod="openshift-ingress/payload-processing-e2e-ait-85e90bf8-7c44767c7c-qqvgc"
Sep 10 19:28:00.587501 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:00.587485    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a6a3b8b1-9656-46e4-97fc-bca77ec9d941-plugins-config-volume\") pod \"payload-processing-e2e-ait-85e90bf8-7c44767c7c-qqvgc\" (UID: \"a6a3b8b1-9656-46e4-97fc-bca77ec9d941\") " pod="openshift-ingress/payload-processing-e2e-ait-85e90bf8-7c44767c7c-qqvgc"
Sep 10 19:28:00.606711 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:00.606681    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bpj85\" (UniqueName: \"kubernetes.io/projected/a6a3b8b1-9656-46e4-97fc-bca77ec9d941-kube-api-access-bpj85\") pod \"payload-processing-e2e-ait-85e90bf8-7c44767c7c-qqvgc\" (UID: \"a6a3b8b1-9656-46e4-97fc-bca77ec9d941\") " pod="openshift-ingress/payload-processing-e2e-ait-85e90bf8-7c44767c7c-qqvgc"
Sep 10 19:28:00.687181 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:00.687112    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-8300b9-855c8cfdc7-wb9qf"
Sep 10 19:28:00.708323 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:00.708292    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-85e90bf8-7c44767c7c-qqvgc"
Sep 10 19:28:00.986851 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:00.986822    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-85e90bf8-7c44767c7c-qqvgc"]
Sep 10 19:28:00.990701 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:28:00.990674    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6a3b8b1_9656_46e4_97fc_bca77ec9d941.slice/crio-77df2d23a9322083b45032864b55fae79b2a12dd7e9461e5a774e44e98a92c27 WatchSource:0}: Error finding container 77df2d23a9322083b45032864b55fae79b2a12dd7e9461e5a774e44e98a92c27: Status 404 returned error can't find the container with id 77df2d23a9322083b45032864b55fae79b2a12dd7e9461e5a774e44e98a92c27
Sep 10 19:28:01.287285 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:01.287211    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-conflict-a-479f6157-kserve-7d76b7f788-bmb57"]
Sep 10 19:28:01.290850 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:01.290833    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-a-479f6157-kserve-7d76b7f788-bmb57"
Sep 10 19:28:01.294925 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:01.294909    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-conflict-a-479f6157-kserve-self-signed-certs\""
Sep 10 19:28:01.304744 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:01.304721    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-conflict-a-479f6157-kserve-7d76b7f788-bmb57"]
Sep 10 19:28:01.394164 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:01.394137    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9c34b550-2700-4103-bea0-5493d30c13d9-home\") pod \"e2e-conflict-a-479f6157-kserve-7d76b7f788-bmb57\" (UID: \"9c34b550-2700-4103-bea0-5493d30c13d9\") " pod="llm/e2e-conflict-a-479f6157-kserve-7d76b7f788-bmb57"
Sep 10 19:28:01.394303 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:01.394176    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9c34b550-2700-4103-bea0-5493d30c13d9-model-cache\") pod \"e2e-conflict-a-479f6157-kserve-7d76b7f788-bmb57\" (UID: \"9c34b550-2700-4103-bea0-5493d30c13d9\") " pod="llm/e2e-conflict-a-479f6157-kserve-7d76b7f788-bmb57"
Sep 10 19:28:01.394303 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:01.394214    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9c34b550-2700-4103-bea0-5493d30c13d9-tmp-dir\") pod \"e2e-conflict-a-479f6157-kserve-7d76b7f788-bmb57\" (UID: \"9c34b550-2700-4103-bea0-5493d30c13d9\") " pod="llm/e2e-conflict-a-479f6157-kserve-7d76b7f788-bmb57"
Sep 10 19:28:01.394303 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:01.394233    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9c34b550-2700-4103-bea0-5493d30c13d9-tls-certs\") pod \"e2e-conflict-a-479f6157-kserve-7d76b7f788-bmb57\" (UID: \"9c34b550-2700-4103-bea0-5493d30c13d9\") " pod="llm/e2e-conflict-a-479f6157-kserve-7d76b7f788-bmb57"
Sep 10 19:28:01.394303 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:01.394253    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lnvcg\" (UniqueName: \"kubernetes.io/projected/9c34b550-2700-4103-bea0-5493d30c13d9-kube-api-access-lnvcg\") pod \"e2e-conflict-a-479f6157-kserve-7d76b7f788-bmb57\" (UID: \"9c34b550-2700-4103-bea0-5493d30c13d9\") " pod="llm/e2e-conflict-a-479f6157-kserve-7d76b7f788-bmb57"
Sep 10 19:28:01.394303 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:01.394278    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9c34b550-2700-4103-bea0-5493d30c13d9-dshm\") pod \"e2e-conflict-a-479f6157-kserve-7d76b7f788-bmb57\" (UID: \"9c34b550-2700-4103-bea0-5493d30c13d9\") " pod="llm/e2e-conflict-a-479f6157-kserve-7d76b7f788-bmb57"
Sep 10 19:28:01.495495 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:01.495454    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9c34b550-2700-4103-bea0-5493d30c13d9-home\") pod \"e2e-conflict-a-479f6157-kserve-7d76b7f788-bmb57\" (UID: \"9c34b550-2700-4103-bea0-5493d30c13d9\") " pod="llm/e2e-conflict-a-479f6157-kserve-7d76b7f788-bmb57"
Sep 10 19:28:01.495942 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:01.495518    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9c34b550-2700-4103-bea0-5493d30c13d9-model-cache\") pod \"e2e-conflict-a-479f6157-kserve-7d76b7f788-bmb57\" (UID: \"9c34b550-2700-4103-bea0-5493d30c13d9\") " pod="llm/e2e-conflict-a-479f6157-kserve-7d76b7f788-bmb57"
Sep 10 19:28:01.495942 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:01.495808    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9c34b550-2700-4103-bea0-5493d30c13d9-tmp-dir\") pod \"e2e-conflict-a-479f6157-kserve-7d76b7f788-bmb57\" (UID: \"9c34b550-2700-4103-bea0-5493d30c13d9\") " pod="llm/e2e-conflict-a-479f6157-kserve-7d76b7f788-bmb57"
Sep 10 19:28:01.495942 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:01.495840    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9c34b550-2700-4103-bea0-5493d30c13d9-tls-certs\") pod \"e2e-conflict-a-479f6157-kserve-7d76b7f788-bmb57\" (UID: \"9c34b550-2700-4103-bea0-5493d30c13d9\") " pod="llm/e2e-conflict-a-479f6157-kserve-7d76b7f788-bmb57"
Sep 10 19:28:01.495942 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:01.495872    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lnvcg\" (UniqueName: \"kubernetes.io/projected/9c34b550-2700-4103-bea0-5493d30c13d9-kube-api-access-lnvcg\") pod \"e2e-conflict-a-479f6157-kserve-7d76b7f788-bmb57\" (UID: \"9c34b550-2700-4103-bea0-5493d30c13d9\") " pod="llm/e2e-conflict-a-479f6157-kserve-7d76b7f788-bmb57"
Sep 10 19:28:01.495942 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:01.495877    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9c34b550-2700-4103-bea0-5493d30c13d9-model-cache\") pod \"e2e-conflict-a-479f6157-kserve-7d76b7f788-bmb57\" (UID: \"9c34b550-2700-4103-bea0-5493d30c13d9\") " pod="llm/e2e-conflict-a-479f6157-kserve-7d76b7f788-bmb57"
Sep 10 19:28:01.495942 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:01.495916    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9c34b550-2700-4103-bea0-5493d30c13d9-home\") pod \"e2e-conflict-a-479f6157-kserve-7d76b7f788-bmb57\" (UID: \"9c34b550-2700-4103-bea0-5493d30c13d9\") " pod="llm/e2e-conflict-a-479f6157-kserve-7d76b7f788-bmb57"
Sep 10 19:28:01.496235 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:01.495973    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9c34b550-2700-4103-bea0-5493d30c13d9-dshm\") pod \"e2e-conflict-a-479f6157-kserve-7d76b7f788-bmb57\" (UID: \"9c34b550-2700-4103-bea0-5493d30c13d9\") " pod="llm/e2e-conflict-a-479f6157-kserve-7d76b7f788-bmb57"
Sep 10 19:28:01.496235 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:01.496152    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9c34b550-2700-4103-bea0-5493d30c13d9-tmp-dir\") pod \"e2e-conflict-a-479f6157-kserve-7d76b7f788-bmb57\" (UID: \"9c34b550-2700-4103-bea0-5493d30c13d9\") " pod="llm/e2e-conflict-a-479f6157-kserve-7d76b7f788-bmb57"
Sep 10 19:28:01.498131 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:01.498107    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9c34b550-2700-4103-bea0-5493d30c13d9-dshm\") pod \"e2e-conflict-a-479f6157-kserve-7d76b7f788-bmb57\" (UID: \"9c34b550-2700-4103-bea0-5493d30c13d9\") " pod="llm/e2e-conflict-a-479f6157-kserve-7d76b7f788-bmb57"
Sep 10 19:28:01.498288 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:01.498271    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9c34b550-2700-4103-bea0-5493d30c13d9-tls-certs\") pod \"e2e-conflict-a-479f6157-kserve-7d76b7f788-bmb57\" (UID: \"9c34b550-2700-4103-bea0-5493d30c13d9\") " pod="llm/e2e-conflict-a-479f6157-kserve-7d76b7f788-bmb57"
Sep 10 19:28:01.506005 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:01.505983    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lnvcg\" (UniqueName: \"kubernetes.io/projected/9c34b550-2700-4103-bea0-5493d30c13d9-kube-api-access-lnvcg\") pod \"e2e-conflict-a-479f6157-kserve-7d76b7f788-bmb57\" (UID: \"9c34b550-2700-4103-bea0-5493d30c13d9\") " pod="llm/e2e-conflict-a-479f6157-kserve-7d76b7f788-bmb57"
Sep 10 19:28:01.599453 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:01.599369    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-a-479f6157-kserve-7d76b7f788-bmb57"
Sep 10 19:28:01.747866 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:01.747820    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-85e90bf8-7c44767c7c-qqvgc" event={"ID":"a6a3b8b1-9656-46e4-97fc-bca77ec9d941","Type":"ContainerStarted","Data":"60552f8310568c1cc93f3dd68e942cbe33642d2ce9ff1b1bab103fadba58f24f"}
Sep 10 19:28:01.748008 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:01.747886    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-85e90bf8-7c44767c7c-qqvgc" event={"ID":"a6a3b8b1-9656-46e4-97fc-bca77ec9d941","Type":"ContainerStarted","Data":"77df2d23a9322083b45032864b55fae79b2a12dd7e9461e5a774e44e98a92c27"}
Sep 10 19:28:01.748074 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:01.748001    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-85e90bf8-7c44767c7c-qqvgc"
Sep 10 19:28:01.774803 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:01.774757    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-85e90bf8-7c44767c7c-qqvgc" podStartSLOduration=1.774741681 podStartE2EDuration="1.774741681s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:28:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:28:01.773049337 +0000 UTC m=+752.832877005" watchObservedRunningTime="2026-09-10 19:28:01.774741681 +0000 UTC m=+752.834569349"
Sep 10 19:28:01.859141 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:01.859041    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:28:01.882126 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:01.882103    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-conflict-a-479f6157-kserve-7d76b7f788-bmb57"]
Sep 10 19:28:01.884580 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:28:01.884553    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9c34b550_2700_4103_bea0_5493d30c13d9.slice/crio-76faf436388958763fb3c31b5f8c723304f7224739e435f1c7e1bec53bbefbee WatchSource:0}: Error finding container 76faf436388958763fb3c31b5f8c723304f7224739e435f1c7e1bec53bbefbee: Status 404 returned error can't find the container with id 76faf436388958763fb3c31b5f8c723304f7224739e435f1c7e1bec53bbefbee
Sep 10 19:28:01.965384 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:01.965355    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:28:02.091142 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:02.091091    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:28:02.183725 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:02.183636    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:28:02.284052 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:02.284018    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:28:02.754449 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:02.754411    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-a-479f6157-kserve-7d76b7f788-bmb57" event={"ID":"9c34b550-2700-4103-bea0-5493d30c13d9","Type":"ContainerStarted","Data":"db7413ef65497bb0a34e7fd19cb9ab86eb9a4a31d1e3ace278c667f5ed33ada0"}
Sep 10 19:28:02.754449 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:02.754455    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-a-479f6157-kserve-7d76b7f788-bmb57" event={"ID":"9c34b550-2700-4103-bea0-5493d30c13d9","Type":"ContainerStarted","Data":"76faf436388958763fb3c31b5f8c723304f7224739e435f1c7e1bec53bbefbee"}
Sep 10 19:28:02.772637 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:02.772589    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-conflict-a-479f6157-kserve-7d76b7f788-bmb57" podStartSLOduration=1.772575372 podStartE2EDuration="1.772575372s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:28:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:28:02.770599551 +0000 UTC m=+753.830427219" watchObservedRunningTime="2026-09-10 19:28:02.772575372 +0000 UTC m=+753.832403040"
Sep 10 19:28:04.890255 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:04.890216    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:28:07.172761 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:07.172675    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:28:07.964255 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:07.964213    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:28:08.053136 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:08.053099    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:28:08.182380 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:08.182328    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:28:08.691854 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:08.691820    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:28:11.255500 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:11.255454    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:28:11.575318 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:11.575224    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:28:11.600318 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:11.600295    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-conflict-a-479f6157-kserve-7d76b7f788-bmb57"
Sep 10 19:28:11.600460 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:11.600355    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-conflict-a-479f6157-kserve-7d76b7f788-bmb57"
Sep 10 19:28:11.612920 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:11.612901    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-conflict-a-479f6157-kserve-7d76b7f788-bmb57"
Sep 10 19:28:11.808763 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:11.808733    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-conflict-a-479f6157-kserve-7d76b7f788-bmb57"
Sep 10 19:28:11.835193 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:11.834323    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-4247c7-openshift-default-6c679cc99d-lk98v"]
Sep 10 19:28:11.844590 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:11.844569    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-4247c7-openshift-default-6c679cc99d-lk98v"
Sep 10 19:28:11.848603 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:11.848583    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-4247c7-openshift-default-dockercfg-dvm6m\""
Sep 10 19:28:11.856098 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:11.856077    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-4247c7-openshift-default-6c679cc99d-lk98v"]
Sep 10 19:28:11.868205 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:11.868178    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:28:11.987799 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:11.987768    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3aea00e-ff8b-4739-9e90-3c5838c124a2-credential-socket\") pod \"e2e-inf-b-4247c7-openshift-default-6c679cc99d-lk98v\" (UID: \"c3aea00e-ff8b-4739-9e90-3c5838c124a2\") " pod="openshift-ingress/e2e-inf-b-4247c7-openshift-default-6c679cc99d-lk98v"
Sep 10 19:28:11.987971 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:11.987818    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c3aea00e-ff8b-4739-9e90-3c5838c124a2-istio-token\") pod \"e2e-inf-b-4247c7-openshift-default-6c679cc99d-lk98v\" (UID: \"c3aea00e-ff8b-4739-9e90-3c5838c124a2\") " pod="openshift-ingress/e2e-inf-b-4247c7-openshift-default-6c679cc99d-lk98v"
Sep 10 19:28:11.987971 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:11.987940    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c3aea00e-ff8b-4739-9e90-3c5838c124a2-istio-envoy\") pod \"e2e-inf-b-4247c7-openshift-default-6c679cc99d-lk98v\" (UID: \"c3aea00e-ff8b-4739-9e90-3c5838c124a2\") " pod="openshift-ingress/e2e-inf-b-4247c7-openshift-default-6c679cc99d-lk98v"
Sep 10 19:28:11.988046 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:11.988005    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3aea00e-ff8b-4739-9e90-3c5838c124a2-workload-socket\") pod \"e2e-inf-b-4247c7-openshift-default-6c679cc99d-lk98v\" (UID: \"c3aea00e-ff8b-4739-9e90-3c5838c124a2\") " pod="openshift-ingress/e2e-inf-b-4247c7-openshift-default-6c679cc99d-lk98v"
Sep 10 19:28:11.988046 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:11.988028    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c3aea00e-ff8b-4739-9e90-3c5838c124a2-istio-data\") pod \"e2e-inf-b-4247c7-openshift-default-6c679cc99d-lk98v\" (UID: \"c3aea00e-ff8b-4739-9e90-3c5838c124a2\") " pod="openshift-ingress/e2e-inf-b-4247c7-openshift-default-6c679cc99d-lk98v"
Sep 10 19:28:11.988106 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:11.988046    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c3aea00e-ff8b-4739-9e90-3c5838c124a2-workload-certs\") pod \"e2e-inf-b-4247c7-openshift-default-6c679cc99d-lk98v\" (UID: \"c3aea00e-ff8b-4739-9e90-3c5838c124a2\") " pod="openshift-ingress/e2e-inf-b-4247c7-openshift-default-6c679cc99d-lk98v"
Sep 10 19:28:11.988106 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:11.988061    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c3aea00e-ff8b-4739-9e90-3c5838c124a2-istio-podinfo\") pod \"e2e-inf-b-4247c7-openshift-default-6c679cc99d-lk98v\" (UID: \"c3aea00e-ff8b-4739-9e90-3c5838c124a2\") " pod="openshift-ingress/e2e-inf-b-4247c7-openshift-default-6c679cc99d-lk98v"
Sep 10 19:28:11.988106 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:11.988079    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c3aea00e-ff8b-4739-9e90-3c5838c124a2-istiod-ca-cert\") pod \"e2e-inf-b-4247c7-openshift-default-6c679cc99d-lk98v\" (UID: \"c3aea00e-ff8b-4739-9e90-3c5838c124a2\") " pod="openshift-ingress/e2e-inf-b-4247c7-openshift-default-6c679cc99d-lk98v"
Sep 10 19:28:11.988198 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:11.988107    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5zkqd\" (UniqueName: \"kubernetes.io/projected/c3aea00e-ff8b-4739-9e90-3c5838c124a2-kube-api-access-5zkqd\") pod \"e2e-inf-b-4247c7-openshift-default-6c679cc99d-lk98v\" (UID: \"c3aea00e-ff8b-4739-9e90-3c5838c124a2\") " pod="openshift-ingress/e2e-inf-b-4247c7-openshift-default-6c679cc99d-lk98v"
Sep 10 19:28:12.088628 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:12.088554    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3aea00e-ff8b-4739-9e90-3c5838c124a2-workload-socket\") pod \"e2e-inf-b-4247c7-openshift-default-6c679cc99d-lk98v\" (UID: \"c3aea00e-ff8b-4739-9e90-3c5838c124a2\") " pod="openshift-ingress/e2e-inf-b-4247c7-openshift-default-6c679cc99d-lk98v"
Sep 10 19:28:12.088628 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:12.088589    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c3aea00e-ff8b-4739-9e90-3c5838c124a2-istio-data\") pod \"e2e-inf-b-4247c7-openshift-default-6c679cc99d-lk98v\" (UID: \"c3aea00e-ff8b-4739-9e90-3c5838c124a2\") " pod="openshift-ingress/e2e-inf-b-4247c7-openshift-default-6c679cc99d-lk98v"
Sep 10 19:28:12.088628 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:12.088608    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c3aea00e-ff8b-4739-9e90-3c5838c124a2-workload-certs\") pod \"e2e-inf-b-4247c7-openshift-default-6c679cc99d-lk98v\" (UID: \"c3aea00e-ff8b-4739-9e90-3c5838c124a2\") " pod="openshift-ingress/e2e-inf-b-4247c7-openshift-default-6c679cc99d-lk98v"
Sep 10 19:28:12.088628 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:12.088626    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c3aea00e-ff8b-4739-9e90-3c5838c124a2-istio-podinfo\") pod \"e2e-inf-b-4247c7-openshift-default-6c679cc99d-lk98v\" (UID: \"c3aea00e-ff8b-4739-9e90-3c5838c124a2\") " pod="openshift-ingress/e2e-inf-b-4247c7-openshift-default-6c679cc99d-lk98v"
Sep 10 19:28:12.088884 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:12.088677    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c3aea00e-ff8b-4739-9e90-3c5838c124a2-istiod-ca-cert\") pod \"e2e-inf-b-4247c7-openshift-default-6c679cc99d-lk98v\" (UID: \"c3aea00e-ff8b-4739-9e90-3c5838c124a2\") " pod="openshift-ingress/e2e-inf-b-4247c7-openshift-default-6c679cc99d-lk98v"
Sep 10 19:28:12.088884 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:12.088713    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5zkqd\" (UniqueName: \"kubernetes.io/projected/c3aea00e-ff8b-4739-9e90-3c5838c124a2-kube-api-access-5zkqd\") pod \"e2e-inf-b-4247c7-openshift-default-6c679cc99d-lk98v\" (UID: \"c3aea00e-ff8b-4739-9e90-3c5838c124a2\") " pod="openshift-ingress/e2e-inf-b-4247c7-openshift-default-6c679cc99d-lk98v"
Sep 10 19:28:12.088884 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:12.088761    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3aea00e-ff8b-4739-9e90-3c5838c124a2-credential-socket\") pod \"e2e-inf-b-4247c7-openshift-default-6c679cc99d-lk98v\" (UID: \"c3aea00e-ff8b-4739-9e90-3c5838c124a2\") " pod="openshift-ingress/e2e-inf-b-4247c7-openshift-default-6c679cc99d-lk98v"
Sep 10 19:28:12.088884 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:12.088821    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c3aea00e-ff8b-4739-9e90-3c5838c124a2-istio-token\") pod \"e2e-inf-b-4247c7-openshift-default-6c679cc99d-lk98v\" (UID: \"c3aea00e-ff8b-4739-9e90-3c5838c124a2\") " pod="openshift-ingress/e2e-inf-b-4247c7-openshift-default-6c679cc99d-lk98v"
Sep 10 19:28:12.089069 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:12.088890    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c3aea00e-ff8b-4739-9e90-3c5838c124a2-istio-envoy\") pod \"e2e-inf-b-4247c7-openshift-default-6c679cc99d-lk98v\" (UID: \"c3aea00e-ff8b-4739-9e90-3c5838c124a2\") " pod="openshift-ingress/e2e-inf-b-4247c7-openshift-default-6c679cc99d-lk98v"
Sep 10 19:28:12.089069 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:12.089006    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3aea00e-ff8b-4739-9e90-3c5838c124a2-workload-socket\") pod \"e2e-inf-b-4247c7-openshift-default-6c679cc99d-lk98v\" (UID: \"c3aea00e-ff8b-4739-9e90-3c5838c124a2\") " pod="openshift-ingress/e2e-inf-b-4247c7-openshift-default-6c679cc99d-lk98v"
Sep 10 19:28:12.089069 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:12.089045    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c3aea00e-ff8b-4739-9e90-3c5838c124a2-workload-certs\") pod \"e2e-inf-b-4247c7-openshift-default-6c679cc99d-lk98v\" (UID: \"c3aea00e-ff8b-4739-9e90-3c5838c124a2\") " pod="openshift-ingress/e2e-inf-b-4247c7-openshift-default-6c679cc99d-lk98v"
Sep 10 19:28:12.089234 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:12.089214    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c3aea00e-ff8b-4739-9e90-3c5838c124a2-istio-data\") pod \"e2e-inf-b-4247c7-openshift-default-6c679cc99d-lk98v\" (UID: \"c3aea00e-ff8b-4739-9e90-3c5838c124a2\") " pod="openshift-ingress/e2e-inf-b-4247c7-openshift-default-6c679cc99d-lk98v"
Sep 10 19:28:12.089293 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:12.089268    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3aea00e-ff8b-4739-9e90-3c5838c124a2-credential-socket\") pod \"e2e-inf-b-4247c7-openshift-default-6c679cc99d-lk98v\" (UID: \"c3aea00e-ff8b-4739-9e90-3c5838c124a2\") " pod="openshift-ingress/e2e-inf-b-4247c7-openshift-default-6c679cc99d-lk98v"
Sep 10 19:28:12.089600 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:12.089572    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c3aea00e-ff8b-4739-9e90-3c5838c124a2-istiod-ca-cert\") pod \"e2e-inf-b-4247c7-openshift-default-6c679cc99d-lk98v\" (UID: \"c3aea00e-ff8b-4739-9e90-3c5838c124a2\") " pod="openshift-ingress/e2e-inf-b-4247c7-openshift-default-6c679cc99d-lk98v"
Sep 10 19:28:12.091179 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:12.091155    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c3aea00e-ff8b-4739-9e90-3c5838c124a2-istio-envoy\") pod \"e2e-inf-b-4247c7-openshift-default-6c679cc99d-lk98v\" (UID: \"c3aea00e-ff8b-4739-9e90-3c5838c124a2\") " pod="openshift-ingress/e2e-inf-b-4247c7-openshift-default-6c679cc99d-lk98v"
Sep 10 19:28:12.091423 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:12.091405    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c3aea00e-ff8b-4739-9e90-3c5838c124a2-istio-podinfo\") pod \"e2e-inf-b-4247c7-openshift-default-6c679cc99d-lk98v\" (UID: \"c3aea00e-ff8b-4739-9e90-3c5838c124a2\") " pod="openshift-ingress/e2e-inf-b-4247c7-openshift-default-6c679cc99d-lk98v"
Sep 10 19:28:12.096493 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:12.096472    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c3aea00e-ff8b-4739-9e90-3c5838c124a2-istio-token\") pod \"e2e-inf-b-4247c7-openshift-default-6c679cc99d-lk98v\" (UID: \"c3aea00e-ff8b-4739-9e90-3c5838c124a2\") " pod="openshift-ingress/e2e-inf-b-4247c7-openshift-default-6c679cc99d-lk98v"
Sep 10 19:28:12.096738 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:12.096721    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5zkqd\" (UniqueName: \"kubernetes.io/projected/c3aea00e-ff8b-4739-9e90-3c5838c124a2-kube-api-access-5zkqd\") pod \"e2e-inf-b-4247c7-openshift-default-6c679cc99d-lk98v\" (UID: \"c3aea00e-ff8b-4739-9e90-3c5838c124a2\") " pod="openshift-ingress/e2e-inf-b-4247c7-openshift-default-6c679cc99d-lk98v"
Sep 10 19:28:12.162196 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:12.162162    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-4247c7-openshift-default-6c679cc99d-lk98v"
Sep 10 19:28:12.260021 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:12.259954    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:28:12.459447 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:12.459423    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-4247c7-openshift-default-6c679cc99d-lk98v"]
Sep 10 19:28:12.461680 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:28:12.461649    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3aea00e_ff8b_4739_9e90_3c5838c124a2.slice/crio-1355786c4246c8944d544f24479233090c26ebcaff53ef9f1ab687d7bc00e299 WatchSource:0}: Error finding container 1355786c4246c8944d544f24479233090c26ebcaff53ef9f1ab687d7bc00e299: Status 404 returned error can't find the container with id 1355786c4246c8944d544f24479233090c26ebcaff53ef9f1ab687d7bc00e299
Sep 10 19:28:12.464125 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:12.464085    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 19:28:12.464190 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:12.464174    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 19:28:12.464235 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:12.464218    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 19:28:12.757335 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:12.756472    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-85e90bf8-7c44767c7c-qqvgc"
Sep 10 19:28:12.804822 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:12.804792    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-4247c7-openshift-default-6c679cc99d-lk98v" event={"ID":"c3aea00e-ff8b-4739-9e90-3c5838c124a2","Type":"ContainerStarted","Data":"66ec3d473001ce2815592ee14441ef7c2e1a52c261e5f2145b355e6df64bf27c"}
Sep 10 19:28:12.804996 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:12.804856    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-4247c7-openshift-default-6c679cc99d-lk98v" event={"ID":"c3aea00e-ff8b-4739-9e90-3c5838c124a2","Type":"ContainerStarted","Data":"1355786c4246c8944d544f24479233090c26ebcaff53ef9f1ab687d7bc00e299"}
Sep 10 19:28:12.829697 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:12.827812    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-4247c7-openshift-default-6c679cc99d-lk98v" podStartSLOduration=1.827791602 podStartE2EDuration="1.827791602s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:28:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:28:12.823678401 +0000 UTC m=+763.883506070" watchObservedRunningTime="2026-09-10 19:28:12.827791602 +0000 UTC m=+763.887619272"
Sep 10 19:28:13.162787 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:13.162710    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-4247c7-openshift-default-6c679cc99d-lk98v"
Sep 10 19:28:13.167736 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:13.167713    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-4247c7-openshift-default-6c679cc99d-lk98v"
Sep 10 19:28:13.810386 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:13.810355    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-4247c7-openshift-default-6c679cc99d-lk98v"
Sep 10 19:28:13.811415 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:13.811396    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-4247c7-openshift-default-6c679cc99d-lk98v"
Sep 10 19:28:14.462897 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:14.462866    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:28:14.573152 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:14.573116    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:28:14.670941 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:14.670905    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:28:14.794003 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:14.793925    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:28:14.892109 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:14.892073    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:28:16.162590 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:16.162554    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qr7rl"]
Sep 10 19:28:16.188324 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:16.188270    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:28:16.301598 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:16.301560    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:28:16.504498 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:16.504460    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:28:16.794869 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:16.794790    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:28:16.893454 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:16.893409    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:28:17.270137 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:17.270084    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:28:18.218366 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:18.218315    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:28:18.840258 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:18.840228    2575 generic.go:356] "Generic (PLEG): container finished" podID="878f8b91-4648-4800-9031-576eb949ae08" containerID="1af16d615123b089010ca82da0a2a50f78aad9c0e41637e2aa48172b2af006d6" exitCode=0
Sep 10 19:28:18.840676 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:18.840302    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-85e90bf8-openshift-default-6cddcfb646-t6zzp" event={"ID":"878f8b91-4648-4800-9031-576eb949ae08","Type":"ContainerDied","Data":"1af16d615123b089010ca82da0a2a50f78aad9c0e41637e2aa48172b2af006d6"}
Sep 10 19:28:18.840755 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:18.840738    2575 scope.go:117] "RemoveContainer" containerID="1af16d615123b089010ca82da0a2a50f78aad9c0e41637e2aa48172b2af006d6"
Sep 10 19:28:18.841787 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:18.841754    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 19:28:18.841860 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:18.841847    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 19:28:18.841898 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:18.841880    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 19:28:19.054035 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:19.053997    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:28:19.849407 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:19.849374    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-85e90bf8-openshift-default-6cddcfb646-t6zzp" event={"ID":"878f8b91-4648-4800-9031-576eb949ae08","Type":"ContainerStarted","Data":"8f45b34325e7f5e5f582046ed2e08212a820f027997cceb08e0ed43f08a74465"}
Sep 10 19:28:20.796778 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:20.796742    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-85e90bf8-openshift-default-6cddcfb646-t6zzp"
Sep 10 19:28:20.796930 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:20.796912    2575 patch_prober.go:28] interesting pod/e2e-ait-85e90bf8-openshift-default-6cddcfb646-t6zzp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.34:15021/healthz/ready\": dial tcp 10.132.0.34:15021: connect: connection refused" start-of-body=
Sep 10 19:28:20.796987 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:20.796959    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-85e90bf8-openshift-default-6cddcfb646-t6zzp" podUID="878f8b91-4648-4800-9031-576eb949ae08" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.34:15021/healthz/ready\": dial tcp 10.132.0.34:15021: connect: connection refused"
Sep 10 19:28:20.956294 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:20.956246    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:28:21.057054 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:21.056960    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:28:21.162650 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:21.162615    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:28:21.796319 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:21.796282    2575 patch_prober.go:28] interesting pod/e2e-ait-85e90bf8-openshift-default-6cddcfb646-t6zzp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.34:15021/healthz/ready\": dial tcp 10.132.0.34:15021: connect: connection refused" start-of-body=
Sep 10 19:28:21.796517 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:21.796331    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-85e90bf8-openshift-default-6cddcfb646-t6zzp" podUID="878f8b91-4648-4800-9031-576eb949ae08" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.34:15021/healthz/ready\": dial tcp 10.132.0.34:15021: connect: connection refused"
Sep 10 19:28:22.795972 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:22.795939    2575 patch_prober.go:28] interesting pod/e2e-ait-85e90bf8-openshift-default-6cddcfb646-t6zzp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.34:15021/healthz/ready\": dial tcp 10.132.0.34:15021: connect: connection refused" start-of-body=
Sep 10 19:28:22.796374 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:22.796002    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-85e90bf8-openshift-default-6cddcfb646-t6zzp" podUID="878f8b91-4648-4800-9031-576eb949ae08" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.34:15021/healthz/ready\": dial tcp 10.132.0.34:15021: connect: connection refused"
Sep 10 19:28:23.419459 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:23.419422    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:28:23.796222 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:23.796191    2575 patch_prober.go:28] interesting pod/e2e-ait-85e90bf8-openshift-default-6cddcfb646-t6zzp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.34:15021/healthz/ready\": dial tcp 10.132.0.34:15021: connect: connection refused" start-of-body=
Sep 10 19:28:23.796627 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:23.796243    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-85e90bf8-openshift-default-6cddcfb646-t6zzp" podUID="878f8b91-4648-4800-9031-576eb949ae08" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.34:15021/healthz/ready\": dial tcp 10.132.0.34:15021: connect: connection refused"
Sep 10 19:28:24.959032 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:24.958996    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-85e90bf8-openshift-default-6cddcfb646-t6zzp"
Sep 10 19:28:24.959604 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:24.959467    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-85e90bf8-openshift-default-6cddcfb646-t6zzp"
Sep 10 19:28:24.961673 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:24.961637    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-85e90bf8-openshift-default-6cddcfb646-t6zzp"
Sep 10 19:28:35.459156 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:35.459121    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qr7rl"]
Sep 10 19:28:35.928982 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:35.928896    2575 generic.go:356] "Generic (PLEG): container finished" podID="c3aea00e-ff8b-4739-9e90-3c5838c124a2" containerID="66ec3d473001ce2815592ee14441ef7c2e1a52c261e5f2145b355e6df64bf27c" exitCode=0
Sep 10 19:28:35.929144 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:35.928973    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-4247c7-openshift-default-6c679cc99d-lk98v" event={"ID":"c3aea00e-ff8b-4739-9e90-3c5838c124a2","Type":"ContainerDied","Data":"66ec3d473001ce2815592ee14441ef7c2e1a52c261e5f2145b355e6df64bf27c"}
Sep 10 19:28:35.929427 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:35.929406    2575 scope.go:117] "RemoveContainer" containerID="66ec3d473001ce2815592ee14441ef7c2e1a52c261e5f2145b355e6df64bf27c"
Sep 10 19:28:35.930476 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:35.930448    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 19:28:35.930574 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:35.930512    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 19:28:35.930574 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:35.930544    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 19:28:36.939630 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:36.938756    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-4247c7-openshift-default-6c679cc99d-lk98v" event={"ID":"c3aea00e-ff8b-4739-9e90-3c5838c124a2","Type":"ContainerStarted","Data":"493add7f5bcc1df6a5d509d3d607bbfdc3073baf018ce010188007e973aa53ec"}
Sep 10 19:28:37.163288 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:37.163253    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-4247c7-openshift-default-6c679cc99d-lk98v"
Sep 10 19:28:37.163544 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:37.163526    2575 patch_prober.go:28] interesting pod/e2e-inf-b-4247c7-openshift-default-6c679cc99d-lk98v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body=
Sep 10 19:28:37.163597 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:37.163567    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-4247c7-openshift-default-6c679cc99d-lk98v" podUID="c3aea00e-ff8b-4739-9e90-3c5838c124a2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused"
Sep 10 19:28:37.315060 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:37.315020    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:28:38.163423 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:38.163386    2575 patch_prober.go:28] interesting pod/e2e-inf-b-4247c7-openshift-default-6c679cc99d-lk98v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body=
Sep 10 19:28:38.163894 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:38.163448    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-4247c7-openshift-default-6c679cc99d-lk98v" podUID="c3aea00e-ff8b-4739-9e90-3c5838c124a2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused"
Sep 10 19:28:39.163032 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:39.162991    2575 patch_prober.go:28] interesting pod/e2e-inf-b-4247c7-openshift-default-6c679cc99d-lk98v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body=
Sep 10 19:28:39.163213 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:39.163061    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-4247c7-openshift-default-6c679cc99d-lk98v" podUID="c3aea00e-ff8b-4739-9e90-3c5838c124a2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused"
Sep 10 19:28:39.551463 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:39.551407    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:28:39.717567 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:39.717514    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:28:40.162608 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:40.162569    2575 patch_prober.go:28] interesting pod/e2e-inf-b-4247c7-openshift-default-6c679cc99d-lk98v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body=
Sep 10 19:28:40.162791 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:40.162636    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-4247c7-openshift-default-6c679cc99d-lk98v" podUID="c3aea00e-ff8b-4739-9e90-3c5838c124a2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused"
Sep 10 19:28:40.416610 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:40.416529    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:28:40.450127 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:40.450092    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-85e90bf8-7c44767c7c-qqvgc"]
Sep 10 19:28:40.450319 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:40.450297    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-85e90bf8-7c44767c7c-qqvgc" podUID="a6a3b8b1-9656-46e4-97fc-bca77ec9d941" containerName="payload-processing" containerID="cri-o://60552f8310568c1cc93f3dd68e942cbe33642d2ce9ff1b1bab103fadba58f24f" gracePeriod=30
Sep 10 19:28:40.543659 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:40.542915    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:28:40.620506 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:40.617471    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:28:40.686789 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:40.686706    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:28:40.730366 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:40.722546    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-85e90bf8-7c44767c7c-qqvgc"
Sep 10 19:28:40.773716 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:40.773479    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bpj85\" (UniqueName: \"kubernetes.io/projected/a6a3b8b1-9656-46e4-97fc-bca77ec9d941-kube-api-access-bpj85\") pod \"a6a3b8b1-9656-46e4-97fc-bca77ec9d941\" (UID: \"a6a3b8b1-9656-46e4-97fc-bca77ec9d941\") "
Sep 10 19:28:40.773716 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:40.773609    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a6a3b8b1-9656-46e4-97fc-bca77ec9d941-plugins-config-volume\") pod \"a6a3b8b1-9656-46e4-97fc-bca77ec9d941\" (UID: \"a6a3b8b1-9656-46e4-97fc-bca77ec9d941\") "
Sep 10 19:28:40.774242 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:40.774217    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a6a3b8b1-9656-46e4-97fc-bca77ec9d941-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a6a3b8b1-9656-46e4-97fc-bca77ec9d941" (UID: "a6a3b8b1-9656-46e4-97fc-bca77ec9d941"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:28:40.777507 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:40.777481    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a6a3b8b1-9656-46e4-97fc-bca77ec9d941-kube-api-access-bpj85" (OuterVolumeSpecName: "kube-api-access-bpj85") pod "a6a3b8b1-9656-46e4-97fc-bca77ec9d941" (UID: "a6a3b8b1-9656-46e4-97fc-bca77ec9d941"). InnerVolumeSpecName "kube-api-access-bpj85". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:28:40.801994 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:40.801968    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-85e90bf8-openshift-default-6cddcfb646-t6zzp"]
Sep 10 19:28:40.802208 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:40.802185    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-85e90bf8-openshift-default-6cddcfb646-t6zzp" podUID="878f8b91-4648-4800-9031-576eb949ae08" containerName="istio-proxy" containerID="cri-o://8f45b34325e7f5e5f582046ed2e08212a820f027997cceb08e0ed43f08a74465" gracePeriod=30
Sep 10 19:28:40.841365 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:40.839444    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:28:40.875431 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:40.875397    2575 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a6a3b8b1-9656-46e4-97fc-bca77ec9d941-plugins-config-volume\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:28:40.875431 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:40.875422    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bpj85\" (UniqueName: \"kubernetes.io/projected/a6a3b8b1-9656-46e4-97fc-bca77ec9d941-kube-api-access-bpj85\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:28:40.961852 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:40.961817    2575 generic.go:356] "Generic (PLEG): container finished" podID="a6a3b8b1-9656-46e4-97fc-bca77ec9d941" containerID="60552f8310568c1cc93f3dd68e942cbe33642d2ce9ff1b1bab103fadba58f24f" exitCode=0
Sep 10 19:28:40.962031 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:40.961861    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-85e90bf8-7c44767c7c-qqvgc"
Sep 10 19:28:40.962031 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:40.961861    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-85e90bf8-7c44767c7c-qqvgc" event={"ID":"a6a3b8b1-9656-46e4-97fc-bca77ec9d941","Type":"ContainerDied","Data":"60552f8310568c1cc93f3dd68e942cbe33642d2ce9ff1b1bab103fadba58f24f"}
Sep 10 19:28:40.962031 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:40.961922    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-85e90bf8-7c44767c7c-qqvgc" event={"ID":"a6a3b8b1-9656-46e4-97fc-bca77ec9d941","Type":"ContainerDied","Data":"77df2d23a9322083b45032864b55fae79b2a12dd7e9461e5a774e44e98a92c27"}
Sep 10 19:28:40.962031 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:40.961952    2575 scope.go:117] "RemoveContainer" containerID="60552f8310568c1cc93f3dd68e942cbe33642d2ce9ff1b1bab103fadba58f24f"
Sep 10 19:28:40.971632 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:40.971590    2575 scope.go:117] "RemoveContainer" containerID="60552f8310568c1cc93f3dd68e942cbe33642d2ce9ff1b1bab103fadba58f24f"
Sep 10 19:28:40.971928 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:28:40.971909    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"60552f8310568c1cc93f3dd68e942cbe33642d2ce9ff1b1bab103fadba58f24f\": container with ID starting with 60552f8310568c1cc93f3dd68e942cbe33642d2ce9ff1b1bab103fadba58f24f not found: ID does not exist" containerID="60552f8310568c1cc93f3dd68e942cbe33642d2ce9ff1b1bab103fadba58f24f"
Sep 10 19:28:40.971998 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:40.971939    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"60552f8310568c1cc93f3dd68e942cbe33642d2ce9ff1b1bab103fadba58f24f"} err="failed to get container status \"60552f8310568c1cc93f3dd68e942cbe33642d2ce9ff1b1bab103fadba58f24f\": rpc error: code = NotFound desc = could not find container \"60552f8310568c1cc93f3dd68e942cbe33642d2ce9ff1b1bab103fadba58f24f\": container with ID starting with 60552f8310568c1cc93f3dd68e942cbe33642d2ce9ff1b1bab103fadba58f24f not found: ID does not exist"
Sep 10 19:28:40.982462 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:40.982440    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-85e90bf8-7c44767c7c-qqvgc"]
Sep 10 19:28:40.988033 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:40.988014    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-85e90bf8-7c44767c7c-qqvgc"]
Sep 10 19:28:41.163564 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:41.163531    2575 patch_prober.go:28] interesting pod/e2e-inf-b-4247c7-openshift-default-6c679cc99d-lk98v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body=
Sep 10 19:28:41.163727 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:41.163583    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-4247c7-openshift-default-6c679cc99d-lk98v" podUID="c3aea00e-ff8b-4739-9e90-3c5838c124a2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused"
Sep 10 19:28:41.283130 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:41.283042    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:28:41.381352 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:41.381310    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:28:41.385020 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:41.384989    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a6a3b8b1-9656-46e4-97fc-bca77ec9d941" path="/var/lib/kubelet/pods/a6a3b8b1-9656-46e4-97fc-bca77ec9d941/volumes"
Sep 10 19:28:41.482162 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:41.482127    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:28:41.765957 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:41.765912    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-80c77821-gw-openshift-default-6b5cc6f576-hz6cr"]
Sep 10 19:28:41.766722 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:41.766698    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a6a3b8b1-9656-46e4-97fc-bca77ec9d941" containerName="payload-processing"
Sep 10 19:28:41.766775 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:41.766728    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="a6a3b8b1-9656-46e4-97fc-bca77ec9d941" containerName="payload-processing"
Sep 10 19:28:41.766908 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:41.766894    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="a6a3b8b1-9656-46e4-97fc-bca77ec9d941" containerName="payload-processing"
Sep 10 19:28:41.771720 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:41.771695    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-80c77821-gw-openshift-default-6b5cc6f576-hz6cr"
Sep 10 19:28:41.773795 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:41.773775    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-80c77821-gw-openshift-default-dockercfg-nbk8t\""
Sep 10 19:28:41.780662 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:41.780633    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-80c77821-gw-openshift-default-6b5cc6f576-hz6cr"]
Sep 10 19:28:41.794860 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:41.794823    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:28:41.884733 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:41.884696    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4e2b654-340b-49d3-bb7d-be131be3c7f8-credential-socket\") pod \"e2e-migrate-80c77821-gw-openshift-default-6b5cc6f576-hz6cr\" (UID: \"e4e2b654-340b-49d3-bb7d-be131be3c7f8\") " pod="openshift-ingress/e2e-migrate-80c77821-gw-openshift-default-6b5cc6f576-hz6cr"
Sep 10 19:28:41.884911 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:41.884741    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e4e2b654-340b-49d3-bb7d-be131be3c7f8-istiod-ca-cert\") pod \"e2e-migrate-80c77821-gw-openshift-default-6b5cc6f576-hz6cr\" (UID: \"e4e2b654-340b-49d3-bb7d-be131be3c7f8\") " pod="openshift-ingress/e2e-migrate-80c77821-gw-openshift-default-6b5cc6f576-hz6cr"
Sep 10 19:28:41.884911 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:41.884774    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e4e2b654-340b-49d3-bb7d-be131be3c7f8-istio-token\") pod \"e2e-migrate-80c77821-gw-openshift-default-6b5cc6f576-hz6cr\" (UID: \"e4e2b654-340b-49d3-bb7d-be131be3c7f8\") " pod="openshift-ingress/e2e-migrate-80c77821-gw-openshift-default-6b5cc6f576-hz6cr"
Sep 10 19:28:41.884911 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:41.884868    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6v89g\" (UniqueName: \"kubernetes.io/projected/e4e2b654-340b-49d3-bb7d-be131be3c7f8-kube-api-access-6v89g\") pod \"e2e-migrate-80c77821-gw-openshift-default-6b5cc6f576-hz6cr\" (UID: \"e4e2b654-340b-49d3-bb7d-be131be3c7f8\") " pod="openshift-ingress/e2e-migrate-80c77821-gw-openshift-default-6b5cc6f576-hz6cr"
Sep 10 19:28:41.885037 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:41.884981    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e4e2b654-340b-49d3-bb7d-be131be3c7f8-istio-envoy\") pod \"e2e-migrate-80c77821-gw-openshift-default-6b5cc6f576-hz6cr\" (UID: \"e4e2b654-340b-49d3-bb7d-be131be3c7f8\") " pod="openshift-ingress/e2e-migrate-80c77821-gw-openshift-default-6b5cc6f576-hz6cr"
Sep 10 19:28:41.885083 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:41.885044    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e4e2b654-340b-49d3-bb7d-be131be3c7f8-istio-data\") pod \"e2e-migrate-80c77821-gw-openshift-default-6b5cc6f576-hz6cr\" (UID: \"e4e2b654-340b-49d3-bb7d-be131be3c7f8\") " pod="openshift-ingress/e2e-migrate-80c77821-gw-openshift-default-6b5cc6f576-hz6cr"
Sep 10 19:28:41.885139 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:41.885091    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4e2b654-340b-49d3-bb7d-be131be3c7f8-workload-socket\") pod \"e2e-migrate-80c77821-gw-openshift-default-6b5cc6f576-hz6cr\" (UID: \"e4e2b654-340b-49d3-bb7d-be131be3c7f8\") " pod="openshift-ingress/e2e-migrate-80c77821-gw-openshift-default-6b5cc6f576-hz6cr"
Sep 10 19:28:41.885139 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:41.885123    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e4e2b654-340b-49d3-bb7d-be131be3c7f8-workload-certs\") pod \"e2e-migrate-80c77821-gw-openshift-default-6b5cc6f576-hz6cr\" (UID: \"e4e2b654-340b-49d3-bb7d-be131be3c7f8\") " pod="openshift-ingress/e2e-migrate-80c77821-gw-openshift-default-6b5cc6f576-hz6cr"
Sep 10 19:28:41.885223 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:41.885153    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e4e2b654-340b-49d3-bb7d-be131be3c7f8-istio-podinfo\") pod \"e2e-migrate-80c77821-gw-openshift-default-6b5cc6f576-hz6cr\" (UID: \"e4e2b654-340b-49d3-bb7d-be131be3c7f8\") " pod="openshift-ingress/e2e-migrate-80c77821-gw-openshift-default-6b5cc6f576-hz6cr"
Sep 10 19:28:41.986186 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:41.986158    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6v89g\" (UniqueName: \"kubernetes.io/projected/e4e2b654-340b-49d3-bb7d-be131be3c7f8-kube-api-access-6v89g\") pod \"e2e-migrate-80c77821-gw-openshift-default-6b5cc6f576-hz6cr\" (UID: \"e4e2b654-340b-49d3-bb7d-be131be3c7f8\") " pod="openshift-ingress/e2e-migrate-80c77821-gw-openshift-default-6b5cc6f576-hz6cr"
Sep 10 19:28:41.986391 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:41.986220    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e4e2b654-340b-49d3-bb7d-be131be3c7f8-istio-envoy\") pod \"e2e-migrate-80c77821-gw-openshift-default-6b5cc6f576-hz6cr\" (UID: \"e4e2b654-340b-49d3-bb7d-be131be3c7f8\") " pod="openshift-ingress/e2e-migrate-80c77821-gw-openshift-default-6b5cc6f576-hz6cr"
Sep 10 19:28:41.986391 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:41.986253    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e4e2b654-340b-49d3-bb7d-be131be3c7f8-istio-data\") pod \"e2e-migrate-80c77821-gw-openshift-default-6b5cc6f576-hz6cr\" (UID: \"e4e2b654-340b-49d3-bb7d-be131be3c7f8\") " pod="openshift-ingress/e2e-migrate-80c77821-gw-openshift-default-6b5cc6f576-hz6cr"
Sep 10 19:28:41.986391 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:41.986280    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4e2b654-340b-49d3-bb7d-be131be3c7f8-workload-socket\") pod \"e2e-migrate-80c77821-gw-openshift-default-6b5cc6f576-hz6cr\" (UID: \"e4e2b654-340b-49d3-bb7d-be131be3c7f8\") " pod="openshift-ingress/e2e-migrate-80c77821-gw-openshift-default-6b5cc6f576-hz6cr"
Sep 10 19:28:41.986391 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:41.986299    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e4e2b654-340b-49d3-bb7d-be131be3c7f8-workload-certs\") pod \"e2e-migrate-80c77821-gw-openshift-default-6b5cc6f576-hz6cr\" (UID: \"e4e2b654-340b-49d3-bb7d-be131be3c7f8\") " pod="openshift-ingress/e2e-migrate-80c77821-gw-openshift-default-6b5cc6f576-hz6cr"
Sep 10 19:28:41.986391 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:41.986329    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e4e2b654-340b-49d3-bb7d-be131be3c7f8-istio-podinfo\") pod \"e2e-migrate-80c77821-gw-openshift-default-6b5cc6f576-hz6cr\" (UID: \"e4e2b654-340b-49d3-bb7d-be131be3c7f8\") " pod="openshift-ingress/e2e-migrate-80c77821-gw-openshift-default-6b5cc6f576-hz6cr"
Sep 10 19:28:41.986662 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:41.986398    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4e2b654-340b-49d3-bb7d-be131be3c7f8-credential-socket\") pod \"e2e-migrate-80c77821-gw-openshift-default-6b5cc6f576-hz6cr\" (UID: \"e4e2b654-340b-49d3-bb7d-be131be3c7f8\") " pod="openshift-ingress/e2e-migrate-80c77821-gw-openshift-default-6b5cc6f576-hz6cr"
Sep 10 19:28:41.986662 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:41.986423    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e4e2b654-340b-49d3-bb7d-be131be3c7f8-istiod-ca-cert\") pod \"e2e-migrate-80c77821-gw-openshift-default-6b5cc6f576-hz6cr\" (UID: \"e4e2b654-340b-49d3-bb7d-be131be3c7f8\") " pod="openshift-ingress/e2e-migrate-80c77821-gw-openshift-default-6b5cc6f576-hz6cr"
Sep 10 19:28:41.986662 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:41.986456    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e4e2b654-340b-49d3-bb7d-be131be3c7f8-istio-token\") pod \"e2e-migrate-80c77821-gw-openshift-default-6b5cc6f576-hz6cr\" (UID: \"e4e2b654-340b-49d3-bb7d-be131be3c7f8\") " pod="openshift-ingress/e2e-migrate-80c77821-gw-openshift-default-6b5cc6f576-hz6cr"
Sep 10 19:28:41.986811 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:41.986752    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4e2b654-340b-49d3-bb7d-be131be3c7f8-workload-socket\") pod \"e2e-migrate-80c77821-gw-openshift-default-6b5cc6f576-hz6cr\" (UID: \"e4e2b654-340b-49d3-bb7d-be131be3c7f8\") " pod="openshift-ingress/e2e-migrate-80c77821-gw-openshift-default-6b5cc6f576-hz6cr"
Sep 10 19:28:41.986910 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:41.986868    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e4e2b654-340b-49d3-bb7d-be131be3c7f8-istio-data\") pod \"e2e-migrate-80c77821-gw-openshift-default-6b5cc6f576-hz6cr\" (UID: \"e4e2b654-340b-49d3-bb7d-be131be3c7f8\") " pod="openshift-ingress/e2e-migrate-80c77821-gw-openshift-default-6b5cc6f576-hz6cr"
Sep 10 19:28:41.987111 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:41.987064    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e4e2b654-340b-49d3-bb7d-be131be3c7f8-workload-certs\") pod \"e2e-migrate-80c77821-gw-openshift-default-6b5cc6f576-hz6cr\" (UID: \"e4e2b654-340b-49d3-bb7d-be131be3c7f8\") " pod="openshift-ingress/e2e-migrate-80c77821-gw-openshift-default-6b5cc6f576-hz6cr"
Sep 10 19:28:41.987205 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:41.987187    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4e2b654-340b-49d3-bb7d-be131be3c7f8-credential-socket\") pod \"e2e-migrate-80c77821-gw-openshift-default-6b5cc6f576-hz6cr\" (UID: \"e4e2b654-340b-49d3-bb7d-be131be3c7f8\") " pod="openshift-ingress/e2e-migrate-80c77821-gw-openshift-default-6b5cc6f576-hz6cr"
Sep 10 19:28:41.987438 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:41.987419    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e4e2b654-340b-49d3-bb7d-be131be3c7f8-istiod-ca-cert\") pod \"e2e-migrate-80c77821-gw-openshift-default-6b5cc6f576-hz6cr\" (UID: \"e4e2b654-340b-49d3-bb7d-be131be3c7f8\") " pod="openshift-ingress/e2e-migrate-80c77821-gw-openshift-default-6b5cc6f576-hz6cr"
Sep 10 19:28:41.988570 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:41.988553    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e4e2b654-340b-49d3-bb7d-be131be3c7f8-istio-envoy\") pod \"e2e-migrate-80c77821-gw-openshift-default-6b5cc6f576-hz6cr\" (UID: \"e4e2b654-340b-49d3-bb7d-be131be3c7f8\") " pod="openshift-ingress/e2e-migrate-80c77821-gw-openshift-default-6b5cc6f576-hz6cr"
Sep 10 19:28:41.989208 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:41.989184    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e4e2b654-340b-49d3-bb7d-be131be3c7f8-istio-podinfo\") pod \"e2e-migrate-80c77821-gw-openshift-default-6b5cc6f576-hz6cr\" (UID: \"e4e2b654-340b-49d3-bb7d-be131be3c7f8\") " pod="openshift-ingress/e2e-migrate-80c77821-gw-openshift-default-6b5cc6f576-hz6cr"
Sep 10 19:28:41.994466 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:41.994439    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e4e2b654-340b-49d3-bb7d-be131be3c7f8-istio-token\") pod \"e2e-migrate-80c77821-gw-openshift-default-6b5cc6f576-hz6cr\" (UID: \"e4e2b654-340b-49d3-bb7d-be131be3c7f8\") " pod="openshift-ingress/e2e-migrate-80c77821-gw-openshift-default-6b5cc6f576-hz6cr"
Sep 10 19:28:41.994651 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:41.994634    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6v89g\" (UniqueName: \"kubernetes.io/projected/e4e2b654-340b-49d3-bb7d-be131be3c7f8-kube-api-access-6v89g\") pod \"e2e-migrate-80c77821-gw-openshift-default-6b5cc6f576-hz6cr\" (UID: \"e4e2b654-340b-49d3-bb7d-be131be3c7f8\") " pod="openshift-ingress/e2e-migrate-80c77821-gw-openshift-default-6b5cc6f576-hz6cr"
Sep 10 19:28:42.084531 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:42.084447    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-80c77821-gw-openshift-default-6b5cc6f576-hz6cr"
Sep 10 19:28:42.167200 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:42.167161    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-4247c7-openshift-default-6c679cc99d-lk98v"
Sep 10 19:28:42.167685 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:42.167666    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-4247c7-openshift-default-6c679cc99d-lk98v"
Sep 10 19:28:42.168155 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:42.168134    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-4247c7-openshift-default-6c679cc99d-lk98v"
Sep 10 19:28:42.179828 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:42.179800    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:28:42.365151 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:42.365128    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-80c77821-gw-openshift-default-6b5cc6f576-hz6cr"]
Sep 10 19:28:42.368008 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:28:42.367937    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4e2b654_340b_49d3_bb7d_be131be3c7f8.slice/crio-83b33255d6cb172dbf7e3dbbeb378ee56be5321d4169b6efe3ebaf8f979a7353 WatchSource:0}: Error finding container 83b33255d6cb172dbf7e3dbbeb378ee56be5321d4169b6efe3ebaf8f979a7353: Status 404 returned error can't find the container with id 83b33255d6cb172dbf7e3dbbeb378ee56be5321d4169b6efe3ebaf8f979a7353
Sep 10 19:28:42.370559 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:42.370524    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 19:28:42.370655 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:42.370590    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 19:28:42.370655 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:42.370623    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 19:28:42.973490 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:42.973455    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-80c77821-gw-openshift-default-6b5cc6f576-hz6cr" event={"ID":"e4e2b654-340b-49d3-bb7d-be131be3c7f8","Type":"ContainerStarted","Data":"92b2840bde931366d98809b43c5d9b7f72518356050759cdac76b293f2628f82"}
Sep 10 19:28:42.973985 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:42.973510    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-80c77821-gw-openshift-default-6b5cc6f576-hz6cr" event={"ID":"e4e2b654-340b-49d3-bb7d-be131be3c7f8","Type":"ContainerStarted","Data":"83b33255d6cb172dbf7e3dbbeb378ee56be5321d4169b6efe3ebaf8f979a7353"}
Sep 10 19:28:42.996106 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:42.996053    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-80c77821-gw-openshift-default-6b5cc6f576-hz6cr" podStartSLOduration=1.9960326290000001 podStartE2EDuration="1.996032629s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:28:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:28:42.992775346 +0000 UTC m=+794.052603014" watchObservedRunningTime="2026-09-10 19:28:42.996032629 +0000 UTC m=+794.055860297"
Sep 10 19:28:43.085518 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:43.085494    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-80c77821-gw-openshift-default-6b5cc6f576-hz6cr"
Sep 10 19:28:43.090321 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:43.090299    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-80c77821-gw-openshift-default-6b5cc6f576-hz6cr"
Sep 10 19:28:43.977896 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:43.977872    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-80c77821-gw-openshift-default-6b5cc6f576-hz6cr"
Sep 10 19:28:43.978917 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:43.978898    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-80c77821-gw-openshift-default-6b5cc6f576-hz6cr"
Sep 10 19:28:44.451211 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:44.451151    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:28:44.551094 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:44.551058    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:28:44.647670 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:44.647630    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:28:44.747922 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:44.747840    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:28:44.808552 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:44.808526    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:28:44.862988 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:44.862960    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-conflict-b-479f6157-kserve-858f766cd6-g9knt"]
Sep 10 19:28:44.866608 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:44.866590    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-479f6157-kserve-858f766cd6-g9knt"
Sep 10 19:28:44.868931 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:44.868910    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-conflict-b-479f6157-kserve-self-signed-certs\""
Sep 10 19:28:44.880308 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:44.880284    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-conflict-b-479f6157-kserve-858f766cd6-g9knt"]
Sep 10 19:28:44.915748 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:44.915721    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sdwz6\" (UniqueName: \"kubernetes.io/projected/5529e50e-798e-4f48-bcbb-7a032f7650b6-kube-api-access-sdwz6\") pod \"e2e-conflict-b-479f6157-kserve-858f766cd6-g9knt\" (UID: \"5529e50e-798e-4f48-bcbb-7a032f7650b6\") " pod="llm/e2e-conflict-b-479f6157-kserve-858f766cd6-g9knt"
Sep 10 19:28:44.915857 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:44.915760    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5529e50e-798e-4f48-bcbb-7a032f7650b6-dshm\") pod \"e2e-conflict-b-479f6157-kserve-858f766cd6-g9knt\" (UID: \"5529e50e-798e-4f48-bcbb-7a032f7650b6\") " pod="llm/e2e-conflict-b-479f6157-kserve-858f766cd6-g9knt"
Sep 10 19:28:44.915857 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:44.915826    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5529e50e-798e-4f48-bcbb-7a032f7650b6-tmp-dir\") pod \"e2e-conflict-b-479f6157-kserve-858f766cd6-g9knt\" (UID: \"5529e50e-798e-4f48-bcbb-7a032f7650b6\") " pod="llm/e2e-conflict-b-479f6157-kserve-858f766cd6-g9knt"
Sep 10 19:28:44.915976 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:44.915855    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5529e50e-798e-4f48-bcbb-7a032f7650b6-home\") pod \"e2e-conflict-b-479f6157-kserve-858f766cd6-g9knt\" (UID: \"5529e50e-798e-4f48-bcbb-7a032f7650b6\") " pod="llm/e2e-conflict-b-479f6157-kserve-858f766cd6-g9knt"
Sep 10 19:28:44.915976 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:44.915912    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5529e50e-798e-4f48-bcbb-7a032f7650b6-tls-certs\") pod \"e2e-conflict-b-479f6157-kserve-858f766cd6-g9knt\" (UID: \"5529e50e-798e-4f48-bcbb-7a032f7650b6\") " pod="llm/e2e-conflict-b-479f6157-kserve-858f766cd6-g9knt"
Sep 10 19:28:44.916067 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:44.915992    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5529e50e-798e-4f48-bcbb-7a032f7650b6-model-cache\") pod \"e2e-conflict-b-479f6157-kserve-858f766cd6-g9knt\" (UID: \"5529e50e-798e-4f48-bcbb-7a032f7650b6\") " pod="llm/e2e-conflict-b-479f6157-kserve-858f766cd6-g9knt"
Sep 10 19:28:44.971685 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:44.971660    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:28:45.017230 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:45.017172    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5529e50e-798e-4f48-bcbb-7a032f7650b6-model-cache\") pod \"e2e-conflict-b-479f6157-kserve-858f766cd6-g9knt\" (UID: \"5529e50e-798e-4f48-bcbb-7a032f7650b6\") " pod="llm/e2e-conflict-b-479f6157-kserve-858f766cd6-g9knt"
Sep 10 19:28:45.017230 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:45.017219    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sdwz6\" (UniqueName: \"kubernetes.io/projected/5529e50e-798e-4f48-bcbb-7a032f7650b6-kube-api-access-sdwz6\") pod \"e2e-conflict-b-479f6157-kserve-858f766cd6-g9knt\" (UID: \"5529e50e-798e-4f48-bcbb-7a032f7650b6\") " pod="llm/e2e-conflict-b-479f6157-kserve-858f766cd6-g9knt"
Sep 10 19:28:45.017678 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:45.017239    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5529e50e-798e-4f48-bcbb-7a032f7650b6-dshm\") pod \"e2e-conflict-b-479f6157-kserve-858f766cd6-g9knt\" (UID: \"5529e50e-798e-4f48-bcbb-7a032f7650b6\") " pod="llm/e2e-conflict-b-479f6157-kserve-858f766cd6-g9knt"
Sep 10 19:28:45.017678 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:45.017288    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5529e50e-798e-4f48-bcbb-7a032f7650b6-tmp-dir\") pod \"e2e-conflict-b-479f6157-kserve-858f766cd6-g9knt\" (UID: \"5529e50e-798e-4f48-bcbb-7a032f7650b6\") " pod="llm/e2e-conflict-b-479f6157-kserve-858f766cd6-g9knt"
Sep 10 19:28:45.017678 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:45.017317    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5529e50e-798e-4f48-bcbb-7a032f7650b6-home\") pod \"e2e-conflict-b-479f6157-kserve-858f766cd6-g9knt\" (UID: \"5529e50e-798e-4f48-bcbb-7a032f7650b6\") " pod="llm/e2e-conflict-b-479f6157-kserve-858f766cd6-g9knt"
Sep 10 19:28:45.017678 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:45.017380    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5529e50e-798e-4f48-bcbb-7a032f7650b6-tls-certs\") pod \"e2e-conflict-b-479f6157-kserve-858f766cd6-g9knt\" (UID: \"5529e50e-798e-4f48-bcbb-7a032f7650b6\") " pod="llm/e2e-conflict-b-479f6157-kserve-858f766cd6-g9knt"
Sep 10 19:28:45.017678 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:45.017632    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5529e50e-798e-4f48-bcbb-7a032f7650b6-model-cache\") pod \"e2e-conflict-b-479f6157-kserve-858f766cd6-g9knt\" (UID: \"5529e50e-798e-4f48-bcbb-7a032f7650b6\") " pod="llm/e2e-conflict-b-479f6157-kserve-858f766cd6-g9knt"
Sep 10 19:28:45.017933 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:45.017721    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5529e50e-798e-4f48-bcbb-7a032f7650b6-tmp-dir\") pod \"e2e-conflict-b-479f6157-kserve-858f766cd6-g9knt\" (UID: \"5529e50e-798e-4f48-bcbb-7a032f7650b6\") " pod="llm/e2e-conflict-b-479f6157-kserve-858f766cd6-g9knt"
Sep 10 19:28:45.017933 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:45.017774    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5529e50e-798e-4f48-bcbb-7a032f7650b6-home\") pod \"e2e-conflict-b-479f6157-kserve-858f766cd6-g9knt\" (UID: \"5529e50e-798e-4f48-bcbb-7a032f7650b6\") " pod="llm/e2e-conflict-b-479f6157-kserve-858f766cd6-g9knt"
Sep 10 19:28:45.019603 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:45.019581    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5529e50e-798e-4f48-bcbb-7a032f7650b6-dshm\") pod \"e2e-conflict-b-479f6157-kserve-858f766cd6-g9knt\" (UID: \"5529e50e-798e-4f48-bcbb-7a032f7650b6\") " pod="llm/e2e-conflict-b-479f6157-kserve-858f766cd6-g9knt"
Sep 10 19:28:45.019857 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:45.019837    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5529e50e-798e-4f48-bcbb-7a032f7650b6-tls-certs\") pod \"e2e-conflict-b-479f6157-kserve-858f766cd6-g9knt\" (UID: \"5529e50e-798e-4f48-bcbb-7a032f7650b6\") " pod="llm/e2e-conflict-b-479f6157-kserve-858f766cd6-g9knt"
Sep 10 19:28:45.025283 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:45.025264    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sdwz6\" (UniqueName: \"kubernetes.io/projected/5529e50e-798e-4f48-bcbb-7a032f7650b6-kube-api-access-sdwz6\") pod \"e2e-conflict-b-479f6157-kserve-858f766cd6-g9knt\" (UID: \"5529e50e-798e-4f48-bcbb-7a032f7650b6\") " pod="llm/e2e-conflict-b-479f6157-kserve-858f766cd6-g9knt"
Sep 10 19:28:45.182853 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:45.182823    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-479f6157-kserve-858f766cd6-g9knt"
Sep 10 19:28:45.273835 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:45.273763    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:28:45.374014 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:45.373972    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:28:45.457743 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:45.457719    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-conflict-b-479f6157-kserve-858f766cd6-g9knt"]
Sep 10 19:28:45.460022 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:28:45.459994    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5529e50e_798e_4f48_bcbb_7a032f7650b6.slice/crio-d32db3d66f52c9ae27309c8d5dcaef5e77d621c38333420d67dc5a18bbf83ac1 WatchSource:0}: Error finding container d32db3d66f52c9ae27309c8d5dcaef5e77d621c38333420d67dc5a18bbf83ac1: Status 404 returned error can't find the container with id d32db3d66f52c9ae27309c8d5dcaef5e77d621c38333420d67dc5a18bbf83ac1
Sep 10 19:28:45.991298 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:45.991259    2575 generic.go:356] "Generic (PLEG): container finished" podID="878f8b91-4648-4800-9031-576eb949ae08" containerID="8f45b34325e7f5e5f582046ed2e08212a820f027997cceb08e0ed43f08a74465" exitCode=0
Sep 10 19:28:45.991480 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:45.991319    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-85e90bf8-openshift-default-6cddcfb646-t6zzp" event={"ID":"878f8b91-4648-4800-9031-576eb949ae08","Type":"ContainerDied","Data":"8f45b34325e7f5e5f582046ed2e08212a820f027997cceb08e0ed43f08a74465"}
Sep 10 19:28:45.991480 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:45.991394    2575 scope.go:117] "RemoveContainer" containerID="1af16d615123b089010ca82da0a2a50f78aad9c0e41637e2aa48172b2af006d6"
Sep 10 19:28:45.993667 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:45.993640    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-479f6157-kserve-858f766cd6-g9knt" event={"ID":"5529e50e-798e-4f48-bcbb-7a032f7650b6","Type":"ContainerStarted","Data":"0b11a7fa87a1205b824784e078dee0375c7577f26b46df831bdbb7f3211066f2"}
Sep 10 19:28:45.993810 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:45.993675    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-479f6157-kserve-858f766cd6-g9knt" event={"ID":"5529e50e-798e-4f48-bcbb-7a032f7650b6","Type":"ContainerStarted","Data":"d32db3d66f52c9ae27309c8d5dcaef5e77d621c38333420d67dc5a18bbf83ac1"}
Sep 10 19:28:46.014878 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:46.014828    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-conflict-b-479f6157-kserve-858f766cd6-g9knt" podStartSLOduration=2.014809267 podStartE2EDuration="2.014809267s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:28:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:28:46.010989305 +0000 UTC m=+797.070816973" watchObservedRunningTime="2026-09-10 19:28:46.014809267 +0000 UTC m=+797.074636936"
Sep 10 19:28:46.055025 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:46.055005    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-85e90bf8-openshift-default-6cddcfb646-t6zzp"
Sep 10 19:28:46.128398 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:46.128304    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/878f8b91-4648-4800-9031-576eb949ae08-workload-certs\") pod \"878f8b91-4648-4800-9031-576eb949ae08\" (UID: \"878f8b91-4648-4800-9031-576eb949ae08\") "
Sep 10 19:28:46.128398 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:46.128381    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/878f8b91-4648-4800-9031-576eb949ae08-istio-data\") pod \"878f8b91-4648-4800-9031-576eb949ae08\" (UID: \"878f8b91-4648-4800-9031-576eb949ae08\") "
Sep 10 19:28:46.128592 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:46.128418    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/878f8b91-4648-4800-9031-576eb949ae08-istio-podinfo\") pod \"878f8b91-4648-4800-9031-576eb949ae08\" (UID: \"878f8b91-4648-4800-9031-576eb949ae08\") "
Sep 10 19:28:46.128592 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:46.128478    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/878f8b91-4648-4800-9031-576eb949ae08-istiod-ca-cert\") pod \"878f8b91-4648-4800-9031-576eb949ae08\" (UID: \"878f8b91-4648-4800-9031-576eb949ae08\") "
Sep 10 19:28:46.128592 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:46.128520    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/878f8b91-4648-4800-9031-576eb949ae08-istio-token\") pod \"878f8b91-4648-4800-9031-576eb949ae08\" (UID: \"878f8b91-4648-4800-9031-576eb949ae08\") "
Sep 10 19:28:46.128592 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:46.128551    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/878f8b91-4648-4800-9031-576eb949ae08-istio-envoy\") pod \"878f8b91-4648-4800-9031-576eb949ae08\" (UID: \"878f8b91-4648-4800-9031-576eb949ae08\") "
Sep 10 19:28:46.128592 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:46.128581    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/878f8b91-4648-4800-9031-576eb949ae08-credential-socket\") pod \"878f8b91-4648-4800-9031-576eb949ae08\" (UID: \"878f8b91-4648-4800-9031-576eb949ae08\") "
Sep 10 19:28:46.128833 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:46.128635    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/878f8b91-4648-4800-9031-576eb949ae08-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "878f8b91-4648-4800-9031-576eb949ae08" (UID: "878f8b91-4648-4800-9031-576eb949ae08"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:28:46.128833 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:46.128643    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zww2b\" (UniqueName: \"kubernetes.io/projected/878f8b91-4648-4800-9031-576eb949ae08-kube-api-access-zww2b\") pod \"878f8b91-4648-4800-9031-576eb949ae08\" (UID: \"878f8b91-4648-4800-9031-576eb949ae08\") "
Sep 10 19:28:46.128833 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:46.128672    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/878f8b91-4648-4800-9031-576eb949ae08-workload-socket\") pod \"878f8b91-4648-4800-9031-576eb949ae08\" (UID: \"878f8b91-4648-4800-9031-576eb949ae08\") "
Sep 10 19:28:46.128833 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:46.128708    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/878f8b91-4648-4800-9031-576eb949ae08-istio-data" (OuterVolumeSpecName: "istio-data") pod "878f8b91-4648-4800-9031-576eb949ae08" (UID: "878f8b91-4648-4800-9031-576eb949ae08"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:28:46.129041 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:46.128850    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/878f8b91-4648-4800-9031-576eb949ae08-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "878f8b91-4648-4800-9031-576eb949ae08" (UID: "878f8b91-4648-4800-9031-576eb949ae08"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:28:46.129041 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:46.128997    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/878f8b91-4648-4800-9031-576eb949ae08-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "878f8b91-4648-4800-9031-576eb949ae08" (UID: "878f8b91-4648-4800-9031-576eb949ae08"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:28:46.129041 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:46.129030    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/878f8b91-4648-4800-9031-576eb949ae08-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "878f8b91-4648-4800-9031-576eb949ae08" (UID: "878f8b91-4648-4800-9031-576eb949ae08"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:28:46.129486 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:46.129367    2575 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/878f8b91-4648-4800-9031-576eb949ae08-workload-socket\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:28:46.129745 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:46.129532    2575 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/878f8b91-4648-4800-9031-576eb949ae08-workload-certs\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:28:46.129745 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:46.129556    2575 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/878f8b91-4648-4800-9031-576eb949ae08-istio-data\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:28:46.129745 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:46.129571    2575 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/878f8b91-4648-4800-9031-576eb949ae08-istiod-ca-cert\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:28:46.129745 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:46.129585    2575 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/878f8b91-4648-4800-9031-576eb949ae08-credential-socket\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:28:46.130962 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:46.130940    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/878f8b91-4648-4800-9031-576eb949ae08-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "878f8b91-4648-4800-9031-576eb949ae08" (UID: "878f8b91-4648-4800-9031-576eb949ae08"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 19:28:46.131033 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:46.131011    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/878f8b91-4648-4800-9031-576eb949ae08-istio-token" (OuterVolumeSpecName: "istio-token") pod "878f8b91-4648-4800-9031-576eb949ae08" (UID: "878f8b91-4648-4800-9031-576eb949ae08"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:28:46.131184 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:46.131170    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/878f8b91-4648-4800-9031-576eb949ae08-kube-api-access-zww2b" (OuterVolumeSpecName: "kube-api-access-zww2b") pod "878f8b91-4648-4800-9031-576eb949ae08" (UID: "878f8b91-4648-4800-9031-576eb949ae08"). InnerVolumeSpecName "kube-api-access-zww2b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:28:46.131507 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:46.131482    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/878f8b91-4648-4800-9031-576eb949ae08-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "878f8b91-4648-4800-9031-576eb949ae08" (UID: "878f8b91-4648-4800-9031-576eb949ae08"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:28:46.230244 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:46.230215    2575 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/878f8b91-4648-4800-9031-576eb949ae08-istio-token\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:28:46.230244 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:46.230239    2575 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/878f8b91-4648-4800-9031-576eb949ae08-istio-envoy\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:28:46.230412 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:46.230253    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zww2b\" (UniqueName: \"kubernetes.io/projected/878f8b91-4648-4800-9031-576eb949ae08-kube-api-access-zww2b\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:28:46.230412 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:46.230282    2575 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/878f8b91-4648-4800-9031-576eb949ae08-istio-podinfo\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:28:46.999863 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:46.999835    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-85e90bf8-openshift-default-6cddcfb646-t6zzp"
Sep 10 19:28:47.000024 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:46.999877    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-85e90bf8-openshift-default-6cddcfb646-t6zzp" event={"ID":"878f8b91-4648-4800-9031-576eb949ae08","Type":"ContainerDied","Data":"7ef0613c96269ab60a94f85dbf96493d40166496e574dc32a7a4e6f1e5a803a4"}
Sep 10 19:28:47.000024 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:46.999928    2575 scope.go:117] "RemoveContainer" containerID="8f45b34325e7f5e5f582046ed2e08212a820f027997cceb08e0ed43f08a74465"
Sep 10 19:28:47.033951 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:47.033921    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-85e90bf8-openshift-default-6cddcfb646-t6zzp"]
Sep 10 19:28:47.038998 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:47.038971    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-85e90bf8-openshift-default-6cddcfb646-t6zzp"]
Sep 10 19:28:47.385055 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:47.384981    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="878f8b91-4648-4800-9031-576eb949ae08" path="/var/lib/kubelet/pods/878f8b91-4648-4800-9031-576eb949ae08/volumes"
Sep 10 19:28:51.857743 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:51.857698    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qr7rl"]
Sep 10 19:28:55.183039 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:55.183003    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-conflict-b-479f6157-kserve-858f766cd6-g9knt"
Sep 10 19:28:55.183536 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:55.183284    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-conflict-b-479f6157-kserve-858f766cd6-g9knt"
Sep 10 19:28:55.195801 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:55.195778    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-conflict-b-479f6157-kserve-858f766cd6-g9knt"
Sep 10 19:28:56.061644 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:28:56.061615    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-conflict-b-479f6157-kserve-858f766cd6-g9knt"
Sep 10 19:29:00.019568 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:00.019520    2575 patch_prober.go:28] interesting pod/e2e-migrate-80c77821-gw-openshift-default-6b5cc6f576-hz6cr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 10 19:29:00.020117 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:00.019616    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-80c77821-gw-openshift-default-6b5cc6f576-hz6cr" podUID="e4e2b654-340b-49d3-bb7d-be131be3c7f8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 10 19:29:02.082013 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:02.081976    2575 generic.go:356] "Generic (PLEG): container finished" podID="e4e2b654-340b-49d3-bb7d-be131be3c7f8" containerID="92b2840bde931366d98809b43c5d9b7f72518356050759cdac76b293f2628f82" exitCode=0
Sep 10 19:29:02.082398 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:02.082037    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-80c77821-gw-openshift-default-6b5cc6f576-hz6cr" event={"ID":"e4e2b654-340b-49d3-bb7d-be131be3c7f8","Type":"ContainerDied","Data":"92b2840bde931366d98809b43c5d9b7f72518356050759cdac76b293f2628f82"}
Sep 10 19:29:02.082548 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:02.082534    2575 scope.go:117] "RemoveContainer" containerID="92b2840bde931366d98809b43c5d9b7f72518356050759cdac76b293f2628f82"
Sep 10 19:29:02.083594 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:02.083565    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 19:29:02.083665 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:02.083634    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 19:29:02.083703 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:02.083665    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 19:29:03.089946 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:03.089915    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-80c77821-gw-openshift-default-6b5cc6f576-hz6cr" event={"ID":"e4e2b654-340b-49d3-bb7d-be131be3c7f8","Type":"ContainerStarted","Data":"e86dfc11d4e0aca5a32046d61e24158da06ffdb4ec94be3d99d9ed2a9e660288"}
Sep 10 19:29:04.084931 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:04.084890    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-80c77821-gw-openshift-default-6b5cc6f576-hz6cr"
Sep 10 19:29:04.085175 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:04.085150    2575 patch_prober.go:28] interesting pod/e2e-migrate-80c77821-gw-openshift-default-6b5cc6f576-hz6cr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" start-of-body=
Sep 10 19:29:04.085270 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:04.085209    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-80c77821-gw-openshift-default-6b5cc6f576-hz6cr" podUID="e4e2b654-340b-49d3-bb7d-be131be3c7f8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused"
Sep 10 19:29:05.085168 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:05.085131    2575 patch_prober.go:28] interesting pod/e2e-migrate-80c77821-gw-openshift-default-6b5cc6f576-hz6cr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" start-of-body=
Sep 10 19:29:05.085662 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:05.085193    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-80c77821-gw-openshift-default-6b5cc6f576-hz6cr" podUID="e4e2b654-340b-49d3-bb7d-be131be3c7f8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused"
Sep 10 19:29:05.519401 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:05.519366    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:05.590052 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:05.590006    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:06.085362 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:06.085314    2575 patch_prober.go:28] interesting pod/e2e-migrate-80c77821-gw-openshift-default-6b5cc6f576-hz6cr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" start-of-body=
Sep 10 19:29:06.085740 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:06.085390    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-80c77821-gw-openshift-default-6b5cc6f576-hz6cr" podUID="e4e2b654-340b-49d3-bb7d-be131be3c7f8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused"
Sep 10 19:29:07.085049 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:07.085018    2575 patch_prober.go:28] interesting pod/e2e-migrate-80c77821-gw-openshift-default-6b5cc6f576-hz6cr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" start-of-body=
Sep 10 19:29:07.085214 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:07.085081    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-80c77821-gw-openshift-default-6b5cc6f576-hz6cr" podUID="e4e2b654-340b-49d3-bb7d-be131be3c7f8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused"
Sep 10 19:29:08.085351 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:08.085301    2575 patch_prober.go:28] interesting pod/e2e-migrate-80c77821-gw-openshift-default-6b5cc6f576-hz6cr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" start-of-body=
Sep 10 19:29:08.085761 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:08.085379    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-80c77821-gw-openshift-default-6b5cc6f576-hz6cr" podUID="e4e2b654-340b-49d3-bb7d-be131be3c7f8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused"
Sep 10 19:29:09.046060 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:09.046021    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qr7rl"]
Sep 10 19:29:09.088168 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:09.088142    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-80c77821-gw-openshift-default-6b5cc6f576-hz6cr"
Sep 10 19:29:09.088625 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:09.088546    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-80c77821-gw-openshift-default-6b5cc6f576-hz6cr"
Sep 10 19:29:09.089118 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:09.089098    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-80c77821-gw-openshift-default-6b5cc6f576-hz6cr"
Sep 10 19:29:16.078875 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:16.078843    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:16.179599 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:16.179562    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:16.411228 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:16.411144    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-80c77821-gw-openshift-default-6b5cc6f576-hz6cr"]
Sep 10 19:29:16.411452 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:16.411406    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-80c77821-gw-openshift-default-6b5cc6f576-hz6cr" podUID="e4e2b654-340b-49d3-bb7d-be131be3c7f8" containerName="istio-proxy" containerID="cri-o://e86dfc11d4e0aca5a32046d61e24158da06ffdb4ec94be3d99d9ed2a9e660288" gracePeriod=30
Sep 10 19:29:16.515280 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:16.515241    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:16.614557 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:16.614529    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:16.877870 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:16.877827    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:17.173571 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:17.173498    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-f0b1e2fe-openshift-default-887748fc4-pk6ww"]
Sep 10 19:29:17.173918 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:17.173904    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="878f8b91-4648-4800-9031-576eb949ae08" containerName="istio-proxy"
Sep 10 19:29:17.173961 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:17.173920    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="878f8b91-4648-4800-9031-576eb949ae08" containerName="istio-proxy"
Sep 10 19:29:17.173961 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:17.173934    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="878f8b91-4648-4800-9031-576eb949ae08" containerName="istio-proxy"
Sep 10 19:29:17.173961 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:17.173940    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="878f8b91-4648-4800-9031-576eb949ae08" containerName="istio-proxy"
Sep 10 19:29:17.174092 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:17.174039    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="878f8b91-4648-4800-9031-576eb949ae08" containerName="istio-proxy"
Sep 10 19:29:17.174092 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:17.174051    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="878f8b91-4648-4800-9031-576eb949ae08" containerName="istio-proxy"
Sep 10 19:29:17.178295 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:17.178279    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f0b1e2fe-openshift-default-887748fc4-pk6ww"
Sep 10 19:29:17.181028 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:17.181010    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-f0b1e2fe-openshift-default-dockercfg-ztsbl\""
Sep 10 19:29:17.192558 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:17.192535    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-f0b1e2fe-openshift-default-887748fc4-pk6ww"]
Sep 10 19:29:17.211642 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:17.211618    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:17.230145 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:17.230121    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s5lc4\" (UniqueName: \"kubernetes.io/projected/1cb07606-a586-46f1-8cb0-d5bae5be695a-kube-api-access-s5lc4\") pod \"e2e-ait-f0b1e2fe-openshift-default-887748fc4-pk6ww\" (UID: \"1cb07606-a586-46f1-8cb0-d5bae5be695a\") " pod="openshift-ingress/e2e-ait-f0b1e2fe-openshift-default-887748fc4-pk6ww"
Sep 10 19:29:17.230243 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:17.230150    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1cb07606-a586-46f1-8cb0-d5bae5be695a-workload-certs\") pod \"e2e-ait-f0b1e2fe-openshift-default-887748fc4-pk6ww\" (UID: \"1cb07606-a586-46f1-8cb0-d5bae5be695a\") " pod="openshift-ingress/e2e-ait-f0b1e2fe-openshift-default-887748fc4-pk6ww"
Sep 10 19:29:17.230243 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:17.230229    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1cb07606-a586-46f1-8cb0-d5bae5be695a-istio-token\") pod \"e2e-ait-f0b1e2fe-openshift-default-887748fc4-pk6ww\" (UID: \"1cb07606-a586-46f1-8cb0-d5bae5be695a\") " pod="openshift-ingress/e2e-ait-f0b1e2fe-openshift-default-887748fc4-pk6ww"
Sep 10 19:29:17.230352 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:17.230245    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1cb07606-a586-46f1-8cb0-d5bae5be695a-credential-socket\") pod \"e2e-ait-f0b1e2fe-openshift-default-887748fc4-pk6ww\" (UID: \"1cb07606-a586-46f1-8cb0-d5bae5be695a\") " pod="openshift-ingress/e2e-ait-f0b1e2fe-openshift-default-887748fc4-pk6ww"
Sep 10 19:29:17.230352 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:17.230265    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1cb07606-a586-46f1-8cb0-d5bae5be695a-istiod-ca-cert\") pod \"e2e-ait-f0b1e2fe-openshift-default-887748fc4-pk6ww\" (UID: \"1cb07606-a586-46f1-8cb0-d5bae5be695a\") " pod="openshift-ingress/e2e-ait-f0b1e2fe-openshift-default-887748fc4-pk6ww"
Sep 10 19:29:17.230352 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:17.230315    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1cb07606-a586-46f1-8cb0-d5bae5be695a-workload-socket\") pod \"e2e-ait-f0b1e2fe-openshift-default-887748fc4-pk6ww\" (UID: \"1cb07606-a586-46f1-8cb0-d5bae5be695a\") " pod="openshift-ingress/e2e-ait-f0b1e2fe-openshift-default-887748fc4-pk6ww"
Sep 10 19:29:17.230511 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:17.230358    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1cb07606-a586-46f1-8cb0-d5bae5be695a-istio-envoy\") pod \"e2e-ait-f0b1e2fe-openshift-default-887748fc4-pk6ww\" (UID: \"1cb07606-a586-46f1-8cb0-d5bae5be695a\") " pod="openshift-ingress/e2e-ait-f0b1e2fe-openshift-default-887748fc4-pk6ww"
Sep 10 19:29:17.230511 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:17.230418    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1cb07606-a586-46f1-8cb0-d5bae5be695a-istio-podinfo\") pod \"e2e-ait-f0b1e2fe-openshift-default-887748fc4-pk6ww\" (UID: \"1cb07606-a586-46f1-8cb0-d5bae5be695a\") " pod="openshift-ingress/e2e-ait-f0b1e2fe-openshift-default-887748fc4-pk6ww"
Sep 10 19:29:17.230511 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:17.230459    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1cb07606-a586-46f1-8cb0-d5bae5be695a-istio-data\") pod \"e2e-ait-f0b1e2fe-openshift-default-887748fc4-pk6ww\" (UID: \"1cb07606-a586-46f1-8cb0-d5bae5be695a\") " pod="openshift-ingress/e2e-ait-f0b1e2fe-openshift-default-887748fc4-pk6ww"
Sep 10 19:29:17.330944 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:17.330913    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1cb07606-a586-46f1-8cb0-d5bae5be695a-workload-socket\") pod \"e2e-ait-f0b1e2fe-openshift-default-887748fc4-pk6ww\" (UID: \"1cb07606-a586-46f1-8cb0-d5bae5be695a\") " pod="openshift-ingress/e2e-ait-f0b1e2fe-openshift-default-887748fc4-pk6ww"
Sep 10 19:29:17.330944 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:17.330953    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1cb07606-a586-46f1-8cb0-d5bae5be695a-istio-envoy\") pod \"e2e-ait-f0b1e2fe-openshift-default-887748fc4-pk6ww\" (UID: \"1cb07606-a586-46f1-8cb0-d5bae5be695a\") " pod="openshift-ingress/e2e-ait-f0b1e2fe-openshift-default-887748fc4-pk6ww"
Sep 10 19:29:17.331192 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:17.331062    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1cb07606-a586-46f1-8cb0-d5bae5be695a-istio-podinfo\") pod \"e2e-ait-f0b1e2fe-openshift-default-887748fc4-pk6ww\" (UID: \"1cb07606-a586-46f1-8cb0-d5bae5be695a\") " pod="openshift-ingress/e2e-ait-f0b1e2fe-openshift-default-887748fc4-pk6ww"
Sep 10 19:29:17.331192 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:17.331115    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1cb07606-a586-46f1-8cb0-d5bae5be695a-istio-data\") pod \"e2e-ait-f0b1e2fe-openshift-default-887748fc4-pk6ww\" (UID: \"1cb07606-a586-46f1-8cb0-d5bae5be695a\") " pod="openshift-ingress/e2e-ait-f0b1e2fe-openshift-default-887748fc4-pk6ww"
Sep 10 19:29:17.331192 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:17.331165    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s5lc4\" (UniqueName: \"kubernetes.io/projected/1cb07606-a586-46f1-8cb0-d5bae5be695a-kube-api-access-s5lc4\") pod \"e2e-ait-f0b1e2fe-openshift-default-887748fc4-pk6ww\" (UID: \"1cb07606-a586-46f1-8cb0-d5bae5be695a\") " pod="openshift-ingress/e2e-ait-f0b1e2fe-openshift-default-887748fc4-pk6ww"
Sep 10 19:29:17.331362 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:17.331195    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1cb07606-a586-46f1-8cb0-d5bae5be695a-workload-certs\") pod \"e2e-ait-f0b1e2fe-openshift-default-887748fc4-pk6ww\" (UID: \"1cb07606-a586-46f1-8cb0-d5bae5be695a\") " pod="openshift-ingress/e2e-ait-f0b1e2fe-openshift-default-887748fc4-pk6ww"
Sep 10 19:29:17.331362 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:17.331305    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1cb07606-a586-46f1-8cb0-d5bae5be695a-istio-token\") pod \"e2e-ait-f0b1e2fe-openshift-default-887748fc4-pk6ww\" (UID: \"1cb07606-a586-46f1-8cb0-d5bae5be695a\") " pod="openshift-ingress/e2e-ait-f0b1e2fe-openshift-default-887748fc4-pk6ww"
Sep 10 19:29:17.331362 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:17.331330    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1cb07606-a586-46f1-8cb0-d5bae5be695a-credential-socket\") pod \"e2e-ait-f0b1e2fe-openshift-default-887748fc4-pk6ww\" (UID: \"1cb07606-a586-46f1-8cb0-d5bae5be695a\") " pod="openshift-ingress/e2e-ait-f0b1e2fe-openshift-default-887748fc4-pk6ww"
Sep 10 19:29:17.331494 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:17.331384    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1cb07606-a586-46f1-8cb0-d5bae5be695a-istiod-ca-cert\") pod \"e2e-ait-f0b1e2fe-openshift-default-887748fc4-pk6ww\" (UID: \"1cb07606-a586-46f1-8cb0-d5bae5be695a\") " pod="openshift-ingress/e2e-ait-f0b1e2fe-openshift-default-887748fc4-pk6ww"
Sep 10 19:29:17.331494 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:17.331396    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1cb07606-a586-46f1-8cb0-d5bae5be695a-workload-socket\") pod \"e2e-ait-f0b1e2fe-openshift-default-887748fc4-pk6ww\" (UID: \"1cb07606-a586-46f1-8cb0-d5bae5be695a\") " pod="openshift-ingress/e2e-ait-f0b1e2fe-openshift-default-887748fc4-pk6ww"
Sep 10 19:29:17.331494 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:17.331461    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1cb07606-a586-46f1-8cb0-d5bae5be695a-istio-data\") pod \"e2e-ait-f0b1e2fe-openshift-default-887748fc4-pk6ww\" (UID: \"1cb07606-a586-46f1-8cb0-d5bae5be695a\") " pod="openshift-ingress/e2e-ait-f0b1e2fe-openshift-default-887748fc4-pk6ww"
Sep 10 19:29:17.331719 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:17.331694    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1cb07606-a586-46f1-8cb0-d5bae5be695a-workload-certs\") pod \"e2e-ait-f0b1e2fe-openshift-default-887748fc4-pk6ww\" (UID: \"1cb07606-a586-46f1-8cb0-d5bae5be695a\") " pod="openshift-ingress/e2e-ait-f0b1e2fe-openshift-default-887748fc4-pk6ww"
Sep 10 19:29:17.332000 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:17.331967    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:17.332102 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:17.332021    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1cb07606-a586-46f1-8cb0-d5bae5be695a-credential-socket\") pod \"e2e-ait-f0b1e2fe-openshift-default-887748fc4-pk6ww\" (UID: \"1cb07606-a586-46f1-8cb0-d5bae5be695a\") " pod="openshift-ingress/e2e-ait-f0b1e2fe-openshift-default-887748fc4-pk6ww"
Sep 10 19:29:17.332335 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:17.332314    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1cb07606-a586-46f1-8cb0-d5bae5be695a-istiod-ca-cert\") pod \"e2e-ait-f0b1e2fe-openshift-default-887748fc4-pk6ww\" (UID: \"1cb07606-a586-46f1-8cb0-d5bae5be695a\") " pod="openshift-ingress/e2e-ait-f0b1e2fe-openshift-default-887748fc4-pk6ww"
Sep 10 19:29:17.333454 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:17.333431    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1cb07606-a586-46f1-8cb0-d5bae5be695a-istio-envoy\") pod \"e2e-ait-f0b1e2fe-openshift-default-887748fc4-pk6ww\" (UID: \"1cb07606-a586-46f1-8cb0-d5bae5be695a\") " pod="openshift-ingress/e2e-ait-f0b1e2fe-openshift-default-887748fc4-pk6ww"
Sep 10 19:29:17.334044 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:17.334025    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1cb07606-a586-46f1-8cb0-d5bae5be695a-istio-podinfo\") pod \"e2e-ait-f0b1e2fe-openshift-default-887748fc4-pk6ww\" (UID: \"1cb07606-a586-46f1-8cb0-d5bae5be695a\") " pod="openshift-ingress/e2e-ait-f0b1e2fe-openshift-default-887748fc4-pk6ww"
Sep 10 19:29:17.341163 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:17.341142    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1cb07606-a586-46f1-8cb0-d5bae5be695a-istio-token\") pod \"e2e-ait-f0b1e2fe-openshift-default-887748fc4-pk6ww\" (UID: \"1cb07606-a586-46f1-8cb0-d5bae5be695a\") " pod="openshift-ingress/e2e-ait-f0b1e2fe-openshift-default-887748fc4-pk6ww"
Sep 10 19:29:17.341350 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:17.341317    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s5lc4\" (UniqueName: \"kubernetes.io/projected/1cb07606-a586-46f1-8cb0-d5bae5be695a-kube-api-access-s5lc4\") pod \"e2e-ait-f0b1e2fe-openshift-default-887748fc4-pk6ww\" (UID: \"1cb07606-a586-46f1-8cb0-d5bae5be695a\") " pod="openshift-ingress/e2e-ait-f0b1e2fe-openshift-default-887748fc4-pk6ww"
Sep 10 19:29:17.489482 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:17.489446    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f0b1e2fe-openshift-default-887748fc4-pk6ww"
Sep 10 19:29:17.776015 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:17.775990    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-f0b1e2fe-openshift-default-887748fc4-pk6ww"]
Sep 10 19:29:17.777978 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:29:17.777943    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1cb07606_a586_46f1_8cb0_d5bae5be695a.slice/crio-e56592af285233614042f46d5d376fb9a8cc83d3f1271c99ddac2067fe28baa8 WatchSource:0}: Error finding container e56592af285233614042f46d5d376fb9a8cc83d3f1271c99ddac2067fe28baa8: Status 404 returned error can't find the container with id e56592af285233614042f46d5d376fb9a8cc83d3f1271c99ddac2067fe28baa8
Sep 10 19:29:17.780465 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:17.780422    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 19:29:17.780561 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:17.780519    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 19:29:17.780597 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:17.780563    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 19:29:18.168115 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:18.168023    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f0b1e2fe-openshift-default-887748fc4-pk6ww" event={"ID":"1cb07606-a586-46f1-8cb0-d5bae5be695a","Type":"ContainerStarted","Data":"74a59f5f405340f9dd4f6ea42e5bbdcb0a4f118712bb30d891b7e1e33de0f5af"}
Sep 10 19:29:18.168115 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:18.168076    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f0b1e2fe-openshift-default-887748fc4-pk6ww" event={"ID":"1cb07606-a586-46f1-8cb0-d5bae5be695a","Type":"ContainerStarted","Data":"e56592af285233614042f46d5d376fb9a8cc83d3f1271c99ddac2067fe28baa8"}
Sep 10 19:29:18.189046 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:18.189002    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-f0b1e2fe-openshift-default-887748fc4-pk6ww" podStartSLOduration=1.188987135 podStartE2EDuration="1.188987135s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:29:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:29:18.186947732 +0000 UTC m=+829.246775402" watchObservedRunningTime="2026-09-10 19:29:18.188987135 +0000 UTC m=+829.248814865"
Sep 10 19:29:18.489997 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:18.489963    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-f0b1e2fe-openshift-default-887748fc4-pk6ww"
Sep 10 19:29:18.494852 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:18.494829    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-f0b1e2fe-openshift-default-887748fc4-pk6ww"
Sep 10 19:29:19.172917 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:19.172884    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-f0b1e2fe-openshift-default-887748fc4-pk6ww"
Sep 10 19:29:19.173873 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:19.173854    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-f0b1e2fe-openshift-default-887748fc4-pk6ww"
Sep 10 19:29:19.524173 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:19.524141    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:19.617944 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:19.617913    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:20.694472 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:20.694440    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:20.793916 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:20.793882    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:21.206409 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:21.206370    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:21.308384 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:21.308357    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:21.395669 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:21.395641    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:21.661853 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:21.661830    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-80c77821-gw-openshift-default-6b5cc6f576-hz6cr"
Sep 10 19:29:21.777112 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:21.777081    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e4e2b654-340b-49d3-bb7d-be131be3c7f8-workload-certs\") pod \"e4e2b654-340b-49d3-bb7d-be131be3c7f8\" (UID: \"e4e2b654-340b-49d3-bb7d-be131be3c7f8\") "
Sep 10 19:29:21.777581 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:21.777125    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e4e2b654-340b-49d3-bb7d-be131be3c7f8-istio-envoy\") pod \"e4e2b654-340b-49d3-bb7d-be131be3c7f8\" (UID: \"e4e2b654-340b-49d3-bb7d-be131be3c7f8\") "
Sep 10 19:29:21.777581 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:21.777156    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e4e2b654-340b-49d3-bb7d-be131be3c7f8-istiod-ca-cert\") pod \"e4e2b654-340b-49d3-bb7d-be131be3c7f8\" (UID: \"e4e2b654-340b-49d3-bb7d-be131be3c7f8\") "
Sep 10 19:29:21.777581 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:21.777175    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6v89g\" (UniqueName: \"kubernetes.io/projected/e4e2b654-340b-49d3-bb7d-be131be3c7f8-kube-api-access-6v89g\") pod \"e4e2b654-340b-49d3-bb7d-be131be3c7f8\" (UID: \"e4e2b654-340b-49d3-bb7d-be131be3c7f8\") "
Sep 10 19:29:21.777581 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:21.777206    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e4e2b654-340b-49d3-bb7d-be131be3c7f8-istio-data\") pod \"e4e2b654-340b-49d3-bb7d-be131be3c7f8\" (UID: \"e4e2b654-340b-49d3-bb7d-be131be3c7f8\") "
Sep 10 19:29:21.777581 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:21.777238    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4e2b654-340b-49d3-bb7d-be131be3c7f8-workload-socket\") pod \"e4e2b654-340b-49d3-bb7d-be131be3c7f8\" (UID: \"e4e2b654-340b-49d3-bb7d-be131be3c7f8\") "
Sep 10 19:29:21.777581 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:21.777278    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e4e2b654-340b-49d3-bb7d-be131be3c7f8-istio-token\") pod \"e4e2b654-340b-49d3-bb7d-be131be3c7f8\" (UID: \"e4e2b654-340b-49d3-bb7d-be131be3c7f8\") "
Sep 10 19:29:21.777581 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:21.777369    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e4e2b654-340b-49d3-bb7d-be131be3c7f8-istio-podinfo\") pod \"e4e2b654-340b-49d3-bb7d-be131be3c7f8\" (UID: \"e4e2b654-340b-49d3-bb7d-be131be3c7f8\") "
Sep 10 19:29:21.777581 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:21.777372    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4e2b654-340b-49d3-bb7d-be131be3c7f8-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e4e2b654-340b-49d3-bb7d-be131be3c7f8" (UID: "e4e2b654-340b-49d3-bb7d-be131be3c7f8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:29:21.777581 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:21.777400    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4e2b654-340b-49d3-bb7d-be131be3c7f8-credential-socket\") pod \"e4e2b654-340b-49d3-bb7d-be131be3c7f8\" (UID: \"e4e2b654-340b-49d3-bb7d-be131be3c7f8\") "
Sep 10 19:29:21.777581 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:21.777536    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4e2b654-340b-49d3-bb7d-be131be3c7f8-istio-data" (OuterVolumeSpecName: "istio-data") pod "e4e2b654-340b-49d3-bb7d-be131be3c7f8" (UID: "e4e2b654-340b-49d3-bb7d-be131be3c7f8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:29:21.778101 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:21.777627    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e4e2b654-340b-49d3-bb7d-be131be3c7f8-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e4e2b654-340b-49d3-bb7d-be131be3c7f8" (UID: "e4e2b654-340b-49d3-bb7d-be131be3c7f8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:29:21.778101 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:21.777771    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4e2b654-340b-49d3-bb7d-be131be3c7f8-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e4e2b654-340b-49d3-bb7d-be131be3c7f8" (UID: "e4e2b654-340b-49d3-bb7d-be131be3c7f8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:29:21.778101 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:21.777792    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4e2b654-340b-49d3-bb7d-be131be3c7f8-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e4e2b654-340b-49d3-bb7d-be131be3c7f8" (UID: "e4e2b654-340b-49d3-bb7d-be131be3c7f8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:29:21.778101 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:21.777851    2575 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e4e2b654-340b-49d3-bb7d-be131be3c7f8-workload-certs\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:29:21.778101 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:21.777870    2575 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e4e2b654-340b-49d3-bb7d-be131be3c7f8-istiod-ca-cert\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:29:21.778101 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:21.777883    2575 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e4e2b654-340b-49d3-bb7d-be131be3c7f8-istio-data\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:29:21.779593 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:21.779520    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e4e2b654-340b-49d3-bb7d-be131be3c7f8-kube-api-access-6v89g" (OuterVolumeSpecName: "kube-api-access-6v89g") pod "e4e2b654-340b-49d3-bb7d-be131be3c7f8" (UID: "e4e2b654-340b-49d3-bb7d-be131be3c7f8"). InnerVolumeSpecName "kube-api-access-6v89g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:29:21.779593 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:21.779554    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e4e2b654-340b-49d3-bb7d-be131be3c7f8-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e4e2b654-340b-49d3-bb7d-be131be3c7f8" (UID: "e4e2b654-340b-49d3-bb7d-be131be3c7f8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 19:29:21.779727 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:21.779599    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e4e2b654-340b-49d3-bb7d-be131be3c7f8-istio-token" (OuterVolumeSpecName: "istio-token") pod "e4e2b654-340b-49d3-bb7d-be131be3c7f8" (UID: "e4e2b654-340b-49d3-bb7d-be131be3c7f8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:29:21.779727 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:21.779603    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4e2b654-340b-49d3-bb7d-be131be3c7f8-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e4e2b654-340b-49d3-bb7d-be131be3c7f8" (UID: "e4e2b654-340b-49d3-bb7d-be131be3c7f8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:29:21.878416 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:21.878393    2575 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e4e2b654-340b-49d3-bb7d-be131be3c7f8-istio-envoy\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:29:21.878416 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:21.878414    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6v89g\" (UniqueName: \"kubernetes.io/projected/e4e2b654-340b-49d3-bb7d-be131be3c7f8-kube-api-access-6v89g\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:29:21.878593 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:21.878425    2575 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4e2b654-340b-49d3-bb7d-be131be3c7f8-workload-socket\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:29:21.878593 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:21.878434    2575 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e4e2b654-340b-49d3-bb7d-be131be3c7f8-istio-token\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:29:21.878593 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:21.878442    2575 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e4e2b654-340b-49d3-bb7d-be131be3c7f8-istio-podinfo\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:29:21.878593 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:21.878450    2575 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4e2b654-340b-49d3-bb7d-be131be3c7f8-credential-socket\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:29:22.190358 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:22.190252    2575 generic.go:356] "Generic (PLEG): container finished" podID="e4e2b654-340b-49d3-bb7d-be131be3c7f8" containerID="e86dfc11d4e0aca5a32046d61e24158da06ffdb4ec94be3d99d9ed2a9e660288" exitCode=0
Sep 10 19:29:22.190358 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:22.190301    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-80c77821-gw-openshift-default-6b5cc6f576-hz6cr"
Sep 10 19:29:22.190572 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:22.190335    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-80c77821-gw-openshift-default-6b5cc6f576-hz6cr" event={"ID":"e4e2b654-340b-49d3-bb7d-be131be3c7f8","Type":"ContainerDied","Data":"e86dfc11d4e0aca5a32046d61e24158da06ffdb4ec94be3d99d9ed2a9e660288"}
Sep 10 19:29:22.190572 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:22.190407    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-80c77821-gw-openshift-default-6b5cc6f576-hz6cr" event={"ID":"e4e2b654-340b-49d3-bb7d-be131be3c7f8","Type":"ContainerDied","Data":"83b33255d6cb172dbf7e3dbbeb378ee56be5321d4169b6efe3ebaf8f979a7353"}
Sep 10 19:29:22.190572 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:22.190430    2575 scope.go:117] "RemoveContainer" containerID="e86dfc11d4e0aca5a32046d61e24158da06ffdb4ec94be3d99d9ed2a9e660288"
Sep 10 19:29:22.200218 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:22.200203    2575 scope.go:117] "RemoveContainer" containerID="92b2840bde931366d98809b43c5d9b7f72518356050759cdac76b293f2628f82"
Sep 10 19:29:22.208535 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:22.208520    2575 scope.go:117] "RemoveContainer" containerID="e86dfc11d4e0aca5a32046d61e24158da06ffdb4ec94be3d99d9ed2a9e660288"
Sep 10 19:29:22.208798 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:29:22.208779    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e86dfc11d4e0aca5a32046d61e24158da06ffdb4ec94be3d99d9ed2a9e660288\": container with ID starting with e86dfc11d4e0aca5a32046d61e24158da06ffdb4ec94be3d99d9ed2a9e660288 not found: ID does not exist" containerID="e86dfc11d4e0aca5a32046d61e24158da06ffdb4ec94be3d99d9ed2a9e660288"
Sep 10 19:29:22.208845 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:22.208806    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e86dfc11d4e0aca5a32046d61e24158da06ffdb4ec94be3d99d9ed2a9e660288"} err="failed to get container status \"e86dfc11d4e0aca5a32046d61e24158da06ffdb4ec94be3d99d9ed2a9e660288\": rpc error: code = NotFound desc = could not find container \"e86dfc11d4e0aca5a32046d61e24158da06ffdb4ec94be3d99d9ed2a9e660288\": container with ID starting with e86dfc11d4e0aca5a32046d61e24158da06ffdb4ec94be3d99d9ed2a9e660288 not found: ID does not exist"
Sep 10 19:29:22.208845 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:22.208821    2575 scope.go:117] "RemoveContainer" containerID="92b2840bde931366d98809b43c5d9b7f72518356050759cdac76b293f2628f82"
Sep 10 19:29:22.209144 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:29:22.209115    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"92b2840bde931366d98809b43c5d9b7f72518356050759cdac76b293f2628f82\": container with ID starting with 92b2840bde931366d98809b43c5d9b7f72518356050759cdac76b293f2628f82 not found: ID does not exist" containerID="92b2840bde931366d98809b43c5d9b7f72518356050759cdac76b293f2628f82"
Sep 10 19:29:22.209233 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:22.209146    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"92b2840bde931366d98809b43c5d9b7f72518356050759cdac76b293f2628f82"} err="failed to get container status \"92b2840bde931366d98809b43c5d9b7f72518356050759cdac76b293f2628f82\": rpc error: code = NotFound desc = could not find container \"92b2840bde931366d98809b43c5d9b7f72518356050759cdac76b293f2628f82\": container with ID starting with 92b2840bde931366d98809b43c5d9b7f72518356050759cdac76b293f2628f82 not found: ID does not exist"
Sep 10 19:29:22.211691 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:22.211670    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-80c77821-gw-openshift-default-6b5cc6f576-hz6cr"]
Sep 10 19:29:22.216006 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:22.215987    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-80c77821-gw-openshift-default-6b5cc6f576-hz6cr"]
Sep 10 19:29:23.384402 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:23.384371    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e4e2b654-340b-49d3-bb7d-be131be3c7f8" path="/var/lib/kubelet/pods/e4e2b654-340b-49d3-bb7d-be131be3c7f8/volumes"
Sep 10 19:29:25.157176 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:25.157142    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qr7rl"]
Sep 10 19:29:30.582679 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:30.582646    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-autores-b3108b/autores-model-b3108b-kserve-7dff498ff5-sv4lz"]
Sep 10 19:29:30.583081 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:30.583033    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e4e2b654-340b-49d3-bb7d-be131be3c7f8" containerName="istio-proxy"
Sep 10 19:29:30.583081 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:30.583043    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="e4e2b654-340b-49d3-bb7d-be131be3c7f8" containerName="istio-proxy"
Sep 10 19:29:30.583081 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:30.583051    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e4e2b654-340b-49d3-bb7d-be131be3c7f8" containerName="istio-proxy"
Sep 10 19:29:30.583081 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:30.583056    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="e4e2b654-340b-49d3-bb7d-be131be3c7f8" containerName="istio-proxy"
Sep 10 19:29:30.583233 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:30.583120    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="e4e2b654-340b-49d3-bb7d-be131be3c7f8" containerName="istio-proxy"
Sep 10 19:29:30.587501 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:30.587482    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-b3108b/autores-model-b3108b-kserve-7dff498ff5-sv4lz"
Sep 10 19:29:30.589927 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:30.589898    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-autores-b3108b\"/\"openshift-service-ca.crt\""
Sep 10 19:29:30.589927 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:30.589914    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-autores-b3108b\"/\"kube-root-ca.crt\""
Sep 10 19:29:30.590117 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:30.589978    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-autores-b3108b\"/\"autores-model-b3108b-kserve-self-signed-certs\""
Sep 10 19:29:30.590117 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:30.589909    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-autores-b3108b\"/\"default-dockercfg-2xl9j\""
Sep 10 19:29:30.595619 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:30.595209    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-autores-b3108b/autores-model-b3108b-kserve-7dff498ff5-sv4lz"]
Sep 10 19:29:30.653673 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:30.653638    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eb9e6693-d98b-4098-ae05-b181f5ab9fbe-home\") pod \"autores-model-b3108b-kserve-7dff498ff5-sv4lz\" (UID: \"eb9e6693-d98b-4098-ae05-b181f5ab9fbe\") " pod="ai-tenant-e2e-autores-b3108b/autores-model-b3108b-kserve-7dff498ff5-sv4lz"
Sep 10 19:29:30.653837 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:30.653756    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eb9e6693-d98b-4098-ae05-b181f5ab9fbe-dshm\") pod \"autores-model-b3108b-kserve-7dff498ff5-sv4lz\" (UID: \"eb9e6693-d98b-4098-ae05-b181f5ab9fbe\") " pod="ai-tenant-e2e-autores-b3108b/autores-model-b3108b-kserve-7dff498ff5-sv4lz"
Sep 10 19:29:30.653837 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:30.653783    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eb9e6693-d98b-4098-ae05-b181f5ab9fbe-model-cache\") pod \"autores-model-b3108b-kserve-7dff498ff5-sv4lz\" (UID: \"eb9e6693-d98b-4098-ae05-b181f5ab9fbe\") " pod="ai-tenant-e2e-autores-b3108b/autores-model-b3108b-kserve-7dff498ff5-sv4lz"
Sep 10 19:29:30.653920 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:30.653877    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eb9e6693-d98b-4098-ae05-b181f5ab9fbe-tmp-dir\") pod \"autores-model-b3108b-kserve-7dff498ff5-sv4lz\" (UID: \"eb9e6693-d98b-4098-ae05-b181f5ab9fbe\") " pod="ai-tenant-e2e-autores-b3108b/autores-model-b3108b-kserve-7dff498ff5-sv4lz"
Sep 10 19:29:30.653957 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:30.653918    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wvzdg\" (UniqueName: \"kubernetes.io/projected/eb9e6693-d98b-4098-ae05-b181f5ab9fbe-kube-api-access-wvzdg\") pod \"autores-model-b3108b-kserve-7dff498ff5-sv4lz\" (UID: \"eb9e6693-d98b-4098-ae05-b181f5ab9fbe\") " pod="ai-tenant-e2e-autores-b3108b/autores-model-b3108b-kserve-7dff498ff5-sv4lz"
Sep 10 19:29:30.654035 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:30.654017    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eb9e6693-d98b-4098-ae05-b181f5ab9fbe-tls-certs\") pod \"autores-model-b3108b-kserve-7dff498ff5-sv4lz\" (UID: \"eb9e6693-d98b-4098-ae05-b181f5ab9fbe\") " pod="ai-tenant-e2e-autores-b3108b/autores-model-b3108b-kserve-7dff498ff5-sv4lz"
Sep 10 19:29:30.755430 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:30.755398    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eb9e6693-d98b-4098-ae05-b181f5ab9fbe-dshm\") pod \"autores-model-b3108b-kserve-7dff498ff5-sv4lz\" (UID: \"eb9e6693-d98b-4098-ae05-b181f5ab9fbe\") " pod="ai-tenant-e2e-autores-b3108b/autores-model-b3108b-kserve-7dff498ff5-sv4lz"
Sep 10 19:29:30.755430 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:30.755433    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eb9e6693-d98b-4098-ae05-b181f5ab9fbe-model-cache\") pod \"autores-model-b3108b-kserve-7dff498ff5-sv4lz\" (UID: \"eb9e6693-d98b-4098-ae05-b181f5ab9fbe\") " pod="ai-tenant-e2e-autores-b3108b/autores-model-b3108b-kserve-7dff498ff5-sv4lz"
Sep 10 19:29:30.755663 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:30.755472    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eb9e6693-d98b-4098-ae05-b181f5ab9fbe-tmp-dir\") pod \"autores-model-b3108b-kserve-7dff498ff5-sv4lz\" (UID: \"eb9e6693-d98b-4098-ae05-b181f5ab9fbe\") " pod="ai-tenant-e2e-autores-b3108b/autores-model-b3108b-kserve-7dff498ff5-sv4lz"
Sep 10 19:29:30.755663 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:30.755499    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wvzdg\" (UniqueName: \"kubernetes.io/projected/eb9e6693-d98b-4098-ae05-b181f5ab9fbe-kube-api-access-wvzdg\") pod \"autores-model-b3108b-kserve-7dff498ff5-sv4lz\" (UID: \"eb9e6693-d98b-4098-ae05-b181f5ab9fbe\") " pod="ai-tenant-e2e-autores-b3108b/autores-model-b3108b-kserve-7dff498ff5-sv4lz"
Sep 10 19:29:30.755663 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:30.755555    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eb9e6693-d98b-4098-ae05-b181f5ab9fbe-tls-certs\") pod \"autores-model-b3108b-kserve-7dff498ff5-sv4lz\" (UID: \"eb9e6693-d98b-4098-ae05-b181f5ab9fbe\") " pod="ai-tenant-e2e-autores-b3108b/autores-model-b3108b-kserve-7dff498ff5-sv4lz"
Sep 10 19:29:30.755663 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:30.755574    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eb9e6693-d98b-4098-ae05-b181f5ab9fbe-home\") pod \"autores-model-b3108b-kserve-7dff498ff5-sv4lz\" (UID: \"eb9e6693-d98b-4098-ae05-b181f5ab9fbe\") " pod="ai-tenant-e2e-autores-b3108b/autores-model-b3108b-kserve-7dff498ff5-sv4lz"
Sep 10 19:29:30.755917 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:30.755891    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eb9e6693-d98b-4098-ae05-b181f5ab9fbe-model-cache\") pod \"autores-model-b3108b-kserve-7dff498ff5-sv4lz\" (UID: \"eb9e6693-d98b-4098-ae05-b181f5ab9fbe\") " pod="ai-tenant-e2e-autores-b3108b/autores-model-b3108b-kserve-7dff498ff5-sv4lz"
Sep 10 19:29:30.755986 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:30.755926    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eb9e6693-d98b-4098-ae05-b181f5ab9fbe-tmp-dir\") pod \"autores-model-b3108b-kserve-7dff498ff5-sv4lz\" (UID: \"eb9e6693-d98b-4098-ae05-b181f5ab9fbe\") " pod="ai-tenant-e2e-autores-b3108b/autores-model-b3108b-kserve-7dff498ff5-sv4lz"
Sep 10 19:29:30.756056 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:30.756031    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eb9e6693-d98b-4098-ae05-b181f5ab9fbe-home\") pod \"autores-model-b3108b-kserve-7dff498ff5-sv4lz\" (UID: \"eb9e6693-d98b-4098-ae05-b181f5ab9fbe\") " pod="ai-tenant-e2e-autores-b3108b/autores-model-b3108b-kserve-7dff498ff5-sv4lz"
Sep 10 19:29:30.757972 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:30.757949    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eb9e6693-d98b-4098-ae05-b181f5ab9fbe-dshm\") pod \"autores-model-b3108b-kserve-7dff498ff5-sv4lz\" (UID: \"eb9e6693-d98b-4098-ae05-b181f5ab9fbe\") " pod="ai-tenant-e2e-autores-b3108b/autores-model-b3108b-kserve-7dff498ff5-sv4lz"
Sep 10 19:29:30.758220 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:30.758201    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eb9e6693-d98b-4098-ae05-b181f5ab9fbe-tls-certs\") pod \"autores-model-b3108b-kserve-7dff498ff5-sv4lz\" (UID: \"eb9e6693-d98b-4098-ae05-b181f5ab9fbe\") " pod="ai-tenant-e2e-autores-b3108b/autores-model-b3108b-kserve-7dff498ff5-sv4lz"
Sep 10 19:29:30.767174 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:30.767154    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wvzdg\" (UniqueName: \"kubernetes.io/projected/eb9e6693-d98b-4098-ae05-b181f5ab9fbe-kube-api-access-wvzdg\") pod \"autores-model-b3108b-kserve-7dff498ff5-sv4lz\" (UID: \"eb9e6693-d98b-4098-ae05-b181f5ab9fbe\") " pod="ai-tenant-e2e-autores-b3108b/autores-model-b3108b-kserve-7dff498ff5-sv4lz"
Sep 10 19:29:30.785398 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:30.785375    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:30.898568 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:30.898496    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-b3108b/autores-model-b3108b-kserve-7dff498ff5-sv4lz"
Sep 10 19:29:31.201306 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:31.201283    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-autores-b3108b/autores-model-b3108b-kserve-7dff498ff5-sv4lz"]
Sep 10 19:29:31.203791 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:29:31.203762    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podeb9e6693_d98b_4098_ae05_b181f5ab9fbe.slice/crio-5c9af8d0c39a7992a7a1af890f944d29f2cb10ba9641d1885591d087af628c40 WatchSource:0}: Error finding container 5c9af8d0c39a7992a7a1af890f944d29f2cb10ba9641d1885591d087af628c40: Status 404 returned error can't find the container with id 5c9af8d0c39a7992a7a1af890f944d29f2cb10ba9641d1885591d087af628c40
Sep 10 19:29:31.246898 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:31.246846    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-b3108b/autores-model-b3108b-kserve-7dff498ff5-sv4lz" event={"ID":"eb9e6693-d98b-4098-ae05-b181f5ab9fbe","Type":"ContainerStarted","Data":"5c9af8d0c39a7992a7a1af890f944d29f2cb10ba9641d1885591d087af628c40"}
Sep 10 19:29:32.253469 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:32.253439    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-b3108b/autores-model-b3108b-kserve-7dff498ff5-sv4lz" event={"ID":"eb9e6693-d98b-4098-ae05-b181f5ab9fbe","Type":"ContainerStarted","Data":"f2b9259d59042d0238878ae1137afc0f69420b2734966eb2128ac532b610eb8c"}
Sep 10 19:29:32.271563 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:32.271512    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-autores-b3108b/autores-model-b3108b-kserve-7dff498ff5-sv4lz" podStartSLOduration=2.271498543 podStartE2EDuration="2.271498543s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:29:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:29:32.270125279 +0000 UTC m=+843.329952948" watchObservedRunningTime="2026-09-10 19:29:32.271498543 +0000 UTC m=+843.331326210"
Sep 10 19:29:35.214475 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:35.214438    2575 patch_prober.go:28] interesting pod/e2e-ait-f0b1e2fe-openshift-default-887748fc4-pk6ww container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 10 19:29:35.214856 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:35.214496    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-f0b1e2fe-openshift-default-887748fc4-pk6ww" podUID="1cb07606-a586-46f1-8cb0-d5bae5be695a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 10 19:29:37.280942 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:37.280911    2575 generic.go:356] "Generic (PLEG): container finished" podID="1cb07606-a586-46f1-8cb0-d5bae5be695a" containerID="74a59f5f405340f9dd4f6ea42e5bbdcb0a4f118712bb30d891b7e1e33de0f5af" exitCode=0
Sep 10 19:29:37.281315 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:37.280985    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f0b1e2fe-openshift-default-887748fc4-pk6ww" event={"ID":"1cb07606-a586-46f1-8cb0-d5bae5be695a","Type":"ContainerDied","Data":"74a59f5f405340f9dd4f6ea42e5bbdcb0a4f118712bb30d891b7e1e33de0f5af"}
Sep 10 19:29:37.281391 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:37.281365    2575 scope.go:117] "RemoveContainer" containerID="74a59f5f405340f9dd4f6ea42e5bbdcb0a4f118712bb30d891b7e1e33de0f5af"
Sep 10 19:29:37.282368 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:37.282321    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 19:29:37.282424 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:37.282411    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 19:29:37.282459 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:37.282450    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 19:29:38.288111 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:38.288072    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f0b1e2fe-openshift-default-887748fc4-pk6ww" event={"ID":"1cb07606-a586-46f1-8cb0-d5bae5be695a","Type":"ContainerStarted","Data":"0979ce6f872ccd30d20ddfc4ad10cacc77009fea923c77f633d25bd8f4db50ca"}
Sep 10 19:29:38.490037 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:38.490005    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-f0b1e2fe-openshift-default-887748fc4-pk6ww"
Sep 10 19:29:38.490313 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:38.490294    2575 patch_prober.go:28] interesting pod/e2e-ait-f0b1e2fe-openshift-default-887748fc4-pk6ww container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body=
Sep 10 19:29:38.490386 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:38.490336    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-f0b1e2fe-openshift-default-887748fc4-pk6ww" podUID="1cb07606-a586-46f1-8cb0-d5bae5be695a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused"
Sep 10 19:29:39.490789 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:39.490756    2575 patch_prober.go:28] interesting pod/e2e-ait-f0b1e2fe-openshift-default-887748fc4-pk6ww container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body=
Sep 10 19:29:39.491154 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:39.490808    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-f0b1e2fe-openshift-default-887748fc4-pk6ww" podUID="1cb07606-a586-46f1-8cb0-d5bae5be695a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused"
Sep 10 19:29:39.953414 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:39.953378    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qr7rl"]
Sep 10 19:29:40.490557 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:40.490521    2575 patch_prober.go:28] interesting pod/e2e-ait-f0b1e2fe-openshift-default-887748fc4-pk6ww container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body=
Sep 10 19:29:40.490727 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:40.490577    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-f0b1e2fe-openshift-default-887748fc4-pk6ww" podUID="1cb07606-a586-46f1-8cb0-d5bae5be695a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused"
Sep 10 19:29:40.898772 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:40.898669    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-autores-b3108b/autores-model-b3108b-kserve-7dff498ff5-sv4lz"
Sep 10 19:29:40.898772 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:40.898727    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-autores-b3108b/autores-model-b3108b-kserve-7dff498ff5-sv4lz"
Sep 10 19:29:40.911003 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:40.910974    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-autores-b3108b/autores-model-b3108b-kserve-7dff498ff5-sv4lz"
Sep 10 19:29:41.313921 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:41.313892    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-autores-b3108b/autores-model-b3108b-kserve-7dff498ff5-sv4lz"
Sep 10 19:29:41.490515 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:41.490480    2575 patch_prober.go:28] interesting pod/e2e-ait-f0b1e2fe-openshift-default-887748fc4-pk6ww container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body=
Sep 10 19:29:41.490694 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:41.490543    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-f0b1e2fe-openshift-default-887748fc4-pk6ww" podUID="1cb07606-a586-46f1-8cb0-d5bae5be695a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused"
Sep 10 19:29:42.490404 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:42.490371    2575 patch_prober.go:28] interesting pod/e2e-ait-f0b1e2fe-openshift-default-887748fc4-pk6ww container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body=
Sep 10 19:29:42.490794 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:42.490421    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-f0b1e2fe-openshift-default-887748fc4-pk6ww" podUID="1cb07606-a586-46f1-8cb0-d5bae5be695a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused"
Sep 10 19:29:43.642014 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:43.641981    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-f0b1e2fe-openshift-default-887748fc4-pk6ww"
Sep 10 19:29:43.642664 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:43.642614    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-f0b1e2fe-openshift-default-887748fc4-pk6ww"
Sep 10 19:29:43.643581 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:43.643564    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-f0b1e2fe-openshift-default-887748fc4-pk6ww"
Sep 10 19:29:45.370057 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:45.370017    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-8300b9/test-model-8300b9-kserve-9574dfb49-h6rrs"]
Sep 10 19:29:45.370539 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:45.370524    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="e4e2b654-340b-49d3-bb7d-be131be3c7f8" containerName="istio-proxy"
Sep 10 19:29:45.373452 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:45.373423    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-8300b9/test-model-8300b9-kserve-9574dfb49-h6rrs"
Sep 10 19:29:45.376483 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:45.376447    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-8300b9\"/\"kube-root-ca.crt\""
Sep 10 19:29:45.376483 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:45.376467    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-8300b9\"/\"openshift-service-ca.crt\""
Sep 10 19:29:45.376483 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:45.376449    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-8300b9\"/\"test-model-8300b9-kserve-self-signed-certs\""
Sep 10 19:29:45.376695 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:45.376456    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-8300b9\"/\"default-dockercfg-gkw9g\""
Sep 10 19:29:45.386857 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:45.386838    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-8300b9/test-model-8300b9-kserve-9574dfb49-h6rrs"]
Sep 10 19:29:45.397913 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:45.397869    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:45.499266 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:45.499228    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/65c44357-30b7-47c0-8957-0e9fb86019c3-tls-certs\") pod \"test-model-8300b9-kserve-9574dfb49-h6rrs\" (UID: \"65c44357-30b7-47c0-8957-0e9fb86019c3\") " pod="ai-tenant-e2e-inf-a-8300b9/test-model-8300b9-kserve-9574dfb49-h6rrs"
Sep 10 19:29:45.499266 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:45.499279    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/65c44357-30b7-47c0-8957-0e9fb86019c3-model-cache\") pod \"test-model-8300b9-kserve-9574dfb49-h6rrs\" (UID: \"65c44357-30b7-47c0-8957-0e9fb86019c3\") " pod="ai-tenant-e2e-inf-a-8300b9/test-model-8300b9-kserve-9574dfb49-h6rrs"
Sep 10 19:29:45.499485 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:45.499411    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-69n9t\" (UniqueName: \"kubernetes.io/projected/65c44357-30b7-47c0-8957-0e9fb86019c3-kube-api-access-69n9t\") pod \"test-model-8300b9-kserve-9574dfb49-h6rrs\" (UID: \"65c44357-30b7-47c0-8957-0e9fb86019c3\") " pod="ai-tenant-e2e-inf-a-8300b9/test-model-8300b9-kserve-9574dfb49-h6rrs"
Sep 10 19:29:45.499485 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:45.499468    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/65c44357-30b7-47c0-8957-0e9fb86019c3-home\") pod \"test-model-8300b9-kserve-9574dfb49-h6rrs\" (UID: \"65c44357-30b7-47c0-8957-0e9fb86019c3\") " pod="ai-tenant-e2e-inf-a-8300b9/test-model-8300b9-kserve-9574dfb49-h6rrs"
Sep 10 19:29:45.499562 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:45.499512    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/65c44357-30b7-47c0-8957-0e9fb86019c3-tmp-dir\") pod \"test-model-8300b9-kserve-9574dfb49-h6rrs\" (UID: \"65c44357-30b7-47c0-8957-0e9fb86019c3\") " pod="ai-tenant-e2e-inf-a-8300b9/test-model-8300b9-kserve-9574dfb49-h6rrs"
Sep 10 19:29:45.499562 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:45.499536    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/65c44357-30b7-47c0-8957-0e9fb86019c3-dshm\") pod \"test-model-8300b9-kserve-9574dfb49-h6rrs\" (UID: \"65c44357-30b7-47c0-8957-0e9fb86019c3\") " pod="ai-tenant-e2e-inf-a-8300b9/test-model-8300b9-kserve-9574dfb49-h6rrs"
Sep 10 19:29:45.600005 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:45.599974    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/65c44357-30b7-47c0-8957-0e9fb86019c3-tmp-dir\") pod \"test-model-8300b9-kserve-9574dfb49-h6rrs\" (UID: \"65c44357-30b7-47c0-8957-0e9fb86019c3\") " pod="ai-tenant-e2e-inf-a-8300b9/test-model-8300b9-kserve-9574dfb49-h6rrs"
Sep 10 19:29:45.600165 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:45.600009    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/65c44357-30b7-47c0-8957-0e9fb86019c3-dshm\") pod \"test-model-8300b9-kserve-9574dfb49-h6rrs\" (UID: \"65c44357-30b7-47c0-8957-0e9fb86019c3\") " pod="ai-tenant-e2e-inf-a-8300b9/test-model-8300b9-kserve-9574dfb49-h6rrs"
Sep 10 19:29:45.600165 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:45.600143    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/65c44357-30b7-47c0-8957-0e9fb86019c3-tls-certs\") pod \"test-model-8300b9-kserve-9574dfb49-h6rrs\" (UID: \"65c44357-30b7-47c0-8957-0e9fb86019c3\") " pod="ai-tenant-e2e-inf-a-8300b9/test-model-8300b9-kserve-9574dfb49-h6rrs"
Sep 10 19:29:45.600284 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:45.600198    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/65c44357-30b7-47c0-8957-0e9fb86019c3-model-cache\") pod \"test-model-8300b9-kserve-9574dfb49-h6rrs\" (UID: \"65c44357-30b7-47c0-8957-0e9fb86019c3\") " pod="ai-tenant-e2e-inf-a-8300b9/test-model-8300b9-kserve-9574dfb49-h6rrs"
Sep 10 19:29:45.600370 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:45.600296    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/65c44357-30b7-47c0-8957-0e9fb86019c3-tmp-dir\") pod \"test-model-8300b9-kserve-9574dfb49-h6rrs\" (UID: \"65c44357-30b7-47c0-8957-0e9fb86019c3\") " pod="ai-tenant-e2e-inf-a-8300b9/test-model-8300b9-kserve-9574dfb49-h6rrs"
Sep 10 19:29:45.600370 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:45.600298    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-69n9t\" (UniqueName: \"kubernetes.io/projected/65c44357-30b7-47c0-8957-0e9fb86019c3-kube-api-access-69n9t\") pod \"test-model-8300b9-kserve-9574dfb49-h6rrs\" (UID: \"65c44357-30b7-47c0-8957-0e9fb86019c3\") " pod="ai-tenant-e2e-inf-a-8300b9/test-model-8300b9-kserve-9574dfb49-h6rrs"
Sep 10 19:29:45.600481 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:45.600404    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/65c44357-30b7-47c0-8957-0e9fb86019c3-home\") pod \"test-model-8300b9-kserve-9574dfb49-h6rrs\" (UID: \"65c44357-30b7-47c0-8957-0e9fb86019c3\") " pod="ai-tenant-e2e-inf-a-8300b9/test-model-8300b9-kserve-9574dfb49-h6rrs"
Sep 10 19:29:45.600520 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:45.600489    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/65c44357-30b7-47c0-8957-0e9fb86019c3-model-cache\") pod \"test-model-8300b9-kserve-9574dfb49-h6rrs\" (UID: \"65c44357-30b7-47c0-8957-0e9fb86019c3\") " pod="ai-tenant-e2e-inf-a-8300b9/test-model-8300b9-kserve-9574dfb49-h6rrs"
Sep 10 19:29:45.600689 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:45.600671    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/65c44357-30b7-47c0-8957-0e9fb86019c3-home\") pod \"test-model-8300b9-kserve-9574dfb49-h6rrs\" (UID: \"65c44357-30b7-47c0-8957-0e9fb86019c3\") " pod="ai-tenant-e2e-inf-a-8300b9/test-model-8300b9-kserve-9574dfb49-h6rrs"
Sep 10 19:29:45.602334 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:45.602313    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/65c44357-30b7-47c0-8957-0e9fb86019c3-dshm\") pod \"test-model-8300b9-kserve-9574dfb49-h6rrs\" (UID: \"65c44357-30b7-47c0-8957-0e9fb86019c3\") " pod="ai-tenant-e2e-inf-a-8300b9/test-model-8300b9-kserve-9574dfb49-h6rrs"
Sep 10 19:29:45.602547 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:45.602533    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/65c44357-30b7-47c0-8957-0e9fb86019c3-tls-certs\") pod \"test-model-8300b9-kserve-9574dfb49-h6rrs\" (UID: \"65c44357-30b7-47c0-8957-0e9fb86019c3\") " pod="ai-tenant-e2e-inf-a-8300b9/test-model-8300b9-kserve-9574dfb49-h6rrs"
Sep 10 19:29:45.609815 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:45.609796    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-69n9t\" (UniqueName: \"kubernetes.io/projected/65c44357-30b7-47c0-8957-0e9fb86019c3-kube-api-access-69n9t\") pod \"test-model-8300b9-kserve-9574dfb49-h6rrs\" (UID: \"65c44357-30b7-47c0-8957-0e9fb86019c3\") " pod="ai-tenant-e2e-inf-a-8300b9/test-model-8300b9-kserve-9574dfb49-h6rrs"
Sep 10 19:29:45.684511 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:45.684422    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-8300b9/test-model-8300b9-kserve-9574dfb49-h6rrs"
Sep 10 19:29:45.957530 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:45.957473    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-8300b9/test-model-8300b9-kserve-9574dfb49-h6rrs"]
Sep 10 19:29:45.959919 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:29:45.959890    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod65c44357_30b7_47c0_8957_0e9fb86019c3.slice/crio-41a17cee222d554152fb0d0a325ec2c65a32236b63bbacb5cffb730c59bf3b8f WatchSource:0}: Error finding container 41a17cee222d554152fb0d0a325ec2c65a32236b63bbacb5cffb730c59bf3b8f: Status 404 returned error can't find the container with id 41a17cee222d554152fb0d0a325ec2c65a32236b63bbacb5cffb730c59bf3b8f
Sep 10 19:29:46.333565 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:46.333485    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-8300b9/test-model-8300b9-kserve-9574dfb49-h6rrs" event={"ID":"65c44357-30b7-47c0-8957-0e9fb86019c3","Type":"ContainerStarted","Data":"bf445cd5678e01ce98ff5cbd69d18544f82df463a5e99589637c386e0fcb23f3"}
Sep 10 19:29:46.333565 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:46.333525    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-8300b9/test-model-8300b9-kserve-9574dfb49-h6rrs" event={"ID":"65c44357-30b7-47c0-8957-0e9fb86019c3","Type":"ContainerStarted","Data":"41a17cee222d554152fb0d0a325ec2c65a32236b63bbacb5cffb730c59bf3b8f"}
Sep 10 19:29:46.350015 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:46.349969    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-8300b9/test-model-8300b9-kserve-9574dfb49-h6rrs" podStartSLOduration=1.349952936 podStartE2EDuration="1.349952936s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:29:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:29:46.348765443 +0000 UTC m=+857.408593113" watchObservedRunningTime="2026-09-10 19:29:46.349952936 +0000 UTC m=+857.409780623"
Sep 10 19:29:47.216995 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:47.216964    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-conflict-b-479f6157-kserve-858f766cd6-g9knt"]
Sep 10 19:29:47.284294 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:47.284254    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:47.870215 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:47.870186    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-conflict-a-479f6157-kserve-7d76b7f788-bmb57"]
Sep 10 19:29:47.902606 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:47.902562    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:49.312534 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:49.312501    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-autores-b3108b/autores-model-b3108b-kserve-7dff498ff5-sv4lz"]
Sep 10 19:29:49.325732 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:49.325690    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:49.567153 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:49.567060    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:51.024523 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:51.024474    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:51.140719 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:51.140680    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:51.227287 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:51.227248    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:51.301953 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:51.301873    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:52.317251 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:52.317209    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-f0b1e2fe-openshift-default-887748fc4-pk6ww"]
Sep 10 19:29:52.317706 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:52.317491    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-f0b1e2fe-openshift-default-887748fc4-pk6ww" podUID="1cb07606-a586-46f1-8cb0-d5bae5be695a" containerName="istio-proxy" containerID="cri-o://0979ce6f872ccd30d20ddfc4ad10cacc77009fea923c77f633d25bd8f4db50ca" gracePeriod=30
Sep 10 19:29:52.349120 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:52.349091    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:52.765356 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:52.765317    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:53.075268 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:53.075180    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:53.285910 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:53.285878    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:55.509586 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:55.509553    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:55.605178 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:55.605147    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:55.685143 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:55.685116    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-8300b9/test-model-8300b9-kserve-9574dfb49-h6rrs"
Sep 10 19:29:55.685269 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:55.685158    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-8300b9/test-model-8300b9-kserve-9574dfb49-h6rrs"
Sep 10 19:29:55.697284 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:55.697258    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-8300b9/test-model-8300b9-kserve-9574dfb49-h6rrs"
Sep 10 19:29:56.394447 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:56.394419    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-8300b9/test-model-8300b9-kserve-9574dfb49-h6rrs"
Sep 10 19:29:56.708311 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:56.708276    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:56.806880 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:56.806838    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:56.882586 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:56.882560    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:57.262680 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:57.262642    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:57.364535 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:57.364504    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:57.580592 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:57.580569    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f0b1e2fe-openshift-default-887748fc4-pk6ww"
Sep 10 19:29:57.716542 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:57.716500    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1cb07606-a586-46f1-8cb0-d5bae5be695a-istiod-ca-cert\") pod \"1cb07606-a586-46f1-8cb0-d5bae5be695a\" (UID: \"1cb07606-a586-46f1-8cb0-d5bae5be695a\") "
Sep 10 19:29:57.716945 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:57.716568    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1cb07606-a586-46f1-8cb0-d5bae5be695a-istio-envoy\") pod \"1cb07606-a586-46f1-8cb0-d5bae5be695a\" (UID: \"1cb07606-a586-46f1-8cb0-d5bae5be695a\") "
Sep 10 19:29:57.716945 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:57.716737    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s5lc4\" (UniqueName: \"kubernetes.io/projected/1cb07606-a586-46f1-8cb0-d5bae5be695a-kube-api-access-s5lc4\") pod \"1cb07606-a586-46f1-8cb0-d5bae5be695a\" (UID: \"1cb07606-a586-46f1-8cb0-d5bae5be695a\") "
Sep 10 19:29:57.716945 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:57.716805    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1cb07606-a586-46f1-8cb0-d5bae5be695a-istio-data\") pod \"1cb07606-a586-46f1-8cb0-d5bae5be695a\" (UID: \"1cb07606-a586-46f1-8cb0-d5bae5be695a\") "
Sep 10 19:29:57.716945 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:57.716848    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1cb07606-a586-46f1-8cb0-d5bae5be695a-credential-socket\") pod \"1cb07606-a586-46f1-8cb0-d5bae5be695a\" (UID: \"1cb07606-a586-46f1-8cb0-d5bae5be695a\") "
Sep 10 19:29:57.716945 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:57.716887    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1cb07606-a586-46f1-8cb0-d5bae5be695a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1cb07606-a586-46f1-8cb0-d5bae5be695a" (UID: "1cb07606-a586-46f1-8cb0-d5bae5be695a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:29:57.716945 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:57.716930    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1cb07606-a586-46f1-8cb0-d5bae5be695a-istio-token\") pod \"1cb07606-a586-46f1-8cb0-d5bae5be695a\" (UID: \"1cb07606-a586-46f1-8cb0-d5bae5be695a\") "
Sep 10 19:29:57.717273 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:57.716993    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1cb07606-a586-46f1-8cb0-d5bae5be695a-workload-socket\") pod \"1cb07606-a586-46f1-8cb0-d5bae5be695a\" (UID: \"1cb07606-a586-46f1-8cb0-d5bae5be695a\") "
Sep 10 19:29:57.717273 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:57.717097    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1cb07606-a586-46f1-8cb0-d5bae5be695a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1cb07606-a586-46f1-8cb0-d5bae5be695a" (UID: "1cb07606-a586-46f1-8cb0-d5bae5be695a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:29:57.717273 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:57.717121    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1cb07606-a586-46f1-8cb0-d5bae5be695a-istio-data" (OuterVolumeSpecName: "istio-data") pod "1cb07606-a586-46f1-8cb0-d5bae5be695a" (UID: "1cb07606-a586-46f1-8cb0-d5bae5be695a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:29:57.717273 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:57.717154    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1cb07606-a586-46f1-8cb0-d5bae5be695a-workload-certs\") pod \"1cb07606-a586-46f1-8cb0-d5bae5be695a\" (UID: \"1cb07606-a586-46f1-8cb0-d5bae5be695a\") "
Sep 10 19:29:57.717273 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:57.717182    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1cb07606-a586-46f1-8cb0-d5bae5be695a-istio-podinfo\") pod \"1cb07606-a586-46f1-8cb0-d5bae5be695a\" (UID: \"1cb07606-a586-46f1-8cb0-d5bae5be695a\") "
Sep 10 19:29:57.717563 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:57.717327    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1cb07606-a586-46f1-8cb0-d5bae5be695a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1cb07606-a586-46f1-8cb0-d5bae5be695a" (UID: "1cb07606-a586-46f1-8cb0-d5bae5be695a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:29:57.717563 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:57.717412    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1cb07606-a586-46f1-8cb0-d5bae5be695a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1cb07606-a586-46f1-8cb0-d5bae5be695a" (UID: "1cb07606-a586-46f1-8cb0-d5bae5be695a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:29:57.717665 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:57.717586    2575 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1cb07606-a586-46f1-8cb0-d5bae5be695a-istiod-ca-cert\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:29:57.717665 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:57.717608    2575 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1cb07606-a586-46f1-8cb0-d5bae5be695a-istio-data\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:29:57.717665 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:57.717620    2575 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1cb07606-a586-46f1-8cb0-d5bae5be695a-credential-socket\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:29:57.717665 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:57.717633    2575 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1cb07606-a586-46f1-8cb0-d5bae5be695a-workload-socket\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:29:57.717665 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:57.717647    2575 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1cb07606-a586-46f1-8cb0-d5bae5be695a-workload-certs\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:29:57.719369 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:57.719312    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1cb07606-a586-46f1-8cb0-d5bae5be695a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1cb07606-a586-46f1-8cb0-d5bae5be695a" (UID: "1cb07606-a586-46f1-8cb0-d5bae5be695a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 19:29:57.719480 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:57.719396    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1cb07606-a586-46f1-8cb0-d5bae5be695a-kube-api-access-s5lc4" (OuterVolumeSpecName: "kube-api-access-s5lc4") pod "1cb07606-a586-46f1-8cb0-d5bae5be695a" (UID: "1cb07606-a586-46f1-8cb0-d5bae5be695a"). InnerVolumeSpecName "kube-api-access-s5lc4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:29:57.719480 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:57.719424    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1cb07606-a586-46f1-8cb0-d5bae5be695a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1cb07606-a586-46f1-8cb0-d5bae5be695a" (UID: "1cb07606-a586-46f1-8cb0-d5bae5be695a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:29:57.719622 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:57.719496    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1cb07606-a586-46f1-8cb0-d5bae5be695a-istio-token" (OuterVolumeSpecName: "istio-token") pod "1cb07606-a586-46f1-8cb0-d5bae5be695a" (UID: "1cb07606-a586-46f1-8cb0-d5bae5be695a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:29:57.818776 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:57.818701    2575 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1cb07606-a586-46f1-8cb0-d5bae5be695a-istio-token\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:29:57.818776 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:57.818727    2575 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1cb07606-a586-46f1-8cb0-d5bae5be695a-istio-podinfo\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:29:57.818776 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:57.818739    2575 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1cb07606-a586-46f1-8cb0-d5bae5be695a-istio-envoy\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:29:57.818776 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:57.818751    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s5lc4\" (UniqueName: \"kubernetes.io/projected/1cb07606-a586-46f1-8cb0-d5bae5be695a-kube-api-access-s5lc4\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:29:58.279240 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:58.279199    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:58.394896 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:58.394858    2575 generic.go:356] "Generic (PLEG): container finished" podID="1cb07606-a586-46f1-8cb0-d5bae5be695a" containerID="0979ce6f872ccd30d20ddfc4ad10cacc77009fea923c77f633d25bd8f4db50ca" exitCode=0
Sep 10 19:29:58.394896 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:58.394896    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f0b1e2fe-openshift-default-887748fc4-pk6ww"
Sep 10 19:29:58.395135 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:58.394945    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f0b1e2fe-openshift-default-887748fc4-pk6ww" event={"ID":"1cb07606-a586-46f1-8cb0-d5bae5be695a","Type":"ContainerDied","Data":"0979ce6f872ccd30d20ddfc4ad10cacc77009fea923c77f633d25bd8f4db50ca"}
Sep 10 19:29:58.395135 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:58.394992    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f0b1e2fe-openshift-default-887748fc4-pk6ww" event={"ID":"1cb07606-a586-46f1-8cb0-d5bae5be695a","Type":"ContainerDied","Data":"e56592af285233614042f46d5d376fb9a8cc83d3f1271c99ddac2067fe28baa8"}
Sep 10 19:29:58.395135 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:58.395015    2575 scope.go:117] "RemoveContainer" containerID="0979ce6f872ccd30d20ddfc4ad10cacc77009fea923c77f633d25bd8f4db50ca"
Sep 10 19:29:58.404465 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:58.404451    2575 scope.go:117] "RemoveContainer" containerID="74a59f5f405340f9dd4f6ea42e5bbdcb0a4f118712bb30d891b7e1e33de0f5af"
Sep 10 19:29:58.412472 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:58.412449    2575 scope.go:117] "RemoveContainer" containerID="0979ce6f872ccd30d20ddfc4ad10cacc77009fea923c77f633d25bd8f4db50ca"
Sep 10 19:29:58.412708 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:29:58.412688    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0979ce6f872ccd30d20ddfc4ad10cacc77009fea923c77f633d25bd8f4db50ca\": container with ID starting with 0979ce6f872ccd30d20ddfc4ad10cacc77009fea923c77f633d25bd8f4db50ca not found: ID does not exist" containerID="0979ce6f872ccd30d20ddfc4ad10cacc77009fea923c77f633d25bd8f4db50ca"
Sep 10 19:29:58.412757 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:58.412718    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0979ce6f872ccd30d20ddfc4ad10cacc77009fea923c77f633d25bd8f4db50ca"} err="failed to get container status \"0979ce6f872ccd30d20ddfc4ad10cacc77009fea923c77f633d25bd8f4db50ca\": rpc error: code = NotFound desc = could not find container \"0979ce6f872ccd30d20ddfc4ad10cacc77009fea923c77f633d25bd8f4db50ca\": container with ID starting with 0979ce6f872ccd30d20ddfc4ad10cacc77009fea923c77f633d25bd8f4db50ca not found: ID does not exist"
Sep 10 19:29:58.412757 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:58.412734    2575 scope.go:117] "RemoveContainer" containerID="74a59f5f405340f9dd4f6ea42e5bbdcb0a4f118712bb30d891b7e1e33de0f5af"
Sep 10 19:29:58.412978 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:29:58.412964    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"74a59f5f405340f9dd4f6ea42e5bbdcb0a4f118712bb30d891b7e1e33de0f5af\": container with ID starting with 74a59f5f405340f9dd4f6ea42e5bbdcb0a4f118712bb30d891b7e1e33de0f5af not found: ID does not exist" containerID="74a59f5f405340f9dd4f6ea42e5bbdcb0a4f118712bb30d891b7e1e33de0f5af"
Sep 10 19:29:58.413018 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:58.412981    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"74a59f5f405340f9dd4f6ea42e5bbdcb0a4f118712bb30d891b7e1e33de0f5af"} err="failed to get container status \"74a59f5f405340f9dd4f6ea42e5bbdcb0a4f118712bb30d891b7e1e33de0f5af\": rpc error: code = NotFound desc = could not find container \"74a59f5f405340f9dd4f6ea42e5bbdcb0a4f118712bb30d891b7e1e33de0f5af\": container with ID starting with 74a59f5f405340f9dd4f6ea42e5bbdcb0a4f118712bb30d891b7e1e33de0f5af not found: ID does not exist"
Sep 10 19:29:58.415844 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:58.415821    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-f0b1e2fe-openshift-default-887748fc4-pk6ww"]
Sep 10 19:29:58.424996 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:58.421224    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-f0b1e2fe-openshift-default-887748fc4-pk6ww"]
Sep 10 19:29:59.150213 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:59.150172    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:59.384455 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:59.384422    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1cb07606-a586-46f1-8cb0-d5bae5be695a" path="/var/lib/kubelet/pods/1cb07606-a586-46f1-8cb0-d5bae5be695a/volumes"
Sep 10 19:29:59.828208 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:59.828173    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:59.923790 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:29:59.923755    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:30:00.023354 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:00.023309    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:30:00.104028 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:00.103949    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:30:00.154573 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:00.154538    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-588a50a1-29817810-n86z5"]
Sep 10 19:30:00.154994 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:00.154929    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1cb07606-a586-46f1-8cb0-d5bae5be695a" containerName="istio-proxy"
Sep 10 19:30:00.154994 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:00.154940    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="1cb07606-a586-46f1-8cb0-d5bae5be695a" containerName="istio-proxy"
Sep 10 19:30:00.154994 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:00.154948    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1cb07606-a586-46f1-8cb0-d5bae5be695a" containerName="istio-proxy"
Sep 10 19:30:00.154994 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:00.154955    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="1cb07606-a586-46f1-8cb0-d5bae5be695a" containerName="istio-proxy"
Sep 10 19:30:00.155204 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:00.155012    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="1cb07606-a586-46f1-8cb0-d5bae5be695a" containerName="istio-proxy"
Sep 10 19:30:00.155204 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:00.155022    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="1cb07606-a586-46f1-8cb0-d5bae5be695a" containerName="istio-proxy"
Sep 10 19:30:00.159458 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:00.159440    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-8300b9-29817810-8rmhd"]
Sep 10 19:30:00.159590 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:00.159575    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-588a50a1-29817810-n86z5"
Sep 10 19:30:00.162775 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:00.162756    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-8300b9-29817810-8rmhd"
Sep 10 19:30:00.163005 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:00.162989    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wkgt8\""
Sep 10 19:30:00.166997 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:00.166970    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-8300b9-29817810-8rmhd"]
Sep 10 19:30:00.169721 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:00.169696    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-588a50a1-29817810-n86z5"]
Sep 10 19:30:00.175679 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:00.175660    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-588a50a1-29817810-n86z5"
Sep 10 19:30:00.192104 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:00.189907    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-8300b9-29817810-8rmhd"
Sep 10 19:30:00.444558 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:00.444470    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:30:00.477697 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:00.477675    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-588a50a1-29817810-n86z5"]
Sep 10 19:30:00.479503 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:30:00.479478    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2c9f912_84c4_4bbf_86d5_7d3c8029f412.slice/crio-727da67898fbed3b9ff6327ce33f208ea084fbf2134e18996db9089adbdfb77b WatchSource:0}: Error finding container 727da67898fbed3b9ff6327ce33f208ea084fbf2134e18996db9089adbdfb77b: Status 404 returned error can't find the container with id 727da67898fbed3b9ff6327ce33f208ea084fbf2134e18996db9089adbdfb77b
Sep 10 19:30:00.484711 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:00.484689    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-8300b9-29817810-8rmhd"]
Sep 10 19:30:00.485945 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:30:00.485879    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4b52ee7_8cce_4ecc_a236_196308998f71.slice/crio-cbea5a277cec1687e40515959cf7912dd7aa3feee327bbeae3695175d373b6d2 WatchSource:0}: Error finding container cbea5a277cec1687e40515959cf7912dd7aa3feee327bbeae3695175d373b6d2: Status 404 returned error can't find the container with id cbea5a277cec1687e40515959cf7912dd7aa3feee327bbeae3695175d373b6d2
Sep 10 19:30:00.961681 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:00.961647    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qr7rl"]
Sep 10 19:30:01.415283 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:01.415139    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-8300b9-29817810-8rmhd" event={"ID":"e4b52ee7-8cce-4ecc-a236-196308998f71","Type":"ContainerStarted","Data":"cbea5a277cec1687e40515959cf7912dd7aa3feee327bbeae3695175d373b6d2"}
Sep 10 19:30:01.417233 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:01.417192    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-588a50a1-29817810-n86z5" event={"ID":"a2c9f912-84c4-4bbf-86d5-7d3c8029f412","Type":"ContainerStarted","Data":"727da67898fbed3b9ff6327ce33f208ea084fbf2134e18996db9089adbdfb77b"}
Sep 10 19:30:02.236959 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:02.236917    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-conflict-b-479f6157-kserve-858f766cd6-g9knt" podUID="5529e50e-798e-4f48-bcbb-7a032f7650b6" containerName="main" containerID="cri-o://0b11a7fa87a1205b824784e078dee0375c7577f26b46df831bdbb7f3211066f2" gracePeriod=45
Sep 10 19:30:02.874952 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:02.874925    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-479f6157-kserve-858f766cd6-g9knt"
Sep 10 19:30:02.889737 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:02.889708    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-conflict-a-479f6157-kserve-7d76b7f788-bmb57" podUID="9c34b550-2700-4103-bea0-5493d30c13d9" containerName="main" containerID="cri-o://db7413ef65497bb0a34e7fd19cb9ab86eb9a4a31d1e3ace278c667f5ed33ada0" gracePeriod=45
Sep 10 19:30:02.973186 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:02.973165    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5529e50e-798e-4f48-bcbb-7a032f7650b6-model-cache\") pod \"5529e50e-798e-4f48-bcbb-7a032f7650b6\" (UID: \"5529e50e-798e-4f48-bcbb-7a032f7650b6\") "
Sep 10 19:30:02.973276 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:02.973224    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sdwz6\" (UniqueName: \"kubernetes.io/projected/5529e50e-798e-4f48-bcbb-7a032f7650b6-kube-api-access-sdwz6\") pod \"5529e50e-798e-4f48-bcbb-7a032f7650b6\" (UID: \"5529e50e-798e-4f48-bcbb-7a032f7650b6\") "
Sep 10 19:30:02.973276 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:02.973242    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5529e50e-798e-4f48-bcbb-7a032f7650b6-dshm\") pod \"5529e50e-798e-4f48-bcbb-7a032f7650b6\" (UID: \"5529e50e-798e-4f48-bcbb-7a032f7650b6\") "
Sep 10 19:30:02.973400 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:02.973275    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5529e50e-798e-4f48-bcbb-7a032f7650b6-tls-certs\") pod \"5529e50e-798e-4f48-bcbb-7a032f7650b6\" (UID: \"5529e50e-798e-4f48-bcbb-7a032f7650b6\") "
Sep 10 19:30:02.973400 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:02.973302    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5529e50e-798e-4f48-bcbb-7a032f7650b6-tmp-dir\") pod \"5529e50e-798e-4f48-bcbb-7a032f7650b6\" (UID: \"5529e50e-798e-4f48-bcbb-7a032f7650b6\") "
Sep 10 19:30:02.973400 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:02.973334    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5529e50e-798e-4f48-bcbb-7a032f7650b6-home\") pod \"5529e50e-798e-4f48-bcbb-7a032f7650b6\" (UID: \"5529e50e-798e-4f48-bcbb-7a032f7650b6\") "
Sep 10 19:30:02.973558 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:02.973457    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5529e50e-798e-4f48-bcbb-7a032f7650b6-model-cache" (OuterVolumeSpecName: "model-cache") pod "5529e50e-798e-4f48-bcbb-7a032f7650b6" (UID: "5529e50e-798e-4f48-bcbb-7a032f7650b6"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:30:02.973652 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:02.973610    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5529e50e-798e-4f48-bcbb-7a032f7650b6-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "5529e50e-798e-4f48-bcbb-7a032f7650b6" (UID: "5529e50e-798e-4f48-bcbb-7a032f7650b6"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:30:02.973713 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:02.973664    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5529e50e-798e-4f48-bcbb-7a032f7650b6-home" (OuterVolumeSpecName: "home") pod "5529e50e-798e-4f48-bcbb-7a032f7650b6" (UID: "5529e50e-798e-4f48-bcbb-7a032f7650b6"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:30:02.973758 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:02.973708    2575 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5529e50e-798e-4f48-bcbb-7a032f7650b6-tmp-dir\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:30:02.973758 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:02.973725    2575 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5529e50e-798e-4f48-bcbb-7a032f7650b6-model-cache\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:30:02.975378 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:02.975350    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5529e50e-798e-4f48-bcbb-7a032f7650b6-dshm" (OuterVolumeSpecName: "dshm") pod "5529e50e-798e-4f48-bcbb-7a032f7650b6" (UID: "5529e50e-798e-4f48-bcbb-7a032f7650b6"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:30:02.975481 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:02.975399    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5529e50e-798e-4f48-bcbb-7a032f7650b6-kube-api-access-sdwz6" (OuterVolumeSpecName: "kube-api-access-sdwz6") pod "5529e50e-798e-4f48-bcbb-7a032f7650b6" (UID: "5529e50e-798e-4f48-bcbb-7a032f7650b6"). InnerVolumeSpecName "kube-api-access-sdwz6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:30:02.975481 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:02.975440    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5529e50e-798e-4f48-bcbb-7a032f7650b6-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "5529e50e-798e-4f48-bcbb-7a032f7650b6" (UID: "5529e50e-798e-4f48-bcbb-7a032f7650b6"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:30:03.074702 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:03.074670    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sdwz6\" (UniqueName: \"kubernetes.io/projected/5529e50e-798e-4f48-bcbb-7a032f7650b6-kube-api-access-sdwz6\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:30:03.074828 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:03.074705    2575 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5529e50e-798e-4f48-bcbb-7a032f7650b6-dshm\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:30:03.074828 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:03.074718    2575 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5529e50e-798e-4f48-bcbb-7a032f7650b6-tls-certs\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:30:03.074828 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:03.074731    2575 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5529e50e-798e-4f48-bcbb-7a032f7650b6-home\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:30:03.132766 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:03.132745    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-a-479f6157-kserve-7d76b7f788-bmb57"
Sep 10 19:30:03.276968 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:03.276915    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9c34b550-2700-4103-bea0-5493d30c13d9-dshm\") pod \"9c34b550-2700-4103-bea0-5493d30c13d9\" (UID: \"9c34b550-2700-4103-bea0-5493d30c13d9\") "
Sep 10 19:30:03.276968 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:03.276966    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9c34b550-2700-4103-bea0-5493d30c13d9-tls-certs\") pod \"9c34b550-2700-4103-bea0-5493d30c13d9\" (UID: \"9c34b550-2700-4103-bea0-5493d30c13d9\") "
Sep 10 19:30:03.277149 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:03.276984    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9c34b550-2700-4103-bea0-5493d30c13d9-home\") pod \"9c34b550-2700-4103-bea0-5493d30c13d9\" (UID: \"9c34b550-2700-4103-bea0-5493d30c13d9\") "
Sep 10 19:30:03.277149 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:03.277018    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9c34b550-2700-4103-bea0-5493d30c13d9-model-cache\") pod \"9c34b550-2700-4103-bea0-5493d30c13d9\" (UID: \"9c34b550-2700-4103-bea0-5493d30c13d9\") "
Sep 10 19:30:03.277149 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:03.277042    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lnvcg\" (UniqueName: \"kubernetes.io/projected/9c34b550-2700-4103-bea0-5493d30c13d9-kube-api-access-lnvcg\") pod \"9c34b550-2700-4103-bea0-5493d30c13d9\" (UID: \"9c34b550-2700-4103-bea0-5493d30c13d9\") "
Sep 10 19:30:03.277149 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:03.277067    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9c34b550-2700-4103-bea0-5493d30c13d9-tmp-dir\") pod \"9c34b550-2700-4103-bea0-5493d30c13d9\" (UID: \"9c34b550-2700-4103-bea0-5493d30c13d9\") "
Sep 10 19:30:03.277394 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:03.277294    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9c34b550-2700-4103-bea0-5493d30c13d9-model-cache" (OuterVolumeSpecName: "model-cache") pod "9c34b550-2700-4103-bea0-5493d30c13d9" (UID: "9c34b550-2700-4103-bea0-5493d30c13d9"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:30:03.277394 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:03.277301    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9c34b550-2700-4103-bea0-5493d30c13d9-home" (OuterVolumeSpecName: "home") pod "9c34b550-2700-4103-bea0-5493d30c13d9" (UID: "9c34b550-2700-4103-bea0-5493d30c13d9"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:30:03.277513 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:03.277410    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9c34b550-2700-4103-bea0-5493d30c13d9-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "9c34b550-2700-4103-bea0-5493d30c13d9" (UID: "9c34b550-2700-4103-bea0-5493d30c13d9"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:30:03.277513 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:03.277440    2575 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9c34b550-2700-4103-bea0-5493d30c13d9-home\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:30:03.277513 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:03.277462    2575 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9c34b550-2700-4103-bea0-5493d30c13d9-model-cache\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:30:03.278958 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:03.278935    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9c34b550-2700-4103-bea0-5493d30c13d9-dshm" (OuterVolumeSpecName: "dshm") pod "9c34b550-2700-4103-bea0-5493d30c13d9" (UID: "9c34b550-2700-4103-bea0-5493d30c13d9"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:30:03.279090 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:03.279073    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9c34b550-2700-4103-bea0-5493d30c13d9-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "9c34b550-2700-4103-bea0-5493d30c13d9" (UID: "9c34b550-2700-4103-bea0-5493d30c13d9"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:30:03.279133 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:03.279086    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9c34b550-2700-4103-bea0-5493d30c13d9-kube-api-access-lnvcg" (OuterVolumeSpecName: "kube-api-access-lnvcg") pod "9c34b550-2700-4103-bea0-5493d30c13d9" (UID: "9c34b550-2700-4103-bea0-5493d30c13d9"). InnerVolumeSpecName "kube-api-access-lnvcg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:30:03.378810 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:03.378783    2575 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9c34b550-2700-4103-bea0-5493d30c13d9-dshm\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:30:03.378810 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:03.378806    2575 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9c34b550-2700-4103-bea0-5493d30c13d9-tls-certs\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:30:03.378934 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:03.378815    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lnvcg\" (UniqueName: \"kubernetes.io/projected/9c34b550-2700-4103-bea0-5493d30c13d9-kube-api-access-lnvcg\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:30:03.378934 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:03.378824    2575 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9c34b550-2700-4103-bea0-5493d30c13d9-tmp-dir\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:30:03.429632 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:03.429609    2575 generic.go:356] "Generic (PLEG): container finished" podID="9c34b550-2700-4103-bea0-5493d30c13d9" containerID="db7413ef65497bb0a34e7fd19cb9ab86eb9a4a31d1e3ace278c667f5ed33ada0" exitCode=0
Sep 10 19:30:03.429717 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:03.429647    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-a-479f6157-kserve-7d76b7f788-bmb57"
Sep 10 19:30:03.429717 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:03.429688    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-a-479f6157-kserve-7d76b7f788-bmb57" event={"ID":"9c34b550-2700-4103-bea0-5493d30c13d9","Type":"ContainerDied","Data":"db7413ef65497bb0a34e7fd19cb9ab86eb9a4a31d1e3ace278c667f5ed33ada0"}
Sep 10 19:30:03.429808 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:03.429726    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-a-479f6157-kserve-7d76b7f788-bmb57" event={"ID":"9c34b550-2700-4103-bea0-5493d30c13d9","Type":"ContainerDied","Data":"76faf436388958763fb3c31b5f8c723304f7224739e435f1c7e1bec53bbefbee"}
Sep 10 19:30:03.429808 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:03.429745    2575 scope.go:117] "RemoveContainer" containerID="db7413ef65497bb0a34e7fd19cb9ab86eb9a4a31d1e3ace278c667f5ed33ada0"
Sep 10 19:30:03.431201 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:03.431136    2575 generic.go:356] "Generic (PLEG): container finished" podID="5529e50e-798e-4f48-bcbb-7a032f7650b6" containerID="0b11a7fa87a1205b824784e078dee0375c7577f26b46df831bdbb7f3211066f2" exitCode=0
Sep 10 19:30:03.431201 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:03.431190    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-479f6157-kserve-858f766cd6-g9knt"
Sep 10 19:30:03.431321 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:03.431225    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-479f6157-kserve-858f766cd6-g9knt" event={"ID":"5529e50e-798e-4f48-bcbb-7a032f7650b6","Type":"ContainerDied","Data":"0b11a7fa87a1205b824784e078dee0375c7577f26b46df831bdbb7f3211066f2"}
Sep 10 19:30:03.431321 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:03.431282    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-479f6157-kserve-858f766cd6-g9knt" event={"ID":"5529e50e-798e-4f48-bcbb-7a032f7650b6","Type":"ContainerDied","Data":"d32db3d66f52c9ae27309c8d5dcaef5e77d621c38333420d67dc5a18bbf83ac1"}
Sep 10 19:30:03.433592 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:03.433553    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-8300b9-29817810-8rmhd" event={"ID":"e4b52ee7-8cce-4ecc-a236-196308998f71","Type":"ContainerStarted","Data":"7b818d6616a80bd7e8420a4f0e4d4ebdb53901b20afc28e20d4eff88643e7dbe"}
Sep 10 19:30:03.435196 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:03.435175    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-588a50a1-29817810-n86z5" event={"ID":"a2c9f912-84c4-4bbf-86d5-7d3c8029f412","Type":"ContainerStarted","Data":"1b299f8f7b0fdad9586b33cdaed31519f36b2b802bac56c1b02e64dc1c6e1aa8"}
Sep 10 19:30:03.438879 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:03.438863    2575 scope.go:117] "RemoveContainer" containerID="db7413ef65497bb0a34e7fd19cb9ab86eb9a4a31d1e3ace278c667f5ed33ada0"
Sep 10 19:30:03.439159 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:30:03.439135    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"db7413ef65497bb0a34e7fd19cb9ab86eb9a4a31d1e3ace278c667f5ed33ada0\": container with ID starting with db7413ef65497bb0a34e7fd19cb9ab86eb9a4a31d1e3ace278c667f5ed33ada0 not found: ID does not exist" containerID="db7413ef65497bb0a34e7fd19cb9ab86eb9a4a31d1e3ace278c667f5ed33ada0"
Sep 10 19:30:03.439232 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:03.439166    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"db7413ef65497bb0a34e7fd19cb9ab86eb9a4a31d1e3ace278c667f5ed33ada0"} err="failed to get container status \"db7413ef65497bb0a34e7fd19cb9ab86eb9a4a31d1e3ace278c667f5ed33ada0\": rpc error: code = NotFound desc = could not find container \"db7413ef65497bb0a34e7fd19cb9ab86eb9a4a31d1e3ace278c667f5ed33ada0\": container with ID starting with db7413ef65497bb0a34e7fd19cb9ab86eb9a4a31d1e3ace278c667f5ed33ada0 not found: ID does not exist"
Sep 10 19:30:03.439232 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:03.439183    2575 scope.go:117] "RemoveContainer" containerID="0b11a7fa87a1205b824784e078dee0375c7577f26b46df831bdbb7f3211066f2"
Sep 10 19:30:03.447005 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:03.446990    2575 scope.go:117] "RemoveContainer" containerID="0b11a7fa87a1205b824784e078dee0375c7577f26b46df831bdbb7f3211066f2"
Sep 10 19:30:03.447263 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:30:03.447248    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0b11a7fa87a1205b824784e078dee0375c7577f26b46df831bdbb7f3211066f2\": container with ID starting with 0b11a7fa87a1205b824784e078dee0375c7577f26b46df831bdbb7f3211066f2 not found: ID does not exist" containerID="0b11a7fa87a1205b824784e078dee0375c7577f26b46df831bdbb7f3211066f2"
Sep 10 19:30:03.447315 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:03.447268    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0b11a7fa87a1205b824784e078dee0375c7577f26b46df831bdbb7f3211066f2"} err="failed to get container status \"0b11a7fa87a1205b824784e078dee0375c7577f26b46df831bdbb7f3211066f2\": rpc error: code = NotFound desc = could not find container \"0b11a7fa87a1205b824784e078dee0375c7577f26b46df831bdbb7f3211066f2\": container with ID starting with 0b11a7fa87a1205b824784e078dee0375c7577f26b46df831bdbb7f3211066f2 not found: ID does not exist"
Sep 10 19:30:03.451099 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:03.451063    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-8300b9-29817810-8rmhd" podStartSLOduration=1.128215609 podStartE2EDuration="3.451051946s" totalImagesPullingTime="2.322836337s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:30:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:30:03.449456144 +0000 UTC m=+874.509283814" watchObservedRunningTime="2026-09-10 19:30:03.451051946 +0000 UTC m=+874.510879615"
Sep 10 19:30:03.461251 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:03.461230    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-conflict-a-479f6157-kserve-7d76b7f788-bmb57"]
Sep 10 19:30:03.463921 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:03.463902    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-conflict-a-479f6157-kserve-7d76b7f788-bmb57"]
Sep 10 19:30:03.473052 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:03.473032    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-conflict-b-479f6157-kserve-858f766cd6-g9knt"]
Sep 10 19:30:03.478913 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:03.478892    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-conflict-b-479f6157-kserve-858f766cd6-g9knt"]
Sep 10 19:30:03.490716 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:03.490681    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-588a50a1-29817810-n86z5" podStartSLOduration=1.15050971 podStartE2EDuration="3.49067036s" totalImagesPullingTime="2.34016065s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:30:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:30:03.489649228 +0000 UTC m=+874.549476896" watchObservedRunningTime="2026-09-10 19:30:03.49067036 +0000 UTC m=+874.550498025"
Sep 10 19:30:04.333484 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:04.333446    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-autores-b3108b/autores-model-b3108b-kserve-7dff498ff5-sv4lz" podUID="eb9e6693-d98b-4098-ae05-b181f5ab9fbe" containerName="main" containerID="cri-o://f2b9259d59042d0238878ae1137afc0f69420b2734966eb2128ac532b610eb8c" gracePeriod=45
Sep 10 19:30:04.446775 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:04.446750    2575 generic.go:356] "Generic (PLEG): container finished" podID="e4b52ee7-8cce-4ecc-a236-196308998f71" containerID="7b818d6616a80bd7e8420a4f0e4d4ebdb53901b20afc28e20d4eff88643e7dbe" exitCode=0
Sep 10 19:30:04.446884 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:04.446833    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-8300b9-29817810-8rmhd" event={"ID":"e4b52ee7-8cce-4ecc-a236-196308998f71","Type":"ContainerDied","Data":"7b818d6616a80bd7e8420a4f0e4d4ebdb53901b20afc28e20d4eff88643e7dbe"}
Sep 10 19:30:04.448393 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:04.448371    2575 generic.go:356] "Generic (PLEG): container finished" podID="a2c9f912-84c4-4bbf-86d5-7d3c8029f412" containerID="1b299f8f7b0fdad9586b33cdaed31519f36b2b802bac56c1b02e64dc1c6e1aa8" exitCode=7
Sep 10 19:30:04.448483 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:04.448410    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-588a50a1-29817810-n86z5" event={"ID":"a2c9f912-84c4-4bbf-86d5-7d3c8029f412","Type":"ContainerDied","Data":"1b299f8f7b0fdad9586b33cdaed31519f36b2b802bac56c1b02e64dc1c6e1aa8"}
Sep 10 19:30:04.448609 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:04.448597    2575 scope.go:117] "RemoveContainer" containerID="1b299f8f7b0fdad9586b33cdaed31519f36b2b802bac56c1b02e64dc1c6e1aa8"
Sep 10 19:30:04.574550 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:04.574529    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-b3108b/autores-model-b3108b-kserve-7dff498ff5-sv4lz"
Sep 10 19:30:04.690791 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:04.690715    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eb9e6693-d98b-4098-ae05-b181f5ab9fbe-model-cache\") pod \"eb9e6693-d98b-4098-ae05-b181f5ab9fbe\" (UID: \"eb9e6693-d98b-4098-ae05-b181f5ab9fbe\") "
Sep 10 19:30:04.690791 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:04.690754    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eb9e6693-d98b-4098-ae05-b181f5ab9fbe-dshm\") pod \"eb9e6693-d98b-4098-ae05-b181f5ab9fbe\" (UID: \"eb9e6693-d98b-4098-ae05-b181f5ab9fbe\") "
Sep 10 19:30:04.690999 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:04.690793    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eb9e6693-d98b-4098-ae05-b181f5ab9fbe-home\") pod \"eb9e6693-d98b-4098-ae05-b181f5ab9fbe\" (UID: \"eb9e6693-d98b-4098-ae05-b181f5ab9fbe\") "
Sep 10 19:30:04.690999 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:04.690843    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eb9e6693-d98b-4098-ae05-b181f5ab9fbe-tls-certs\") pod \"eb9e6693-d98b-4098-ae05-b181f5ab9fbe\" (UID: \"eb9e6693-d98b-4098-ae05-b181f5ab9fbe\") "
Sep 10 19:30:04.690999 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:04.690892    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eb9e6693-d98b-4098-ae05-b181f5ab9fbe-tmp-dir\") pod \"eb9e6693-d98b-4098-ae05-b181f5ab9fbe\" (UID: \"eb9e6693-d98b-4098-ae05-b181f5ab9fbe\") "
Sep 10 19:30:04.690999 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:04.690923    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wvzdg\" (UniqueName: \"kubernetes.io/projected/eb9e6693-d98b-4098-ae05-b181f5ab9fbe-kube-api-access-wvzdg\") pod \"eb9e6693-d98b-4098-ae05-b181f5ab9fbe\" (UID: \"eb9e6693-d98b-4098-ae05-b181f5ab9fbe\") "
Sep 10 19:30:04.690999 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:04.690955    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb9e6693-d98b-4098-ae05-b181f5ab9fbe-model-cache" (OuterVolumeSpecName: "model-cache") pod "eb9e6693-d98b-4098-ae05-b181f5ab9fbe" (UID: "eb9e6693-d98b-4098-ae05-b181f5ab9fbe"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:30:04.691226 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:04.691050    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb9e6693-d98b-4098-ae05-b181f5ab9fbe-home" (OuterVolumeSpecName: "home") pod "eb9e6693-d98b-4098-ae05-b181f5ab9fbe" (UID: "eb9e6693-d98b-4098-ae05-b181f5ab9fbe"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:30:04.691226 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:04.691163    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb9e6693-d98b-4098-ae05-b181f5ab9fbe-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "eb9e6693-d98b-4098-ae05-b181f5ab9fbe" (UID: "eb9e6693-d98b-4098-ae05-b181f5ab9fbe"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:30:04.691367 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:04.691254    2575 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eb9e6693-d98b-4098-ae05-b181f5ab9fbe-model-cache\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:30:04.691367 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:04.691271    2575 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eb9e6693-d98b-4098-ae05-b181f5ab9fbe-home\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:30:04.691367 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:04.691283    2575 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eb9e6693-d98b-4098-ae05-b181f5ab9fbe-tmp-dir\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:30:04.692997 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:04.692975    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb9e6693-d98b-4098-ae05-b181f5ab9fbe-dshm" (OuterVolumeSpecName: "dshm") pod "eb9e6693-d98b-4098-ae05-b181f5ab9fbe" (UID: "eb9e6693-d98b-4098-ae05-b181f5ab9fbe"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:30:04.693098 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:04.692998    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eb9e6693-d98b-4098-ae05-b181f5ab9fbe-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "eb9e6693-d98b-4098-ae05-b181f5ab9fbe" (UID: "eb9e6693-d98b-4098-ae05-b181f5ab9fbe"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:30:04.693098 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:04.693031    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb9e6693-d98b-4098-ae05-b181f5ab9fbe-kube-api-access-wvzdg" (OuterVolumeSpecName: "kube-api-access-wvzdg") pod "eb9e6693-d98b-4098-ae05-b181f5ab9fbe" (UID: "eb9e6693-d98b-4098-ae05-b181f5ab9fbe"). InnerVolumeSpecName "kube-api-access-wvzdg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:30:04.792533 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:04.792500    2575 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eb9e6693-d98b-4098-ae05-b181f5ab9fbe-dshm\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:30:04.792533 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:04.792524    2575 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eb9e6693-d98b-4098-ae05-b181f5ab9fbe-tls-certs\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:30:04.792670 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:04.792539    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wvzdg\" (UniqueName: \"kubernetes.io/projected/eb9e6693-d98b-4098-ae05-b181f5ab9fbe-kube-api-access-wvzdg\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:30:05.386416 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:05.386387    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5529e50e-798e-4f48-bcbb-7a032f7650b6" path="/var/lib/kubelet/pods/5529e50e-798e-4f48-bcbb-7a032f7650b6/volumes"
Sep 10 19:30:05.386779 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:05.386767    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9c34b550-2700-4103-bea0-5493d30c13d9" path="/var/lib/kubelet/pods/9c34b550-2700-4103-bea0-5493d30c13d9/volumes"
Sep 10 19:30:05.454460 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:05.454429    2575 generic.go:356] "Generic (PLEG): container finished" podID="eb9e6693-d98b-4098-ae05-b181f5ab9fbe" containerID="f2b9259d59042d0238878ae1137afc0f69420b2734966eb2128ac532b610eb8c" exitCode=0
Sep 10 19:30:05.454619 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:05.454468    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-b3108b/autores-model-b3108b-kserve-7dff498ff5-sv4lz"
Sep 10 19:30:05.454619 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:05.454518    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-b3108b/autores-model-b3108b-kserve-7dff498ff5-sv4lz" event={"ID":"eb9e6693-d98b-4098-ae05-b181f5ab9fbe","Type":"ContainerDied","Data":"f2b9259d59042d0238878ae1137afc0f69420b2734966eb2128ac532b610eb8c"}
Sep 10 19:30:05.454619 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:05.454561    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-b3108b/autores-model-b3108b-kserve-7dff498ff5-sv4lz" event={"ID":"eb9e6693-d98b-4098-ae05-b181f5ab9fbe","Type":"ContainerDied","Data":"5c9af8d0c39a7992a7a1af890f944d29f2cb10ba9641d1885591d087af628c40"}
Sep 10 19:30:05.454619 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:05.454578    2575 scope.go:117] "RemoveContainer" containerID="f2b9259d59042d0238878ae1137afc0f69420b2734966eb2128ac532b610eb8c"
Sep 10 19:30:05.456653 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:05.456633    2575 generic.go:356] "Generic (PLEG): container finished" podID="a2c9f912-84c4-4bbf-86d5-7d3c8029f412" containerID="d7462fdf9ddaa712f5d708d8b6ea8a5d1649f8cad4b8847054f957f877dbef4a" exitCode=0
Sep 10 19:30:05.456749 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:05.456719    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-588a50a1-29817810-n86z5" event={"ID":"a2c9f912-84c4-4bbf-86d5-7d3c8029f412","Type":"ContainerDied","Data":"d7462fdf9ddaa712f5d708d8b6ea8a5d1649f8cad4b8847054f957f877dbef4a"}
Sep 10 19:30:05.473513 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:05.473489    2575 scope.go:117] "RemoveContainer" containerID="f2b9259d59042d0238878ae1137afc0f69420b2734966eb2128ac532b610eb8c"
Sep 10 19:30:05.473826 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:30:05.473804    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f2b9259d59042d0238878ae1137afc0f69420b2734966eb2128ac532b610eb8c\": container with ID starting with f2b9259d59042d0238878ae1137afc0f69420b2734966eb2128ac532b610eb8c not found: ID does not exist" containerID="f2b9259d59042d0238878ae1137afc0f69420b2734966eb2128ac532b610eb8c"
Sep 10 19:30:05.473887 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:05.473841    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f2b9259d59042d0238878ae1137afc0f69420b2734966eb2128ac532b610eb8c"} err="failed to get container status \"f2b9259d59042d0238878ae1137afc0f69420b2734966eb2128ac532b610eb8c\": rpc error: code = NotFound desc = could not find container \"f2b9259d59042d0238878ae1137afc0f69420b2734966eb2128ac532b610eb8c\": container with ID starting with f2b9259d59042d0238878ae1137afc0f69420b2734966eb2128ac532b610eb8c not found: ID does not exist"
Sep 10 19:30:05.473887 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:05.473863    2575 scope.go:117] "RemoveContainer" containerID="1b299f8f7b0fdad9586b33cdaed31519f36b2b802bac56c1b02e64dc1c6e1aa8"
Sep 10 19:30:05.479421 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:05.479387    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:30:05.490401 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:05.490324    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-autores-b3108b/autores-model-b3108b-kserve-7dff498ff5-sv4lz"]
Sep 10 19:30:05.494279 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:05.494258    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-autores-b3108b/autores-model-b3108b-kserve-7dff498ff5-sv4lz"]
Sep 10 19:30:05.595631 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:05.595604    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-8300b9-29817810-8rmhd"
Sep 10 19:30:06.462455 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:06.462427    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-8300b9-29817810-8rmhd"
Sep 10 19:30:06.462455 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:06.462439    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-8300b9-29817810-8rmhd" event={"ID":"e4b52ee7-8cce-4ecc-a236-196308998f71","Type":"ContainerDied","Data":"cbea5a277cec1687e40515959cf7912dd7aa3feee327bbeae3695175d373b6d2"}
Sep 10 19:30:06.462907 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:06.462484    2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cbea5a277cec1687e40515959cf7912dd7aa3feee327bbeae3695175d373b6d2"
Sep 10 19:30:06.592525 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:06.592503    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-588a50a1-29817810-n86z5"
Sep 10 19:30:06.831682 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:06.831598    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:30:07.386313 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:07.386281    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eb9e6693-d98b-4098-ae05-b181f5ab9fbe" path="/var/lib/kubelet/pods/eb9e6693-d98b-4098-ae05-b181f5ab9fbe/volumes"
Sep 10 19:30:07.472061 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:07.472031    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-588a50a1-29817810-n86z5" event={"ID":"a2c9f912-84c4-4bbf-86d5-7d3c8029f412","Type":"ContainerDied","Data":"727da67898fbed3b9ff6327ce33f208ea084fbf2134e18996db9089adbdfb77b"}
Sep 10 19:30:07.472061 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:07.472051    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-588a50a1-29817810-n86z5"
Sep 10 19:30:07.472445 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:07.472073    2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="727da67898fbed3b9ff6327ce33f208ea084fbf2134e18996db9089adbdfb77b"
Sep 10 19:30:07.543169 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:07.543142    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:30:07.645011 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:07.644916    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:30:07.800631 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:07.800603    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-explicit-faed1b-openshift-default-644ddf89b4-jsrj9"]
Sep 10 19:30:07.800981 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:07.800970    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5529e50e-798e-4f48-bcbb-7a032f7650b6" containerName="main"
Sep 10 19:30:07.801021 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:07.800982    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="5529e50e-798e-4f48-bcbb-7a032f7650b6" containerName="main"
Sep 10 19:30:07.801021 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:07.800991    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a2c9f912-84c4-4bbf-86d5-7d3c8029f412" containerName="cleanup"
Sep 10 19:30:07.801021 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:07.800996    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="a2c9f912-84c4-4bbf-86d5-7d3c8029f412" containerName="cleanup"
Sep 10 19:30:07.801021 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:07.801004    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e4b52ee7-8cce-4ecc-a236-196308998f71" containerName="cleanup"
Sep 10 19:30:07.801021 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:07.801010    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="e4b52ee7-8cce-4ecc-a236-196308998f71" containerName="cleanup"
Sep 10 19:30:07.801021 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:07.801017    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a2c9f912-84c4-4bbf-86d5-7d3c8029f412" containerName="cleanup"
Sep 10 19:30:07.801021 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:07.801023    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="a2c9f912-84c4-4bbf-86d5-7d3c8029f412" containerName="cleanup"
Sep 10 19:30:07.801221 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:07.801033    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eb9e6693-d98b-4098-ae05-b181f5ab9fbe" containerName="main"
Sep 10 19:30:07.801221 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:07.801038    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="eb9e6693-d98b-4098-ae05-b181f5ab9fbe" containerName="main"
Sep 10 19:30:07.801221 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:07.801045    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9c34b550-2700-4103-bea0-5493d30c13d9" containerName="main"
Sep 10 19:30:07.801221 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:07.801049    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="9c34b550-2700-4103-bea0-5493d30c13d9" containerName="main"
Sep 10 19:30:07.801221 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:07.801104    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="a2c9f912-84c4-4bbf-86d5-7d3c8029f412" containerName="cleanup"
Sep 10 19:30:07.801221 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:07.801111    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="9c34b550-2700-4103-bea0-5493d30c13d9" containerName="main"
Sep 10 19:30:07.801221 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:07.801118    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="a2c9f912-84c4-4bbf-86d5-7d3c8029f412" containerName="cleanup"
Sep 10 19:30:07.801221 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:07.801125    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="5529e50e-798e-4f48-bcbb-7a032f7650b6" containerName="main"
Sep 10 19:30:07.801221 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:07.801131    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="e4b52ee7-8cce-4ecc-a236-196308998f71" containerName="cleanup"
Sep 10 19:30:07.801221 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:07.801135    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="eb9e6693-d98b-4098-ae05-b181f5ab9fbe" containerName="main"
Sep 10 19:30:07.804195 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:07.804180    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-faed1b-openshift-default-644ddf89b4-jsrj9"
Sep 10 19:30:07.806292 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:07.806273    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-explicit-faed1b-openshift-default-dockercfg-88p8c\""
Sep 10 19:30:07.819615 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:07.819596    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-faed1b-openshift-default-644ddf89b4-jsrj9"]
Sep 10 19:30:07.828640 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:07.828617    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:30:07.926732 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:07.926644    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/82f513d0-95e0-42e4-abc4-22c31668664b-workload-socket\") pod \"e2e-explicit-faed1b-openshift-default-644ddf89b4-jsrj9\" (UID: \"82f513d0-95e0-42e4-abc4-22c31668664b\") " pod="openshift-ingress/e2e-explicit-faed1b-openshift-default-644ddf89b4-jsrj9"
Sep 10 19:30:07.926732 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:07.926699    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/82f513d0-95e0-42e4-abc4-22c31668664b-istio-envoy\") pod \"e2e-explicit-faed1b-openshift-default-644ddf89b4-jsrj9\" (UID: \"82f513d0-95e0-42e4-abc4-22c31668664b\") " pod="openshift-ingress/e2e-explicit-faed1b-openshift-default-644ddf89b4-jsrj9"
Sep 10 19:30:07.926897 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:07.926762    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/82f513d0-95e0-42e4-abc4-22c31668664b-istiod-ca-cert\") pod \"e2e-explicit-faed1b-openshift-default-644ddf89b4-jsrj9\" (UID: \"82f513d0-95e0-42e4-abc4-22c31668664b\") " pod="openshift-ingress/e2e-explicit-faed1b-openshift-default-644ddf89b4-jsrj9"
Sep 10 19:30:07.926897 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:07.926792    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/82f513d0-95e0-42e4-abc4-22c31668664b-workload-certs\") pod \"e2e-explicit-faed1b-openshift-default-644ddf89b4-jsrj9\" (UID: \"82f513d0-95e0-42e4-abc4-22c31668664b\") " pod="openshift-ingress/e2e-explicit-faed1b-openshift-default-644ddf89b4-jsrj9"
Sep 10 19:30:07.926897 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:07.926821    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/82f513d0-95e0-42e4-abc4-22c31668664b-istio-data\") pod \"e2e-explicit-faed1b-openshift-default-644ddf89b4-jsrj9\" (UID: \"82f513d0-95e0-42e4-abc4-22c31668664b\") " pod="openshift-ingress/e2e-explicit-faed1b-openshift-default-644ddf89b4-jsrj9"
Sep 10 19:30:07.926897 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:07.926864    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/82f513d0-95e0-42e4-abc4-22c31668664b-istio-podinfo\") pod \"e2e-explicit-faed1b-openshift-default-644ddf89b4-jsrj9\" (UID: \"82f513d0-95e0-42e4-abc4-22c31668664b\") " pod="openshift-ingress/e2e-explicit-faed1b-openshift-default-644ddf89b4-jsrj9"
Sep 10 19:30:07.926897 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:07.926880    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qdj7b\" (UniqueName: \"kubernetes.io/projected/82f513d0-95e0-42e4-abc4-22c31668664b-kube-api-access-qdj7b\") pod \"e2e-explicit-faed1b-openshift-default-644ddf89b4-jsrj9\" (UID: \"82f513d0-95e0-42e4-abc4-22c31668664b\") " pod="openshift-ingress/e2e-explicit-faed1b-openshift-default-644ddf89b4-jsrj9"
Sep 10 19:30:07.927055 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:07.926902    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/82f513d0-95e0-42e4-abc4-22c31668664b-credential-socket\") pod \"e2e-explicit-faed1b-openshift-default-644ddf89b4-jsrj9\" (UID: \"82f513d0-95e0-42e4-abc4-22c31668664b\") " pod="openshift-ingress/e2e-explicit-faed1b-openshift-default-644ddf89b4-jsrj9"
Sep 10 19:30:07.927055 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:07.926926    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/82f513d0-95e0-42e4-abc4-22c31668664b-istio-token\") pod \"e2e-explicit-faed1b-openshift-default-644ddf89b4-jsrj9\" (UID: \"82f513d0-95e0-42e4-abc4-22c31668664b\") " pod="openshift-ingress/e2e-explicit-faed1b-openshift-default-644ddf89b4-jsrj9"
Sep 10 19:30:08.027418 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:08.027390    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/82f513d0-95e0-42e4-abc4-22c31668664b-workload-certs\") pod \"e2e-explicit-faed1b-openshift-default-644ddf89b4-jsrj9\" (UID: \"82f513d0-95e0-42e4-abc4-22c31668664b\") " pod="openshift-ingress/e2e-explicit-faed1b-openshift-default-644ddf89b4-jsrj9"
Sep 10 19:30:08.027565 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:08.027433    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/82f513d0-95e0-42e4-abc4-22c31668664b-istio-data\") pod \"e2e-explicit-faed1b-openshift-default-644ddf89b4-jsrj9\" (UID: \"82f513d0-95e0-42e4-abc4-22c31668664b\") " pod="openshift-ingress/e2e-explicit-faed1b-openshift-default-644ddf89b4-jsrj9"
Sep 10 19:30:08.027565 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:08.027458    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/82f513d0-95e0-42e4-abc4-22c31668664b-istio-podinfo\") pod \"e2e-explicit-faed1b-openshift-default-644ddf89b4-jsrj9\" (UID: \"82f513d0-95e0-42e4-abc4-22c31668664b\") " pod="openshift-ingress/e2e-explicit-faed1b-openshift-default-644ddf89b4-jsrj9"
Sep 10 19:30:08.027565 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:08.027475    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qdj7b\" (UniqueName: \"kubernetes.io/projected/82f513d0-95e0-42e4-abc4-22c31668664b-kube-api-access-qdj7b\") pod \"e2e-explicit-faed1b-openshift-default-644ddf89b4-jsrj9\" (UID: \"82f513d0-95e0-42e4-abc4-22c31668664b\") " pod="openshift-ingress/e2e-explicit-faed1b-openshift-default-644ddf89b4-jsrj9"
Sep 10 19:30:08.027565 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:08.027503    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/82f513d0-95e0-42e4-abc4-22c31668664b-credential-socket\") pod \"e2e-explicit-faed1b-openshift-default-644ddf89b4-jsrj9\" (UID: \"82f513d0-95e0-42e4-abc4-22c31668664b\") " pod="openshift-ingress/e2e-explicit-faed1b-openshift-default-644ddf89b4-jsrj9"
Sep 10 19:30:08.027565 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:08.027521    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/82f513d0-95e0-42e4-abc4-22c31668664b-istio-token\") pod \"e2e-explicit-faed1b-openshift-default-644ddf89b4-jsrj9\" (UID: \"82f513d0-95e0-42e4-abc4-22c31668664b\") " pod="openshift-ingress/e2e-explicit-faed1b-openshift-default-644ddf89b4-jsrj9"
Sep 10 19:30:08.027815 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:08.027564    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/82f513d0-95e0-42e4-abc4-22c31668664b-workload-socket\") pod \"e2e-explicit-faed1b-openshift-default-644ddf89b4-jsrj9\" (UID: \"82f513d0-95e0-42e4-abc4-22c31668664b\") " pod="openshift-ingress/e2e-explicit-faed1b-openshift-default-644ddf89b4-jsrj9"
Sep 10 19:30:08.027815 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:08.027728    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/82f513d0-95e0-42e4-abc4-22c31668664b-istio-envoy\") pod \"e2e-explicit-faed1b-openshift-default-644ddf89b4-jsrj9\" (UID: \"82f513d0-95e0-42e4-abc4-22c31668664b\") " pod="openshift-ingress/e2e-explicit-faed1b-openshift-default-644ddf89b4-jsrj9"
Sep 10 19:30:08.027815 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:08.027806    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/82f513d0-95e0-42e4-abc4-22c31668664b-istiod-ca-cert\") pod \"e2e-explicit-faed1b-openshift-default-644ddf89b4-jsrj9\" (UID: \"82f513d0-95e0-42e4-abc4-22c31668664b\") " pod="openshift-ingress/e2e-explicit-faed1b-openshift-default-644ddf89b4-jsrj9"
Sep 10 19:30:08.028004 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:08.027836    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/82f513d0-95e0-42e4-abc4-22c31668664b-workload-certs\") pod \"e2e-explicit-faed1b-openshift-default-644ddf89b4-jsrj9\" (UID: \"82f513d0-95e0-42e4-abc4-22c31668664b\") " pod="openshift-ingress/e2e-explicit-faed1b-openshift-default-644ddf89b4-jsrj9"
Sep 10 19:30:08.028004 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:08.027885    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/82f513d0-95e0-42e4-abc4-22c31668664b-credential-socket\") pod \"e2e-explicit-faed1b-openshift-default-644ddf89b4-jsrj9\" (UID: \"82f513d0-95e0-42e4-abc4-22c31668664b\") " pod="openshift-ingress/e2e-explicit-faed1b-openshift-default-644ddf89b4-jsrj9"
Sep 10 19:30:08.028004 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:08.027954    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/82f513d0-95e0-42e4-abc4-22c31668664b-workload-socket\") pod \"e2e-explicit-faed1b-openshift-default-644ddf89b4-jsrj9\" (UID: \"82f513d0-95e0-42e4-abc4-22c31668664b\") " pod="openshift-ingress/e2e-explicit-faed1b-openshift-default-644ddf89b4-jsrj9"
Sep 10 19:30:08.028125 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:08.028105    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/82f513d0-95e0-42e4-abc4-22c31668664b-istio-data\") pod \"e2e-explicit-faed1b-openshift-default-644ddf89b4-jsrj9\" (UID: \"82f513d0-95e0-42e4-abc4-22c31668664b\") " pod="openshift-ingress/e2e-explicit-faed1b-openshift-default-644ddf89b4-jsrj9"
Sep 10 19:30:08.028405 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:08.028388    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/82f513d0-95e0-42e4-abc4-22c31668664b-istiod-ca-cert\") pod \"e2e-explicit-faed1b-openshift-default-644ddf89b4-jsrj9\" (UID: \"82f513d0-95e0-42e4-abc4-22c31668664b\") " pod="openshift-ingress/e2e-explicit-faed1b-openshift-default-644ddf89b4-jsrj9"
Sep 10 19:30:08.029860 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:08.029839    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/82f513d0-95e0-42e4-abc4-22c31668664b-istio-envoy\") pod \"e2e-explicit-faed1b-openshift-default-644ddf89b4-jsrj9\" (UID: \"82f513d0-95e0-42e4-abc4-22c31668664b\") " pod="openshift-ingress/e2e-explicit-faed1b-openshift-default-644ddf89b4-jsrj9"
Sep 10 19:30:08.030058 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:08.030035    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/82f513d0-95e0-42e4-abc4-22c31668664b-istio-podinfo\") pod \"e2e-explicit-faed1b-openshift-default-644ddf89b4-jsrj9\" (UID: \"82f513d0-95e0-42e4-abc4-22c31668664b\") " pod="openshift-ingress/e2e-explicit-faed1b-openshift-default-644ddf89b4-jsrj9"
Sep 10 19:30:08.043556 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:08.043533    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qdj7b\" (UniqueName: \"kubernetes.io/projected/82f513d0-95e0-42e4-abc4-22c31668664b-kube-api-access-qdj7b\") pod \"e2e-explicit-faed1b-openshift-default-644ddf89b4-jsrj9\" (UID: \"82f513d0-95e0-42e4-abc4-22c31668664b\") " pod="openshift-ingress/e2e-explicit-faed1b-openshift-default-644ddf89b4-jsrj9"
Sep 10 19:30:08.043738 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:08.043713    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/82f513d0-95e0-42e4-abc4-22c31668664b-istio-token\") pod \"e2e-explicit-faed1b-openshift-default-644ddf89b4-jsrj9\" (UID: \"82f513d0-95e0-42e4-abc4-22c31668664b\") " pod="openshift-ingress/e2e-explicit-faed1b-openshift-default-644ddf89b4-jsrj9"
Sep 10 19:30:08.114032 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:08.114014    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-faed1b-openshift-default-644ddf89b4-jsrj9"
Sep 10 19:30:08.388178 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:08.388154    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-faed1b-openshift-default-644ddf89b4-jsrj9"]
Sep 10 19:30:08.390007 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:30:08.389983    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82f513d0_95e0_42e4_abc4_22c31668664b.slice/crio-ba16646f73573c5b7b1471bac37a989ee9e1d43b7b4cee1ccc9c6f0ed08fbde6 WatchSource:0}: Error finding container ba16646f73573c5b7b1471bac37a989ee9e1d43b7b4cee1ccc9c6f0ed08fbde6: Status 404 returned error can't find the container with id ba16646f73573c5b7b1471bac37a989ee9e1d43b7b4cee1ccc9c6f0ed08fbde6
Sep 10 19:30:08.392166 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:08.392137    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 19:30:08.392223 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:08.392197    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 19:30:08.392265 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:08.392229    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 19:30:08.478434 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:08.478385    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-faed1b-openshift-default-644ddf89b4-jsrj9" event={"ID":"82f513d0-95e0-42e4-abc4-22c31668664b","Type":"ContainerStarted","Data":"fc25bc9562678c5b737e1fbbc3eb09107638eebd893e79cb0f2099e29a118f6e"}
Sep 10 19:30:08.478775 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:08.478466    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-faed1b-openshift-default-644ddf89b4-jsrj9" event={"ID":"82f513d0-95e0-42e4-abc4-22c31668664b","Type":"ContainerStarted","Data":"ba16646f73573c5b7b1471bac37a989ee9e1d43b7b4cee1ccc9c6f0ed08fbde6"}
Sep 10 19:30:08.500474 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:08.500418    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-explicit-faed1b-openshift-default-644ddf89b4-jsrj9" podStartSLOduration=1.500398066 podStartE2EDuration="1.500398066s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:30:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:30:08.49843474 +0000 UTC m=+879.558262411" watchObservedRunningTime="2026-09-10 19:30:08.500398066 +0000 UTC m=+879.560225734"
Sep 10 19:30:09.092107 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:09.092078    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:30:09.114884 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:09.114863    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-explicit-faed1b-openshift-default-644ddf89b4-jsrj9"
Sep 10 19:30:09.120390 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:09.120371    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-explicit-faed1b-openshift-default-644ddf89b4-jsrj9"
Sep 10 19:30:09.483203 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:09.483168    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-explicit-faed1b-openshift-default-644ddf89b4-jsrj9"
Sep 10 19:30:09.484061 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:09.484041    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-explicit-faed1b-openshift-default-644ddf89b4-jsrj9"
Sep 10 19:30:10.464818 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:10.464787    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:30:10.562525 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:10.562489    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:30:11.768454 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:11.768416    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:30:11.868293 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:11.868256    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:30:11.967716 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:11.967679    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:30:12.290878 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:12.290844    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:30:12.390689 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:12.390647    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:30:12.493655 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:12.493627    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-faed1b-6756ffcc7d-lvs7d"]
Sep 10 19:30:12.497657 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:12.497636    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-faed1b-6756ffcc7d-lvs7d"
Sep 10 19:30:12.499702 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:12.499681    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-explicit-faed1b\""
Sep 10 19:30:12.499902 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:12.499888    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-explicit-faed1b-dockercfg-5zjqz\""
Sep 10 19:30:12.506393 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:12.506370    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-faed1b-6756ffcc7d-lvs7d"]
Sep 10 19:30:12.668680 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:12.668593    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/16d359d7-2baa-4fc7-aee2-70d88578e5f4-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-faed1b-6756ffcc7d-lvs7d\" (UID: \"16d359d7-2baa-4fc7-aee2-70d88578e5f4\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-faed1b-6756ffcc7d-lvs7d"
Sep 10 19:30:12.668819 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:12.668705    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ctbwk\" (UniqueName: \"kubernetes.io/projected/16d359d7-2baa-4fc7-aee2-70d88578e5f4-kube-api-access-ctbwk\") pod \"payload-pre-processing-e2e-explicit-faed1b-6756ffcc7d-lvs7d\" (UID: \"16d359d7-2baa-4fc7-aee2-70d88578e5f4\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-faed1b-6756ffcc7d-lvs7d"
Sep 10 19:30:12.769966 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:12.769931    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ctbwk\" (UniqueName: \"kubernetes.io/projected/16d359d7-2baa-4fc7-aee2-70d88578e5f4-kube-api-access-ctbwk\") pod \"payload-pre-processing-e2e-explicit-faed1b-6756ffcc7d-lvs7d\" (UID: \"16d359d7-2baa-4fc7-aee2-70d88578e5f4\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-faed1b-6756ffcc7d-lvs7d"
Sep 10 19:30:12.770420 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:12.770016    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/16d359d7-2baa-4fc7-aee2-70d88578e5f4-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-faed1b-6756ffcc7d-lvs7d\" (UID: \"16d359d7-2baa-4fc7-aee2-70d88578e5f4\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-faed1b-6756ffcc7d-lvs7d"
Sep 10 19:30:12.770920 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:12.770899    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/16d359d7-2baa-4fc7-aee2-70d88578e5f4-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-faed1b-6756ffcc7d-lvs7d\" (UID: \"16d359d7-2baa-4fc7-aee2-70d88578e5f4\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-faed1b-6756ffcc7d-lvs7d"
Sep 10 19:30:12.777277 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:12.777249    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ctbwk\" (UniqueName: \"kubernetes.io/projected/16d359d7-2baa-4fc7-aee2-70d88578e5f4-kube-api-access-ctbwk\") pod \"payload-pre-processing-e2e-explicit-faed1b-6756ffcc7d-lvs7d\" (UID: \"16d359d7-2baa-4fc7-aee2-70d88578e5f4\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-faed1b-6756ffcc7d-lvs7d"
Sep 10 19:30:12.808504 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:12.808479    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-faed1b-6756ffcc7d-lvs7d"
Sep 10 19:30:13.079529 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:13.079504    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-faed1b-6756ffcc7d-lvs7d"]
Sep 10 19:30:13.081380 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:30:13.081333    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16d359d7_2baa_4fc7_aee2_70d88578e5f4.slice/crio-ba55c59ae8a19efe956e17ef22fa39fe6c3cd18aedda13d7584d3d52d714b1ee WatchSource:0}: Error finding container ba55c59ae8a19efe956e17ef22fa39fe6c3cd18aedda13d7584d3d52d714b1ee: Status 404 returned error can't find the container with id ba55c59ae8a19efe956e17ef22fa39fe6c3cd18aedda13d7584d3d52d714b1ee
Sep 10 19:30:13.507642 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:13.507610    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-faed1b-6756ffcc7d-lvs7d" event={"ID":"16d359d7-2baa-4fc7-aee2-70d88578e5f4","Type":"ContainerStarted","Data":"3e7466470ff28522d1eb9dfcb2a79292b98aa89eb34e178ad0fca4566ffc3852"}
Sep 10 19:30:13.507829 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:13.507662    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-faed1b-6756ffcc7d-lvs7d" event={"ID":"16d359d7-2baa-4fc7-aee2-70d88578e5f4","Type":"ContainerStarted","Data":"ba55c59ae8a19efe956e17ef22fa39fe6c3cd18aedda13d7584d3d52d714b1ee"}
Sep 10 19:30:13.507829 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:13.507689    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-explicit-faed1b-6756ffcc7d-lvs7d"
Sep 10 19:30:13.525193 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:13.525145    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-explicit-faed1b-6756ffcc7d-lvs7d" podStartSLOduration=1.525131088 podStartE2EDuration="1.525131088s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:30:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:30:13.523296492 +0000 UTC m=+884.583124161" watchObservedRunningTime="2026-09-10 19:30:13.525131088 +0000 UTC m=+884.584958795"
Sep 10 19:30:19.264532 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:19.264497    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qr7rl"]
Sep 10 19:30:19.943597 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:19.943558    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:30:24.514256 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:24.514179    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-explicit-faed1b-6756ffcc7d-lvs7d"
Sep 10 19:30:29.372560 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:29.372533    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rrxfj_c7511dfe-c588-4be1-bd16-1dce34740809/ovn-acl-logging/0.log"
Sep 10 19:30:29.373001 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:29.372795    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rrxfj_c7511dfe-c588-4be1-bd16-1dce34740809/ovn-acl-logging/0.log"
Sep 10 19:30:29.601936 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:29.601905    2575 generic.go:356] "Generic (PLEG): container finished" podID="82f513d0-95e0-42e4-abc4-22c31668664b" containerID="fc25bc9562678c5b737e1fbbc3eb09107638eebd893e79cb0f2099e29a118f6e" exitCode=0
Sep 10 19:30:29.602097 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:29.601979    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-faed1b-openshift-default-644ddf89b4-jsrj9" event={"ID":"82f513d0-95e0-42e4-abc4-22c31668664b","Type":"ContainerDied","Data":"fc25bc9562678c5b737e1fbbc3eb09107638eebd893e79cb0f2099e29a118f6e"}
Sep 10 19:30:29.602394 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:29.602380    2575 scope.go:117] "RemoveContainer" containerID="fc25bc9562678c5b737e1fbbc3eb09107638eebd893e79cb0f2099e29a118f6e"
Sep 10 19:30:29.603394 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:29.603367    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 19:30:29.603468 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:29.603441    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 19:30:29.603504 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:29.603471    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 19:30:30.609109 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:30.609069    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-faed1b-openshift-default-644ddf89b4-jsrj9" event={"ID":"82f513d0-95e0-42e4-abc4-22c31668664b","Type":"ContainerStarted","Data":"1a96670fda8269b4a1f1a983096acf24805318bd2dfa51d9aab4635d33e953b8"}
Sep 10 19:30:31.114930 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:31.114893    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-explicit-faed1b-openshift-default-644ddf89b4-jsrj9"
Sep 10 19:30:31.115163 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:31.115139    2575 patch_prober.go:28] interesting pod/e2e-explicit-faed1b-openshift-default-644ddf89b4-jsrj9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body=
Sep 10 19:30:31.115214 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:31.115190    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-faed1b-openshift-default-644ddf89b4-jsrj9" podUID="82f513d0-95e0-42e4-abc4-22c31668664b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused"
Sep 10 19:30:32.115693 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:32.115659    2575 patch_prober.go:28] interesting pod/e2e-explicit-faed1b-openshift-default-644ddf89b4-jsrj9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body=
Sep 10 19:30:32.116066 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:32.115721    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-faed1b-openshift-default-644ddf89b4-jsrj9" podUID="82f513d0-95e0-42e4-abc4-22c31668664b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused"
Sep 10 19:30:33.114597 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:33.114563    2575 patch_prober.go:28] interesting pod/e2e-explicit-faed1b-openshift-default-644ddf89b4-jsrj9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body=
Sep 10 19:30:33.114905 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:33.114636    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-faed1b-openshift-default-644ddf89b4-jsrj9" podUID="82f513d0-95e0-42e4-abc4-22c31668664b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused"
Sep 10 19:30:34.104538 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:34.104504    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-4247c7/test-model-4247c7-kserve-746dcdc87-vnj4w"]
Sep 10 19:30:34.113471 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:34.113448    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-4247c7/test-model-4247c7-kserve-746dcdc87-vnj4w"
Sep 10 19:30:34.114648 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:34.114611    2575 patch_prober.go:28] interesting pod/e2e-explicit-faed1b-openshift-default-644ddf89b4-jsrj9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body=
Sep 10 19:30:34.114770 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:34.114674    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-faed1b-openshift-default-644ddf89b4-jsrj9" podUID="82f513d0-95e0-42e4-abc4-22c31668664b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused"
Sep 10 19:30:34.115665 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:34.115642    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-4247c7\"/\"default-dockercfg-d42gp\""
Sep 10 19:30:34.115783 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:34.115750    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-4247c7\"/\"test-model-4247c7-kserve-self-signed-certs\""
Sep 10 19:30:34.116018 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:34.116000    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-4247c7\"/\"openshift-service-ca.crt\""
Sep 10 19:30:34.118965 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:34.118929    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-4247c7/test-model-4247c7-kserve-746dcdc87-vnj4w"]
Sep 10 19:30:34.122103 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:34.122085    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-4247c7\"/\"kube-root-ca.crt\""
Sep 10 19:30:34.163301 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:34.163267    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hjknp\" (UniqueName: \"kubernetes.io/projected/50e45adb-8cbc-4820-9c39-c4896e4a2ddd-kube-api-access-hjknp\") pod \"test-model-4247c7-kserve-746dcdc87-vnj4w\" (UID: \"50e45adb-8cbc-4820-9c39-c4896e4a2ddd\") " pod="ai-tenant-e2e-inf-b-4247c7/test-model-4247c7-kserve-746dcdc87-vnj4w"
Sep 10 19:30:34.163301 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:34.163305    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/50e45adb-8cbc-4820-9c39-c4896e4a2ddd-tmp-dir\") pod \"test-model-4247c7-kserve-746dcdc87-vnj4w\" (UID: \"50e45adb-8cbc-4820-9c39-c4896e4a2ddd\") " pod="ai-tenant-e2e-inf-b-4247c7/test-model-4247c7-kserve-746dcdc87-vnj4w"
Sep 10 19:30:34.163518 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:34.163324    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/50e45adb-8cbc-4820-9c39-c4896e4a2ddd-dshm\") pod \"test-model-4247c7-kserve-746dcdc87-vnj4w\" (UID: \"50e45adb-8cbc-4820-9c39-c4896e4a2ddd\") " pod="ai-tenant-e2e-inf-b-4247c7/test-model-4247c7-kserve-746dcdc87-vnj4w"
Sep 10 19:30:34.163518 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:34.163364    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/50e45adb-8cbc-4820-9c39-c4896e4a2ddd-model-cache\") pod \"test-model-4247c7-kserve-746dcdc87-vnj4w\" (UID: \"50e45adb-8cbc-4820-9c39-c4896e4a2ddd\") " pod="ai-tenant-e2e-inf-b-4247c7/test-model-4247c7-kserve-746dcdc87-vnj4w"
Sep 10 19:30:34.163518 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:34.163392    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/50e45adb-8cbc-4820-9c39-c4896e4a2ddd-tls-certs\") pod \"test-model-4247c7-kserve-746dcdc87-vnj4w\" (UID: \"50e45adb-8cbc-4820-9c39-c4896e4a2ddd\") " pod="ai-tenant-e2e-inf-b-4247c7/test-model-4247c7-kserve-746dcdc87-vnj4w"
Sep 10 19:30:34.163518 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:34.163446    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/50e45adb-8cbc-4820-9c39-c4896e4a2ddd-home\") pod \"test-model-4247c7-kserve-746dcdc87-vnj4w\" (UID: \"50e45adb-8cbc-4820-9c39-c4896e4a2ddd\") " pod="ai-tenant-e2e-inf-b-4247c7/test-model-4247c7-kserve-746dcdc87-vnj4w"
Sep 10 19:30:34.231366 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:34.230264    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:30:34.264375 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:34.264335    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/50e45adb-8cbc-4820-9c39-c4896e4a2ddd-dshm\") pod \"test-model-4247c7-kserve-746dcdc87-vnj4w\" (UID: \"50e45adb-8cbc-4820-9c39-c4896e4a2ddd\") " pod="ai-tenant-e2e-inf-b-4247c7/test-model-4247c7-kserve-746dcdc87-vnj4w"
Sep 10 19:30:34.264555 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:34.264449    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/50e45adb-8cbc-4820-9c39-c4896e4a2ddd-model-cache\") pod \"test-model-4247c7-kserve-746dcdc87-vnj4w\" (UID: \"50e45adb-8cbc-4820-9c39-c4896e4a2ddd\") " pod="ai-tenant-e2e-inf-b-4247c7/test-model-4247c7-kserve-746dcdc87-vnj4w"
Sep 10 19:30:34.264555 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:34.264507    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/50e45adb-8cbc-4820-9c39-c4896e4a2ddd-tls-certs\") pod \"test-model-4247c7-kserve-746dcdc87-vnj4w\" (UID: \"50e45adb-8cbc-4820-9c39-c4896e4a2ddd\") " pod="ai-tenant-e2e-inf-b-4247c7/test-model-4247c7-kserve-746dcdc87-vnj4w"
Sep 10 19:30:34.264555 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:34.264549    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/50e45adb-8cbc-4820-9c39-c4896e4a2ddd-home\") pod \"test-model-4247c7-kserve-746dcdc87-vnj4w\" (UID: \"50e45adb-8cbc-4820-9c39-c4896e4a2ddd\") " pod="ai-tenant-e2e-inf-b-4247c7/test-model-4247c7-kserve-746dcdc87-vnj4w"
Sep 10 19:30:34.264727 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:34.264640    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hjknp\" (UniqueName: \"kubernetes.io/projected/50e45adb-8cbc-4820-9c39-c4896e4a2ddd-kube-api-access-hjknp\") pod \"test-model-4247c7-kserve-746dcdc87-vnj4w\" (UID: \"50e45adb-8cbc-4820-9c39-c4896e4a2ddd\") " pod="ai-tenant-e2e-inf-b-4247c7/test-model-4247c7-kserve-746dcdc87-vnj4w"
Sep 10 19:30:34.264727 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:34.264695    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/50e45adb-8cbc-4820-9c39-c4896e4a2ddd-tmp-dir\") pod \"test-model-4247c7-kserve-746dcdc87-vnj4w\" (UID: \"50e45adb-8cbc-4820-9c39-c4896e4a2ddd\") " pod="ai-tenant-e2e-inf-b-4247c7/test-model-4247c7-kserve-746dcdc87-vnj4w"
Sep 10 19:30:34.264727 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:34.264705    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/50e45adb-8cbc-4820-9c39-c4896e4a2ddd-model-cache\") pod \"test-model-4247c7-kserve-746dcdc87-vnj4w\" (UID: \"50e45adb-8cbc-4820-9c39-c4896e4a2ddd\") " pod="ai-tenant-e2e-inf-b-4247c7/test-model-4247c7-kserve-746dcdc87-vnj4w"
Sep 10 19:30:34.265168 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:34.265149    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/50e45adb-8cbc-4820-9c39-c4896e4a2ddd-home\") pod \"test-model-4247c7-kserve-746dcdc87-vnj4w\" (UID: \"50e45adb-8cbc-4820-9c39-c4896e4a2ddd\") " pod="ai-tenant-e2e-inf-b-4247c7/test-model-4247c7-kserve-746dcdc87-vnj4w"
Sep 10 19:30:34.265315 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:34.265291    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/50e45adb-8cbc-4820-9c39-c4896e4a2ddd-tmp-dir\") pod \"test-model-4247c7-kserve-746dcdc87-vnj4w\" (UID: \"50e45adb-8cbc-4820-9c39-c4896e4a2ddd\") " pod="ai-tenant-e2e-inf-b-4247c7/test-model-4247c7-kserve-746dcdc87-vnj4w"
Sep 10 19:30:34.267078 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:34.267057    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/50e45adb-8cbc-4820-9c39-c4896e4a2ddd-dshm\") pod \"test-model-4247c7-kserve-746dcdc87-vnj4w\" (UID: \"50e45adb-8cbc-4820-9c39-c4896e4a2ddd\") " pod="ai-tenant-e2e-inf-b-4247c7/test-model-4247c7-kserve-746dcdc87-vnj4w"
Sep 10 19:30:34.267171 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:34.267100    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/50e45adb-8cbc-4820-9c39-c4896e4a2ddd-tls-certs\") pod \"test-model-4247c7-kserve-746dcdc87-vnj4w\" (UID: \"50e45adb-8cbc-4820-9c39-c4896e4a2ddd\") " pod="ai-tenant-e2e-inf-b-4247c7/test-model-4247c7-kserve-746dcdc87-vnj4w"
Sep 10 19:30:34.272736 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:34.272701    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hjknp\" (UniqueName: \"kubernetes.io/projected/50e45adb-8cbc-4820-9c39-c4896e4a2ddd-kube-api-access-hjknp\") pod \"test-model-4247c7-kserve-746dcdc87-vnj4w\" (UID: \"50e45adb-8cbc-4820-9c39-c4896e4a2ddd\") " pod="ai-tenant-e2e-inf-b-4247c7/test-model-4247c7-kserve-746dcdc87-vnj4w"
Sep 10 19:30:34.330716 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:34.330691    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:30:34.435666 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:34.435591    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-4247c7/test-model-4247c7-kserve-746dcdc87-vnj4w"
Sep 10 19:30:34.711455 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:34.711423    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-4247c7/test-model-4247c7-kserve-746dcdc87-vnj4w"]
Sep 10 19:30:34.714801 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:30:34.714774    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod50e45adb_8cbc_4820_9c39_c4896e4a2ddd.slice/crio-9d4a21f4d59089f52fc69711d2c9430cf04ec7790b0fe5236a128590a26556c2 WatchSource:0}: Error finding container 9d4a21f4d59089f52fc69711d2c9430cf04ec7790b0fe5236a128590a26556c2: Status 404 returned error can't find the container with id 9d4a21f4d59089f52fc69711d2c9430cf04ec7790b0fe5236a128590a26556c2
Sep 10 19:30:35.115303 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:35.115209    2575 patch_prober.go:28] interesting pod/e2e-explicit-faed1b-openshift-default-644ddf89b4-jsrj9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body=
Sep 10 19:30:35.115303 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:35.115262    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-faed1b-openshift-default-644ddf89b4-jsrj9" podUID="82f513d0-95e0-42e4-abc4-22c31668664b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused"
Sep 10 19:30:35.356521 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:35.356484    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:30:35.647539 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:35.647501    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-4247c7/test-model-4247c7-kserve-746dcdc87-vnj4w" event={"ID":"50e45adb-8cbc-4820-9c39-c4896e4a2ddd","Type":"ContainerStarted","Data":"8c8816571e7d33a6b2903b669d295055b5d87eec4532757beef9561122d1be5c"}
Sep 10 19:30:35.647539 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:35.647543    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-4247c7/test-model-4247c7-kserve-746dcdc87-vnj4w" event={"ID":"50e45adb-8cbc-4820-9c39-c4896e4a2ddd","Type":"ContainerStarted","Data":"9d4a21f4d59089f52fc69711d2c9430cf04ec7790b0fe5236a128590a26556c2"}
Sep 10 19:30:35.665619 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:35.665566    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-4247c7/test-model-4247c7-kserve-746dcdc87-vnj4w" podStartSLOduration=1.6655479930000001 podStartE2EDuration="1.665547993s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:30:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:30:35.664397714 +0000 UTC m=+906.724225379" watchObservedRunningTime="2026-09-10 19:30:35.665547993 +0000 UTC m=+906.725375663"
Sep 10 19:30:36.115140 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:36.115106    2575 patch_prober.go:28] interesting pod/e2e-explicit-faed1b-openshift-default-644ddf89b4-jsrj9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body=
Sep 10 19:30:36.115313 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:36.115156    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-faed1b-openshift-default-644ddf89b4-jsrj9" podUID="82f513d0-95e0-42e4-abc4-22c31668664b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused"
Sep 10 19:30:36.887182 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:36.887149    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:30:37.119464 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:37.119430    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-explicit-faed1b-openshift-default-644ddf89b4-jsrj9"
Sep 10 19:30:37.119867 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:37.119840    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-explicit-faed1b-openshift-default-644ddf89b4-jsrj9"
Sep 10 19:30:37.120519 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:37.120500    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-explicit-faed1b-openshift-default-644ddf89b4-jsrj9"
Sep 10 19:30:37.660480 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:37.660445    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:30:37.661210 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:37.661177    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-588a50a1-29817810-n86z5"]
Sep 10 19:30:37.665726 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:37.665696    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-588a50a1-29817810-n86z5"]
Sep 10 19:30:37.759746 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:37.759705    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:30:37.767120 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:37.767093    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qr7rl"]
Sep 10 19:30:37.868058 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:37.868017    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:30:37.969775 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:37.969740    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:30:38.066360 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:38.066008    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:30:38.703398 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:38.703360    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:30:38.804566 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:38.804528    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:30:38.906472 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:38.906439    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:30:39.386006 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:39.385974    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a2c9f912-84c4-4bbf-86d5-7d3c8029f412" path="/var/lib/kubelet/pods/a2c9f912-84c4-4bbf-86d5-7d3c8029f412/volumes"
Sep 10 19:30:40.365179 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:40.365142    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:30:44.436746 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:44.436715    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-4247c7/test-model-4247c7-kserve-746dcdc87-vnj4w"
Sep 10 19:30:44.437101 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:44.436759    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-4247c7/test-model-4247c7-kserve-746dcdc87-vnj4w"
Sep 10 19:30:44.451670 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:44.451647    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-4247c7/test-model-4247c7-kserve-746dcdc87-vnj4w"
Sep 10 19:30:44.705719 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:44.705692    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-4247c7/test-model-4247c7-kserve-746dcdc87-vnj4w"
Sep 10 19:30:46.341784 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:46.341742    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:30:46.477987 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:46.477950    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:30:46.574899 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:46.574868    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:30:46.675138 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:46.675063    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:30:47.030856 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:47.030822    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:30:53.360472 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:53.360437    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qr7rl"]
Sep 10 19:30:59.037274 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:30:59.037230    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:31:04.880154 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:31:04.880117    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:31:05.271053 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:31:05.271020    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:31:11.448964 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:31:11.448928    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qr7rl"]
Sep 10 19:31:12.931273 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:31:12.931240    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:31:15.545592 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:31:15.545560    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:31:15.581412 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:31:15.581374    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-faed1b-6756ffcc7d-lvs7d"]
Sep 10 19:31:15.582254 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:31:15.582205    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-explicit-faed1b-6756ffcc7d-lvs7d" podUID="16d359d7-2baa-4fc7-aee2-70d88578e5f4" containerName="payload-pre-processing" containerID="cri-o://3e7466470ff28522d1eb9dfcb2a79292b98aa89eb34e178ad0fca4566ffc3852" gracePeriod=30
Sep 10 19:31:15.646070 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:31:15.646029    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:31:15.747384 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:31:15.747318    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:31:15.834305 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:31:15.834279    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-faed1b-6756ffcc7d-lvs7d"
Sep 10 19:31:15.850395 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:31:15.850367    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:31:15.859677 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:31:15.859648    2575 generic.go:356] "Generic (PLEG): container finished" podID="16d359d7-2baa-4fc7-aee2-70d88578e5f4" containerID="3e7466470ff28522d1eb9dfcb2a79292b98aa89eb34e178ad0fca4566ffc3852" exitCode=0
Sep 10 19:31:15.859863 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:31:15.859717    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-faed1b-6756ffcc7d-lvs7d"
Sep 10 19:31:15.859863 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:31:15.859774    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-faed1b-6756ffcc7d-lvs7d" event={"ID":"16d359d7-2baa-4fc7-aee2-70d88578e5f4","Type":"ContainerDied","Data":"3e7466470ff28522d1eb9dfcb2a79292b98aa89eb34e178ad0fca4566ffc3852"}
Sep 10 19:31:15.859863 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:31:15.859822    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-faed1b-6756ffcc7d-lvs7d" event={"ID":"16d359d7-2baa-4fc7-aee2-70d88578e5f4","Type":"ContainerDied","Data":"ba55c59ae8a19efe956e17ef22fa39fe6c3cd18aedda13d7584d3d52d714b1ee"}
Sep 10 19:31:15.859863 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:31:15.859861    2575 scope.go:117] "RemoveContainer" containerID="3e7466470ff28522d1eb9dfcb2a79292b98aa89eb34e178ad0fca4566ffc3852"
Sep 10 19:31:15.869831 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:31:15.869803    2575 scope.go:117] "RemoveContainer" containerID="3e7466470ff28522d1eb9dfcb2a79292b98aa89eb34e178ad0fca4566ffc3852"
Sep 10 19:31:15.870116 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:31:15.870098    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3e7466470ff28522d1eb9dfcb2a79292b98aa89eb34e178ad0fca4566ffc3852\": container with ID starting with 3e7466470ff28522d1eb9dfcb2a79292b98aa89eb34e178ad0fca4566ffc3852 not found: ID does not exist" containerID="3e7466470ff28522d1eb9dfcb2a79292b98aa89eb34e178ad0fca4566ffc3852"
Sep 10 19:31:15.870180 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:31:15.870127    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3e7466470ff28522d1eb9dfcb2a79292b98aa89eb34e178ad0fca4566ffc3852"} err="failed to get container status \"3e7466470ff28522d1eb9dfcb2a79292b98aa89eb34e178ad0fca4566ffc3852\": rpc error: code = NotFound desc = could not find container \"3e7466470ff28522d1eb9dfcb2a79292b98aa89eb34e178ad0fca4566ffc3852\": container with ID starting with 3e7466470ff28522d1eb9dfcb2a79292b98aa89eb34e178ad0fca4566ffc3852 not found: ID does not exist"
Sep 10 19:31:15.925552 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:31:15.925519    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/16d359d7-2baa-4fc7-aee2-70d88578e5f4-plugins-config-volume\") pod \"16d359d7-2baa-4fc7-aee2-70d88578e5f4\" (UID: \"16d359d7-2baa-4fc7-aee2-70d88578e5f4\") "
Sep 10 19:31:15.925735 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:31:15.925583    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ctbwk\" (UniqueName: \"kubernetes.io/projected/16d359d7-2baa-4fc7-aee2-70d88578e5f4-kube-api-access-ctbwk\") pod \"16d359d7-2baa-4fc7-aee2-70d88578e5f4\" (UID: \"16d359d7-2baa-4fc7-aee2-70d88578e5f4\") "
Sep 10 19:31:15.925946 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:31:15.925857    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/16d359d7-2baa-4fc7-aee2-70d88578e5f4-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "16d359d7-2baa-4fc7-aee2-70d88578e5f4" (UID: "16d359d7-2baa-4fc7-aee2-70d88578e5f4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:31:15.927617 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:31:15.927595    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/16d359d7-2baa-4fc7-aee2-70d88578e5f4-kube-api-access-ctbwk" (OuterVolumeSpecName: "kube-api-access-ctbwk") pod "16d359d7-2baa-4fc7-aee2-70d88578e5f4" (UID: "16d359d7-2baa-4fc7-aee2-70d88578e5f4"). InnerVolumeSpecName "kube-api-access-ctbwk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:31:16.027230 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:31:16.027204    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ctbwk\" (UniqueName: \"kubernetes.io/projected/16d359d7-2baa-4fc7-aee2-70d88578e5f4-kube-api-access-ctbwk\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:31:16.027407 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:31:16.027228    2575 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/16d359d7-2baa-4fc7-aee2-70d88578e5f4-plugins-config-volume\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:31:16.118505 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:31:16.118417    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:31:16.181417 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:31:16.181379    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-faed1b-6756ffcc7d-lvs7d"]
Sep 10 19:31:16.187621 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:31:16.187598    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-faed1b-6756ffcc7d-lvs7d"]
Sep 10 19:31:17.384395 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:31:17.384368    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="16d359d7-2baa-4fc7-aee2-70d88578e5f4" path="/var/lib/kubelet/pods/16d359d7-2baa-4fc7-aee2-70d88578e5f4/volumes"
Sep 10 19:31:19.254961 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:31:19.254925    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:31:19.358460 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:31:19.358424    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:31:19.460847 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:31:19.460817    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:31:22.357870 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:31:22.357831    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:31:22.432312 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:31:22.432276    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:31:22.583087 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:31:22.583055    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-faed1b-openshift-default-644ddf89b4-jsrj9"]
Sep 10 19:31:22.583309 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:31:22.583282    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-explicit-faed1b-openshift-default-644ddf89b4-jsrj9" podUID="82f513d0-95e0-42e4-abc4-22c31668664b" containerName="istio-proxy" containerID="cri-o://1a96670fda8269b4a1f1a983096acf24805318bd2dfa51d9aab4635d33e953b8" gracePeriod=30
Sep 10 19:31:22.603967 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:31:22.603935    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:31:23.034713 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:31:23.034679    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:31:23.353818 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:31:23.353717    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:31:23.828439 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:31:23.828407    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:31:25.066967 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:31:25.066928    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:31:25.162483 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:31:25.162456    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:31:25.266557 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:31:25.266528    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:31:26.555385 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:31:26.555352    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qr7rl"]
Sep 10 19:31:27.844220 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:31:27.844166    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-faed1b-openshift-default-644ddf89b4-jsrj9"
Sep 10 19:31:27.924929 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:31:27.924894    2575 generic.go:356] "Generic (PLEG): container finished" podID="82f513d0-95e0-42e4-abc4-22c31668664b" containerID="1a96670fda8269b4a1f1a983096acf24805318bd2dfa51d9aab4635d33e953b8" exitCode=0
Sep 10 19:31:27.925089 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:31:27.924937    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-faed1b-openshift-default-644ddf89b4-jsrj9"
Sep 10 19:31:27.925089 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:31:27.924967    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-faed1b-openshift-default-644ddf89b4-jsrj9" event={"ID":"82f513d0-95e0-42e4-abc4-22c31668664b","Type":"ContainerDied","Data":"1a96670fda8269b4a1f1a983096acf24805318bd2dfa51d9aab4635d33e953b8"}
Sep 10 19:31:27.925089 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:31:27.925014    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-faed1b-openshift-default-644ddf89b4-jsrj9" event={"ID":"82f513d0-95e0-42e4-abc4-22c31668664b","Type":"ContainerDied","Data":"ba16646f73573c5b7b1471bac37a989ee9e1d43b7b4cee1ccc9c6f0ed08fbde6"}
Sep 10 19:31:27.925089 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:31:27.925036    2575 scope.go:117] "RemoveContainer" containerID="1a96670fda8269b4a1f1a983096acf24805318bd2dfa51d9aab4635d33e953b8"
Sep 10 19:31:27.931210 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:31:27.931192    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/82f513d0-95e0-42e4-abc4-22c31668664b-istio-podinfo\") pod \"82f513d0-95e0-42e4-abc4-22c31668664b\" (UID: \"82f513d0-95e0-42e4-abc4-22c31668664b\") "
Sep 10 19:31:27.931333 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:31:27.931253    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/82f513d0-95e0-42e4-abc4-22c31668664b-istio-envoy\") pod \"82f513d0-95e0-42e4-abc4-22c31668664b\" (UID: \"82f513d0-95e0-42e4-abc4-22c31668664b\") "
Sep 10 19:31:27.931333 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:31:27.931282    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/82f513d0-95e0-42e4-abc4-22c31668664b-istio-token\") pod \"82f513d0-95e0-42e4-abc4-22c31668664b\" (UID: \"82f513d0-95e0-42e4-abc4-22c31668664b\") "
Sep 10 19:31:27.931333 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:31:27.931307    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/82f513d0-95e0-42e4-abc4-22c31668664b-credential-socket\") pod \"82f513d0-95e0-42e4-abc4-22c31668664b\" (UID: \"82f513d0-95e0-42e4-abc4-22c31668664b\") "
Sep 10 19:31:27.931333 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:31:27.931328    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/82f513d0-95e0-42e4-abc4-22c31668664b-istiod-ca-cert\") pod \"82f513d0-95e0-42e4-abc4-22c31668664b\" (UID: \"82f513d0-95e0-42e4-abc4-22c31668664b\") "
Sep 10 19:31:27.931592 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:31:27.931374    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/82f513d0-95e0-42e4-abc4-22c31668664b-istio-data\") pod \"82f513d0-95e0-42e4-abc4-22c31668664b\" (UID: \"82f513d0-95e0-42e4-abc4-22c31668664b\") "
Sep 10 19:31:27.931592 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:31:27.931407    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/82f513d0-95e0-42e4-abc4-22c31668664b-workload-certs\") pod \"82f513d0-95e0-42e4-abc4-22c31668664b\" (UID: \"82f513d0-95e0-42e4-abc4-22c31668664b\") "
Sep 10 19:31:27.931592 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:31:27.931466    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qdj7b\" (UniqueName: \"kubernetes.io/projected/82f513d0-95e0-42e4-abc4-22c31668664b-kube-api-access-qdj7b\") pod \"82f513d0-95e0-42e4-abc4-22c31668664b\" (UID: \"82f513d0-95e0-42e4-abc4-22c31668664b\") "
Sep 10 19:31:27.931592 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:31:27.931491    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/82f513d0-95e0-42e4-abc4-22c31668664b-workload-socket\") pod \"82f513d0-95e0-42e4-abc4-22c31668664b\" (UID: \"82f513d0-95e0-42e4-abc4-22c31668664b\") "
Sep 10 19:31:27.931832 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:31:27.931803    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/82f513d0-95e0-42e4-abc4-22c31668664b-istio-data" (OuterVolumeSpecName: "istio-data") pod "82f513d0-95e0-42e4-abc4-22c31668664b" (UID: "82f513d0-95e0-42e4-abc4-22c31668664b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:31:27.931918 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:31:27.931893    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/82f513d0-95e0-42e4-abc4-22c31668664b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "82f513d0-95e0-42e4-abc4-22c31668664b" (UID: "82f513d0-95e0-42e4-abc4-22c31668664b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:31:27.931974 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:31:27.931942    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/82f513d0-95e0-42e4-abc4-22c31668664b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "82f513d0-95e0-42e4-abc4-22c31668664b" (UID: "82f513d0-95e0-42e4-abc4-22c31668664b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:31:27.931974 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:31:27.931927    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/82f513d0-95e0-42e4-abc4-22c31668664b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "82f513d0-95e0-42e4-abc4-22c31668664b" (UID: "82f513d0-95e0-42e4-abc4-22c31668664b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:31:27.932093 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:31:27.932004    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/82f513d0-95e0-42e4-abc4-22c31668664b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "82f513d0-95e0-42e4-abc4-22c31668664b" (UID: "82f513d0-95e0-42e4-abc4-22c31668664b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:31:27.933468 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:31:27.933443    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/82f513d0-95e0-42e4-abc4-22c31668664b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "82f513d0-95e0-42e4-abc4-22c31668664b" (UID: "82f513d0-95e0-42e4-abc4-22c31668664b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 19:31:27.933904 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:31:27.933874    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/82f513d0-95e0-42e4-abc4-22c31668664b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "82f513d0-95e0-42e4-abc4-22c31668664b" (UID: "82f513d0-95e0-42e4-abc4-22c31668664b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:31:27.934004 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:31:27.933946    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/82f513d0-95e0-42e4-abc4-22c31668664b-istio-token" (OuterVolumeSpecName: "istio-token") pod "82f513d0-95e0-42e4-abc4-22c31668664b" (UID: "82f513d0-95e0-42e4-abc4-22c31668664b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:31:27.934004 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:31:27.933963    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/82f513d0-95e0-42e4-abc4-22c31668664b-kube-api-access-qdj7b" (OuterVolumeSpecName: "kube-api-access-qdj7b") pod "82f513d0-95e0-42e4-abc4-22c31668664b" (UID: "82f513d0-95e0-42e4-abc4-22c31668664b"). InnerVolumeSpecName "kube-api-access-qdj7b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:31:27.934474 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:31:27.934452    2575 scope.go:117] "RemoveContainer" containerID="fc25bc9562678c5b737e1fbbc3eb09107638eebd893e79cb0f2099e29a118f6e"
Sep 10 19:31:27.952274 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:31:27.952256    2575 scope.go:117] "RemoveContainer" containerID="1a96670fda8269b4a1f1a983096acf24805318bd2dfa51d9aab4635d33e953b8"
Sep 10 19:31:27.952541 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:31:27.952521    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1a96670fda8269b4a1f1a983096acf24805318bd2dfa51d9aab4635d33e953b8\": container with ID starting with 1a96670fda8269b4a1f1a983096acf24805318bd2dfa51d9aab4635d33e953b8 not found: ID does not exist" containerID="1a96670fda8269b4a1f1a983096acf24805318bd2dfa51d9aab4635d33e953b8"
Sep 10 19:31:27.952590 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:31:27.952555    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1a96670fda8269b4a1f1a983096acf24805318bd2dfa51d9aab4635d33e953b8"} err="failed to get container status \"1a96670fda8269b4a1f1a983096acf24805318bd2dfa51d9aab4635d33e953b8\": rpc error: code = NotFound desc = could not find container \"1a96670fda8269b4a1f1a983096acf24805318bd2dfa51d9aab4635d33e953b8\": container with ID starting with 1a96670fda8269b4a1f1a983096acf24805318bd2dfa51d9aab4635d33e953b8 not found: ID does not exist"
Sep 10 19:31:27.952590 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:31:27.952578    2575 scope.go:117] "RemoveContainer" containerID="fc25bc9562678c5b737e1fbbc3eb09107638eebd893e79cb0f2099e29a118f6e"
Sep 10 19:31:27.952795 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:31:27.952780    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fc25bc9562678c5b737e1fbbc3eb09107638eebd893e79cb0f2099e29a118f6e\": container with ID starting with fc25bc9562678c5b737e1fbbc3eb09107638eebd893e79cb0f2099e29a118f6e not found: ID does not exist" containerID="fc25bc9562678c5b737e1fbbc3eb09107638eebd893e79cb0f2099e29a118f6e"
Sep 10 19:31:27.952833 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:31:27.952800    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc25bc9562678c5b737e1fbbc3eb09107638eebd893e79cb0f2099e29a118f6e"} err="failed to get container status \"fc25bc9562678c5b737e1fbbc3eb09107638eebd893e79cb0f2099e29a118f6e\": rpc error: code = NotFound desc = could not find container \"fc25bc9562678c5b737e1fbbc3eb09107638eebd893e79cb0f2099e29a118f6e\": container with ID starting with fc25bc9562678c5b737e1fbbc3eb09107638eebd893e79cb0f2099e29a118f6e not found: ID does not exist"
Sep 10 19:31:28.033176 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:31:28.033145    2575 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/82f513d0-95e0-42e4-abc4-22c31668664b-workload-certs\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:31:28.033176 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:31:28.033170    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qdj7b\" (UniqueName: \"kubernetes.io/projected/82f513d0-95e0-42e4-abc4-22c31668664b-kube-api-access-qdj7b\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:31:28.033176 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:31:28.033179    2575 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/82f513d0-95e0-42e4-abc4-22c31668664b-workload-socket\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:31:28.033392 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:31:28.033187    2575 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/82f513d0-95e0-42e4-abc4-22c31668664b-istio-podinfo\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:31:28.033392 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:31:28.033196    2575 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/82f513d0-95e0-42e4-abc4-22c31668664b-istio-envoy\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:31:28.033392 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:31:28.033203    2575 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/82f513d0-95e0-42e4-abc4-22c31668664b-istio-token\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:31:28.033392 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:31:28.033211    2575 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/82f513d0-95e0-42e4-abc4-22c31668664b-credential-socket\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:31:28.033392 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:31:28.033219    2575 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/82f513d0-95e0-42e4-abc4-22c31668664b-istiod-ca-cert\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:31:28.033392 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:31:28.033228    2575 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/82f513d0-95e0-42e4-abc4-22c31668664b-istio-data\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:31:28.247302 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:31:28.247273    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-faed1b-openshift-default-644ddf89b4-jsrj9"]
Sep 10 19:31:28.251268 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:31:28.251241    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-explicit-faed1b-openshift-default-644ddf89b4-jsrj9"]
Sep 10 19:31:29.384964 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:31:29.384932    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="82f513d0-95e0-42e4-abc4-22c31668664b" path="/var/lib/kubelet/pods/82f513d0-95e0-42e4-abc4-22c31668664b/volumes"
Sep 10 19:31:32.322468 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:31:32.322404    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:31:35.255309 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:31:35.255265    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:31:41.441166 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:31:41.441126    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:31:41.631972 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:31:41.631939    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:31:41.734657 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:31:41.734623    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:31:42.353189 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:31:42.353158    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qr7rl"]
Sep 10 19:31:52.150713 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:31:52.150627    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:31:52.909779 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:31:52.909741    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:31:59.306688 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:31:59.306652    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:31:59.418172 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:31:59.418127    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:31:59.508209 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:31:59.508179    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:31:59.607979 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:31:59.607902    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:32:00.216020 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:00.215974    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:32:00.312799 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:00.312764    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:32:00.413334 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:00.413306    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:32:01.838674 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:01.838629    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:32:03.263396 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:03.263365    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-8300b9/test-model-8300b9-kserve-9574dfb49-h6rrs"]
Sep 10 19:32:03.296721 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:03.296691    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:32:04.195066 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:04.195011    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:32:04.342223 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:04.342193    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:32:04.500619 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:04.500586    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:32:04.616047 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:04.616019    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:32:05.049504 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:05.049466    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qr7rl"]
Sep 10 19:32:05.167603 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:05.167573    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-4247c7/test-model-4247c7-kserve-746dcdc87-vnj4w"]
Sep 10 19:32:05.227978 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:05.227945    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:32:05.549883 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:05.549840    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:32:08.168165 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:08.168119    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:32:08.269093 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:08.269069    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:32:08.806651 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:08.806598    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:32:09.689762 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:09.689727    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:32:14.155560 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:14.155527    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qr7rl"]
Sep 10 19:32:15.002222 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:15.002185    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:32:15.072175 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:15.072143    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:32:15.234292 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:15.234265    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:32:15.562851 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:15.562814    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-9d2e2715-openshift-default-5dbb96d66d-t7967"]
Sep 10 19:32:15.563260 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:15.563243    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="82f513d0-95e0-42e4-abc4-22c31668664b" containerName="istio-proxy"
Sep 10 19:32:15.563374 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:15.563263    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="82f513d0-95e0-42e4-abc4-22c31668664b" containerName="istio-proxy"
Sep 10 19:32:15.563374 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:15.563281    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="16d359d7-2baa-4fc7-aee2-70d88578e5f4" containerName="payload-pre-processing"
Sep 10 19:32:15.563374 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:15.563289    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="16d359d7-2baa-4fc7-aee2-70d88578e5f4" containerName="payload-pre-processing"
Sep 10 19:32:15.563540 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:15.563429    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="82f513d0-95e0-42e4-abc4-22c31668664b" containerName="istio-proxy"
Sep 10 19:32:15.563540 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:15.563447    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="82f513d0-95e0-42e4-abc4-22c31668664b" containerName="istio-proxy"
Sep 10 19:32:15.563540 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:15.563458    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="16d359d7-2baa-4fc7-aee2-70d88578e5f4" containerName="payload-pre-processing"
Sep 10 19:32:15.566848 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:15.566828    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-9d2e2715-openshift-default-5dbb96d66d-t7967"
Sep 10 19:32:15.569087 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:15.569068    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-9d2e2715-openshift-default-dockercfg-jstdd\""
Sep 10 19:32:15.595698 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:15.595673    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-9d2e2715-openshift-default-5dbb96d66d-t7967"]
Sep 10 19:32:15.651774 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:15.651733    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:32:15.717636 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:15.717606    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cad351b9-3b12-4d82-8f0f-d24ee08fcfbb-istio-token\") pod \"e2e-mt-9d2e2715-openshift-default-5dbb96d66d-t7967\" (UID: \"cad351b9-3b12-4d82-8f0f-d24ee08fcfbb\") " pod="openshift-ingress/e2e-mt-9d2e2715-openshift-default-5dbb96d66d-t7967"
Sep 10 19:32:15.717782 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:15.717641    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cad351b9-3b12-4d82-8f0f-d24ee08fcfbb-workload-certs\") pod \"e2e-mt-9d2e2715-openshift-default-5dbb96d66d-t7967\" (UID: \"cad351b9-3b12-4d82-8f0f-d24ee08fcfbb\") " pod="openshift-ingress/e2e-mt-9d2e2715-openshift-default-5dbb96d66d-t7967"
Sep 10 19:32:15.717782 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:15.717667    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cad351b9-3b12-4d82-8f0f-d24ee08fcfbb-istio-envoy\") pod \"e2e-mt-9d2e2715-openshift-default-5dbb96d66d-t7967\" (UID: \"cad351b9-3b12-4d82-8f0f-d24ee08fcfbb\") " pod="openshift-ingress/e2e-mt-9d2e2715-openshift-default-5dbb96d66d-t7967"
Sep 10 19:32:15.717782 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:15.717735    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cad351b9-3b12-4d82-8f0f-d24ee08fcfbb-workload-socket\") pod \"e2e-mt-9d2e2715-openshift-default-5dbb96d66d-t7967\" (UID: \"cad351b9-3b12-4d82-8f0f-d24ee08fcfbb\") " pod="openshift-ingress/e2e-mt-9d2e2715-openshift-default-5dbb96d66d-t7967"
Sep 10 19:32:15.717782 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:15.717764    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cad351b9-3b12-4d82-8f0f-d24ee08fcfbb-istio-podinfo\") pod \"e2e-mt-9d2e2715-openshift-default-5dbb96d66d-t7967\" (UID: \"cad351b9-3b12-4d82-8f0f-d24ee08fcfbb\") " pod="openshift-ingress/e2e-mt-9d2e2715-openshift-default-5dbb96d66d-t7967"
Sep 10 19:32:15.717782 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:15.717783    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cad351b9-3b12-4d82-8f0f-d24ee08fcfbb-istiod-ca-cert\") pod \"e2e-mt-9d2e2715-openshift-default-5dbb96d66d-t7967\" (UID: \"cad351b9-3b12-4d82-8f0f-d24ee08fcfbb\") " pod="openshift-ingress/e2e-mt-9d2e2715-openshift-default-5dbb96d66d-t7967"
Sep 10 19:32:15.718061 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:15.717808    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cad351b9-3b12-4d82-8f0f-d24ee08fcfbb-istio-data\") pod \"e2e-mt-9d2e2715-openshift-default-5dbb96d66d-t7967\" (UID: \"cad351b9-3b12-4d82-8f0f-d24ee08fcfbb\") " pod="openshift-ingress/e2e-mt-9d2e2715-openshift-default-5dbb96d66d-t7967"
Sep 10 19:32:15.718061 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:15.717872    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cad351b9-3b12-4d82-8f0f-d24ee08fcfbb-credential-socket\") pod \"e2e-mt-9d2e2715-openshift-default-5dbb96d66d-t7967\" (UID: \"cad351b9-3b12-4d82-8f0f-d24ee08fcfbb\") " pod="openshift-ingress/e2e-mt-9d2e2715-openshift-default-5dbb96d66d-t7967"
Sep 10 19:32:15.718061 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:15.717893    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-45m45\" (UniqueName: \"kubernetes.io/projected/cad351b9-3b12-4d82-8f0f-d24ee08fcfbb-kube-api-access-45m45\") pod \"e2e-mt-9d2e2715-openshift-default-5dbb96d66d-t7967\" (UID: \"cad351b9-3b12-4d82-8f0f-d24ee08fcfbb\") " pod="openshift-ingress/e2e-mt-9d2e2715-openshift-default-5dbb96d66d-t7967"
Sep 10 19:32:15.819003 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:15.818929    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cad351b9-3b12-4d82-8f0f-d24ee08fcfbb-istio-envoy\") pod \"e2e-mt-9d2e2715-openshift-default-5dbb96d66d-t7967\" (UID: \"cad351b9-3b12-4d82-8f0f-d24ee08fcfbb\") " pod="openshift-ingress/e2e-mt-9d2e2715-openshift-default-5dbb96d66d-t7967"
Sep 10 19:32:15.819003 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:15.818969    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cad351b9-3b12-4d82-8f0f-d24ee08fcfbb-workload-socket\") pod \"e2e-mt-9d2e2715-openshift-default-5dbb96d66d-t7967\" (UID: \"cad351b9-3b12-4d82-8f0f-d24ee08fcfbb\") " pod="openshift-ingress/e2e-mt-9d2e2715-openshift-default-5dbb96d66d-t7967"
Sep 10 19:32:15.819003 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:15.818988    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cad351b9-3b12-4d82-8f0f-d24ee08fcfbb-istio-podinfo\") pod \"e2e-mt-9d2e2715-openshift-default-5dbb96d66d-t7967\" (UID: \"cad351b9-3b12-4d82-8f0f-d24ee08fcfbb\") " pod="openshift-ingress/e2e-mt-9d2e2715-openshift-default-5dbb96d66d-t7967"
Sep 10 19:32:15.819247 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:15.819013    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cad351b9-3b12-4d82-8f0f-d24ee08fcfbb-istiod-ca-cert\") pod \"e2e-mt-9d2e2715-openshift-default-5dbb96d66d-t7967\" (UID: \"cad351b9-3b12-4d82-8f0f-d24ee08fcfbb\") " pod="openshift-ingress/e2e-mt-9d2e2715-openshift-default-5dbb96d66d-t7967"
Sep 10 19:32:15.819247 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:15.819047    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cad351b9-3b12-4d82-8f0f-d24ee08fcfbb-istio-data\") pod \"e2e-mt-9d2e2715-openshift-default-5dbb96d66d-t7967\" (UID: \"cad351b9-3b12-4d82-8f0f-d24ee08fcfbb\") " pod="openshift-ingress/e2e-mt-9d2e2715-openshift-default-5dbb96d66d-t7967"
Sep 10 19:32:15.819247 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:15.819079    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cad351b9-3b12-4d82-8f0f-d24ee08fcfbb-credential-socket\") pod \"e2e-mt-9d2e2715-openshift-default-5dbb96d66d-t7967\" (UID: \"cad351b9-3b12-4d82-8f0f-d24ee08fcfbb\") " pod="openshift-ingress/e2e-mt-9d2e2715-openshift-default-5dbb96d66d-t7967"
Sep 10 19:32:15.819247 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:15.819108    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-45m45\" (UniqueName: \"kubernetes.io/projected/cad351b9-3b12-4d82-8f0f-d24ee08fcfbb-kube-api-access-45m45\") pod \"e2e-mt-9d2e2715-openshift-default-5dbb96d66d-t7967\" (UID: \"cad351b9-3b12-4d82-8f0f-d24ee08fcfbb\") " pod="openshift-ingress/e2e-mt-9d2e2715-openshift-default-5dbb96d66d-t7967"
Sep 10 19:32:15.819247 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:15.819179    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cad351b9-3b12-4d82-8f0f-d24ee08fcfbb-istio-token\") pod \"e2e-mt-9d2e2715-openshift-default-5dbb96d66d-t7967\" (UID: \"cad351b9-3b12-4d82-8f0f-d24ee08fcfbb\") " pod="openshift-ingress/e2e-mt-9d2e2715-openshift-default-5dbb96d66d-t7967"
Sep 10 19:32:15.819247 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:15.819213    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cad351b9-3b12-4d82-8f0f-d24ee08fcfbb-workload-certs\") pod \"e2e-mt-9d2e2715-openshift-default-5dbb96d66d-t7967\" (UID: \"cad351b9-3b12-4d82-8f0f-d24ee08fcfbb\") " pod="openshift-ingress/e2e-mt-9d2e2715-openshift-default-5dbb96d66d-t7967"
Sep 10 19:32:15.819570 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:15.819404    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cad351b9-3b12-4d82-8f0f-d24ee08fcfbb-workload-socket\") pod \"e2e-mt-9d2e2715-openshift-default-5dbb96d66d-t7967\" (UID: \"cad351b9-3b12-4d82-8f0f-d24ee08fcfbb\") " pod="openshift-ingress/e2e-mt-9d2e2715-openshift-default-5dbb96d66d-t7967"
Sep 10 19:32:15.819798 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:15.819774    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cad351b9-3b12-4d82-8f0f-d24ee08fcfbb-istio-data\") pod \"e2e-mt-9d2e2715-openshift-default-5dbb96d66d-t7967\" (UID: \"cad351b9-3b12-4d82-8f0f-d24ee08fcfbb\") " pod="openshift-ingress/e2e-mt-9d2e2715-openshift-default-5dbb96d66d-t7967"
Sep 10 19:32:15.819931 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:15.819909    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cad351b9-3b12-4d82-8f0f-d24ee08fcfbb-credential-socket\") pod \"e2e-mt-9d2e2715-openshift-default-5dbb96d66d-t7967\" (UID: \"cad351b9-3b12-4d82-8f0f-d24ee08fcfbb\") " pod="openshift-ingress/e2e-mt-9d2e2715-openshift-default-5dbb96d66d-t7967"
Sep 10 19:32:15.820003 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:15.819916    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cad351b9-3b12-4d82-8f0f-d24ee08fcfbb-workload-certs\") pod \"e2e-mt-9d2e2715-openshift-default-5dbb96d66d-t7967\" (UID: \"cad351b9-3b12-4d82-8f0f-d24ee08fcfbb\") " pod="openshift-ingress/e2e-mt-9d2e2715-openshift-default-5dbb96d66d-t7967"
Sep 10 19:32:15.820003 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:15.819954    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cad351b9-3b12-4d82-8f0f-d24ee08fcfbb-istiod-ca-cert\") pod \"e2e-mt-9d2e2715-openshift-default-5dbb96d66d-t7967\" (UID: \"cad351b9-3b12-4d82-8f0f-d24ee08fcfbb\") " pod="openshift-ingress/e2e-mt-9d2e2715-openshift-default-5dbb96d66d-t7967"
Sep 10 19:32:15.821732 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:15.821713    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cad351b9-3b12-4d82-8f0f-d24ee08fcfbb-istio-envoy\") pod \"e2e-mt-9d2e2715-openshift-default-5dbb96d66d-t7967\" (UID: \"cad351b9-3b12-4d82-8f0f-d24ee08fcfbb\") " pod="openshift-ingress/e2e-mt-9d2e2715-openshift-default-5dbb96d66d-t7967"
Sep 10 19:32:15.822058 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:15.822042    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cad351b9-3b12-4d82-8f0f-d24ee08fcfbb-istio-podinfo\") pod \"e2e-mt-9d2e2715-openshift-default-5dbb96d66d-t7967\" (UID: \"cad351b9-3b12-4d82-8f0f-d24ee08fcfbb\") " pod="openshift-ingress/e2e-mt-9d2e2715-openshift-default-5dbb96d66d-t7967"
Sep 10 19:32:15.830469 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:15.830443    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cad351b9-3b12-4d82-8f0f-d24ee08fcfbb-istio-token\") pod \"e2e-mt-9d2e2715-openshift-default-5dbb96d66d-t7967\" (UID: \"cad351b9-3b12-4d82-8f0f-d24ee08fcfbb\") " pod="openshift-ingress/e2e-mt-9d2e2715-openshift-default-5dbb96d66d-t7967"
Sep 10 19:32:15.831011 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:15.830987    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-45m45\" (UniqueName: \"kubernetes.io/projected/cad351b9-3b12-4d82-8f0f-d24ee08fcfbb-kube-api-access-45m45\") pod \"e2e-mt-9d2e2715-openshift-default-5dbb96d66d-t7967\" (UID: \"cad351b9-3b12-4d82-8f0f-d24ee08fcfbb\") " pod="openshift-ingress/e2e-mt-9d2e2715-openshift-default-5dbb96d66d-t7967"
Sep 10 19:32:15.831510 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:15.831485    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:32:15.878625 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:15.878602    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-9d2e2715-openshift-default-5dbb96d66d-t7967"
Sep 10 19:32:16.104757 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:16.104671    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:32:16.110674 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:16.110647    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-8300b9-29817810-8rmhd"]
Sep 10 19:32:16.112715 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:16.112694    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-8300b9-29817810-8rmhd"]
Sep 10 19:32:16.138991 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:16.138963    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-8300b9-855c8cfdc7-wb9qf"]
Sep 10 19:32:16.139192 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:16.139158    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-8300b9-855c8cfdc7-wb9qf" podUID="913ee6b0-8326-493c-bb1f-db2ebda11283" containerName="payload-processing" containerID="cri-o://6bd6ae959549747c2d7e0424bc578ac68675956717ca8542957f0430f8a4f180" gracePeriod=30
Sep 10 19:32:16.156674 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:16.156604    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-9d2e2715-openshift-default-5dbb96d66d-t7967"]
Sep 10 19:32:16.159281 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:32:16.159152    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcad351b9_3b12_4d82_8f0f_d24ee08fcfbb.slice/crio-255e70e543cdc3d96290ca7a337372316ac4ba7ce1d55d5d02c32416393e7c9b WatchSource:0}: Error finding container 255e70e543cdc3d96290ca7a337372316ac4ba7ce1d55d5d02c32416393e7c9b: Status 404 returned error can't find the container with id 255e70e543cdc3d96290ca7a337372316ac4ba7ce1d55d5d02c32416393e7c9b
Sep 10 19:32:16.202859 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:16.202830    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:32:16.244613 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:16.244582    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 19:32:16.244879 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:16.244648    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 19:32:16.244879 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:16.244676    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 19:32:16.303431 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:16.303398    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:32:16.380835 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:16.380811    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-8300b9-855c8cfdc7-wb9qf"
Sep 10 19:32:16.403252 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:16.403189    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:32:16.526261 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:16.526234    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k58jw\" (UniqueName: \"kubernetes.io/projected/913ee6b0-8326-493c-bb1f-db2ebda11283-kube-api-access-k58jw\") pod \"913ee6b0-8326-493c-bb1f-db2ebda11283\" (UID: \"913ee6b0-8326-493c-bb1f-db2ebda11283\") "
Sep 10 19:32:16.526462 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:16.526294    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/913ee6b0-8326-493c-bb1f-db2ebda11283-plugins-config-volume\") pod \"913ee6b0-8326-493c-bb1f-db2ebda11283\" (UID: \"913ee6b0-8326-493c-bb1f-db2ebda11283\") "
Sep 10 19:32:16.526728 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:16.526698    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/913ee6b0-8326-493c-bb1f-db2ebda11283-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "913ee6b0-8326-493c-bb1f-db2ebda11283" (UID: "913ee6b0-8326-493c-bb1f-db2ebda11283"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:32:16.528741 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:16.528714    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/913ee6b0-8326-493c-bb1f-db2ebda11283-kube-api-access-k58jw" (OuterVolumeSpecName: "kube-api-access-k58jw") pod "913ee6b0-8326-493c-bb1f-db2ebda11283" (UID: "913ee6b0-8326-493c-bb1f-db2ebda11283"). InnerVolumeSpecName "kube-api-access-k58jw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:32:16.627332 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:16.627257    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k58jw\" (UniqueName: \"kubernetes.io/projected/913ee6b0-8326-493c-bb1f-db2ebda11283-kube-api-access-k58jw\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:32:16.627332 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:16.627282    2575 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/913ee6b0-8326-493c-bb1f-db2ebda11283-plugins-config-volume\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:32:16.660082 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:16.660049    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:32:17.174997 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:17.174954    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9d2e2715-openshift-default-5dbb96d66d-t7967" event={"ID":"cad351b9-3b12-4d82-8f0f-d24ee08fcfbb","Type":"ContainerStarted","Data":"154a8a044cdf4c450f6fbee139be216667b28b3ccf81ef9319c01ef1b494104e"}
Sep 10 19:32:17.175178 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:17.175016    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9d2e2715-openshift-default-5dbb96d66d-t7967" event={"ID":"cad351b9-3b12-4d82-8f0f-d24ee08fcfbb","Type":"ContainerStarted","Data":"255e70e543cdc3d96290ca7a337372316ac4ba7ce1d55d5d02c32416393e7c9b"}
Sep 10 19:32:17.176274 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:17.176245    2575 generic.go:356] "Generic (PLEG): container finished" podID="913ee6b0-8326-493c-bb1f-db2ebda11283" containerID="6bd6ae959549747c2d7e0424bc578ac68675956717ca8542957f0430f8a4f180" exitCode=0
Sep 10 19:32:17.176403 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:17.176285    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-8300b9-855c8cfdc7-wb9qf"
Sep 10 19:32:17.176403 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:17.176317    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-8300b9-855c8cfdc7-wb9qf" event={"ID":"913ee6b0-8326-493c-bb1f-db2ebda11283","Type":"ContainerDied","Data":"6bd6ae959549747c2d7e0424bc578ac68675956717ca8542957f0430f8a4f180"}
Sep 10 19:32:17.176403 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:17.176383    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-8300b9-855c8cfdc7-wb9qf" event={"ID":"913ee6b0-8326-493c-bb1f-db2ebda11283","Type":"ContainerDied","Data":"46f9bd223f370ecb58af62def798a6692cf4bedcdf91568111c56988ec2bedbb"}
Sep 10 19:32:17.176513 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:17.176412    2575 scope.go:117] "RemoveContainer" containerID="6bd6ae959549747c2d7e0424bc578ac68675956717ca8542957f0430f8a4f180"
Sep 10 19:32:17.184854 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:17.184838    2575 scope.go:117] "RemoveContainer" containerID="6bd6ae959549747c2d7e0424bc578ac68675956717ca8542957f0430f8a4f180"
Sep 10 19:32:17.185088 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:32:17.185069    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6bd6ae959549747c2d7e0424bc578ac68675956717ca8542957f0430f8a4f180\": container with ID starting with 6bd6ae959549747c2d7e0424bc578ac68675956717ca8542957f0430f8a4f180 not found: ID does not exist" containerID="6bd6ae959549747c2d7e0424bc578ac68675956717ca8542957f0430f8a4f180"
Sep 10 19:32:17.185151 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:17.185096    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6bd6ae959549747c2d7e0424bc578ac68675956717ca8542957f0430f8a4f180"} err="failed to get container status \"6bd6ae959549747c2d7e0424bc578ac68675956717ca8542957f0430f8a4f180\": rpc error: code = NotFound desc = could not find container \"6bd6ae959549747c2d7e0424bc578ac68675956717ca8542957f0430f8a4f180\": container with ID starting with 6bd6ae959549747c2d7e0424bc578ac68675956717ca8542957f0430f8a4f180 not found: ID does not exist"
Sep 10 19:32:17.200228 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:17.200185    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-9d2e2715-openshift-default-5dbb96d66d-t7967" podStartSLOduration=2.200173067 podStartE2EDuration="2.200173067s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:32:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:32:17.199124868 +0000 UTC m=+1008.258952535" watchObservedRunningTime="2026-09-10 19:32:17.200173067 +0000 UTC m=+1008.260000735"
Sep 10 19:32:17.212055 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:17.212034    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-8300b9-855c8cfdc7-wb9qf"]
Sep 10 19:32:17.217948 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:17.217930    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-8300b9-855c8cfdc7-wb9qf"]
Sep 10 19:32:17.385440 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:17.385411    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="913ee6b0-8326-493c-bb1f-db2ebda11283" path="/var/lib/kubelet/pods/913ee6b0-8326-493c-bb1f-db2ebda11283/volumes"
Sep 10 19:32:17.385775 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:17.385724    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e4b52ee7-8cce-4ecc-a236-196308998f71" path="/var/lib/kubelet/pods/e4b52ee7-8cce-4ecc-a236-196308998f71/volumes"
Sep 10 19:32:17.879385 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:17.879332    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-9d2e2715-openshift-default-5dbb96d66d-t7967"
Sep 10 19:32:17.884104 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:17.884082    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-9d2e2715-openshift-default-5dbb96d66d-t7967"
Sep 10 19:32:18.182567 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:18.182491    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-9d2e2715-openshift-default-5dbb96d66d-t7967"
Sep 10 19:32:18.183303 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:18.183283    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-9d2e2715-openshift-default-5dbb96d66d-t7967"
Sep 10 19:32:18.282945 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:18.282906    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-8300b9/test-model-8300b9-kserve-9574dfb49-h6rrs" podUID="65c44357-30b7-47c0-8957-0e9fb86019c3" containerName="main" containerID="cri-o://bf445cd5678e01ce98ff5cbd69d18544f82df463a5e99589637c386e0fcb23f3" gracePeriod=45
Sep 10 19:32:18.537916 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:18.537894    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-8300b9/test-model-8300b9-kserve-9574dfb49-h6rrs"
Sep 10 19:32:18.644714 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:18.644686    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/65c44357-30b7-47c0-8957-0e9fb86019c3-home\") pod \"65c44357-30b7-47c0-8957-0e9fb86019c3\" (UID: \"65c44357-30b7-47c0-8957-0e9fb86019c3\") "
Sep 10 19:32:18.644903 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:18.644739    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/65c44357-30b7-47c0-8957-0e9fb86019c3-dshm\") pod \"65c44357-30b7-47c0-8957-0e9fb86019c3\" (UID: \"65c44357-30b7-47c0-8957-0e9fb86019c3\") "
Sep 10 19:32:18.644903 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:18.644798    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-69n9t\" (UniqueName: \"kubernetes.io/projected/65c44357-30b7-47c0-8957-0e9fb86019c3-kube-api-access-69n9t\") pod \"65c44357-30b7-47c0-8957-0e9fb86019c3\" (UID: \"65c44357-30b7-47c0-8957-0e9fb86019c3\") "
Sep 10 19:32:18.644903 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:18.644822    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/65c44357-30b7-47c0-8957-0e9fb86019c3-tmp-dir\") pod \"65c44357-30b7-47c0-8957-0e9fb86019c3\" (UID: \"65c44357-30b7-47c0-8957-0e9fb86019c3\") "
Sep 10 19:32:18.644903 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:18.644873    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/65c44357-30b7-47c0-8957-0e9fb86019c3-tls-certs\") pod \"65c44357-30b7-47c0-8957-0e9fb86019c3\" (UID: \"65c44357-30b7-47c0-8957-0e9fb86019c3\") "
Sep 10 19:32:18.644903 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:18.644902    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/65c44357-30b7-47c0-8957-0e9fb86019c3-model-cache\") pod \"65c44357-30b7-47c0-8957-0e9fb86019c3\" (UID: \"65c44357-30b7-47c0-8957-0e9fb86019c3\") "
Sep 10 19:32:18.645141 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:18.644957    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/65c44357-30b7-47c0-8957-0e9fb86019c3-home" (OuterVolumeSpecName: "home") pod "65c44357-30b7-47c0-8957-0e9fb86019c3" (UID: "65c44357-30b7-47c0-8957-0e9fb86019c3"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:32:18.645141 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:18.645127    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/65c44357-30b7-47c0-8957-0e9fb86019c3-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "65c44357-30b7-47c0-8957-0e9fb86019c3" (UID: "65c44357-30b7-47c0-8957-0e9fb86019c3"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:32:18.645256 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:18.645173    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/65c44357-30b7-47c0-8957-0e9fb86019c3-model-cache" (OuterVolumeSpecName: "model-cache") pod "65c44357-30b7-47c0-8957-0e9fb86019c3" (UID: "65c44357-30b7-47c0-8957-0e9fb86019c3"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:32:18.645312 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:18.645252    2575 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/65c44357-30b7-47c0-8957-0e9fb86019c3-home\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:32:18.645312 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:18.645270    2575 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/65c44357-30b7-47c0-8957-0e9fb86019c3-tmp-dir\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:32:18.647081 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:18.647048    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/65c44357-30b7-47c0-8957-0e9fb86019c3-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "65c44357-30b7-47c0-8957-0e9fb86019c3" (UID: "65c44357-30b7-47c0-8957-0e9fb86019c3"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:32:18.647183 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:18.647085    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/65c44357-30b7-47c0-8957-0e9fb86019c3-kube-api-access-69n9t" (OuterVolumeSpecName: "kube-api-access-69n9t") pod "65c44357-30b7-47c0-8957-0e9fb86019c3" (UID: "65c44357-30b7-47c0-8957-0e9fb86019c3"). InnerVolumeSpecName "kube-api-access-69n9t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:32:18.647183 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:18.647092    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/65c44357-30b7-47c0-8957-0e9fb86019c3-dshm" (OuterVolumeSpecName: "dshm") pod "65c44357-30b7-47c0-8957-0e9fb86019c3" (UID: "65c44357-30b7-47c0-8957-0e9fb86019c3"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:32:18.746576 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:18.746508    2575 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/65c44357-30b7-47c0-8957-0e9fb86019c3-dshm\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:32:18.746576 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:18.746533    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-69n9t\" (UniqueName: \"kubernetes.io/projected/65c44357-30b7-47c0-8957-0e9fb86019c3-kube-api-access-69n9t\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:32:18.746576 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:18.746547    2575 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/65c44357-30b7-47c0-8957-0e9fb86019c3-tls-certs\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:32:18.746576 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:18.746558    2575 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/65c44357-30b7-47c0-8957-0e9fb86019c3-model-cache\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:32:19.075030 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:19.074948    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:32:19.171046 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:19.171006    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:32:19.187964 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:19.187932    2575 generic.go:356] "Generic (PLEG): container finished" podID="65c44357-30b7-47c0-8957-0e9fb86019c3" containerID="bf445cd5678e01ce98ff5cbd69d18544f82df463a5e99589637c386e0fcb23f3" exitCode=0
Sep 10 19:32:19.188083 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:19.187973    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-8300b9/test-model-8300b9-kserve-9574dfb49-h6rrs"
Sep 10 19:32:19.188083 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:19.188009    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-8300b9/test-model-8300b9-kserve-9574dfb49-h6rrs" event={"ID":"65c44357-30b7-47c0-8957-0e9fb86019c3","Type":"ContainerDied","Data":"bf445cd5678e01ce98ff5cbd69d18544f82df463a5e99589637c386e0fcb23f3"}
Sep 10 19:32:19.188083 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:19.188043    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-8300b9/test-model-8300b9-kserve-9574dfb49-h6rrs" event={"ID":"65c44357-30b7-47c0-8957-0e9fb86019c3","Type":"ContainerDied","Data":"41a17cee222d554152fb0d0a325ec2c65a32236b63bbacb5cffb730c59bf3b8f"}
Sep 10 19:32:19.188083 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:19.188062    2575 scope.go:117] "RemoveContainer" containerID="bf445cd5678e01ce98ff5cbd69d18544f82df463a5e99589637c386e0fcb23f3"
Sep 10 19:32:19.197748 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:19.197732    2575 scope.go:117] "RemoveContainer" containerID="bf445cd5678e01ce98ff5cbd69d18544f82df463a5e99589637c386e0fcb23f3"
Sep 10 19:32:19.198001 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:32:19.197984    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bf445cd5678e01ce98ff5cbd69d18544f82df463a5e99589637c386e0fcb23f3\": container with ID starting with bf445cd5678e01ce98ff5cbd69d18544f82df463a5e99589637c386e0fcb23f3 not found: ID does not exist" containerID="bf445cd5678e01ce98ff5cbd69d18544f82df463a5e99589637c386e0fcb23f3"
Sep 10 19:32:19.198054 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:19.198007    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bf445cd5678e01ce98ff5cbd69d18544f82df463a5e99589637c386e0fcb23f3"} err="failed to get container status \"bf445cd5678e01ce98ff5cbd69d18544f82df463a5e99589637c386e0fcb23f3\": rpc error: code = NotFound desc = could not find container \"bf445cd5678e01ce98ff5cbd69d18544f82df463a5e99589637c386e0fcb23f3\": container with ID starting with bf445cd5678e01ce98ff5cbd69d18544f82df463a5e99589637c386e0fcb23f3 not found: ID does not exist"
Sep 10 19:32:19.211135 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:19.211110    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-8300b9/test-model-8300b9-kserve-9574dfb49-h6rrs"]
Sep 10 19:32:19.216013 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:19.215989    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-8300b9/test-model-8300b9-kserve-9574dfb49-h6rrs"]
Sep 10 19:32:19.386794 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:19.386724    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="65c44357-30b7-47c0-8957-0e9fb86019c3" path="/var/lib/kubelet/pods/65c44357-30b7-47c0-8957-0e9fb86019c3/volumes"
Sep 10 19:32:20.186813 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:20.186771    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-4247c7/test-model-4247c7-kserve-746dcdc87-vnj4w" podUID="50e45adb-8cbc-4820-9c39-c4896e4a2ddd" containerName="main" containerID="cri-o://8c8816571e7d33a6b2903b669d295055b5d87eec4532757beef9561122d1be5c" gracePeriod=45
Sep 10 19:32:20.254466 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:20.254434    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:32:20.355954 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:20.355920    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:32:20.437946 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:20.437889    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-4247c7/test-model-4247c7-kserve-746dcdc87-vnj4w"
Sep 10 19:32:20.560812 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:20.560777    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/50e45adb-8cbc-4820-9c39-c4896e4a2ddd-dshm\") pod \"50e45adb-8cbc-4820-9c39-c4896e4a2ddd\" (UID: \"50e45adb-8cbc-4820-9c39-c4896e4a2ddd\") "
Sep 10 19:32:20.560991 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:20.560825    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/50e45adb-8cbc-4820-9c39-c4896e4a2ddd-tls-certs\") pod \"50e45adb-8cbc-4820-9c39-c4896e4a2ddd\" (UID: \"50e45adb-8cbc-4820-9c39-c4896e4a2ddd\") "
Sep 10 19:32:20.560991 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:20.560891    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/50e45adb-8cbc-4820-9c39-c4896e4a2ddd-home\") pod \"50e45adb-8cbc-4820-9c39-c4896e4a2ddd\" (UID: \"50e45adb-8cbc-4820-9c39-c4896e4a2ddd\") "
Sep 10 19:32:20.560991 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:20.560935    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/50e45adb-8cbc-4820-9c39-c4896e4a2ddd-model-cache\") pod \"50e45adb-8cbc-4820-9c39-c4896e4a2ddd\" (UID: \"50e45adb-8cbc-4820-9c39-c4896e4a2ddd\") "
Sep 10 19:32:20.560991 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:20.560959    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/50e45adb-8cbc-4820-9c39-c4896e4a2ddd-tmp-dir\") pod \"50e45adb-8cbc-4820-9c39-c4896e4a2ddd\" (UID: \"50e45adb-8cbc-4820-9c39-c4896e4a2ddd\") "
Sep 10 19:32:20.561192 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:20.560994    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hjknp\" (UniqueName: \"kubernetes.io/projected/50e45adb-8cbc-4820-9c39-c4896e4a2ddd-kube-api-access-hjknp\") pod \"50e45adb-8cbc-4820-9c39-c4896e4a2ddd\" (UID: \"50e45adb-8cbc-4820-9c39-c4896e4a2ddd\") "
Sep 10 19:32:20.561242 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:20.561203    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50e45adb-8cbc-4820-9c39-c4896e4a2ddd-model-cache" (OuterVolumeSpecName: "model-cache") pod "50e45adb-8cbc-4820-9c39-c4896e4a2ddd" (UID: "50e45adb-8cbc-4820-9c39-c4896e4a2ddd"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:32:20.561242 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:20.561219    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50e45adb-8cbc-4820-9c39-c4896e4a2ddd-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "50e45adb-8cbc-4820-9c39-c4896e4a2ddd" (UID: "50e45adb-8cbc-4820-9c39-c4896e4a2ddd"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:32:20.561242 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:20.561228    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50e45adb-8cbc-4820-9c39-c4896e4a2ddd-home" (OuterVolumeSpecName: "home") pod "50e45adb-8cbc-4820-9c39-c4896e4a2ddd" (UID: "50e45adb-8cbc-4820-9c39-c4896e4a2ddd"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:32:20.561437 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:20.561417    2575 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/50e45adb-8cbc-4820-9c39-c4896e4a2ddd-home\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:32:20.561478 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:20.561443    2575 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/50e45adb-8cbc-4820-9c39-c4896e4a2ddd-model-cache\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:32:20.561478 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:20.561460    2575 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/50e45adb-8cbc-4820-9c39-c4896e4a2ddd-tmp-dir\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:32:20.563073 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:20.563050    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/50e45adb-8cbc-4820-9c39-c4896e4a2ddd-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "50e45adb-8cbc-4820-9c39-c4896e4a2ddd" (UID: "50e45adb-8cbc-4820-9c39-c4896e4a2ddd"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:32:20.563179 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:20.563088    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50e45adb-8cbc-4820-9c39-c4896e4a2ddd-dshm" (OuterVolumeSpecName: "dshm") pod "50e45adb-8cbc-4820-9c39-c4896e4a2ddd" (UID: "50e45adb-8cbc-4820-9c39-c4896e4a2ddd"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:32:20.563179 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:20.563104    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/50e45adb-8cbc-4820-9c39-c4896e4a2ddd-kube-api-access-hjknp" (OuterVolumeSpecName: "kube-api-access-hjknp") pod "50e45adb-8cbc-4820-9c39-c4896e4a2ddd" (UID: "50e45adb-8cbc-4820-9c39-c4896e4a2ddd"). InnerVolumeSpecName "kube-api-access-hjknp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:32:20.662486 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:20.662460    2575 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/50e45adb-8cbc-4820-9c39-c4896e4a2ddd-dshm\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:32:20.662486 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:20.662482    2575 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/50e45adb-8cbc-4820-9c39-c4896e4a2ddd-tls-certs\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:32:20.662635 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:20.662493    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hjknp\" (UniqueName: \"kubernetes.io/projected/50e45adb-8cbc-4820-9c39-c4896e4a2ddd-kube-api-access-hjknp\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:32:20.863142 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:20.863089    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:32:20.963146 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:20.963098    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:32:21.023595 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:21.023559    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-9d2e2715-77fc999f9c-4866g"]
Sep 10 19:32:21.023976 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:21.023960    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="913ee6b0-8326-493c-bb1f-db2ebda11283" containerName="payload-processing"
Sep 10 19:32:21.023976 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:21.023976    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="913ee6b0-8326-493c-bb1f-db2ebda11283" containerName="payload-processing"
Sep 10 19:32:21.024120 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:21.023998    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="65c44357-30b7-47c0-8957-0e9fb86019c3" containerName="main"
Sep 10 19:32:21.024120 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:21.024004    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="65c44357-30b7-47c0-8957-0e9fb86019c3" containerName="main"
Sep 10 19:32:21.024120 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:21.024017    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="82f513d0-95e0-42e4-abc4-22c31668664b" containerName="istio-proxy"
Sep 10 19:32:21.024120 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:21.024023    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="82f513d0-95e0-42e4-abc4-22c31668664b" containerName="istio-proxy"
Sep 10 19:32:21.024120 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:21.024031    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="50e45adb-8cbc-4820-9c39-c4896e4a2ddd" containerName="main"
Sep 10 19:32:21.024120 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:21.024036    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="50e45adb-8cbc-4820-9c39-c4896e4a2ddd" containerName="main"
Sep 10 19:32:21.024120 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:21.024093    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="913ee6b0-8326-493c-bb1f-db2ebda11283" containerName="payload-processing"
Sep 10 19:32:21.024120 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:21.024103    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="65c44357-30b7-47c0-8957-0e9fb86019c3" containerName="main"
Sep 10 19:32:21.024120 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:21.024113    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="50e45adb-8cbc-4820-9c39-c4896e4a2ddd" containerName="main"
Sep 10 19:32:21.027627 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:21.027608    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-9d2e2715-77fc999f9c-4866g"
Sep 10 19:32:21.029956 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:21.029933    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-9d2e2715-dockercfg-djlsg\""
Sep 10 19:32:21.030217 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:21.030202    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-9d2e2715\""
Sep 10 19:32:21.036404 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:21.036383    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-9d2e2715-77fc999f9c-4866g"]
Sep 10 19:32:21.041910 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:21.041888    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:32:21.167453 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:21.167333    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4bf3ad13-04d5-4cd6-96d3-45336c0f44f5-plugins-config-volume\") pod \"payload-processing-e2e-mt-9d2e2715-77fc999f9c-4866g\" (UID: \"4bf3ad13-04d5-4cd6-96d3-45336c0f44f5\") " pod="openshift-ingress/payload-processing-e2e-mt-9d2e2715-77fc999f9c-4866g"
Sep 10 19:32:21.167616 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:21.167489    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gnz9l\" (UniqueName: \"kubernetes.io/projected/4bf3ad13-04d5-4cd6-96d3-45336c0f44f5-kube-api-access-gnz9l\") pod \"payload-processing-e2e-mt-9d2e2715-77fc999f9c-4866g\" (UID: \"4bf3ad13-04d5-4cd6-96d3-45336c0f44f5\") " pod="openshift-ingress/payload-processing-e2e-mt-9d2e2715-77fc999f9c-4866g"
Sep 10 19:32:21.203263 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:21.203225    2575 generic.go:356] "Generic (PLEG): container finished" podID="50e45adb-8cbc-4820-9c39-c4896e4a2ddd" containerID="8c8816571e7d33a6b2903b669d295055b5d87eec4532757beef9561122d1be5c" exitCode=0
Sep 10 19:32:21.203701 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:21.203273    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-4247c7/test-model-4247c7-kserve-746dcdc87-vnj4w"
Sep 10 19:32:21.203701 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:21.203317    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-4247c7/test-model-4247c7-kserve-746dcdc87-vnj4w" event={"ID":"50e45adb-8cbc-4820-9c39-c4896e4a2ddd","Type":"ContainerDied","Data":"8c8816571e7d33a6b2903b669d295055b5d87eec4532757beef9561122d1be5c"}
Sep 10 19:32:21.203701 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:21.203371    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-4247c7/test-model-4247c7-kserve-746dcdc87-vnj4w" event={"ID":"50e45adb-8cbc-4820-9c39-c4896e4a2ddd","Type":"ContainerDied","Data":"9d4a21f4d59089f52fc69711d2c9430cf04ec7790b0fe5236a128590a26556c2"}
Sep 10 19:32:21.203701 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:21.203389    2575 scope.go:117] "RemoveContainer" containerID="8c8816571e7d33a6b2903b669d295055b5d87eec4532757beef9561122d1be5c"
Sep 10 19:32:21.212853 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:21.212833    2575 scope.go:117] "RemoveContainer" containerID="8c8816571e7d33a6b2903b669d295055b5d87eec4532757beef9561122d1be5c"
Sep 10 19:32:21.213099 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:32:21.213083    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8c8816571e7d33a6b2903b669d295055b5d87eec4532757beef9561122d1be5c\": container with ID starting with 8c8816571e7d33a6b2903b669d295055b5d87eec4532757beef9561122d1be5c not found: ID does not exist" containerID="8c8816571e7d33a6b2903b669d295055b5d87eec4532757beef9561122d1be5c"
Sep 10 19:32:21.213151 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:21.213106    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c8816571e7d33a6b2903b669d295055b5d87eec4532757beef9561122d1be5c"} err="failed to get container status \"8c8816571e7d33a6b2903b669d295055b5d87eec4532757beef9561122d1be5c\": rpc error: code = NotFound desc = could not find container \"8c8816571e7d33a6b2903b669d295055b5d87eec4532757beef9561122d1be5c\": container with ID starting with 8c8816571e7d33a6b2903b669d295055b5d87eec4532757beef9561122d1be5c not found: ID does not exist"
Sep 10 19:32:21.224531 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:21.224511    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-4247c7/test-model-4247c7-kserve-746dcdc87-vnj4w"]
Sep 10 19:32:21.228196 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:21.228176    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-4247c7/test-model-4247c7-kserve-746dcdc87-vnj4w"]
Sep 10 19:32:21.268862 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:21.268843    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4bf3ad13-04d5-4cd6-96d3-45336c0f44f5-plugins-config-volume\") pod \"payload-processing-e2e-mt-9d2e2715-77fc999f9c-4866g\" (UID: \"4bf3ad13-04d5-4cd6-96d3-45336c0f44f5\") " pod="openshift-ingress/payload-processing-e2e-mt-9d2e2715-77fc999f9c-4866g"
Sep 10 19:32:21.268957 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:21.268906    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gnz9l\" (UniqueName: \"kubernetes.io/projected/4bf3ad13-04d5-4cd6-96d3-45336c0f44f5-kube-api-access-gnz9l\") pod \"payload-processing-e2e-mt-9d2e2715-77fc999f9c-4866g\" (UID: \"4bf3ad13-04d5-4cd6-96d3-45336c0f44f5\") " pod="openshift-ingress/payload-processing-e2e-mt-9d2e2715-77fc999f9c-4866g"
Sep 10 19:32:21.269563 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:21.269547    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4bf3ad13-04d5-4cd6-96d3-45336c0f44f5-plugins-config-volume\") pod \"payload-processing-e2e-mt-9d2e2715-77fc999f9c-4866g\" (UID: \"4bf3ad13-04d5-4cd6-96d3-45336c0f44f5\") " pod="openshift-ingress/payload-processing-e2e-mt-9d2e2715-77fc999f9c-4866g"
Sep 10 19:32:21.277667 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:21.277637    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gnz9l\" (UniqueName: \"kubernetes.io/projected/4bf3ad13-04d5-4cd6-96d3-45336c0f44f5-kube-api-access-gnz9l\") pod \"payload-processing-e2e-mt-9d2e2715-77fc999f9c-4866g\" (UID: \"4bf3ad13-04d5-4cd6-96d3-45336c0f44f5\") " pod="openshift-ingress/payload-processing-e2e-mt-9d2e2715-77fc999f9c-4866g"
Sep 10 19:32:21.339479 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:21.339457    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-9d2e2715-77fc999f9c-4866g"
Sep 10 19:32:21.385300 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:21.385270    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="50e45adb-8cbc-4820-9c39-c4896e4a2ddd" path="/var/lib/kubelet/pods/50e45adb-8cbc-4820-9c39-c4896e4a2ddd/volumes"
Sep 10 19:32:21.628612 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:21.628590    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-9d2e2715-77fc999f9c-4866g"]
Sep 10 19:32:21.630998 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:32:21.630976    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4bf3ad13_04d5_4cd6_96d3_45336c0f44f5.slice/crio-70dd83e5e65a84468daf656e440f80f6c4252d5724a0c8aa5e66a53996c88780 WatchSource:0}: Error finding container 70dd83e5e65a84468daf656e440f80f6c4252d5724a0c8aa5e66a53996c88780: Status 404 returned error can't find the container with id 70dd83e5e65a84468daf656e440f80f6c4252d5724a0c8aa5e66a53996c88780
Sep 10 19:32:22.210439 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:22.210407    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-9d2e2715-77fc999f9c-4866g" event={"ID":"4bf3ad13-04d5-4cd6-96d3-45336c0f44f5","Type":"ContainerStarted","Data":"aa761c22485649d5eb20139becb93e10b9c284996bf1e64bff73aac96961c0ff"}
Sep 10 19:32:22.210870 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:22.210452    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-9d2e2715-77fc999f9c-4866g" event={"ID":"4bf3ad13-04d5-4cd6-96d3-45336c0f44f5","Type":"ContainerStarted","Data":"70dd83e5e65a84468daf656e440f80f6c4252d5724a0c8aa5e66a53996c88780"}
Sep 10 19:32:22.210870 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:22.210481    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-9d2e2715-77fc999f9c-4866g"
Sep 10 19:32:22.224985 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:22.224933    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-9d2e2715-77fc999f9c-4866g" podStartSLOduration=1.224916599 podStartE2EDuration="1.224916599s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:32:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:32:22.224908764 +0000 UTC m=+1013.284736457" watchObservedRunningTime="2026-09-10 19:32:22.224916599 +0000 UTC m=+1013.284744268"
Sep 10 19:32:23.058048 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:23.058013    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:32:23.160199 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:23.160166    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:32:23.321805 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:23.321716    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:32:23.755691 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:23.755656    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:32:25.554621 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:25.554588    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qr7rl"]
Sep 10 19:32:31.871018 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:31.870981    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:32:33.216955 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:33.216927    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-9d2e2715-77fc999f9c-4866g"
Sep 10 19:32:34.224614 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:34.224568    2575 patch_prober.go:28] interesting pod/e2e-mt-9d2e2715-openshift-default-5dbb96d66d-t7967 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 10 19:32:34.225031 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:34.224647    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-9d2e2715-openshift-default-5dbb96d66d-t7967" podUID="cad351b9-3b12-4d82-8f0f-d24ee08fcfbb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 10 19:32:34.360720 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:34.360685    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:32:34.472851 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:34.472807    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:32:34.620708 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:34.620621    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:32:34.694012 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:34.693976    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:32:34.758238 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:34.758186    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:32:35.042674 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:35.042641    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:32:36.288720 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:36.288636    2575 generic.go:356] "Generic (PLEG): container finished" podID="cad351b9-3b12-4d82-8f0f-d24ee08fcfbb" containerID="154a8a044cdf4c450f6fbee139be216667b28b3ccf81ef9319c01ef1b494104e" exitCode=0
Sep 10 19:32:36.289101 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:36.288712    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9d2e2715-openshift-default-5dbb96d66d-t7967" event={"ID":"cad351b9-3b12-4d82-8f0f-d24ee08fcfbb","Type":"ContainerDied","Data":"154a8a044cdf4c450f6fbee139be216667b28b3ccf81ef9319c01ef1b494104e"}
Sep 10 19:32:36.289101 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:36.289071    2575 scope.go:117] "RemoveContainer" containerID="154a8a044cdf4c450f6fbee139be216667b28b3ccf81ef9319c01ef1b494104e"
Sep 10 19:32:36.290040 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:36.290008    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 19:32:36.290132 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:36.290115    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 19:32:36.290195 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:36.290169    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 19:32:37.295533 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:37.295497    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9d2e2715-openshift-default-5dbb96d66d-t7967" event={"ID":"cad351b9-3b12-4d82-8f0f-d24ee08fcfbb","Type":"ContainerStarted","Data":"7deaa8fbab3429b0326b593963d7ce561a6c3e55bbc5d6df3b8d308a906a4a88"}
Sep 10 19:32:37.356092 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:37.356057    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qr7rl"]
Sep 10 19:32:37.878960 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:37.878927    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-9d2e2715-openshift-default-5dbb96d66d-t7967"
Sep 10 19:32:37.879124 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:37.879061    2575 patch_prober.go:28] interesting pod/e2e-mt-9d2e2715-openshift-default-5dbb96d66d-t7967 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body=
Sep 10 19:32:37.879124 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:37.879110    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-9d2e2715-openshift-default-5dbb96d66d-t7967" podUID="cad351b9-3b12-4d82-8f0f-d24ee08fcfbb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused"
Sep 10 19:32:38.879910 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:38.879873    2575 patch_prober.go:28] interesting pod/e2e-mt-9d2e2715-openshift-default-5dbb96d66d-t7967 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body=
Sep 10 19:32:38.880295 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:38.879937    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-9d2e2715-openshift-default-5dbb96d66d-t7967" podUID="cad351b9-3b12-4d82-8f0f-d24ee08fcfbb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused"
Sep 10 19:32:39.879600 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:39.879565    2575 patch_prober.go:28] interesting pod/e2e-mt-9d2e2715-openshift-default-5dbb96d66d-t7967 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body=
Sep 10 19:32:39.879799 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:39.879634    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-9d2e2715-openshift-default-5dbb96d66d-t7967" podUID="cad351b9-3b12-4d82-8f0f-d24ee08fcfbb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused"
Sep 10 19:32:40.879287 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:40.879249    2575 patch_prober.go:28] interesting pod/e2e-mt-9d2e2715-openshift-default-5dbb96d66d-t7967 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body=
Sep 10 19:32:40.879679 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:40.879306    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-9d2e2715-openshift-default-5dbb96d66d-t7967" podUID="cad351b9-3b12-4d82-8f0f-d24ee08fcfbb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused"
Sep 10 19:32:41.322751 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:41.322717    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:32:41.428085 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:41.428049    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:32:41.528277 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:41.528225    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:32:41.667693 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:41.667602    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-4247c7-openshift-default-6c679cc99d-lk98v"]
Sep 10 19:32:41.668202 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:41.668167    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-4247c7-openshift-default-6c679cc99d-lk98v" podUID="c3aea00e-ff8b-4739-9e90-3c5838c124a2" containerName="istio-proxy" containerID="cri-o://493add7f5bcc1df6a5d509d3d607bbfdc3073baf018ce010188007e973aa53ec" gracePeriod=30
Sep 10 19:32:41.698237 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:41.698212    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:32:41.879664 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:41.879634    2575 patch_prober.go:28] interesting pod/e2e-mt-9d2e2715-openshift-default-5dbb96d66d-t7967 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body=
Sep 10 19:32:41.880028 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:41.879683    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-9d2e2715-openshift-default-5dbb96d66d-t7967" podUID="cad351b9-3b12-4d82-8f0f-d24ee08fcfbb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused"
Sep 10 19:32:42.169730 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:42.169696    2575 patch_prober.go:28] interesting pod/e2e-inf-b-4247c7-openshift-default-6c679cc99d-lk98v container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 10 19:32:42.169907 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:42.169750    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-4247c7-openshift-default-6c679cc99d-lk98v" podUID="c3aea00e-ff8b-4739-9e90-3c5838c124a2" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 10 19:32:42.213800 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:42.213769    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:32:42.537365 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:42.531829    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:32:42.970443 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:42.970412    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-9d2e2715-openshift-default-5dbb96d66d-t7967"
Sep 10 19:32:42.970951 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:42.970927    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-9d2e2715-openshift-default-5dbb96d66d-t7967"
Sep 10 19:32:42.972264 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:42.972242    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-9d2e2715-openshift-default-5dbb96d66d-t7967"
Sep 10 19:32:43.084725 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:43.084687    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:32:43.633523 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:43.633490    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:32:43.967429 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:43.967392    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:32:44.616139 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:44.616102    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:32:44.708851 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:44.708824    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:32:45.020194 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:45.020154    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:32:45.191736 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:45.191707    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:32:45.285433 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:45.285331    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:32:46.453631 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:46.453599    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:32:46.553730 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:46.553698    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:32:46.739405 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:46.739324    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:32:46.913448 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:46.913426    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-4247c7-openshift-default-6c679cc99d-lk98v"
Sep 10 19:32:47.014412 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:47.014304    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c3aea00e-ff8b-4739-9e90-3c5838c124a2-istio-podinfo\") pod \"c3aea00e-ff8b-4739-9e90-3c5838c124a2\" (UID: \"c3aea00e-ff8b-4739-9e90-3c5838c124a2\") "
Sep 10 19:32:47.014412 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:47.014357    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c3aea00e-ff8b-4739-9e90-3c5838c124a2-istio-token\") pod \"c3aea00e-ff8b-4739-9e90-3c5838c124a2\" (UID: \"c3aea00e-ff8b-4739-9e90-3c5838c124a2\") "
Sep 10 19:32:47.014412 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:47.014388    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c3aea00e-ff8b-4739-9e90-3c5838c124a2-istio-data\") pod \"c3aea00e-ff8b-4739-9e90-3c5838c124a2\" (UID: \"c3aea00e-ff8b-4739-9e90-3c5838c124a2\") "
Sep 10 19:32:47.014680 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:47.014495    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c3aea00e-ff8b-4739-9e90-3c5838c124a2-istio-envoy\") pod \"c3aea00e-ff8b-4739-9e90-3c5838c124a2\" (UID: \"c3aea00e-ff8b-4739-9e90-3c5838c124a2\") "
Sep 10 19:32:47.014680 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:47.014523    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c3aea00e-ff8b-4739-9e90-3c5838c124a2-istiod-ca-cert\") pod \"c3aea00e-ff8b-4739-9e90-3c5838c124a2\" (UID: \"c3aea00e-ff8b-4739-9e90-3c5838c124a2\") "
Sep 10 19:32:47.014680 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:47.014564    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c3aea00e-ff8b-4739-9e90-3c5838c124a2-workload-certs\") pod \"c3aea00e-ff8b-4739-9e90-3c5838c124a2\" (UID: \"c3aea00e-ff8b-4739-9e90-3c5838c124a2\") "
Sep 10 19:32:47.014680 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:47.014587    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5zkqd\" (UniqueName: \"kubernetes.io/projected/c3aea00e-ff8b-4739-9e90-3c5838c124a2-kube-api-access-5zkqd\") pod \"c3aea00e-ff8b-4739-9e90-3c5838c124a2\" (UID: \"c3aea00e-ff8b-4739-9e90-3c5838c124a2\") "
Sep 10 19:32:47.014680 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:47.014619    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3aea00e-ff8b-4739-9e90-3c5838c124a2-credential-socket\") pod \"c3aea00e-ff8b-4739-9e90-3c5838c124a2\" (UID: \"c3aea00e-ff8b-4739-9e90-3c5838c124a2\") "
Sep 10 19:32:47.014680 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:47.014646    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3aea00e-ff8b-4739-9e90-3c5838c124a2-workload-socket\") pod \"c3aea00e-ff8b-4739-9e90-3c5838c124a2\" (UID: \"c3aea00e-ff8b-4739-9e90-3c5838c124a2\") "
Sep 10 19:32:47.014965 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:47.014910    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3aea00e-ff8b-4739-9e90-3c5838c124a2-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c3aea00e-ff8b-4739-9e90-3c5838c124a2" (UID: "c3aea00e-ff8b-4739-9e90-3c5838c124a2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:32:47.015019 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:47.014974    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3aea00e-ff8b-4739-9e90-3c5838c124a2-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c3aea00e-ff8b-4739-9e90-3c5838c124a2" (UID: "c3aea00e-ff8b-4739-9e90-3c5838c124a2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:32:47.015019 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:47.014994    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c3aea00e-ff8b-4739-9e90-3c5838c124a2-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c3aea00e-ff8b-4739-9e90-3c5838c124a2" (UID: "c3aea00e-ff8b-4739-9e90-3c5838c124a2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:32:47.015131 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:47.015058    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3aea00e-ff8b-4739-9e90-3c5838c124a2-istio-data" (OuterVolumeSpecName: "istio-data") pod "c3aea00e-ff8b-4739-9e90-3c5838c124a2" (UID: "c3aea00e-ff8b-4739-9e90-3c5838c124a2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:32:47.015197 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:47.015176    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3aea00e-ff8b-4739-9e90-3c5838c124a2-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c3aea00e-ff8b-4739-9e90-3c5838c124a2" (UID: "c3aea00e-ff8b-4739-9e90-3c5838c124a2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:32:47.016361 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:47.016321    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c3aea00e-ff8b-4739-9e90-3c5838c124a2-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c3aea00e-ff8b-4739-9e90-3c5838c124a2" (UID: "c3aea00e-ff8b-4739-9e90-3c5838c124a2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 19:32:47.016896 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:47.016876    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c3aea00e-ff8b-4739-9e90-3c5838c124a2-istio-token" (OuterVolumeSpecName: "istio-token") pod "c3aea00e-ff8b-4739-9e90-3c5838c124a2" (UID: "c3aea00e-ff8b-4739-9e90-3c5838c124a2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:32:47.017069 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:47.017046    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3aea00e-ff8b-4739-9e90-3c5838c124a2-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c3aea00e-ff8b-4739-9e90-3c5838c124a2" (UID: "c3aea00e-ff8b-4739-9e90-3c5838c124a2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:32:47.017134 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:47.017054    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c3aea00e-ff8b-4739-9e90-3c5838c124a2-kube-api-access-5zkqd" (OuterVolumeSpecName: "kube-api-access-5zkqd") pod "c3aea00e-ff8b-4739-9e90-3c5838c124a2" (UID: "c3aea00e-ff8b-4739-9e90-3c5838c124a2"). InnerVolumeSpecName "kube-api-access-5zkqd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:32:47.077419 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:47.077383    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:32:47.116281 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:47.116258    2575 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c3aea00e-ff8b-4739-9e90-3c5838c124a2-istio-podinfo\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:32:47.116281 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:47.116279    2575 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c3aea00e-ff8b-4739-9e90-3c5838c124a2-istio-token\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:32:47.116498 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:47.116289    2575 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c3aea00e-ff8b-4739-9e90-3c5838c124a2-istio-data\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:32:47.116498 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:47.116297    2575 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c3aea00e-ff8b-4739-9e90-3c5838c124a2-istio-envoy\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:32:47.116498 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:47.116306    2575 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c3aea00e-ff8b-4739-9e90-3c5838c124a2-istiod-ca-cert\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:32:47.116498 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:47.116313    2575 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c3aea00e-ff8b-4739-9e90-3c5838c124a2-workload-certs\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:32:47.116498 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:47.116321    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5zkqd\" (UniqueName: \"kubernetes.io/projected/c3aea00e-ff8b-4739-9e90-3c5838c124a2-kube-api-access-5zkqd\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:32:47.116498 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:47.116329    2575 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3aea00e-ff8b-4739-9e90-3c5838c124a2-credential-socket\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:32:47.116498 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:47.116359    2575 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3aea00e-ff8b-4739-9e90-3c5838c124a2-workload-socket\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:32:47.164128 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:47.164097    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:32:47.351218 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:47.351139    2575 generic.go:356] "Generic (PLEG): container finished" podID="c3aea00e-ff8b-4739-9e90-3c5838c124a2" containerID="493add7f5bcc1df6a5d509d3d607bbfdc3073baf018ce010188007e973aa53ec" exitCode=0
Sep 10 19:32:47.351218 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:47.351195    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-4247c7-openshift-default-6c679cc99d-lk98v"
Sep 10 19:32:47.351421 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:47.351219    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-4247c7-openshift-default-6c679cc99d-lk98v" event={"ID":"c3aea00e-ff8b-4739-9e90-3c5838c124a2","Type":"ContainerDied","Data":"493add7f5bcc1df6a5d509d3d607bbfdc3073baf018ce010188007e973aa53ec"}
Sep 10 19:32:47.351421 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:47.351274    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-4247c7-openshift-default-6c679cc99d-lk98v" event={"ID":"c3aea00e-ff8b-4739-9e90-3c5838c124a2","Type":"ContainerDied","Data":"1355786c4246c8944d544f24479233090c26ebcaff53ef9f1ab687d7bc00e299"}
Sep 10 19:32:47.351421 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:47.351303    2575 scope.go:117] "RemoveContainer" containerID="493add7f5bcc1df6a5d509d3d607bbfdc3073baf018ce010188007e973aa53ec"
Sep 10 19:32:47.362363 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:47.362318    2575 scope.go:117] "RemoveContainer" containerID="66ec3d473001ce2815592ee14441ef7c2e1a52c261e5f2145b355e6df64bf27c"
Sep 10 19:32:47.370801 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:47.370781    2575 scope.go:117] "RemoveContainer" containerID="493add7f5bcc1df6a5d509d3d607bbfdc3073baf018ce010188007e973aa53ec"
Sep 10 19:32:47.371078 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:32:47.371060    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"493add7f5bcc1df6a5d509d3d607bbfdc3073baf018ce010188007e973aa53ec\": container with ID starting with 493add7f5bcc1df6a5d509d3d607bbfdc3073baf018ce010188007e973aa53ec not found: ID does not exist" containerID="493add7f5bcc1df6a5d509d3d607bbfdc3073baf018ce010188007e973aa53ec"
Sep 10 19:32:47.371157 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:47.371092    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"493add7f5bcc1df6a5d509d3d607bbfdc3073baf018ce010188007e973aa53ec"} err="failed to get container status \"493add7f5bcc1df6a5d509d3d607bbfdc3073baf018ce010188007e973aa53ec\": rpc error: code = NotFound desc = could not find container \"493add7f5bcc1df6a5d509d3d607bbfdc3073baf018ce010188007e973aa53ec\": container with ID starting with 493add7f5bcc1df6a5d509d3d607bbfdc3073baf018ce010188007e973aa53ec not found: ID does not exist"
Sep 10 19:32:47.371157 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:47.371116    2575 scope.go:117] "RemoveContainer" containerID="66ec3d473001ce2815592ee14441ef7c2e1a52c261e5f2145b355e6df64bf27c"
Sep 10 19:32:47.371407 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:32:47.371371    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"66ec3d473001ce2815592ee14441ef7c2e1a52c261e5f2145b355e6df64bf27c\": container with ID starting with 66ec3d473001ce2815592ee14441ef7c2e1a52c261e5f2145b355e6df64bf27c not found: ID does not exist" containerID="66ec3d473001ce2815592ee14441ef7c2e1a52c261e5f2145b355e6df64bf27c"
Sep 10 19:32:47.371493 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:47.371410    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"66ec3d473001ce2815592ee14441ef7c2e1a52c261e5f2145b355e6df64bf27c"} err="failed to get container status \"66ec3d473001ce2815592ee14441ef7c2e1a52c261e5f2145b355e6df64bf27c\": rpc error: code = NotFound desc = could not find container \"66ec3d473001ce2815592ee14441ef7c2e1a52c261e5f2145b355e6df64bf27c\": container with ID starting with 66ec3d473001ce2815592ee14441ef7c2e1a52c261e5f2145b355e6df64bf27c not found: ID does not exist"
Sep 10 19:32:47.374840 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:47.374816    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-4247c7-openshift-default-6c679cc99d-lk98v"]
Sep 10 19:32:47.377073 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:47.377056    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-4247c7-openshift-default-6c679cc99d-lk98v"]
Sep 10 19:32:47.384791 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:47.384770    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c3aea00e-ff8b-4739-9e90-3c5838c124a2" path="/var/lib/kubelet/pods/c3aea00e-ff8b-4739-9e90-3c5838c124a2/volumes"
Sep 10 19:32:47.780191 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:47.780155    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:32:49.959563 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:49.959531    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qr7rl"]
Sep 10 19:32:51.488907 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:51.488865    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:32:51.581935 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:51.581900    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:32:51.836187 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:51.836114    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9d2e2715-lttwp"]
Sep 10 19:32:51.836521 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:51.836507    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c3aea00e-ff8b-4739-9e90-3c5838c124a2" containerName="istio-proxy"
Sep 10 19:32:51.836566 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:51.836522    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="c3aea00e-ff8b-4739-9e90-3c5838c124a2" containerName="istio-proxy"
Sep 10 19:32:51.836605 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:51.836595    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="c3aea00e-ff8b-4739-9e90-3c5838c124a2" containerName="istio-proxy"
Sep 10 19:32:51.836641 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:51.836606    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="c3aea00e-ff8b-4739-9e90-3c5838c124a2" containerName="istio-proxy"
Sep 10 19:32:51.840962 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:51.840945    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9d2e2715-lttwp"
Sep 10 19:32:51.843012 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:51.842993    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 19:32:51.843094 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:51.842993    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wkgt8\""
Sep 10 19:32:51.850189 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:51.850161    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9d2e2715-lttwp"]
Sep 10 19:32:51.958952 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:51.958925    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8a7fd53b-d752-4e8b-b901-32f067e703b3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-9d2e2715-lttwp\" (UID: \"8a7fd53b-d752-4e8b-b901-32f067e703b3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9d2e2715-lttwp"
Sep 10 19:32:51.964749 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:51.964723    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:32:52.059392 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:52.059356    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8a7fd53b-d752-4e8b-b901-32f067e703b3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-9d2e2715-lttwp\" (UID: \"8a7fd53b-d752-4e8b-b901-32f067e703b3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9d2e2715-lttwp"
Sep 10 19:32:52.059937 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:52.059920    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8a7fd53b-d752-4e8b-b901-32f067e703b3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-9d2e2715-lttwp\" (UID: \"8a7fd53b-d752-4e8b-b901-32f067e703b3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9d2e2715-lttwp"
Sep 10 19:32:52.151772 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:52.151719    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9d2e2715-lttwp"
Sep 10 19:32:52.420916 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:52.420892    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9d2e2715-lttwp"]
Sep 10 19:32:52.423278 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:32:52.423247    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a7fd53b_d752_4e8b_b901_32f067e703b3.slice/crio-fb87c281b09135243938d8ae2b498ad5852392eebc8780e5ea0ac0546f5a820e WatchSource:0}: Error finding container fb87c281b09135243938d8ae2b498ad5852392eebc8780e5ea0ac0546f5a820e: Status 404 returned error can't find the container with id fb87c281b09135243938d8ae2b498ad5852392eebc8780e5ea0ac0546f5a820e
Sep 10 19:32:53.390944 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:53.390869    2575 generic.go:356] "Generic (PLEG): container finished" podID="8a7fd53b-d752-4e8b-b901-32f067e703b3" containerID="a6fae3a63dfb6b9d778315f028570bb9eb1e58ebd2971502d48db7d9b3017709" exitCode=0
Sep 10 19:32:53.390944 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:53.390927    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9d2e2715-lttwp" event={"ID":"8a7fd53b-d752-4e8b-b901-32f067e703b3","Type":"ContainerDied","Data":"a6fae3a63dfb6b9d778315f028570bb9eb1e58ebd2971502d48db7d9b3017709"}
Sep 10 19:32:53.391334 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:53.390958    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9d2e2715-lttwp" event={"ID":"8a7fd53b-d752-4e8b-b901-32f067e703b3","Type":"ContainerStarted","Data":"fb87c281b09135243938d8ae2b498ad5852392eebc8780e5ea0ac0546f5a820e"}
Sep 10 19:32:54.331819 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:54.331787    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:32:54.522375 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:54.522333    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9d2e2715-lttwp"
Sep 10 19:32:54.679744 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:54.679673    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8a7fd53b-d752-4e8b-b901-32f067e703b3-maas-api-service-ca\") pod \"8a7fd53b-d752-4e8b-b901-32f067e703b3\" (UID: \"8a7fd53b-d752-4e8b-b901-32f067e703b3\") "
Sep 10 19:32:54.680035 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:54.680011    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8a7fd53b-d752-4e8b-b901-32f067e703b3-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "8a7fd53b-d752-4e8b-b901-32f067e703b3" (UID: "8a7fd53b-d752-4e8b-b901-32f067e703b3"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:32:54.780773 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:54.780736    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8a7fd53b-d752-4e8b-b901-32f067e703b3-maas-api-service-ca\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:32:55.404021 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:55.403996    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9d2e2715-lttwp"
Sep 10 19:32:55.404176 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:55.404025    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9d2e2715-lttwp" event={"ID":"8a7fd53b-d752-4e8b-b901-32f067e703b3","Type":"ContainerDied","Data":"fb87c281b09135243938d8ae2b498ad5852392eebc8780e5ea0ac0546f5a820e"}
Sep 10 19:32:55.404176 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:32:55.404063    2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fb87c281b09135243938d8ae2b498ad5852392eebc8780e5ea0ac0546f5a820e"
Sep 10 19:33:02.489846 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:02.489812    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:33:02.523912 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:02.523883    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-9d2e2715-77fc999f9c-4866g"]
Sep 10 19:33:02.524136 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:02.524112    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-9d2e2715-77fc999f9c-4866g" podUID="4bf3ad13-04d5-4cd6-96d3-45336c0f44f5" containerName="payload-processing" containerID="cri-o://aa761c22485649d5eb20139becb93e10b9c284996bf1e64bff73aac96961c0ff" gracePeriod=30
Sep 10 19:33:02.590903 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:02.590855    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:33:02.691132 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:02.691007    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:33:02.721779 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:02.721747    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9d2e2715-lttwp"]
Sep 10 19:33:02.723894 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:02.723867    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9d2e2715-lttwp"]
Sep 10 19:33:02.752573 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:02.752509    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:33:02.773938 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:02.773920    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-9d2e2715-77fc999f9c-4866g"
Sep 10 19:33:02.843191 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:02.843161    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4bf3ad13-04d5-4cd6-96d3-45336c0f44f5-plugins-config-volume\") pod \"4bf3ad13-04d5-4cd6-96d3-45336c0f44f5\" (UID: \"4bf3ad13-04d5-4cd6-96d3-45336c0f44f5\") "
Sep 10 19:33:02.843370 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:02.843287    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gnz9l\" (UniqueName: \"kubernetes.io/projected/4bf3ad13-04d5-4cd6-96d3-45336c0f44f5-kube-api-access-gnz9l\") pod \"4bf3ad13-04d5-4cd6-96d3-45336c0f44f5\" (UID: \"4bf3ad13-04d5-4cd6-96d3-45336c0f44f5\") "
Sep 10 19:33:02.843593 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:02.843572    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4bf3ad13-04d5-4cd6-96d3-45336c0f44f5-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "4bf3ad13-04d5-4cd6-96d3-45336c0f44f5" (UID: "4bf3ad13-04d5-4cd6-96d3-45336c0f44f5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:33:02.845310 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:02.845290    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4bf3ad13-04d5-4cd6-96d3-45336c0f44f5-kube-api-access-gnz9l" (OuterVolumeSpecName: "kube-api-access-gnz9l") pod "4bf3ad13-04d5-4cd6-96d3-45336c0f44f5" (UID: "4bf3ad13-04d5-4cd6-96d3-45336c0f44f5"). InnerVolumeSpecName "kube-api-access-gnz9l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:33:02.944089 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:02.944062    2575 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4bf3ad13-04d5-4cd6-96d3-45336c0f44f5-plugins-config-volume\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:33:02.944089 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:02.944085    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gnz9l\" (UniqueName: \"kubernetes.io/projected/4bf3ad13-04d5-4cd6-96d3-45336c0f44f5-kube-api-access-gnz9l\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:33:03.072387 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:03.072308    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:33:03.173626 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:03.173596    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:33:03.384999 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:03.384930    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8a7fd53b-d752-4e8b-b901-32f067e703b3" path="/var/lib/kubelet/pods/8a7fd53b-d752-4e8b-b901-32f067e703b3/volumes"
Sep 10 19:33:03.447695 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:03.447669    2575 generic.go:356] "Generic (PLEG): container finished" podID="4bf3ad13-04d5-4cd6-96d3-45336c0f44f5" containerID="aa761c22485649d5eb20139becb93e10b9c284996bf1e64bff73aac96961c0ff" exitCode=0
Sep 10 19:33:03.447798 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:03.447713    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-9d2e2715-77fc999f9c-4866g"
Sep 10 19:33:03.447798 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:03.447754    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-9d2e2715-77fc999f9c-4866g" event={"ID":"4bf3ad13-04d5-4cd6-96d3-45336c0f44f5","Type":"ContainerDied","Data":"aa761c22485649d5eb20139becb93e10b9c284996bf1e64bff73aac96961c0ff"}
Sep 10 19:33:03.447875 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:03.447803    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-9d2e2715-77fc999f9c-4866g" event={"ID":"4bf3ad13-04d5-4cd6-96d3-45336c0f44f5","Type":"ContainerDied","Data":"70dd83e5e65a84468daf656e440f80f6c4252d5724a0c8aa5e66a53996c88780"}
Sep 10 19:33:03.447875 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:03.447833    2575 scope.go:117] "RemoveContainer" containerID="aa761c22485649d5eb20139becb93e10b9c284996bf1e64bff73aac96961c0ff"
Sep 10 19:33:03.456223 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:03.456207    2575 scope.go:117] "RemoveContainer" containerID="aa761c22485649d5eb20139becb93e10b9c284996bf1e64bff73aac96961c0ff"
Sep 10 19:33:03.456510 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:33:03.456492    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aa761c22485649d5eb20139becb93e10b9c284996bf1e64bff73aac96961c0ff\": container with ID starting with aa761c22485649d5eb20139becb93e10b9c284996bf1e64bff73aac96961c0ff not found: ID does not exist" containerID="aa761c22485649d5eb20139becb93e10b9c284996bf1e64bff73aac96961c0ff"
Sep 10 19:33:03.456562 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:03.456518    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aa761c22485649d5eb20139becb93e10b9c284996bf1e64bff73aac96961c0ff"} err="failed to get container status \"aa761c22485649d5eb20139becb93e10b9c284996bf1e64bff73aac96961c0ff\": rpc error: code = NotFound desc = could not find container \"aa761c22485649d5eb20139becb93e10b9c284996bf1e64bff73aac96961c0ff\": container with ID starting with aa761c22485649d5eb20139becb93e10b9c284996bf1e64bff73aac96961c0ff not found: ID does not exist"
Sep 10 19:33:03.463252 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:03.463232    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-9d2e2715-77fc999f9c-4866g"]
Sep 10 19:33:03.467062 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:03.467042    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-9d2e2715-77fc999f9c-4866g"]
Sep 10 19:33:05.351831 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:05.351799    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qr7rl"]
Sep 10 19:33:05.384066 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:05.384035    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4bf3ad13-04d5-4cd6-96d3-45336c0f44f5" path="/var/lib/kubelet/pods/4bf3ad13-04d5-4cd6-96d3-45336c0f44f5/volumes"
Sep 10 19:33:07.640773 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:07.640728    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:33:09.317751 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:09.317713    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:33:09.417523 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:09.417483    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:33:09.546957 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:09.546926    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-9d2e2715-openshift-default-5dbb96d66d-t7967"]
Sep 10 19:33:09.547205 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:09.547180    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-9d2e2715-openshift-default-5dbb96d66d-t7967" podUID="cad351b9-3b12-4d82-8f0f-d24ee08fcfbb" containerName="istio-proxy" containerID="cri-o://7deaa8fbab3429b0326b593963d7ce561a6c3e55bbc5d6df3b8d308a906a4a88" gracePeriod=30
Sep 10 19:33:09.567079 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:09.567050    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:33:09.981004 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:09.980975    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:33:10.099878 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:10.099848    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:33:10.193055 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:10.193027    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:33:10.483308 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:10.483266    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:33:10.740949 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:10.740875    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w3-cab4ee-openshift-default-945f9dd46-2dq66"]
Sep 10 19:33:10.741259 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:10.741244    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8a7fd53b-d752-4e8b-b901-32f067e703b3" containerName="revoke-keys"
Sep 10 19:33:10.741307 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:10.741262    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a7fd53b-d752-4e8b-b901-32f067e703b3" containerName="revoke-keys"
Sep 10 19:33:10.741307 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:10.741275    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c3aea00e-ff8b-4739-9e90-3c5838c124a2" containerName="istio-proxy"
Sep 10 19:33:10.741307 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:10.741281    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="c3aea00e-ff8b-4739-9e90-3c5838c124a2" containerName="istio-proxy"
Sep 10 19:33:10.741307 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:10.741306    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4bf3ad13-04d5-4cd6-96d3-45336c0f44f5" containerName="payload-processing"
Sep 10 19:33:10.741450 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:10.741312    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="4bf3ad13-04d5-4cd6-96d3-45336c0f44f5" containerName="payload-processing"
Sep 10 19:33:10.741450 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:10.741392    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="4bf3ad13-04d5-4cd6-96d3-45336c0f44f5" containerName="payload-processing"
Sep 10 19:33:10.741450 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:10.741403    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="8a7fd53b-d752-4e8b-b901-32f067e703b3" containerName="revoke-keys"
Sep 10 19:33:10.745982 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:10.745963    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-cab4ee-openshift-default-945f9dd46-2dq66"
Sep 10 19:33:10.748097 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:10.748077    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w3-cab4ee-openshift-default-dockercfg-bkwtf\""
Sep 10 19:33:10.757010 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:10.756989    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-cab4ee-openshift-default-945f9dd46-2dq66"]
Sep 10 19:33:10.769308 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:10.769286    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:33:10.915138 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:10.915107    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/61a281ba-e630-4357-bb77-64fe9085aadb-istio-data\") pod \"e2e-shared-b-w3-cab4ee-openshift-default-945f9dd46-2dq66\" (UID: \"61a281ba-e630-4357-bb77-64fe9085aadb\") " pod="openshift-ingress/e2e-shared-b-w3-cab4ee-openshift-default-945f9dd46-2dq66"
Sep 10 19:33:10.915239 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:10.915147    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/61a281ba-e630-4357-bb77-64fe9085aadb-workload-socket\") pod \"e2e-shared-b-w3-cab4ee-openshift-default-945f9dd46-2dq66\" (UID: \"61a281ba-e630-4357-bb77-64fe9085aadb\") " pod="openshift-ingress/e2e-shared-b-w3-cab4ee-openshift-default-945f9dd46-2dq66"
Sep 10 19:33:10.915239 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:10.915169    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/61a281ba-e630-4357-bb77-64fe9085aadb-istio-podinfo\") pod \"e2e-shared-b-w3-cab4ee-openshift-default-945f9dd46-2dq66\" (UID: \"61a281ba-e630-4357-bb77-64fe9085aadb\") " pod="openshift-ingress/e2e-shared-b-w3-cab4ee-openshift-default-945f9dd46-2dq66"
Sep 10 19:33:10.915239 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:10.915186    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/61a281ba-e630-4357-bb77-64fe9085aadb-credential-socket\") pod \"e2e-shared-b-w3-cab4ee-openshift-default-945f9dd46-2dq66\" (UID: \"61a281ba-e630-4357-bb77-64fe9085aadb\") " pod="openshift-ingress/e2e-shared-b-w3-cab4ee-openshift-default-945f9dd46-2dq66"
Sep 10 19:33:10.915375 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:10.915246    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/61a281ba-e630-4357-bb77-64fe9085aadb-workload-certs\") pod \"e2e-shared-b-w3-cab4ee-openshift-default-945f9dd46-2dq66\" (UID: \"61a281ba-e630-4357-bb77-64fe9085aadb\") " pod="openshift-ingress/e2e-shared-b-w3-cab4ee-openshift-default-945f9dd46-2dq66"
Sep 10 19:33:10.915375 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:10.915326    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/61a281ba-e630-4357-bb77-64fe9085aadb-istiod-ca-cert\") pod \"e2e-shared-b-w3-cab4ee-openshift-default-945f9dd46-2dq66\" (UID: \"61a281ba-e630-4357-bb77-64fe9085aadb\") " pod="openshift-ingress/e2e-shared-b-w3-cab4ee-openshift-default-945f9dd46-2dq66"
Sep 10 19:33:10.915439 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:10.915393    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/61a281ba-e630-4357-bb77-64fe9085aadb-istio-token\") pod \"e2e-shared-b-w3-cab4ee-openshift-default-945f9dd46-2dq66\" (UID: \"61a281ba-e630-4357-bb77-64fe9085aadb\") " pod="openshift-ingress/e2e-shared-b-w3-cab4ee-openshift-default-945f9dd46-2dq66"
Sep 10 19:33:10.915473 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:10.915459    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/61a281ba-e630-4357-bb77-64fe9085aadb-istio-envoy\") pod \"e2e-shared-b-w3-cab4ee-openshift-default-945f9dd46-2dq66\" (UID: \"61a281ba-e630-4357-bb77-64fe9085aadb\") " pod="openshift-ingress/e2e-shared-b-w3-cab4ee-openshift-default-945f9dd46-2dq66"
Sep 10 19:33:10.915507 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:10.915484    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g24cz\" (UniqueName: \"kubernetes.io/projected/61a281ba-e630-4357-bb77-64fe9085aadb-kube-api-access-g24cz\") pod \"e2e-shared-b-w3-cab4ee-openshift-default-945f9dd46-2dq66\" (UID: \"61a281ba-e630-4357-bb77-64fe9085aadb\") " pod="openshift-ingress/e2e-shared-b-w3-cab4ee-openshift-default-945f9dd46-2dq66"
Sep 10 19:33:11.016929 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:11.016867    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/61a281ba-e630-4357-bb77-64fe9085aadb-istio-data\") pod \"e2e-shared-b-w3-cab4ee-openshift-default-945f9dd46-2dq66\" (UID: \"61a281ba-e630-4357-bb77-64fe9085aadb\") " pod="openshift-ingress/e2e-shared-b-w3-cab4ee-openshift-default-945f9dd46-2dq66"
Sep 10 19:33:11.016929 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:11.016905    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/61a281ba-e630-4357-bb77-64fe9085aadb-workload-socket\") pod \"e2e-shared-b-w3-cab4ee-openshift-default-945f9dd46-2dq66\" (UID: \"61a281ba-e630-4357-bb77-64fe9085aadb\") " pod="openshift-ingress/e2e-shared-b-w3-cab4ee-openshift-default-945f9dd46-2dq66"
Sep 10 19:33:11.017065 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:11.016932    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/61a281ba-e630-4357-bb77-64fe9085aadb-istio-podinfo\") pod \"e2e-shared-b-w3-cab4ee-openshift-default-945f9dd46-2dq66\" (UID: \"61a281ba-e630-4357-bb77-64fe9085aadb\") " pod="openshift-ingress/e2e-shared-b-w3-cab4ee-openshift-default-945f9dd46-2dq66"
Sep 10 19:33:11.017103 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:11.017065    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/61a281ba-e630-4357-bb77-64fe9085aadb-credential-socket\") pod \"e2e-shared-b-w3-cab4ee-openshift-default-945f9dd46-2dq66\" (UID: \"61a281ba-e630-4357-bb77-64fe9085aadb\") " pod="openshift-ingress/e2e-shared-b-w3-cab4ee-openshift-default-945f9dd46-2dq66"
Sep 10 19:33:11.017154 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:11.017106    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/61a281ba-e630-4357-bb77-64fe9085aadb-workload-certs\") pod \"e2e-shared-b-w3-cab4ee-openshift-default-945f9dd46-2dq66\" (UID: \"61a281ba-e630-4357-bb77-64fe9085aadb\") " pod="openshift-ingress/e2e-shared-b-w3-cab4ee-openshift-default-945f9dd46-2dq66"
Sep 10 19:33:11.017209 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:11.017151    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/61a281ba-e630-4357-bb77-64fe9085aadb-istiod-ca-cert\") pod \"e2e-shared-b-w3-cab4ee-openshift-default-945f9dd46-2dq66\" (UID: \"61a281ba-e630-4357-bb77-64fe9085aadb\") " pod="openshift-ingress/e2e-shared-b-w3-cab4ee-openshift-default-945f9dd46-2dq66"
Sep 10 19:33:11.017262 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:11.017208    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/61a281ba-e630-4357-bb77-64fe9085aadb-istio-token\") pod \"e2e-shared-b-w3-cab4ee-openshift-default-945f9dd46-2dq66\" (UID: \"61a281ba-e630-4357-bb77-64fe9085aadb\") " pod="openshift-ingress/e2e-shared-b-w3-cab4ee-openshift-default-945f9dd46-2dq66"
Sep 10 19:33:11.017324 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:11.017281    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/61a281ba-e630-4357-bb77-64fe9085aadb-istio-envoy\") pod \"e2e-shared-b-w3-cab4ee-openshift-default-945f9dd46-2dq66\" (UID: \"61a281ba-e630-4357-bb77-64fe9085aadb\") " pod="openshift-ingress/e2e-shared-b-w3-cab4ee-openshift-default-945f9dd46-2dq66"
Sep 10 19:33:11.017324 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:11.017309    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g24cz\" (UniqueName: \"kubernetes.io/projected/61a281ba-e630-4357-bb77-64fe9085aadb-kube-api-access-g24cz\") pod \"e2e-shared-b-w3-cab4ee-openshift-default-945f9dd46-2dq66\" (UID: \"61a281ba-e630-4357-bb77-64fe9085aadb\") " pod="openshift-ingress/e2e-shared-b-w3-cab4ee-openshift-default-945f9dd46-2dq66"
Sep 10 19:33:11.017324 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:11.017309    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/61a281ba-e630-4357-bb77-64fe9085aadb-istio-data\") pod \"e2e-shared-b-w3-cab4ee-openshift-default-945f9dd46-2dq66\" (UID: \"61a281ba-e630-4357-bb77-64fe9085aadb\") " pod="openshift-ingress/e2e-shared-b-w3-cab4ee-openshift-default-945f9dd46-2dq66"
Sep 10 19:33:11.017530 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:11.017374    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/61a281ba-e630-4357-bb77-64fe9085aadb-workload-socket\") pod \"e2e-shared-b-w3-cab4ee-openshift-default-945f9dd46-2dq66\" (UID: \"61a281ba-e630-4357-bb77-64fe9085aadb\") " pod="openshift-ingress/e2e-shared-b-w3-cab4ee-openshift-default-945f9dd46-2dq66"
Sep 10 19:33:11.017530 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:11.017428    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/61a281ba-e630-4357-bb77-64fe9085aadb-credential-socket\") pod \"e2e-shared-b-w3-cab4ee-openshift-default-945f9dd46-2dq66\" (UID: \"61a281ba-e630-4357-bb77-64fe9085aadb\") " pod="openshift-ingress/e2e-shared-b-w3-cab4ee-openshift-default-945f9dd46-2dq66"
Sep 10 19:33:11.017789 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:11.017763    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/61a281ba-e630-4357-bb77-64fe9085aadb-workload-certs\") pod \"e2e-shared-b-w3-cab4ee-openshift-default-945f9dd46-2dq66\" (UID: \"61a281ba-e630-4357-bb77-64fe9085aadb\") " pod="openshift-ingress/e2e-shared-b-w3-cab4ee-openshift-default-945f9dd46-2dq66"
Sep 10 19:33:11.018075 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:11.018049    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/61a281ba-e630-4357-bb77-64fe9085aadb-istiod-ca-cert\") pod \"e2e-shared-b-w3-cab4ee-openshift-default-945f9dd46-2dq66\" (UID: \"61a281ba-e630-4357-bb77-64fe9085aadb\") " pod="openshift-ingress/e2e-shared-b-w3-cab4ee-openshift-default-945f9dd46-2dq66"
Sep 10 19:33:11.019787 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:11.019763    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/61a281ba-e630-4357-bb77-64fe9085aadb-istio-podinfo\") pod \"e2e-shared-b-w3-cab4ee-openshift-default-945f9dd46-2dq66\" (UID: \"61a281ba-e630-4357-bb77-64fe9085aadb\") " pod="openshift-ingress/e2e-shared-b-w3-cab4ee-openshift-default-945f9dd46-2dq66"
Sep 10 19:33:11.019972 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:11.019951    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/61a281ba-e630-4357-bb77-64fe9085aadb-istio-envoy\") pod \"e2e-shared-b-w3-cab4ee-openshift-default-945f9dd46-2dq66\" (UID: \"61a281ba-e630-4357-bb77-64fe9085aadb\") " pod="openshift-ingress/e2e-shared-b-w3-cab4ee-openshift-default-945f9dd46-2dq66"
Sep 10 19:33:11.031204 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:11.031177    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g24cz\" (UniqueName: \"kubernetes.io/projected/61a281ba-e630-4357-bb77-64fe9085aadb-kube-api-access-g24cz\") pod \"e2e-shared-b-w3-cab4ee-openshift-default-945f9dd46-2dq66\" (UID: \"61a281ba-e630-4357-bb77-64fe9085aadb\") " pod="openshift-ingress/e2e-shared-b-w3-cab4ee-openshift-default-945f9dd46-2dq66"
Sep 10 19:33:11.031204 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:11.031189    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/61a281ba-e630-4357-bb77-64fe9085aadb-istio-token\") pod \"e2e-shared-b-w3-cab4ee-openshift-default-945f9dd46-2dq66\" (UID: \"61a281ba-e630-4357-bb77-64fe9085aadb\") " pod="openshift-ingress/e2e-shared-b-w3-cab4ee-openshift-default-945f9dd46-2dq66"
Sep 10 19:33:11.056182 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:11.056157    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-cab4ee-openshift-default-945f9dd46-2dq66"
Sep 10 19:33:11.061108 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:11.061059    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:33:11.333682 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:11.333656    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-cab4ee-openshift-default-945f9dd46-2dq66"]
Sep 10 19:33:11.335903 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:33:11.335878    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61a281ba_e630_4357_bb77_64fe9085aadb.slice/crio-fee7ad8d02f589369c253d4b17d4440a19de4d95f0315687cf5dc4ac97233463 WatchSource:0}: Error finding container fee7ad8d02f589369c253d4b17d4440a19de4d95f0315687cf5dc4ac97233463: Status 404 returned error can't find the container with id fee7ad8d02f589369c253d4b17d4440a19de4d95f0315687cf5dc4ac97233463
Sep 10 19:33:11.338093 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:11.338059    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 19:33:11.338162 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:11.338130    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 19:33:11.338199 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:11.338163    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 19:33:11.489791 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:11.489753    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-cab4ee-openshift-default-945f9dd46-2dq66" event={"ID":"61a281ba-e630-4357-bb77-64fe9085aadb","Type":"ContainerStarted","Data":"1b568bf9c31904d27f7b7f5c390a7e8e975d0e46edf68f14dc6ba98556945c46"}
Sep 10 19:33:11.490226 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:11.489817    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-cab4ee-openshift-default-945f9dd46-2dq66" event={"ID":"61a281ba-e630-4357-bb77-64fe9085aadb","Type":"ContainerStarted","Data":"fee7ad8d02f589369c253d4b17d4440a19de4d95f0315687cf5dc4ac97233463"}
Sep 10 19:33:11.512508 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:11.512450    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w3-cab4ee-openshift-default-945f9dd46-2dq66" podStartSLOduration=1.512431404 podStartE2EDuration="1.512431404s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:33:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:33:11.510869846 +0000 UTC m=+1062.570697515" watchObservedRunningTime="2026-09-10 19:33:11.512431404 +0000 UTC m=+1062.572259077"
Sep 10 19:33:12.056956 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:12.056922    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-cab4ee-openshift-default-945f9dd46-2dq66"
Sep 10 19:33:12.061937 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:12.061912    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-cab4ee-openshift-default-945f9dd46-2dq66"
Sep 10 19:33:12.494266 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:12.494239    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-cab4ee-openshift-default-945f9dd46-2dq66"
Sep 10 19:33:12.495166 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:12.495147    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-cab4ee-openshift-default-945f9dd46-2dq66"
Sep 10 19:33:12.973113 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:12.973076    2575 patch_prober.go:28] interesting pod/e2e-mt-9d2e2715-openshift-default-5dbb96d66d-t7967 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 10 19:33:12.973279 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:12.973128    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-9d2e2715-openshift-default-5dbb96d66d-t7967" podUID="cad351b9-3b12-4d82-8f0f-d24ee08fcfbb" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 10 19:33:13.487356 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:13.487318    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:33:13.578880 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:13.578852    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:33:14.648719 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:14.648686    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:33:14.748151 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:14.748123    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:33:14.799601 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:14.799575    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-9d2e2715-openshift-default-5dbb96d66d-t7967"
Sep 10 19:33:14.954842 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:14.954817    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cad351b9-3b12-4d82-8f0f-d24ee08fcfbb-workload-socket\") pod \"cad351b9-3b12-4d82-8f0f-d24ee08fcfbb\" (UID: \"cad351b9-3b12-4d82-8f0f-d24ee08fcfbb\") "
Sep 10 19:33:14.954990 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:14.954870    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-45m45\" (UniqueName: \"kubernetes.io/projected/cad351b9-3b12-4d82-8f0f-d24ee08fcfbb-kube-api-access-45m45\") pod \"cad351b9-3b12-4d82-8f0f-d24ee08fcfbb\" (UID: \"cad351b9-3b12-4d82-8f0f-d24ee08fcfbb\") "
Sep 10 19:33:14.954990 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:14.954918    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cad351b9-3b12-4d82-8f0f-d24ee08fcfbb-workload-certs\") pod \"cad351b9-3b12-4d82-8f0f-d24ee08fcfbb\" (UID: \"cad351b9-3b12-4d82-8f0f-d24ee08fcfbb\") "
Sep 10 19:33:14.954990 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:14.954975    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cad351b9-3b12-4d82-8f0f-d24ee08fcfbb-istio-data\") pod \"cad351b9-3b12-4d82-8f0f-d24ee08fcfbb\" (UID: \"cad351b9-3b12-4d82-8f0f-d24ee08fcfbb\") "
Sep 10 19:33:14.955163 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:14.955009    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cad351b9-3b12-4d82-8f0f-d24ee08fcfbb-credential-socket\") pod \"cad351b9-3b12-4d82-8f0f-d24ee08fcfbb\" (UID: \"cad351b9-3b12-4d82-8f0f-d24ee08fcfbb\") "
Sep 10 19:33:14.955163 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:14.955039    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cad351b9-3b12-4d82-8f0f-d24ee08fcfbb-istio-podinfo\") pod \"cad351b9-3b12-4d82-8f0f-d24ee08fcfbb\" (UID: \"cad351b9-3b12-4d82-8f0f-d24ee08fcfbb\") "
Sep 10 19:33:14.955163 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:14.955067    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cad351b9-3b12-4d82-8f0f-d24ee08fcfbb-istio-envoy\") pod \"cad351b9-3b12-4d82-8f0f-d24ee08fcfbb\" (UID: \"cad351b9-3b12-4d82-8f0f-d24ee08fcfbb\") "
Sep 10 19:33:14.955163 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:14.955090    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cad351b9-3b12-4d82-8f0f-d24ee08fcfbb-istiod-ca-cert\") pod \"cad351b9-3b12-4d82-8f0f-d24ee08fcfbb\" (UID: \"cad351b9-3b12-4d82-8f0f-d24ee08fcfbb\") "
Sep 10 19:33:14.955163 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:14.955110    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cad351b9-3b12-4d82-8f0f-d24ee08fcfbb-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "cad351b9-3b12-4d82-8f0f-d24ee08fcfbb" (UID: "cad351b9-3b12-4d82-8f0f-d24ee08fcfbb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:33:14.955163 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:14.955126    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cad351b9-3b12-4d82-8f0f-d24ee08fcfbb-istio-token\") pod \"cad351b9-3b12-4d82-8f0f-d24ee08fcfbb\" (UID: \"cad351b9-3b12-4d82-8f0f-d24ee08fcfbb\") "
Sep 10 19:33:14.955501 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:14.955354    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cad351b9-3b12-4d82-8f0f-d24ee08fcfbb-istio-data" (OuterVolumeSpecName: "istio-data") pod "cad351b9-3b12-4d82-8f0f-d24ee08fcfbb" (UID: "cad351b9-3b12-4d82-8f0f-d24ee08fcfbb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:33:14.955501 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:14.955386    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cad351b9-3b12-4d82-8f0f-d24ee08fcfbb-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "cad351b9-3b12-4d82-8f0f-d24ee08fcfbb" (UID: "cad351b9-3b12-4d82-8f0f-d24ee08fcfbb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:33:14.955501 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:14.955466    2575 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cad351b9-3b12-4d82-8f0f-d24ee08fcfbb-workload-socket\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:33:14.955501 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:14.955485    2575 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cad351b9-3b12-4d82-8f0f-d24ee08fcfbb-workload-certs\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:33:14.955501 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:14.955499    2575 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cad351b9-3b12-4d82-8f0f-d24ee08fcfbb-istio-data\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:33:14.955705 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:14.955511    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cad351b9-3b12-4d82-8f0f-d24ee08fcfbb-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "cad351b9-3b12-4d82-8f0f-d24ee08fcfbb" (UID: "cad351b9-3b12-4d82-8f0f-d24ee08fcfbb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:33:14.955814 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:14.955789    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cad351b9-3b12-4d82-8f0f-d24ee08fcfbb-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "cad351b9-3b12-4d82-8f0f-d24ee08fcfbb" (UID: "cad351b9-3b12-4d82-8f0f-d24ee08fcfbb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:33:14.957271 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:14.957249    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cad351b9-3b12-4d82-8f0f-d24ee08fcfbb-kube-api-access-45m45" (OuterVolumeSpecName: "kube-api-access-45m45") pod "cad351b9-3b12-4d82-8f0f-d24ee08fcfbb" (UID: "cad351b9-3b12-4d82-8f0f-d24ee08fcfbb"). InnerVolumeSpecName "kube-api-access-45m45". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:33:14.957633 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:14.957609    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cad351b9-3b12-4d82-8f0f-d24ee08fcfbb-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "cad351b9-3b12-4d82-8f0f-d24ee08fcfbb" (UID: "cad351b9-3b12-4d82-8f0f-d24ee08fcfbb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:33:14.957870 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:14.957854    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cad351b9-3b12-4d82-8f0f-d24ee08fcfbb-istio-token" (OuterVolumeSpecName: "istio-token") pod "cad351b9-3b12-4d82-8f0f-d24ee08fcfbb" (UID: "cad351b9-3b12-4d82-8f0f-d24ee08fcfbb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:33:14.957930 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:14.957864    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/cad351b9-3b12-4d82-8f0f-d24ee08fcfbb-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "cad351b9-3b12-4d82-8f0f-d24ee08fcfbb" (UID: "cad351b9-3b12-4d82-8f0f-d24ee08fcfbb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 19:33:15.056388 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:15.056362    2575 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cad351b9-3b12-4d82-8f0f-d24ee08fcfbb-credential-socket\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:33:15.056388 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:15.056386    2575 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cad351b9-3b12-4d82-8f0f-d24ee08fcfbb-istio-podinfo\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:33:15.056508 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:15.056395    2575 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cad351b9-3b12-4d82-8f0f-d24ee08fcfbb-istio-envoy\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:33:15.056508 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:15.056404    2575 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cad351b9-3b12-4d82-8f0f-d24ee08fcfbb-istiod-ca-cert\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:33:15.056508 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:15.056412    2575 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cad351b9-3b12-4d82-8f0f-d24ee08fcfbb-istio-token\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:33:15.056508 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:15.056421    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-45m45\" (UniqueName: \"kubernetes.io/projected/cad351b9-3b12-4d82-8f0f-d24ee08fcfbb-kube-api-access-45m45\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:33:15.200680 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:15.200644    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:33:15.259800 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:15.259717    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:33:15.263817 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:15.263796    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-cab4ee-5fc779cb6-f4nbr"]
Sep 10 19:33:15.264213 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:15.264198    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cad351b9-3b12-4d82-8f0f-d24ee08fcfbb" containerName="istio-proxy"
Sep 10 19:33:15.264255 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:15.264215    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="cad351b9-3b12-4d82-8f0f-d24ee08fcfbb" containerName="istio-proxy"
Sep 10 19:33:15.264255 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:15.264226    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cad351b9-3b12-4d82-8f0f-d24ee08fcfbb" containerName="istio-proxy"
Sep 10 19:33:15.264255 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:15.264232    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="cad351b9-3b12-4d82-8f0f-d24ee08fcfbb" containerName="istio-proxy"
Sep 10 19:33:15.264389 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:15.264295    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="cad351b9-3b12-4d82-8f0f-d24ee08fcfbb" containerName="istio-proxy"
Sep 10 19:33:15.267520 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:15.267504    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-cab4ee-5fc779cb6-f4nbr"
Sep 10 19:33:15.269460 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:15.269441    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w3-cab4ee\""
Sep 10 19:33:15.269529 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:15.269441    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w3-cab4ee-dockercfg-cpcgg\""
Sep 10 19:33:15.278747 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:15.278725    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-cab4ee-5fc779cb6-f4nbr"]
Sep 10 19:33:15.358783 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:15.358755    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/24ce5208-91be-4c34-869b-75b0102f6380-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-cab4ee-5fc779cb6-f4nbr\" (UID: \"24ce5208-91be-4c34-869b-75b0102f6380\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-cab4ee-5fc779cb6-f4nbr"
Sep 10 19:33:15.358914 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:15.358825    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-79msj\" (UniqueName: \"kubernetes.io/projected/24ce5208-91be-4c34-869b-75b0102f6380-kube-api-access-79msj\") pod \"payload-pre-processing-e2e-shared-b-w3-cab4ee-5fc779cb6-f4nbr\" (UID: \"24ce5208-91be-4c34-869b-75b0102f6380\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-cab4ee-5fc779cb6-f4nbr"
Sep 10 19:33:15.460068 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:15.460041    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/24ce5208-91be-4c34-869b-75b0102f6380-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-cab4ee-5fc779cb6-f4nbr\" (UID: \"24ce5208-91be-4c34-869b-75b0102f6380\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-cab4ee-5fc779cb6-f4nbr"
Sep 10 19:33:15.460225 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:15.460095    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-79msj\" (UniqueName: \"kubernetes.io/projected/24ce5208-91be-4c34-869b-75b0102f6380-kube-api-access-79msj\") pod \"payload-pre-processing-e2e-shared-b-w3-cab4ee-5fc779cb6-f4nbr\" (UID: \"24ce5208-91be-4c34-869b-75b0102f6380\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-cab4ee-5fc779cb6-f4nbr"
Sep 10 19:33:15.460902 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:15.460874    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/24ce5208-91be-4c34-869b-75b0102f6380-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-cab4ee-5fc779cb6-f4nbr\" (UID: \"24ce5208-91be-4c34-869b-75b0102f6380\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-cab4ee-5fc779cb6-f4nbr"
Sep 10 19:33:15.468647 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:15.468621    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-79msj\" (UniqueName: \"kubernetes.io/projected/24ce5208-91be-4c34-869b-75b0102f6380-kube-api-access-79msj\") pod \"payload-pre-processing-e2e-shared-b-w3-cab4ee-5fc779cb6-f4nbr\" (UID: \"24ce5208-91be-4c34-869b-75b0102f6380\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-cab4ee-5fc779cb6-f4nbr"
Sep 10 19:33:15.510134 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:15.510067    2575 generic.go:356] "Generic (PLEG): container finished" podID="cad351b9-3b12-4d82-8f0f-d24ee08fcfbb" containerID="7deaa8fbab3429b0326b593963d7ce561a6c3e55bbc5d6df3b8d308a906a4a88" exitCode=0
Sep 10 19:33:15.510134 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:15.510108    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-9d2e2715-openshift-default-5dbb96d66d-t7967"
Sep 10 19:33:15.510259 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:15.510151    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9d2e2715-openshift-default-5dbb96d66d-t7967" event={"ID":"cad351b9-3b12-4d82-8f0f-d24ee08fcfbb","Type":"ContainerDied","Data":"7deaa8fbab3429b0326b593963d7ce561a6c3e55bbc5d6df3b8d308a906a4a88"}
Sep 10 19:33:15.510259 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:15.510200    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9d2e2715-openshift-default-5dbb96d66d-t7967" event={"ID":"cad351b9-3b12-4d82-8f0f-d24ee08fcfbb","Type":"ContainerDied","Data":"255e70e543cdc3d96290ca7a337372316ac4ba7ce1d55d5d02c32416393e7c9b"}
Sep 10 19:33:15.510259 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:15.510222    2575 scope.go:117] "RemoveContainer" containerID="7deaa8fbab3429b0326b593963d7ce561a6c3e55bbc5d6df3b8d308a906a4a88"
Sep 10 19:33:15.519937 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:15.519916    2575 scope.go:117] "RemoveContainer" containerID="154a8a044cdf4c450f6fbee139be216667b28b3ccf81ef9319c01ef1b494104e"
Sep 10 19:33:15.525824 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:15.525803    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-9d2e2715-openshift-default-5dbb96d66d-t7967"]
Sep 10 19:33:15.528357 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:15.528304    2575 scope.go:117] "RemoveContainer" containerID="7deaa8fbab3429b0326b593963d7ce561a6c3e55bbc5d6df3b8d308a906a4a88"
Sep 10 19:33:15.528776 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:33:15.528641    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7deaa8fbab3429b0326b593963d7ce561a6c3e55bbc5d6df3b8d308a906a4a88\": container with ID starting with 7deaa8fbab3429b0326b593963d7ce561a6c3e55bbc5d6df3b8d308a906a4a88 not found: ID does not exist" containerID="7deaa8fbab3429b0326b593963d7ce561a6c3e55bbc5d6df3b8d308a906a4a88"
Sep 10 19:33:15.528776 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:15.528675    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7deaa8fbab3429b0326b593963d7ce561a6c3e55bbc5d6df3b8d308a906a4a88"} err="failed to get container status \"7deaa8fbab3429b0326b593963d7ce561a6c3e55bbc5d6df3b8d308a906a4a88\": rpc error: code = NotFound desc = could not find container \"7deaa8fbab3429b0326b593963d7ce561a6c3e55bbc5d6df3b8d308a906a4a88\": container with ID starting with 7deaa8fbab3429b0326b593963d7ce561a6c3e55bbc5d6df3b8d308a906a4a88 not found: ID does not exist"
Sep 10 19:33:15.528776 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:15.528699    2575 scope.go:117] "RemoveContainer" containerID="154a8a044cdf4c450f6fbee139be216667b28b3ccf81ef9319c01ef1b494104e"
Sep 10 19:33:15.529130 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:33:15.529112    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"154a8a044cdf4c450f6fbee139be216667b28b3ccf81ef9319c01ef1b494104e\": container with ID starting with 154a8a044cdf4c450f6fbee139be216667b28b3ccf81ef9319c01ef1b494104e not found: ID does not exist" containerID="154a8a044cdf4c450f6fbee139be216667b28b3ccf81ef9319c01ef1b494104e"
Sep 10 19:33:15.529195 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:15.529138    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"154a8a044cdf4c450f6fbee139be216667b28b3ccf81ef9319c01ef1b494104e"} err="failed to get container status \"154a8a044cdf4c450f6fbee139be216667b28b3ccf81ef9319c01ef1b494104e\": rpc error: code = NotFound desc = could not find container \"154a8a044cdf4c450f6fbee139be216667b28b3ccf81ef9319c01ef1b494104e\": container with ID starting with 154a8a044cdf4c450f6fbee139be216667b28b3ccf81ef9319c01ef1b494104e not found: ID does not exist"
Sep 10 19:33:15.530123 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:15.530107    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-9d2e2715-openshift-default-5dbb96d66d-t7967"]
Sep 10 19:33:15.579445 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:15.579422    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-cab4ee-5fc779cb6-f4nbr"
Sep 10 19:33:15.851139 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:15.851038    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-cab4ee-5fc779cb6-f4nbr"]
Sep 10 19:33:15.853606 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:33:15.853574    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24ce5208_91be_4c34_869b_75b0102f6380.slice/crio-c3e83d05f06ec7119aa48585024e715fca72aee19cb5d42bfe6975781d7f1397 WatchSource:0}: Error finding container c3e83d05f06ec7119aa48585024e715fca72aee19cb5d42bfe6975781d7f1397: Status 404 returned error can't find the container with id c3e83d05f06ec7119aa48585024e715fca72aee19cb5d42bfe6975781d7f1397
Sep 10 19:33:16.517293 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:16.517248    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-cab4ee-5fc779cb6-f4nbr" event={"ID":"24ce5208-91be-4c34-869b-75b0102f6380","Type":"ContainerStarted","Data":"0f57979f80b92e5e3f3b2950597db4c3290edfeab78ea4411e3b5c810b3fcc05"}
Sep 10 19:33:16.517603 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:16.517308    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-cab4ee-5fc779cb6-f4nbr" event={"ID":"24ce5208-91be-4c34-869b-75b0102f6380","Type":"ContainerStarted","Data":"c3e83d05f06ec7119aa48585024e715fca72aee19cb5d42bfe6975781d7f1397"}
Sep 10 19:33:16.517603 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:16.517363    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-cab4ee-5fc779cb6-f4nbr"
Sep 10 19:33:16.533985 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:16.533933    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-cab4ee-5fc779cb6-f4nbr" podStartSLOduration=1.533919424 podStartE2EDuration="1.533919424s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:33:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:33:16.531824044 +0000 UTC m=+1067.591651711" watchObservedRunningTime="2026-09-10 19:33:16.533919424 +0000 UTC m=+1067.593747092"
Sep 10 19:33:17.070970 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:17.070926    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:33:17.178439 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:17.178398    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qr7rl"]
Sep 10 19:33:17.180988 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:17.180955    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:33:17.291070 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:17.291038    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:33:17.384248 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:17.384184    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cad351b9-3b12-4d82-8f0f-d24ee08fcfbb" path="/var/lib/kubelet/pods/cad351b9-3b12-4d82-8f0f-d24ee08fcfbb/volumes"
Sep 10 19:33:17.416023 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:17.415987    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:33:24.153714 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:24.153622    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:33:24.221855 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:24.221783    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:33:25.066904 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:25.066862    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:33:25.340229 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:25.340150    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w2-8dd4a3-openshift-default-856bd4c5f8-24xmz"]
Sep 10 19:33:25.340865 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:25.340845    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="cad351b9-3b12-4d82-8f0f-d24ee08fcfbb" containerName="istio-proxy"
Sep 10 19:33:25.345402 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:25.345378    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-8dd4a3-openshift-default-856bd4c5f8-24xmz"
Sep 10 19:33:25.347969 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:25.347949    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w2-8dd4a3-openshift-default-dockercfg-qsjzx\""
Sep 10 19:33:25.356037 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:25.356007    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-8dd4a3-openshift-default-856bd4c5f8-24xmz"]
Sep 10 19:33:25.370617 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:25.370581    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:33:25.445380 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:25.445317    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/60e22eaa-6ec6-4298-bfcd-3f42bcce4117-istio-data\") pod \"e2e-shared-a-w2-8dd4a3-openshift-default-856bd4c5f8-24xmz\" (UID: \"60e22eaa-6ec6-4298-bfcd-3f42bcce4117\") " pod="openshift-ingress/e2e-shared-a-w2-8dd4a3-openshift-default-856bd4c5f8-24xmz"
Sep 10 19:33:25.445591 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:25.445451    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/60e22eaa-6ec6-4298-bfcd-3f42bcce4117-credential-socket\") pod \"e2e-shared-a-w2-8dd4a3-openshift-default-856bd4c5f8-24xmz\" (UID: \"60e22eaa-6ec6-4298-bfcd-3f42bcce4117\") " pod="openshift-ingress/e2e-shared-a-w2-8dd4a3-openshift-default-856bd4c5f8-24xmz"
Sep 10 19:33:25.445591 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:25.445513    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/60e22eaa-6ec6-4298-bfcd-3f42bcce4117-istiod-ca-cert\") pod \"e2e-shared-a-w2-8dd4a3-openshift-default-856bd4c5f8-24xmz\" (UID: \"60e22eaa-6ec6-4298-bfcd-3f42bcce4117\") " pod="openshift-ingress/e2e-shared-a-w2-8dd4a3-openshift-default-856bd4c5f8-24xmz"
Sep 10 19:33:25.445681 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:25.445599    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/60e22eaa-6ec6-4298-bfcd-3f42bcce4117-workload-certs\") pod \"e2e-shared-a-w2-8dd4a3-openshift-default-856bd4c5f8-24xmz\" (UID: \"60e22eaa-6ec6-4298-bfcd-3f42bcce4117\") " pod="openshift-ingress/e2e-shared-a-w2-8dd4a3-openshift-default-856bd4c5f8-24xmz"
Sep 10 19:33:25.445681 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:25.445651    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/60e22eaa-6ec6-4298-bfcd-3f42bcce4117-istio-token\") pod \"e2e-shared-a-w2-8dd4a3-openshift-default-856bd4c5f8-24xmz\" (UID: \"60e22eaa-6ec6-4298-bfcd-3f42bcce4117\") " pod="openshift-ingress/e2e-shared-a-w2-8dd4a3-openshift-default-856bd4c5f8-24xmz"
Sep 10 19:33:25.445770 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:25.445691    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/60e22eaa-6ec6-4298-bfcd-3f42bcce4117-istio-envoy\") pod \"e2e-shared-a-w2-8dd4a3-openshift-default-856bd4c5f8-24xmz\" (UID: \"60e22eaa-6ec6-4298-bfcd-3f42bcce4117\") " pod="openshift-ingress/e2e-shared-a-w2-8dd4a3-openshift-default-856bd4c5f8-24xmz"
Sep 10 19:33:25.445823 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:25.445798    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/60e22eaa-6ec6-4298-bfcd-3f42bcce4117-istio-podinfo\") pod \"e2e-shared-a-w2-8dd4a3-openshift-default-856bd4c5f8-24xmz\" (UID: \"60e22eaa-6ec6-4298-bfcd-3f42bcce4117\") " pod="openshift-ingress/e2e-shared-a-w2-8dd4a3-openshift-default-856bd4c5f8-24xmz"
Sep 10 19:33:25.445858 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:25.445844    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/60e22eaa-6ec6-4298-bfcd-3f42bcce4117-workload-socket\") pod \"e2e-shared-a-w2-8dd4a3-openshift-default-856bd4c5f8-24xmz\" (UID: \"60e22eaa-6ec6-4298-bfcd-3f42bcce4117\") " pod="openshift-ingress/e2e-shared-a-w2-8dd4a3-openshift-default-856bd4c5f8-24xmz"
Sep 10 19:33:25.445900 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:25.445867    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dkvqw\" (UniqueName: \"kubernetes.io/projected/60e22eaa-6ec6-4298-bfcd-3f42bcce4117-kube-api-access-dkvqw\") pod \"e2e-shared-a-w2-8dd4a3-openshift-default-856bd4c5f8-24xmz\" (UID: \"60e22eaa-6ec6-4298-bfcd-3f42bcce4117\") " pod="openshift-ingress/e2e-shared-a-w2-8dd4a3-openshift-default-856bd4c5f8-24xmz"
Sep 10 19:33:25.477752 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:25.477705    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:33:25.546899 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:25.546857    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/60e22eaa-6ec6-4298-bfcd-3f42bcce4117-istio-podinfo\") pod \"e2e-shared-a-w2-8dd4a3-openshift-default-856bd4c5f8-24xmz\" (UID: \"60e22eaa-6ec6-4298-bfcd-3f42bcce4117\") " pod="openshift-ingress/e2e-shared-a-w2-8dd4a3-openshift-default-856bd4c5f8-24xmz"
Sep 10 19:33:25.546899 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:25.546899    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/60e22eaa-6ec6-4298-bfcd-3f42bcce4117-workload-socket\") pod \"e2e-shared-a-w2-8dd4a3-openshift-default-856bd4c5f8-24xmz\" (UID: \"60e22eaa-6ec6-4298-bfcd-3f42bcce4117\") " pod="openshift-ingress/e2e-shared-a-w2-8dd4a3-openshift-default-856bd4c5f8-24xmz"
Sep 10 19:33:25.547170 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:25.546926    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dkvqw\" (UniqueName: \"kubernetes.io/projected/60e22eaa-6ec6-4298-bfcd-3f42bcce4117-kube-api-access-dkvqw\") pod \"e2e-shared-a-w2-8dd4a3-openshift-default-856bd4c5f8-24xmz\" (UID: \"60e22eaa-6ec6-4298-bfcd-3f42bcce4117\") " pod="openshift-ingress/e2e-shared-a-w2-8dd4a3-openshift-default-856bd4c5f8-24xmz"
Sep 10 19:33:25.547170 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:25.546969    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/60e22eaa-6ec6-4298-bfcd-3f42bcce4117-istio-data\") pod \"e2e-shared-a-w2-8dd4a3-openshift-default-856bd4c5f8-24xmz\" (UID: \"60e22eaa-6ec6-4298-bfcd-3f42bcce4117\") " pod="openshift-ingress/e2e-shared-a-w2-8dd4a3-openshift-default-856bd4c5f8-24xmz"
Sep 10 19:33:25.547170 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:25.547002    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/60e22eaa-6ec6-4298-bfcd-3f42bcce4117-credential-socket\") pod \"e2e-shared-a-w2-8dd4a3-openshift-default-856bd4c5f8-24xmz\" (UID: \"60e22eaa-6ec6-4298-bfcd-3f42bcce4117\") " pod="openshift-ingress/e2e-shared-a-w2-8dd4a3-openshift-default-856bd4c5f8-24xmz"
Sep 10 19:33:25.547333 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:25.547210    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/60e22eaa-6ec6-4298-bfcd-3f42bcce4117-istiod-ca-cert\") pod \"e2e-shared-a-w2-8dd4a3-openshift-default-856bd4c5f8-24xmz\" (UID: \"60e22eaa-6ec6-4298-bfcd-3f42bcce4117\") " pod="openshift-ingress/e2e-shared-a-w2-8dd4a3-openshift-default-856bd4c5f8-24xmz"
Sep 10 19:33:25.547333 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:25.547290    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/60e22eaa-6ec6-4298-bfcd-3f42bcce4117-workload-certs\") pod \"e2e-shared-a-w2-8dd4a3-openshift-default-856bd4c5f8-24xmz\" (UID: \"60e22eaa-6ec6-4298-bfcd-3f42bcce4117\") " pod="openshift-ingress/e2e-shared-a-w2-8dd4a3-openshift-default-856bd4c5f8-24xmz"
Sep 10 19:33:25.547471 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:25.547331    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/60e22eaa-6ec6-4298-bfcd-3f42bcce4117-istio-token\") pod \"e2e-shared-a-w2-8dd4a3-openshift-default-856bd4c5f8-24xmz\" (UID: \"60e22eaa-6ec6-4298-bfcd-3f42bcce4117\") " pod="openshift-ingress/e2e-shared-a-w2-8dd4a3-openshift-default-856bd4c5f8-24xmz"
Sep 10 19:33:25.547471 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:25.547379    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/60e22eaa-6ec6-4298-bfcd-3f42bcce4117-istio-data\") pod \"e2e-shared-a-w2-8dd4a3-openshift-default-856bd4c5f8-24xmz\" (UID: \"60e22eaa-6ec6-4298-bfcd-3f42bcce4117\") " pod="openshift-ingress/e2e-shared-a-w2-8dd4a3-openshift-default-856bd4c5f8-24xmz"
Sep 10 19:33:25.547471 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:25.547407    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/60e22eaa-6ec6-4298-bfcd-3f42bcce4117-istio-envoy\") pod \"e2e-shared-a-w2-8dd4a3-openshift-default-856bd4c5f8-24xmz\" (UID: \"60e22eaa-6ec6-4298-bfcd-3f42bcce4117\") " pod="openshift-ingress/e2e-shared-a-w2-8dd4a3-openshift-default-856bd4c5f8-24xmz"
Sep 10 19:33:25.547663 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:25.547638    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/60e22eaa-6ec6-4298-bfcd-3f42bcce4117-workload-certs\") pod \"e2e-shared-a-w2-8dd4a3-openshift-default-856bd4c5f8-24xmz\" (UID: \"60e22eaa-6ec6-4298-bfcd-3f42bcce4117\") " pod="openshift-ingress/e2e-shared-a-w2-8dd4a3-openshift-default-856bd4c5f8-24xmz"
Sep 10 19:33:25.547726 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:25.547703    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/60e22eaa-6ec6-4298-bfcd-3f42bcce4117-credential-socket\") pod \"e2e-shared-a-w2-8dd4a3-openshift-default-856bd4c5f8-24xmz\" (UID: \"60e22eaa-6ec6-4298-bfcd-3f42bcce4117\") " pod="openshift-ingress/e2e-shared-a-w2-8dd4a3-openshift-default-856bd4c5f8-24xmz"
Sep 10 19:33:25.547783 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:25.547733    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/60e22eaa-6ec6-4298-bfcd-3f42bcce4117-workload-socket\") pod \"e2e-shared-a-w2-8dd4a3-openshift-default-856bd4c5f8-24xmz\" (UID: \"60e22eaa-6ec6-4298-bfcd-3f42bcce4117\") " pod="openshift-ingress/e2e-shared-a-w2-8dd4a3-openshift-default-856bd4c5f8-24xmz"
Sep 10 19:33:25.548157 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:25.548130    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/60e22eaa-6ec6-4298-bfcd-3f42bcce4117-istiod-ca-cert\") pod \"e2e-shared-a-w2-8dd4a3-openshift-default-856bd4c5f8-24xmz\" (UID: \"60e22eaa-6ec6-4298-bfcd-3f42bcce4117\") " pod="openshift-ingress/e2e-shared-a-w2-8dd4a3-openshift-default-856bd4c5f8-24xmz"
Sep 10 19:33:25.549715 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:25.549698    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/60e22eaa-6ec6-4298-bfcd-3f42bcce4117-istio-podinfo\") pod \"e2e-shared-a-w2-8dd4a3-openshift-default-856bd4c5f8-24xmz\" (UID: \"60e22eaa-6ec6-4298-bfcd-3f42bcce4117\") " pod="openshift-ingress/e2e-shared-a-w2-8dd4a3-openshift-default-856bd4c5f8-24xmz"
Sep 10 19:33:25.549890 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:25.549870    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/60e22eaa-6ec6-4298-bfcd-3f42bcce4117-istio-envoy\") pod \"e2e-shared-a-w2-8dd4a3-openshift-default-856bd4c5f8-24xmz\" (UID: \"60e22eaa-6ec6-4298-bfcd-3f42bcce4117\") " pod="openshift-ingress/e2e-shared-a-w2-8dd4a3-openshift-default-856bd4c5f8-24xmz"
Sep 10 19:33:25.555587 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:25.555560    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dkvqw\" (UniqueName: \"kubernetes.io/projected/60e22eaa-6ec6-4298-bfcd-3f42bcce4117-kube-api-access-dkvqw\") pod \"e2e-shared-a-w2-8dd4a3-openshift-default-856bd4c5f8-24xmz\" (UID: \"60e22eaa-6ec6-4298-bfcd-3f42bcce4117\") " pod="openshift-ingress/e2e-shared-a-w2-8dd4a3-openshift-default-856bd4c5f8-24xmz"
Sep 10 19:33:25.555692 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:25.555630    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/60e22eaa-6ec6-4298-bfcd-3f42bcce4117-istio-token\") pod \"e2e-shared-a-w2-8dd4a3-openshift-default-856bd4c5f8-24xmz\" (UID: \"60e22eaa-6ec6-4298-bfcd-3f42bcce4117\") " pod="openshift-ingress/e2e-shared-a-w2-8dd4a3-openshift-default-856bd4c5f8-24xmz"
Sep 10 19:33:25.657852 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:25.657757    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-8dd4a3-openshift-default-856bd4c5f8-24xmz"
Sep 10 19:33:25.947890 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:25.947865    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-8dd4a3-openshift-default-856bd4c5f8-24xmz"]
Sep 10 19:33:25.950082 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:33:25.950051    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60e22eaa_6ec6_4298_bfcd_3f42bcce4117.slice/crio-ca82e3759cfb7f4fb4837b165f1eecefd06edb56ead600e3f0164fd4b9369cfd WatchSource:0}: Error finding container ca82e3759cfb7f4fb4837b165f1eecefd06edb56ead600e3f0164fd4b9369cfd: Status 404 returned error can't find the container with id ca82e3759cfb7f4fb4837b165f1eecefd06edb56ead600e3f0164fd4b9369cfd
Sep 10 19:33:25.952461 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:25.952421    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 19:33:25.952559 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:25.952526    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 19:33:25.952606 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:25.952568    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 19:33:26.578976 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:26.578940    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-8dd4a3-openshift-default-856bd4c5f8-24xmz" event={"ID":"60e22eaa-6ec6-4298-bfcd-3f42bcce4117","Type":"ContainerStarted","Data":"ff6dc3b967c2f1c6a58b4a055e5eb5867f5073af8ebbff11d723ff2ee70f10ab"}
Sep 10 19:33:26.579322 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:26.579000    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-8dd4a3-openshift-default-856bd4c5f8-24xmz" event={"ID":"60e22eaa-6ec6-4298-bfcd-3f42bcce4117","Type":"ContainerStarted","Data":"ca82e3759cfb7f4fb4837b165f1eecefd06edb56ead600e3f0164fd4b9369cfd"}
Sep 10 19:33:26.602055 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:26.602001    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w2-8dd4a3-openshift-default-856bd4c5f8-24xmz" podStartSLOduration=1.6019810589999999 podStartE2EDuration="1.601981059s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:33:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:33:26.599084703 +0000 UTC m=+1077.658912370" watchObservedRunningTime="2026-09-10 19:33:26.601981059 +0000 UTC m=+1077.661808730"
Sep 10 19:33:26.658890 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:26.658860    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-8dd4a3-openshift-default-856bd4c5f8-24xmz"
Sep 10 19:33:27.495478 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:27.495448    2575 patch_prober.go:28] interesting pod/e2e-shared-b-w3-cab4ee-openshift-default-945f9dd46-2dq66 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body=
Sep 10 19:33:27.495627 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:27.495499    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w3-cab4ee-openshift-default-945f9dd46-2dq66" podUID="61a281ba-e630-4357-bb77-64fe9085aadb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused"
Sep 10 19:33:27.525038 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:27.524980    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-cab4ee-5fc779cb6-f4nbr"
Sep 10 19:33:27.586620 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:27.586590    2575 generic.go:356] "Generic (PLEG): container finished" podID="61a281ba-e630-4357-bb77-64fe9085aadb" containerID="1b568bf9c31904d27f7b7f5c390a7e8e975d0e46edf68f14dc6ba98556945c46" exitCode=0
Sep 10 19:33:27.587049 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:27.586656    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-cab4ee-openshift-default-945f9dd46-2dq66" event={"ID":"61a281ba-e630-4357-bb77-64fe9085aadb","Type":"ContainerDied","Data":"1b568bf9c31904d27f7b7f5c390a7e8e975d0e46edf68f14dc6ba98556945c46"}
Sep 10 19:33:27.587161 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:27.587144    2575 scope.go:117] "RemoveContainer" containerID="1b568bf9c31904d27f7b7f5c390a7e8e975d0e46edf68f14dc6ba98556945c46"
Sep 10 19:33:27.588180 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:27.588152    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 19:33:27.588278 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:27.588218    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 19:33:27.588278 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:27.588250    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 19:33:27.663092 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:27.663065    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-8dd4a3-openshift-default-856bd4c5f8-24xmz"
Sep 10 19:33:28.480742 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:28.480706    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:33:28.577269 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:28.577237    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:33:28.594506 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:28.594465    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-cab4ee-openshift-default-945f9dd46-2dq66" event={"ID":"61a281ba-e630-4357-bb77-64fe9085aadb","Type":"ContainerStarted","Data":"ca8403791c2b818e31ce7cf8ca88e3379863d4d8c9f7d2a35546fd0feb811ca2"}
Sep 10 19:33:28.594901 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:28.594849    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-8dd4a3-openshift-default-856bd4c5f8-24xmz"
Sep 10 19:33:28.595741 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:28.595714    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-8dd4a3-openshift-default-856bd4c5f8-24xmz"
Sep 10 19:33:29.056887 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:29.056844    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-cab4ee-openshift-default-945f9dd46-2dq66"
Sep 10 19:33:29.057102 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:29.057078    2575 patch_prober.go:28] interesting pod/e2e-shared-b-w3-cab4ee-openshift-default-945f9dd46-2dq66 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body=
Sep 10 19:33:29.057181 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:29.057148    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-cab4ee-openshift-default-945f9dd46-2dq66" podUID="61a281ba-e630-4357-bb77-64fe9085aadb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused"
Sep 10 19:33:29.642501 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:29.642461    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:33:29.740927 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:29.740841    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:33:30.057105 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:30.057013    2575 patch_prober.go:28] interesting pod/e2e-shared-b-w3-cab4ee-openshift-default-945f9dd46-2dq66 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body=
Sep 10 19:33:30.057276 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:30.057092    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-cab4ee-openshift-default-945f9dd46-2dq66" podUID="61a281ba-e630-4357-bb77-64fe9085aadb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused"
Sep 10 19:33:30.075846 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:30.075807    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:33:30.391699 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:30.391614    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:33:30.492436 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:30.492404    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:33:30.581656 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:30.581622    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-8dd4a3-75545f76bb-pmdbk"]
Sep 10 19:33:30.586434 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:30.586416    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-8dd4a3-75545f76bb-pmdbk"
Sep 10 19:33:30.588578 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:30.588524    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w2-8dd4a3\""
Sep 10 19:33:30.588699 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:30.588683    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w2-8dd4a3-dockercfg-ggqwf\""
Sep 10 19:33:30.597733 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:30.597713    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-8dd4a3-75545f76bb-pmdbk"]
Sep 10 19:33:30.678462 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:30.678388    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-8dd4a3-6ff9bb8949-mbb6m"]
Sep 10 19:33:30.681873 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:30.681857    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-8dd4a3-6ff9bb8949-mbb6m"
Sep 10 19:33:30.692830 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:30.692806    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-smqcm\" (UniqueName: \"kubernetes.io/projected/0dc7c183-7da2-4dc6-8fea-b317f4f2553c-kube-api-access-smqcm\") pod \"payload-pre-processing-e2e-shared-a-w2-8dd4a3-75545f76bb-pmdbk\" (UID: \"0dc7c183-7da2-4dc6-8fea-b317f4f2553c\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-8dd4a3-75545f76bb-pmdbk"
Sep 10 19:33:30.692959 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:30.692862    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0dc7c183-7da2-4dc6-8fea-b317f4f2553c-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-8dd4a3-75545f76bb-pmdbk\" (UID: \"0dc7c183-7da2-4dc6-8fea-b317f4f2553c\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-8dd4a3-75545f76bb-pmdbk"
Sep 10 19:33:30.693818 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:30.693787    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-8dd4a3-6ff9bb8949-mbb6m"]
Sep 10 19:33:30.794091 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:30.794061    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0dc7c183-7da2-4dc6-8fea-b317f4f2553c-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-8dd4a3-75545f76bb-pmdbk\" (UID: \"0dc7c183-7da2-4dc6-8fea-b317f4f2553c\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-8dd4a3-75545f76bb-pmdbk"
Sep 10 19:33:30.794258 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:30.794109    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fdff22b9-2e1f-4e1e-b7af-d929036e4e82-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-8dd4a3-6ff9bb8949-mbb6m\" (UID: \"fdff22b9-2e1f-4e1e-b7af-d929036e4e82\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-8dd4a3-6ff9bb8949-mbb6m"
Sep 10 19:33:30.794258 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:30.794132    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hqcvf\" (UniqueName: \"kubernetes.io/projected/fdff22b9-2e1f-4e1e-b7af-d929036e4e82-kube-api-access-hqcvf\") pod \"payload-processing-e2e-shared-a-w2-8dd4a3-6ff9bb8949-mbb6m\" (UID: \"fdff22b9-2e1f-4e1e-b7af-d929036e4e82\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-8dd4a3-6ff9bb8949-mbb6m"
Sep 10 19:33:30.794258 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:30.794236    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-smqcm\" (UniqueName: \"kubernetes.io/projected/0dc7c183-7da2-4dc6-8fea-b317f4f2553c-kube-api-access-smqcm\") pod \"payload-pre-processing-e2e-shared-a-w2-8dd4a3-75545f76bb-pmdbk\" (UID: \"0dc7c183-7da2-4dc6-8fea-b317f4f2553c\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-8dd4a3-75545f76bb-pmdbk"
Sep 10 19:33:30.794805 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:30.794788    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0dc7c183-7da2-4dc6-8fea-b317f4f2553c-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-8dd4a3-75545f76bb-pmdbk\" (UID: \"0dc7c183-7da2-4dc6-8fea-b317f4f2553c\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-8dd4a3-75545f76bb-pmdbk"
Sep 10 19:33:30.802254 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:30.802228    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-smqcm\" (UniqueName: \"kubernetes.io/projected/0dc7c183-7da2-4dc6-8fea-b317f4f2553c-kube-api-access-smqcm\") pod \"payload-pre-processing-e2e-shared-a-w2-8dd4a3-75545f76bb-pmdbk\" (UID: \"0dc7c183-7da2-4dc6-8fea-b317f4f2553c\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-8dd4a3-75545f76bb-pmdbk"
Sep 10 19:33:30.894850 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:30.894819    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fdff22b9-2e1f-4e1e-b7af-d929036e4e82-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-8dd4a3-6ff9bb8949-mbb6m\" (UID: \"fdff22b9-2e1f-4e1e-b7af-d929036e4e82\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-8dd4a3-6ff9bb8949-mbb6m"
Sep 10 19:33:30.894991 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:30.894857    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hqcvf\" (UniqueName: \"kubernetes.io/projected/fdff22b9-2e1f-4e1e-b7af-d929036e4e82-kube-api-access-hqcvf\") pod \"payload-processing-e2e-shared-a-w2-8dd4a3-6ff9bb8949-mbb6m\" (UID: \"fdff22b9-2e1f-4e1e-b7af-d929036e4e82\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-8dd4a3-6ff9bb8949-mbb6m"
Sep 10 19:33:30.895509 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:30.895489    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fdff22b9-2e1f-4e1e-b7af-d929036e4e82-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-8dd4a3-6ff9bb8949-mbb6m\" (UID: \"fdff22b9-2e1f-4e1e-b7af-d929036e4e82\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-8dd4a3-6ff9bb8949-mbb6m"
Sep 10 19:33:30.897741 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:30.897724    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-8dd4a3-75545f76bb-pmdbk"
Sep 10 19:33:30.902800 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:30.902774    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hqcvf\" (UniqueName: \"kubernetes.io/projected/fdff22b9-2e1f-4e1e-b7af-d929036e4e82-kube-api-access-hqcvf\") pod \"payload-processing-e2e-shared-a-w2-8dd4a3-6ff9bb8949-mbb6m\" (UID: \"fdff22b9-2e1f-4e1e-b7af-d929036e4e82\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-8dd4a3-6ff9bb8949-mbb6m"
Sep 10 19:33:30.998369 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:30.997011    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-8dd4a3-6ff9bb8949-mbb6m"
Sep 10 19:33:31.056794 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:31.056760    2575 patch_prober.go:28] interesting pod/e2e-shared-b-w3-cab4ee-openshift-default-945f9dd46-2dq66 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body=
Sep 10 19:33:31.056961 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:31.056822    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-cab4ee-openshift-default-945f9dd46-2dq66" podUID="61a281ba-e630-4357-bb77-64fe9085aadb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused"
Sep 10 19:33:31.215632 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:31.215549    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-8dd4a3-75545f76bb-pmdbk"]
Sep 10 19:33:31.306908 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:31.306886    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-8dd4a3-6ff9bb8949-mbb6m"]
Sep 10 19:33:31.308772 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:33:31.308751    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfdff22b9_2e1f_4e1e_b7af_d929036e4e82.slice/crio-a8f6271e895438fcc56f823d6c00bd3b313a810c29ce11ef97d7fbbdde9a1f1c WatchSource:0}: Error finding container a8f6271e895438fcc56f823d6c00bd3b313a810c29ce11ef97d7fbbdde9a1f1c: Status 404 returned error can't find the container with id a8f6271e895438fcc56f823d6c00bd3b313a810c29ce11ef97d7fbbdde9a1f1c
Sep 10 19:33:31.611085 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:31.610990    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-8dd4a3-75545f76bb-pmdbk" event={"ID":"0dc7c183-7da2-4dc6-8fea-b317f4f2553c","Type":"ContainerStarted","Data":"5fab4b74e272fd32fa8daef3323c92208073af19230b0426fe50197aa6c714c0"}
Sep 10 19:33:31.611085 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:31.611048    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-8dd4a3-75545f76bb-pmdbk" event={"ID":"0dc7c183-7da2-4dc6-8fea-b317f4f2553c","Type":"ContainerStarted","Data":"5d59161382697312e9b6117c2d660373d42a31d34a88a551029102783acded93"}
Sep 10 19:33:31.611085 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:31.611078    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-8dd4a3-75545f76bb-pmdbk"
Sep 10 19:33:31.612660 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:31.612631    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-8dd4a3-6ff9bb8949-mbb6m" event={"ID":"fdff22b9-2e1f-4e1e-b7af-d929036e4e82","Type":"ContainerStarted","Data":"48c1f7e157fa199d0a3d3bfbd9f84dfae051c0cf7a2975448530d8892f3ad1eb"}
Sep 10 19:33:31.612793 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:31.612676    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-8dd4a3-6ff9bb8949-mbb6m" event={"ID":"fdff22b9-2e1f-4e1e-b7af-d929036e4e82","Type":"ContainerStarted","Data":"a8f6271e895438fcc56f823d6c00bd3b313a810c29ce11ef97d7fbbdde9a1f1c"}
Sep 10 19:33:31.612793 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:31.612698    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-8dd4a3-6ff9bb8949-mbb6m"
Sep 10 19:33:31.648818 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:31.648755    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-8dd4a3-75545f76bb-pmdbk" podStartSLOduration=1.64873727 podStartE2EDuration="1.64873727s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:33:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:33:31.630416363 +0000 UTC m=+1082.690244033" watchObservedRunningTime="2026-09-10 19:33:31.64873727 +0000 UTC m=+1082.708564940"
Sep 10 19:33:32.058010 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:32.057978    2575 patch_prober.go:28] interesting pod/e2e-shared-b-w3-cab4ee-openshift-default-945f9dd46-2dq66 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body=
Sep 10 19:33:32.058373 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:32.058031    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-cab4ee-openshift-default-945f9dd46-2dq66" podUID="61a281ba-e630-4357-bb77-64fe9085aadb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused"
Sep 10 19:33:33.048085 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:33.048056    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:33:33.057115 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:33.057091    2575 patch_prober.go:28] interesting pod/e2e-shared-b-w3-cab4ee-openshift-default-945f9dd46-2dq66 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body=
Sep 10 19:33:33.057173 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:33.057141    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-cab4ee-openshift-default-945f9dd46-2dq66" podUID="61a281ba-e630-4357-bb77-64fe9085aadb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused"
Sep 10 19:33:33.159448 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:33.159394    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-8dd4a3-6ff9bb8949-mbb6m" podStartSLOduration=3.159376294 podStartE2EDuration="3.159376294s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:33:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:33:31.650925697 +0000 UTC m=+1082.710753366" watchObservedRunningTime="2026-09-10 19:33:33.159376294 +0000 UTC m=+1084.219203962"
Sep 10 19:33:33.160461 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:33.160443    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qr7rl"]
Sep 10 19:33:34.061387 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:34.061354    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-cab4ee-openshift-default-945f9dd46-2dq66"
Sep 10 19:33:34.061767 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:34.061746    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-cab4ee-openshift-default-945f9dd46-2dq66"
Sep 10 19:33:34.062585 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:34.062564    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-cab4ee-openshift-default-945f9dd46-2dq66"
Sep 10 19:33:35.644253 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:35.644215    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:33:38.307434 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:38.307395    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:33:42.620217 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:42.620186    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-8dd4a3-75545f76bb-pmdbk"
Sep 10 19:33:42.620779 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:42.620253    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-8dd4a3-6ff9bb8949-mbb6m"
Sep 10 19:33:43.596075 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:43.596046    2575 patch_prober.go:28] interesting pod/e2e-shared-a-w2-8dd4a3-openshift-default-856bd4c5f8-24xmz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Sep 10 19:33:43.596249 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:43.596093    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w2-8dd4a3-openshift-default-856bd4c5f8-24xmz" podUID="60e22eaa-6ec6-4298-bfcd-3f42bcce4117" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Sep 10 19:33:43.684131 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:43.684104    2575 generic.go:356] "Generic (PLEG): container finished" podID="60e22eaa-6ec6-4298-bfcd-3f42bcce4117" containerID="ff6dc3b967c2f1c6a58b4a055e5eb5867f5073af8ebbff11d723ff2ee70f10ab" exitCode=0
Sep 10 19:33:43.684470 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:43.684176    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-8dd4a3-openshift-default-856bd4c5f8-24xmz" event={"ID":"60e22eaa-6ec6-4298-bfcd-3f42bcce4117","Type":"ContainerDied","Data":"ff6dc3b967c2f1c6a58b4a055e5eb5867f5073af8ebbff11d723ff2ee70f10ab"}
Sep 10 19:33:43.684525 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:43.684512    2575 scope.go:117] "RemoveContainer" containerID="ff6dc3b967c2f1c6a58b4a055e5eb5867f5073af8ebbff11d723ff2ee70f10ab"
Sep 10 19:33:43.685507 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:43.685478    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 19:33:43.685631 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:43.685579    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 19:33:43.685697 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:43.685626    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 19:33:44.691640 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:44.691606    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-8dd4a3-openshift-default-856bd4c5f8-24xmz" event={"ID":"60e22eaa-6ec6-4298-bfcd-3f42bcce4117","Type":"ContainerStarted","Data":"dc8ab877844a2e08a88f9dc563d21091d8c1eea9ef2954eb2240e8bfad587ae4"}
Sep 10 19:33:45.658232 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:45.658199    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-8dd4a3-openshift-default-856bd4c5f8-24xmz"
Sep 10 19:33:45.658430 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:45.658411    2575 patch_prober.go:28] interesting pod/e2e-shared-a-w2-8dd4a3-openshift-default-856bd4c5f8-24xmz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Sep 10 19:33:45.658494 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:45.658453    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-8dd4a3-openshift-default-856bd4c5f8-24xmz" podUID="60e22eaa-6ec6-4298-bfcd-3f42bcce4117" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Sep 10 19:33:46.658934 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:46.658901    2575 patch_prober.go:28] interesting pod/e2e-shared-a-w2-8dd4a3-openshift-default-856bd4c5f8-24xmz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Sep 10 19:33:46.659293 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:46.658953    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-8dd4a3-openshift-default-856bd4c5f8-24xmz" podUID="60e22eaa-6ec6-4298-bfcd-3f42bcce4117" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Sep 10 19:33:47.658160 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:47.658120    2575 patch_prober.go:28] interesting pod/e2e-shared-a-w2-8dd4a3-openshift-default-856bd4c5f8-24xmz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Sep 10 19:33:47.658361 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:47.658180    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-8dd4a3-openshift-default-856bd4c5f8-24xmz" podUID="60e22eaa-6ec6-4298-bfcd-3f42bcce4117" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Sep 10 19:33:48.658277 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:48.658240    2575 patch_prober.go:28] interesting pod/e2e-shared-a-w2-8dd4a3-openshift-default-856bd4c5f8-24xmz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Sep 10 19:33:48.658704 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:48.658300    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-8dd4a3-openshift-default-856bd4c5f8-24xmz" podUID="60e22eaa-6ec6-4298-bfcd-3f42bcce4117" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Sep 10 19:33:49.662590 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:49.662564    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-8dd4a3-openshift-default-856bd4c5f8-24xmz"
Sep 10 19:33:49.663051 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:49.662907    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-8dd4a3-openshift-default-856bd4c5f8-24xmz"
Sep 10 19:33:49.663627 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:49.663610    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-8dd4a3-openshift-default-856bd4c5f8-24xmz"
Sep 10 19:33:50.846957 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:50.846916    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:33:53.402133 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:53.402099    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:33:53.771045 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:53.771001    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:33:54.083995 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:54.083915    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:33:54.458469 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:54.458432    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:33:56.667586 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:56.667556    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:33:56.765193 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:56.765156    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:33:57.839160 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:57.839127    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:33:57.939624 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:57.939587    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:33:58.005322 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:58.005293    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:33:58.561679 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:58.561651    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:33:58.665564 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:33:58.665534    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:02.055502 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:02.055459    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qr7rl"]
Sep 10 19:34:18.308820 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:18.308778    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:21.612918 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:21.612878    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:22.261870 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:22.261841    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qr7rl"]
Sep 10 19:34:22.337394 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:22.337354    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7455-514839"]
Sep 10 19:34:22.342188 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:22.342163    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7455-514839"
Sep 10 19:34:22.344254 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:22.344232    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-8vfd9\""
Sep 10 19:34:22.349021 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:22.348994    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7455-514839"]
Sep 10 19:34:22.439685 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:22.439658    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cslmg\" (UniqueName: \"kubernetes.io/projected/17693f58-c507-4301-89c1-07487aa965ac-kube-api-access-cslmg\") pod \"test-curl-7455-514839\" (UID: \"17693f58-c507-4301-89c1-07487aa965ac\") " pod="opendatahub/test-curl-7455-514839"
Sep 10 19:34:22.541271 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:22.541191    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cslmg\" (UniqueName: \"kubernetes.io/projected/17693f58-c507-4301-89c1-07487aa965ac-kube-api-access-cslmg\") pod \"test-curl-7455-514839\" (UID: \"17693f58-c507-4301-89c1-07487aa965ac\") " pod="opendatahub/test-curl-7455-514839"
Sep 10 19:34:22.549658 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:22.549635    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cslmg\" (UniqueName: \"kubernetes.io/projected/17693f58-c507-4301-89c1-07487aa965ac-kube-api-access-cslmg\") pod \"test-curl-7455-514839\" (UID: \"17693f58-c507-4301-89c1-07487aa965ac\") " pod="opendatahub/test-curl-7455-514839"
Sep 10 19:34:22.654326 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:22.654289    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7455-514839"
Sep 10 19:34:22.934786 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:22.934762    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7455-514839"]
Sep 10 19:34:22.937313 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:34:22.937286    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod17693f58_c507_4301_89c1_07487aa965ac.slice/crio-ee1c8aec864b9f411e6e343dd1a15de90a630c88a3eafb111f57b798a74c6ea0 WatchSource:0}: Error finding container ee1c8aec864b9f411e6e343dd1a15de90a630c88a3eafb111f57b798a74c6ea0: Status 404 returned error can't find the container with id ee1c8aec864b9f411e6e343dd1a15de90a630c88a3eafb111f57b798a74c6ea0
Sep 10 19:34:22.939712 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:22.939684    2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 19:34:23.889767 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:23.889735    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7455-514839" event={"ID":"17693f58-c507-4301-89c1-07487aa965ac","Type":"ContainerStarted","Data":"ee1c8aec864b9f411e6e343dd1a15de90a630c88a3eafb111f57b798a74c6ea0"}
Sep 10 19:34:24.895978 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:24.895949    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7455-514839" event={"ID":"17693f58-c507-4301-89c1-07487aa965ac","Type":"ContainerStarted","Data":"ca657870383f342bd9960a0d047164f729f0ebf9a819f90f86c6916317a5500c"}
Sep 10 19:34:24.910106 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:24.910050    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7455-514839" podStartSLOduration=1.112178129 podStartE2EDuration="2.910031674s" totalImagesPullingTime="1.797853545s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:34:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:34:24.909467206 +0000 UTC m=+1135.969294874" watchObservedRunningTime="2026-09-10 19:34:24.910031674 +0000 UTC m=+1135.969859342"
Sep 10 19:34:25.491823 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:25.491793    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7455-514839"]
Sep 10 19:34:25.494010 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:25.493985    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7455-514839"]
Sep 10 19:34:25.607781 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:25.607754    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7455-c498dc"]
Sep 10 19:34:25.608140 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:25.608126    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="17693f58-c507-4301-89c1-07487aa965ac" containerName="test-curl-7455-514839"
Sep 10 19:34:25.608187 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:25.608141    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="17693f58-c507-4301-89c1-07487aa965ac" containerName="test-curl-7455-514839"
Sep 10 19:34:25.608231 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:25.608222    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="17693f58-c507-4301-89c1-07487aa965ac" containerName="test-curl-7455-514839"
Sep 10 19:34:25.611329 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:25.611313    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7455-c498dc"
Sep 10 19:34:25.614949 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:25.614925    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7455-c498dc"]
Sep 10 19:34:25.668582 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:25.668556    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gxml8\" (UniqueName: \"kubernetes.io/projected/6394aa77-b492-405a-885f-9c1c13a77fa7-kube-api-access-gxml8\") pod \"test-curl-7455-c498dc\" (UID: \"6394aa77-b492-405a-885f-9c1c13a77fa7\") " pod="opendatahub/test-curl-7455-c498dc"
Sep 10 19:34:25.769172 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:25.769104    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gxml8\" (UniqueName: \"kubernetes.io/projected/6394aa77-b492-405a-885f-9c1c13a77fa7-kube-api-access-gxml8\") pod \"test-curl-7455-c498dc\" (UID: \"6394aa77-b492-405a-885f-9c1c13a77fa7\") " pod="opendatahub/test-curl-7455-c498dc"
Sep 10 19:34:25.777510 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:25.777410    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gxml8\" (UniqueName: \"kubernetes.io/projected/6394aa77-b492-405a-885f-9c1c13a77fa7-kube-api-access-gxml8\") pod \"test-curl-7455-c498dc\" (UID: \"6394aa77-b492-405a-885f-9c1c13a77fa7\") " pod="opendatahub/test-curl-7455-c498dc"
Sep 10 19:34:25.900767 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:34:25.900744    2575 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7455-514839" container="test-curl-7455-514839"
Sep 10 19:34:25.921278 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:25.921260    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7455-c498dc"
Sep 10 19:34:26.191397 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:26.191330    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7455-c498dc"]
Sep 10 19:34:26.193475 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:34:26.193446    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6394aa77_b492_405a_885f_9c1c13a77fa7.slice/crio-9731bca2f27b99096b8c68636cbc0901dbb37fddd5118f26170ba3b250878fd5 WatchSource:0}: Error finding container 9731bca2f27b99096b8c68636cbc0901dbb37fddd5118f26170ba3b250878fd5: Status 404 returned error can't find the container with id 9731bca2f27b99096b8c68636cbc0901dbb37fddd5118f26170ba3b250878fd5
Sep 10 19:34:26.907893 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:26.907857    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7455-c498dc" event={"ID":"6394aa77-b492-405a-885f-9c1c13a77fa7","Type":"ContainerStarted","Data":"66270b2614582748f8a664f0066f40a1c43c56b36d280c93ec6000c1de574987"}
Sep 10 19:34:26.908279 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:26.907901    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7455-c498dc" event={"ID":"6394aa77-b492-405a-885f-9c1c13a77fa7","Type":"ContainerStarted","Data":"9731bca2f27b99096b8c68636cbc0901dbb37fddd5118f26170ba3b250878fd5"}
Sep 10 19:34:26.908279 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:26.907930    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7455-514839" podUID="17693f58-c507-4301-89c1-07487aa965ac" containerName="test-curl-7455-514839" containerID="cri-o://ca657870383f342bd9960a0d047164f729f0ebf9a819f90f86c6916317a5500c" gracePeriod=2
Sep 10 19:34:26.909976 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:34:26.909952    2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7455-514839\" is forbidden: User \"system:node:ip-10-0-129-114.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-129-114.ec2.internal' and this object" podUID="17693f58-c507-4301-89c1-07487aa965ac" pod="opendatahub/test-curl-7455-514839"
Sep 10 19:34:26.927120 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:34:26.927098    2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7455-514839\" is forbidden: User \"system:node:ip-10-0-129-114.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-129-114.ec2.internal' and this object" podUID="17693f58-c507-4301-89c1-07487aa965ac" pod="opendatahub/test-curl-7455-514839"
Sep 10 19:34:26.927673 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:26.927637    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7455-c498dc" podStartSLOduration=1.927624256 podStartE2EDuration="1.927624256s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:34:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:34:26.925531128 +0000 UTC m=+1137.985358796" watchObservedRunningTime="2026-09-10 19:34:26.927624256 +0000 UTC m=+1137.987451962"
Sep 10 19:34:27.522200 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:27.522170    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7455-c498dc"]
Sep 10 19:34:27.527081 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:27.527048    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7455-c498dc"]
Sep 10 19:34:27.640197 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:27.640132    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7455-966741"]
Sep 10 19:34:27.640537 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:27.640524    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6394aa77-b492-405a-885f-9c1c13a77fa7" containerName="test-curl-7455-c498dc"
Sep 10 19:34:27.640587 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:27.640540    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="6394aa77-b492-405a-885f-9c1c13a77fa7" containerName="test-curl-7455-c498dc"
Sep 10 19:34:27.640634 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:27.640624    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="6394aa77-b492-405a-885f-9c1c13a77fa7" containerName="test-curl-7455-c498dc"
Sep 10 19:34:27.643683 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:27.643666    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7455-966741"
Sep 10 19:34:27.650741 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:27.650714    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7455-966741"]
Sep 10 19:34:27.685795 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:27.685772    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xm74j\" (UniqueName: \"kubernetes.io/projected/8af5b0c5-dd53-402d-bb0e-3c4d5da2fb05-kube-api-access-xm74j\") pod \"test-curl-7455-966741\" (UID: \"8af5b0c5-dd53-402d-bb0e-3c4d5da2fb05\") " pod="opendatahub/test-curl-7455-966741"
Sep 10 19:34:27.786870 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:27.786844    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xm74j\" (UniqueName: \"kubernetes.io/projected/8af5b0c5-dd53-402d-bb0e-3c4d5da2fb05-kube-api-access-xm74j\") pod \"test-curl-7455-966741\" (UID: \"8af5b0c5-dd53-402d-bb0e-3c4d5da2fb05\") " pod="opendatahub/test-curl-7455-966741"
Sep 10 19:34:27.800219 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:27.800200    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xm74j\" (UniqueName: \"kubernetes.io/projected/8af5b0c5-dd53-402d-bb0e-3c4d5da2fb05-kube-api-access-xm74j\") pod \"test-curl-7455-966741\" (UID: \"8af5b0c5-dd53-402d-bb0e-3c4d5da2fb05\") " pod="opendatahub/test-curl-7455-966741"
Sep 10 19:34:27.913222 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:34:27.913166    2575 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7455-c498dc" container="test-curl-7455-c498dc"
Sep 10 19:34:27.954061 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:27.954030    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7455-966741"
Sep 10 19:34:28.222487 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:28.222458    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7455-966741"]
Sep 10 19:34:28.226138 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:34:28.226105    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8af5b0c5_dd53_402d_bb0e_3c4d5da2fb05.slice/crio-bf22c1339dce2d5000385126b03bd6ae05fbca65b361c3cecd12ee2defcd1199 WatchSource:0}: Error finding container bf22c1339dce2d5000385126b03bd6ae05fbca65b361c3cecd12ee2defcd1199: Status 404 returned error can't find the container with id bf22c1339dce2d5000385126b03bd6ae05fbca65b361c3cecd12ee2defcd1199
Sep 10 19:34:28.920659 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:28.920622    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7455-966741" event={"ID":"8af5b0c5-dd53-402d-bb0e-3c4d5da2fb05","Type":"ContainerStarted","Data":"c9a8058160a72348118d1d8b180f67e334f482c337542782799069e270246aac"}
Sep 10 19:34:28.921053 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:28.920672    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7455-966741" event={"ID":"8af5b0c5-dd53-402d-bb0e-3c4d5da2fb05","Type":"ContainerStarted","Data":"bf22c1339dce2d5000385126b03bd6ae05fbca65b361c3cecd12ee2defcd1199"}
Sep 10 19:34:28.921053 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:28.920730    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7455-c498dc" podUID="6394aa77-b492-405a-885f-9c1c13a77fa7" containerName="test-curl-7455-c498dc" containerID="cri-o://66270b2614582748f8a664f0066f40a1c43c56b36d280c93ec6000c1de574987" gracePeriod=2
Sep 10 19:34:28.938105 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:28.937908    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7455-966741" podStartSLOduration=1.937888278 podStartE2EDuration="1.937888278s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:34:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:34:28.936808258 +0000 UTC m=+1139.996635925" watchObservedRunningTime="2026-09-10 19:34:28.937888278 +0000 UTC m=+1139.997715949"
Sep 10 19:34:28.938732 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:34:28.938701    2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7455-c498dc\" is forbidden: User \"system:node:ip-10-0-129-114.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-129-114.ec2.internal' and this object" podUID="6394aa77-b492-405a-885f-9c1c13a77fa7" pod="opendatahub/test-curl-7455-c498dc"
Sep 10 19:34:28.940456 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:34:28.940432    2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7455-c498dc\" is forbidden: User \"system:node:ip-10-0-129-114.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-129-114.ec2.internal' and this object" podUID="6394aa77-b492-405a-885f-9c1c13a77fa7" pod="opendatahub/test-curl-7455-c498dc"
Sep 10 19:34:29.067780 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:29.067758    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7455-514839"
Sep 10 19:34:29.069929 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:34:29.069905    2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7455-c498dc\" is forbidden: User \"system:node:ip-10-0-129-114.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-129-114.ec2.internal' and this object" podUID="6394aa77-b492-405a-885f-9c1c13a77fa7" pod="opendatahub/test-curl-7455-c498dc"
Sep 10 19:34:29.204231 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:29.204183    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cslmg\" (UniqueName: \"kubernetes.io/projected/17693f58-c507-4301-89c1-07487aa965ac-kube-api-access-cslmg\") pod \"17693f58-c507-4301-89c1-07487aa965ac\" (UID: \"17693f58-c507-4301-89c1-07487aa965ac\") "
Sep 10 19:34:29.206214 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:29.206185    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/17693f58-c507-4301-89c1-07487aa965ac-kube-api-access-cslmg" (OuterVolumeSpecName: "kube-api-access-cslmg") pod "17693f58-c507-4301-89c1-07487aa965ac" (UID: "17693f58-c507-4301-89c1-07487aa965ac"). InnerVolumeSpecName "kube-api-access-cslmg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:34:29.305069 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:29.305029    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cslmg\" (UniqueName: \"kubernetes.io/projected/17693f58-c507-4301-89c1-07487aa965ac-kube-api-access-cslmg\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:34:29.383506 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:34:29.383476    2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7455-c498dc\" is forbidden: User \"system:node:ip-10-0-129-114.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-129-114.ec2.internal' and this object" podUID="6394aa77-b492-405a-885f-9c1c13a77fa7" pod="opendatahub/test-curl-7455-c498dc"
Sep 10 19:34:29.385306 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:29.385284    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="17693f58-c507-4301-89c1-07487aa965ac" path="/var/lib/kubelet/pods/17693f58-c507-4301-89c1-07487aa965ac/volumes"
Sep 10 19:34:29.544066 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:29.543980    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7455-966741"]
Sep 10 19:34:29.547235 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:29.547208    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7455-966741"]
Sep 10 19:34:29.662660 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:29.662635    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7455-77fa9d"]
Sep 10 19:34:29.663162 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:29.663143    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8af5b0c5-dd53-402d-bb0e-3c4d5da2fb05" containerName="test-curl-7455-966741"
Sep 10 19:34:29.663162 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:29.663164    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="8af5b0c5-dd53-402d-bb0e-3c4d5da2fb05" containerName="test-curl-7455-966741"
Sep 10 19:34:29.663313 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:29.663272    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="8af5b0c5-dd53-402d-bb0e-3c4d5da2fb05" containerName="test-curl-7455-966741"
Sep 10 19:34:29.666691 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:29.666675    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7455-77fa9d"
Sep 10 19:34:29.675443 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:29.675423    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7455-77fa9d"]
Sep 10 19:34:29.809207 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:29.809138    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-87z6n\" (UniqueName: \"kubernetes.io/projected/8631f4e7-5654-4c8f-9295-17cb8872dfa9-kube-api-access-87z6n\") pod \"test-curl-7455-77fa9d\" (UID: \"8631f4e7-5654-4c8f-9295-17cb8872dfa9\") " pod="opendatahub/test-curl-7455-77fa9d"
Sep 10 19:34:29.909906 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:29.909882    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-87z6n\" (UniqueName: \"kubernetes.io/projected/8631f4e7-5654-4c8f-9295-17cb8872dfa9-kube-api-access-87z6n\") pod \"test-curl-7455-77fa9d\" (UID: \"8631f4e7-5654-4c8f-9295-17cb8872dfa9\") " pod="opendatahub/test-curl-7455-77fa9d"
Sep 10 19:34:29.923724 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:29.923698    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-87z6n\" (UniqueName: \"kubernetes.io/projected/8631f4e7-5654-4c8f-9295-17cb8872dfa9-kube-api-access-87z6n\") pod \"test-curl-7455-77fa9d\" (UID: \"8631f4e7-5654-4c8f-9295-17cb8872dfa9\") " pod="opendatahub/test-curl-7455-77fa9d"
Sep 10 19:34:29.927073 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:29.927048    2575 generic.go:356] "Generic (PLEG): container finished" podID="17693f58-c507-4301-89c1-07487aa965ac" containerID="ca657870383f342bd9960a0d047164f729f0ebf9a819f90f86c6916317a5500c" exitCode=137
Sep 10 19:34:29.927191 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:29.927090    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7455-514839"
Sep 10 19:34:29.927191 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:29.927141    2575 scope.go:117] "RemoveContainer" containerID="ca657870383f342bd9960a0d047164f729f0ebf9a819f90f86c6916317a5500c"
Sep 10 19:34:29.927388 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:34:29.927370    2575 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7455-966741" container="test-curl-7455-966741"
Sep 10 19:34:29.937883 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:29.937862    2575 scope.go:117] "RemoveContainer" containerID="ca657870383f342bd9960a0d047164f729f0ebf9a819f90f86c6916317a5500c"
Sep 10 19:34:29.938168 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:34:29.938151    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ca657870383f342bd9960a0d047164f729f0ebf9a819f90f86c6916317a5500c\": container with ID starting with ca657870383f342bd9960a0d047164f729f0ebf9a819f90f86c6916317a5500c not found: ID does not exist" containerID="ca657870383f342bd9960a0d047164f729f0ebf9a819f90f86c6916317a5500c"
Sep 10 19:34:29.938241 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:29.938177    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca657870383f342bd9960a0d047164f729f0ebf9a819f90f86c6916317a5500c"} err="failed to get container status \"ca657870383f342bd9960a0d047164f729f0ebf9a819f90f86c6916317a5500c\": rpc error: code = NotFound desc = could not find container \"ca657870383f342bd9960a0d047164f729f0ebf9a819f90f86c6916317a5500c\": container with ID starting with ca657870383f342bd9960a0d047164f729f0ebf9a819f90f86c6916317a5500c not found: ID does not exist"
Sep 10 19:34:29.978466 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:29.978419    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7455-77fa9d"
Sep 10 19:34:30.266028 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:30.265866    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7455-77fa9d"]
Sep 10 19:34:30.268856 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:34:30.268827    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8631f4e7_5654_4c8f_9295_17cb8872dfa9.slice/crio-9a1376c6b0ebdce6994764e628127758e9aba067605f89ded8d3cff38aa22c14 WatchSource:0}: Error finding container 9a1376c6b0ebdce6994764e628127758e9aba067605f89ded8d3cff38aa22c14: Status 404 returned error can't find the container with id 9a1376c6b0ebdce6994764e628127758e9aba067605f89ded8d3cff38aa22c14
Sep 10 19:34:30.934068 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:30.934038    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7455-77fa9d" event={"ID":"8631f4e7-5654-4c8f-9295-17cb8872dfa9","Type":"ContainerStarted","Data":"2250eefae4862d5c5a1a5b25ebdab44daf0085e48c46d5e71c9b84b618f271b4"}
Sep 10 19:34:30.934461 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:30.934084    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7455-77fa9d" event={"ID":"8631f4e7-5654-4c8f-9295-17cb8872dfa9","Type":"ContainerStarted","Data":"9a1376c6b0ebdce6994764e628127758e9aba067605f89ded8d3cff38aa22c14"}
Sep 10 19:34:30.935277 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:30.935243    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7455-966741" podUID="8af5b0c5-dd53-402d-bb0e-3c4d5da2fb05" containerName="test-curl-7455-966741" containerID="cri-o://c9a8058160a72348118d1d8b180f67e334f482c337542782799069e270246aac" gracePeriod=2
Sep 10 19:34:30.951744 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:30.951700    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7455-77fa9d" podStartSLOduration=1.951684544 podStartE2EDuration="1.951684544s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:34:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:34:30.950509856 +0000 UTC m=+1142.010337525" watchObservedRunningTime="2026-09-10 19:34:30.951684544 +0000 UTC m=+1142.011512213"
Sep 10 19:34:30.952194 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:34:30.952169    2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7455-966741\" is forbidden: User \"system:node:ip-10-0-129-114.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-129-114.ec2.internal' and this object" podUID="8af5b0c5-dd53-402d-bb0e-3c4d5da2fb05" pod="opendatahub/test-curl-7455-966741"
Sep 10 19:34:31.062687 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:31.062665    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7455-c498dc"
Sep 10 19:34:31.064638 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:34:31.064616    2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7455-966741\" is forbidden: User \"system:node:ip-10-0-129-114.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-129-114.ec2.internal' and this object" podUID="8af5b0c5-dd53-402d-bb0e-3c4d5da2fb05" pod="opendatahub/test-curl-7455-966741"
Sep 10 19:34:31.121462 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:31.121440    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gxml8\" (UniqueName: \"kubernetes.io/projected/6394aa77-b492-405a-885f-9c1c13a77fa7-kube-api-access-gxml8\") pod \"6394aa77-b492-405a-885f-9c1c13a77fa7\" (UID: \"6394aa77-b492-405a-885f-9c1c13a77fa7\") "
Sep 10 19:34:31.123362 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:31.123321    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6394aa77-b492-405a-885f-9c1c13a77fa7-kube-api-access-gxml8" (OuterVolumeSpecName: "kube-api-access-gxml8") pod "6394aa77-b492-405a-885f-9c1c13a77fa7" (UID: "6394aa77-b492-405a-885f-9c1c13a77fa7"). InnerVolumeSpecName "kube-api-access-gxml8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:34:31.222689 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:31.222666    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gxml8\" (UniqueName: \"kubernetes.io/projected/6394aa77-b492-405a-885f-9c1c13a77fa7-kube-api-access-gxml8\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:34:31.385187 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:31.385160    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6394aa77-b492-405a-885f-9c1c13a77fa7" path="/var/lib/kubelet/pods/6394aa77-b492-405a-885f-9c1c13a77fa7/volumes"
Sep 10 19:34:31.589534 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:31.589461    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7455-77fa9d"]
Sep 10 19:34:31.594223 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:31.594199    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7455-77fa9d"]
Sep 10 19:34:31.941641 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:31.941557    2575 generic.go:356] "Generic (PLEG): container finished" podID="6394aa77-b492-405a-885f-9c1c13a77fa7" containerID="66270b2614582748f8a664f0066f40a1c43c56b36d280c93ec6000c1de574987" exitCode=137
Sep 10 19:34:31.941641 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:31.941616    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7455-c498dc"
Sep 10 19:34:31.942085 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:31.941647    2575 scope.go:117] "RemoveContainer" containerID="66270b2614582748f8a664f0066f40a1c43c56b36d280c93ec6000c1de574987"
Sep 10 19:34:31.942085 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:34:31.941859    2575 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7455-77fa9d" container="test-curl-7455-77fa9d"
Sep 10 19:34:31.952280 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:31.952256    2575 scope.go:117] "RemoveContainer" containerID="66270b2614582748f8a664f0066f40a1c43c56b36d280c93ec6000c1de574987"
Sep 10 19:34:31.952560 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:34:31.952543    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"66270b2614582748f8a664f0066f40a1c43c56b36d280c93ec6000c1de574987\": container with ID starting with 66270b2614582748f8a664f0066f40a1c43c56b36d280c93ec6000c1de574987 not found: ID does not exist" containerID="66270b2614582748f8a664f0066f40a1c43c56b36d280c93ec6000c1de574987"
Sep 10 19:34:31.952666 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:31.952569    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"66270b2614582748f8a664f0066f40a1c43c56b36d280c93ec6000c1de574987"} err="failed to get container status \"66270b2614582748f8a664f0066f40a1c43c56b36d280c93ec6000c1de574987\": rpc error: code = NotFound desc = could not find container \"66270b2614582748f8a664f0066f40a1c43c56b36d280c93ec6000c1de574987\": container with ID starting with 66270b2614582748f8a664f0066f40a1c43c56b36d280c93ec6000c1de574987 not found: ID does not exist"
Sep 10 19:34:32.179834 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:32.179807    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:32.470662 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:32.470633    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-c615f98e-openshift-default-7fbbb5c8d-dggrv"]
Sep 10 19:34:32.471041 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:32.471027    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8631f4e7-5654-4c8f-9295-17cb8872dfa9" containerName="test-curl-7455-77fa9d"
Sep 10 19:34:32.471093 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:32.471043    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="8631f4e7-5654-4c8f-9295-17cb8872dfa9" containerName="test-curl-7455-77fa9d"
Sep 10 19:34:32.471126 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:32.471112    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="8631f4e7-5654-4c8f-9295-17cb8872dfa9" containerName="test-curl-7455-77fa9d"
Sep 10 19:34:32.475498 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:32.475478    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-c615f98e-openshift-default-7fbbb5c8d-dggrv"
Sep 10 19:34:32.477643 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:32.477625    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-c615f98e-openshift-default-dockercfg-8tmmt\""
Sep 10 19:34:32.490707 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:32.490685    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-c615f98e-openshift-default-7fbbb5c8d-dggrv"]
Sep 10 19:34:32.495577 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:34:32.495549    2575 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod17693f58_c507_4301_89c1_07487aa965ac.slice/crio-ee1c8aec864b9f411e6e343dd1a15de90a630c88a3eafb111f57b798a74c6ea0\": RecentStats: unable to find data in memory cache]"
Sep 10 19:34:32.514647 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:32.514619    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:32.533644 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:32.533619    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5f2b470f-b238-4321-b5de-ff03dcfd1b79-istio-token\") pod \"e2e-mt-c615f98e-openshift-default-7fbbb5c8d-dggrv\" (UID: \"5f2b470f-b238-4321-b5de-ff03dcfd1b79\") " pod="openshift-ingress/e2e-mt-c615f98e-openshift-default-7fbbb5c8d-dggrv"
Sep 10 19:34:32.533744 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:32.533656    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5f2b470f-b238-4321-b5de-ff03dcfd1b79-istio-envoy\") pod \"e2e-mt-c615f98e-openshift-default-7fbbb5c8d-dggrv\" (UID: \"5f2b470f-b238-4321-b5de-ff03dcfd1b79\") " pod="openshift-ingress/e2e-mt-c615f98e-openshift-default-7fbbb5c8d-dggrv"
Sep 10 19:34:32.533744 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:32.533715    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5f2b470f-b238-4321-b5de-ff03dcfd1b79-istio-data\") pod \"e2e-mt-c615f98e-openshift-default-7fbbb5c8d-dggrv\" (UID: \"5f2b470f-b238-4321-b5de-ff03dcfd1b79\") " pod="openshift-ingress/e2e-mt-c615f98e-openshift-default-7fbbb5c8d-dggrv"
Sep 10 19:34:32.533862 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:32.533749    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f2b470f-b238-4321-b5de-ff03dcfd1b79-credential-socket\") pod \"e2e-mt-c615f98e-openshift-default-7fbbb5c8d-dggrv\" (UID: \"5f2b470f-b238-4321-b5de-ff03dcfd1b79\") " pod="openshift-ingress/e2e-mt-c615f98e-openshift-default-7fbbb5c8d-dggrv"
Sep 10 19:34:32.533862 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:32.533859    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5f2b470f-b238-4321-b5de-ff03dcfd1b79-istiod-ca-cert\") pod \"e2e-mt-c615f98e-openshift-default-7fbbb5c8d-dggrv\" (UID: \"5f2b470f-b238-4321-b5de-ff03dcfd1b79\") " pod="openshift-ingress/e2e-mt-c615f98e-openshift-default-7fbbb5c8d-dggrv"
Sep 10 19:34:32.533979 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:32.533894    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f2b470f-b238-4321-b5de-ff03dcfd1b79-workload-socket\") pod \"e2e-mt-c615f98e-openshift-default-7fbbb5c8d-dggrv\" (UID: \"5f2b470f-b238-4321-b5de-ff03dcfd1b79\") " pod="openshift-ingress/e2e-mt-c615f98e-openshift-default-7fbbb5c8d-dggrv"
Sep 10 19:34:32.533979 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:32.533921    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bw7s7\" (UniqueName: \"kubernetes.io/projected/5f2b470f-b238-4321-b5de-ff03dcfd1b79-kube-api-access-bw7s7\") pod \"e2e-mt-c615f98e-openshift-default-7fbbb5c8d-dggrv\" (UID: \"5f2b470f-b238-4321-b5de-ff03dcfd1b79\") " pod="openshift-ingress/e2e-mt-c615f98e-openshift-default-7fbbb5c8d-dggrv"
Sep 10 19:34:32.533979 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:32.533938    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5f2b470f-b238-4321-b5de-ff03dcfd1b79-workload-certs\") pod \"e2e-mt-c615f98e-openshift-default-7fbbb5c8d-dggrv\" (UID: \"5f2b470f-b238-4321-b5de-ff03dcfd1b79\") " pod="openshift-ingress/e2e-mt-c615f98e-openshift-default-7fbbb5c8d-dggrv"
Sep 10 19:34:32.533979 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:32.533973    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5f2b470f-b238-4321-b5de-ff03dcfd1b79-istio-podinfo\") pod \"e2e-mt-c615f98e-openshift-default-7fbbb5c8d-dggrv\" (UID: \"5f2b470f-b238-4321-b5de-ff03dcfd1b79\") " pod="openshift-ingress/e2e-mt-c615f98e-openshift-default-7fbbb5c8d-dggrv"
Sep 10 19:34:32.634810 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:32.634772    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5f2b470f-b238-4321-b5de-ff03dcfd1b79-istio-token\") pod \"e2e-mt-c615f98e-openshift-default-7fbbb5c8d-dggrv\" (UID: \"5f2b470f-b238-4321-b5de-ff03dcfd1b79\") " pod="openshift-ingress/e2e-mt-c615f98e-openshift-default-7fbbb5c8d-dggrv"
Sep 10 19:34:32.634810 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:32.634818    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5f2b470f-b238-4321-b5de-ff03dcfd1b79-istio-envoy\") pod \"e2e-mt-c615f98e-openshift-default-7fbbb5c8d-dggrv\" (UID: \"5f2b470f-b238-4321-b5de-ff03dcfd1b79\") " pod="openshift-ingress/e2e-mt-c615f98e-openshift-default-7fbbb5c8d-dggrv"
Sep 10 19:34:32.635006 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:32.634934    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5f2b470f-b238-4321-b5de-ff03dcfd1b79-istio-data\") pod \"e2e-mt-c615f98e-openshift-default-7fbbb5c8d-dggrv\" (UID: \"5f2b470f-b238-4321-b5de-ff03dcfd1b79\") " pod="openshift-ingress/e2e-mt-c615f98e-openshift-default-7fbbb5c8d-dggrv"
Sep 10 19:34:32.635006 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:32.634980    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f2b470f-b238-4321-b5de-ff03dcfd1b79-credential-socket\") pod \"e2e-mt-c615f98e-openshift-default-7fbbb5c8d-dggrv\" (UID: \"5f2b470f-b238-4321-b5de-ff03dcfd1b79\") " pod="openshift-ingress/e2e-mt-c615f98e-openshift-default-7fbbb5c8d-dggrv"
Sep 10 19:34:32.635116 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:32.635020    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5f2b470f-b238-4321-b5de-ff03dcfd1b79-istiod-ca-cert\") pod \"e2e-mt-c615f98e-openshift-default-7fbbb5c8d-dggrv\" (UID: \"5f2b470f-b238-4321-b5de-ff03dcfd1b79\") " pod="openshift-ingress/e2e-mt-c615f98e-openshift-default-7fbbb5c8d-dggrv"
Sep 10 19:34:32.635116 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:32.635067    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f2b470f-b238-4321-b5de-ff03dcfd1b79-workload-socket\") pod \"e2e-mt-c615f98e-openshift-default-7fbbb5c8d-dggrv\" (UID: \"5f2b470f-b238-4321-b5de-ff03dcfd1b79\") " pod="openshift-ingress/e2e-mt-c615f98e-openshift-default-7fbbb5c8d-dggrv"
Sep 10 19:34:32.635116 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:32.635111    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bw7s7\" (UniqueName: \"kubernetes.io/projected/5f2b470f-b238-4321-b5de-ff03dcfd1b79-kube-api-access-bw7s7\") pod \"e2e-mt-c615f98e-openshift-default-7fbbb5c8d-dggrv\" (UID: \"5f2b470f-b238-4321-b5de-ff03dcfd1b79\") " pod="openshift-ingress/e2e-mt-c615f98e-openshift-default-7fbbb5c8d-dggrv"
Sep 10 19:34:32.635273 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:32.635132    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5f2b470f-b238-4321-b5de-ff03dcfd1b79-workload-certs\") pod \"e2e-mt-c615f98e-openshift-default-7fbbb5c8d-dggrv\" (UID: \"5f2b470f-b238-4321-b5de-ff03dcfd1b79\") " pod="openshift-ingress/e2e-mt-c615f98e-openshift-default-7fbbb5c8d-dggrv"
Sep 10 19:34:32.635273 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:32.635190    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5f2b470f-b238-4321-b5de-ff03dcfd1b79-istio-podinfo\") pod \"e2e-mt-c615f98e-openshift-default-7fbbb5c8d-dggrv\" (UID: \"5f2b470f-b238-4321-b5de-ff03dcfd1b79\") " pod="openshift-ingress/e2e-mt-c615f98e-openshift-default-7fbbb5c8d-dggrv"
Sep 10 19:34:32.635411 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:32.635331    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5f2b470f-b238-4321-b5de-ff03dcfd1b79-istio-data\") pod \"e2e-mt-c615f98e-openshift-default-7fbbb5c8d-dggrv\" (UID: \"5f2b470f-b238-4321-b5de-ff03dcfd1b79\") " pod="openshift-ingress/e2e-mt-c615f98e-openshift-default-7fbbb5c8d-dggrv"
Sep 10 19:34:32.635411 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:32.635392    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f2b470f-b238-4321-b5de-ff03dcfd1b79-credential-socket\") pod \"e2e-mt-c615f98e-openshift-default-7fbbb5c8d-dggrv\" (UID: \"5f2b470f-b238-4321-b5de-ff03dcfd1b79\") " pod="openshift-ingress/e2e-mt-c615f98e-openshift-default-7fbbb5c8d-dggrv"
Sep 10 19:34:32.635495 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:32.635445    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f2b470f-b238-4321-b5de-ff03dcfd1b79-workload-socket\") pod \"e2e-mt-c615f98e-openshift-default-7fbbb5c8d-dggrv\" (UID: \"5f2b470f-b238-4321-b5de-ff03dcfd1b79\") " pod="openshift-ingress/e2e-mt-c615f98e-openshift-default-7fbbb5c8d-dggrv"
Sep 10 19:34:32.635631 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:32.635609    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5f2b470f-b238-4321-b5de-ff03dcfd1b79-workload-certs\") pod \"e2e-mt-c615f98e-openshift-default-7fbbb5c8d-dggrv\" (UID: \"5f2b470f-b238-4321-b5de-ff03dcfd1b79\") " pod="openshift-ingress/e2e-mt-c615f98e-openshift-default-7fbbb5c8d-dggrv"
Sep 10 19:34:32.635722 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:32.635701    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5f2b470f-b238-4321-b5de-ff03dcfd1b79-istiod-ca-cert\") pod \"e2e-mt-c615f98e-openshift-default-7fbbb5c8d-dggrv\" (UID: \"5f2b470f-b238-4321-b5de-ff03dcfd1b79\") " pod="openshift-ingress/e2e-mt-c615f98e-openshift-default-7fbbb5c8d-dggrv"
Sep 10 19:34:32.636966 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:32.636949    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5f2b470f-b238-4321-b5de-ff03dcfd1b79-istio-envoy\") pod \"e2e-mt-c615f98e-openshift-default-7fbbb5c8d-dggrv\" (UID: \"5f2b470f-b238-4321-b5de-ff03dcfd1b79\") " pod="openshift-ingress/e2e-mt-c615f98e-openshift-default-7fbbb5c8d-dggrv"
Sep 10 19:34:32.637692 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:32.637672    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5f2b470f-b238-4321-b5de-ff03dcfd1b79-istio-podinfo\") pod \"e2e-mt-c615f98e-openshift-default-7fbbb5c8d-dggrv\" (UID: \"5f2b470f-b238-4321-b5de-ff03dcfd1b79\") " pod="openshift-ingress/e2e-mt-c615f98e-openshift-default-7fbbb5c8d-dggrv"
Sep 10 19:34:32.646567 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:32.646547    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bw7s7\" (UniqueName: \"kubernetes.io/projected/5f2b470f-b238-4321-b5de-ff03dcfd1b79-kube-api-access-bw7s7\") pod \"e2e-mt-c615f98e-openshift-default-7fbbb5c8d-dggrv\" (UID: \"5f2b470f-b238-4321-b5de-ff03dcfd1b79\") " pod="openshift-ingress/e2e-mt-c615f98e-openshift-default-7fbbb5c8d-dggrv"
Sep 10 19:34:32.655182 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:32.655162    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5f2b470f-b238-4321-b5de-ff03dcfd1b79-istio-token\") pod \"e2e-mt-c615f98e-openshift-default-7fbbb5c8d-dggrv\" (UID: \"5f2b470f-b238-4321-b5de-ff03dcfd1b79\") " pod="openshift-ingress/e2e-mt-c615f98e-openshift-default-7fbbb5c8d-dggrv"
Sep 10 19:34:32.785787 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:32.785725    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-c615f98e-openshift-default-7fbbb5c8d-dggrv"
Sep 10 19:34:32.947670 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:32.947644    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7455-77fa9d" podUID="8631f4e7-5654-4c8f-9295-17cb8872dfa9" containerName="test-curl-7455-77fa9d" containerID="cri-o://2250eefae4862d5c5a1a5b25ebdab44daf0085e48c46d5e71c9b84b618f271b4" gracePeriod=2
Sep 10 19:34:32.949895 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:34:32.949866    2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7455-77fa9d\" is forbidden: User \"system:node:ip-10-0-129-114.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-129-114.ec2.internal' and this object" podUID="8631f4e7-5654-4c8f-9295-17cb8872dfa9" pod="opendatahub/test-curl-7455-77fa9d"
Sep 10 19:34:33.061943 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:33.061916    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-c615f98e-openshift-default-7fbbb5c8d-dggrv"]
Sep 10 19:34:33.065049 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:34:33.065021    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f2b470f_b238_4321_b5de_ff03dcfd1b79.slice/crio-54ae713484fabff893f1a9f25195226668c02dda3f644fb7fdbc6b15b6fc11a8 WatchSource:0}: Error finding container 54ae713484fabff893f1a9f25195226668c02dda3f644fb7fdbc6b15b6fc11a8: Status 404 returned error can't find the container with id 54ae713484fabff893f1a9f25195226668c02dda3f644fb7fdbc6b15b6fc11a8
Sep 10 19:34:33.067698 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:33.067663    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 19:34:33.067800 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:33.067735    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 19:34:33.067800 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:33.067767    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 19:34:33.103890 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:33.103866    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7455-966741"
Sep 10 19:34:33.105521 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:34:33.105498    2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7455-77fa9d\" is forbidden: User \"system:node:ip-10-0-129-114.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-129-114.ec2.internal' and this object" podUID="8631f4e7-5654-4c8f-9295-17cb8872dfa9" pod="opendatahub/test-curl-7455-77fa9d"
Sep 10 19:34:33.120176 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:33.120144    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:33.239939 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:33.239906    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xm74j\" (UniqueName: \"kubernetes.io/projected/8af5b0c5-dd53-402d-bb0e-3c4d5da2fb05-kube-api-access-xm74j\") pod \"8af5b0c5-dd53-402d-bb0e-3c4d5da2fb05\" (UID: \"8af5b0c5-dd53-402d-bb0e-3c4d5da2fb05\") "
Sep 10 19:34:33.241927 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:33.241898    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8af5b0c5-dd53-402d-bb0e-3c4d5da2fb05-kube-api-access-xm74j" (OuterVolumeSpecName: "kube-api-access-xm74j") pod "8af5b0c5-dd53-402d-bb0e-3c4d5da2fb05" (UID: "8af5b0c5-dd53-402d-bb0e-3c4d5da2fb05"). InnerVolumeSpecName "kube-api-access-xm74j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:34:33.341184 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:33.341117    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xm74j\" (UniqueName: \"kubernetes.io/projected/8af5b0c5-dd53-402d-bb0e-3c4d5da2fb05-kube-api-access-xm74j\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:34:33.386043 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:33.386013    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8af5b0c5-dd53-402d-bb0e-3c4d5da2fb05" path="/var/lib/kubelet/pods/8af5b0c5-dd53-402d-bb0e-3c4d5da2fb05/volumes"
Sep 10 19:34:33.953970 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:33.953939    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-c615f98e-openshift-default-7fbbb5c8d-dggrv" event={"ID":"5f2b470f-b238-4321-b5de-ff03dcfd1b79","Type":"ContainerStarted","Data":"530ebf86ed555c60404d66090baefdafb1afb57b44f627d8431ad33adc8de4bf"}
Sep 10 19:34:33.954367 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:33.953999    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-c615f98e-openshift-default-7fbbb5c8d-dggrv" event={"ID":"5f2b470f-b238-4321-b5de-ff03dcfd1b79","Type":"ContainerStarted","Data":"54ae713484fabff893f1a9f25195226668c02dda3f644fb7fdbc6b15b6fc11a8"}
Sep 10 19:34:33.955197 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:33.955170    2575 generic.go:356] "Generic (PLEG): container finished" podID="8af5b0c5-dd53-402d-bb0e-3c4d5da2fb05" containerID="c9a8058160a72348118d1d8b180f67e334f482c337542782799069e270246aac" exitCode=137
Sep 10 19:34:33.955298 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:33.955196    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7455-966741"
Sep 10 19:34:33.955298 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:33.955220    2575 scope.go:117] "RemoveContainer" containerID="c9a8058160a72348118d1d8b180f67e334f482c337542782799069e270246aac"
Sep 10 19:34:33.963789 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:33.963773    2575 scope.go:117] "RemoveContainer" containerID="c9a8058160a72348118d1d8b180f67e334f482c337542782799069e270246aac"
Sep 10 19:34:33.964034 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:34:33.964018    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c9a8058160a72348118d1d8b180f67e334f482c337542782799069e270246aac\": container with ID starting with c9a8058160a72348118d1d8b180f67e334f482c337542782799069e270246aac not found: ID does not exist" containerID="c9a8058160a72348118d1d8b180f67e334f482c337542782799069e270246aac"
Sep 10 19:34:33.964082 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:33.964040    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c9a8058160a72348118d1d8b180f67e334f482c337542782799069e270246aac"} err="failed to get container status \"c9a8058160a72348118d1d8b180f67e334f482c337542782799069e270246aac\": rpc error: code = NotFound desc = could not find container \"c9a8058160a72348118d1d8b180f67e334f482c337542782799069e270246aac\": container with ID starting with c9a8058160a72348118d1d8b180f67e334f482c337542782799069e270246aac not found: ID does not exist"
Sep 10 19:34:33.973552 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:33.973485    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-c615f98e-openshift-default-7fbbb5c8d-dggrv" podStartSLOduration=1.973467214 podStartE2EDuration="1.973467214s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:34:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:34:33.972798854 +0000 UTC m=+1145.032626522" watchObservedRunningTime="2026-09-10 19:34:33.973467214 +0000 UTC m=+1145.033294882"
Sep 10 19:34:34.224812 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:34:34.224740    2575 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod17693f58_c507_4301_89c1_07487aa965ac.slice/crio-ee1c8aec864b9f411e6e343dd1a15de90a630c88a3eafb111f57b798a74c6ea0\": RecentStats: unable to find data in memory cache]"
Sep 10 19:34:34.786849 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:34.786820    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-c615f98e-openshift-default-7fbbb5c8d-dggrv"
Sep 10 19:34:34.791734 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:34.791712    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-c615f98e-openshift-default-7fbbb5c8d-dggrv"
Sep 10 19:34:34.961698 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:34.961677    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-c615f98e-openshift-default-7fbbb5c8d-dggrv"
Sep 10 19:34:34.962729 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:34.962709    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-c615f98e-openshift-default-7fbbb5c8d-dggrv"
Sep 10 19:34:35.118882 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:35.118858    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7455-77fa9d"
Sep 10 19:34:35.264581 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:35.264553    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-87z6n\" (UniqueName: \"kubernetes.io/projected/8631f4e7-5654-4c8f-9295-17cb8872dfa9-kube-api-access-87z6n\") pod \"8631f4e7-5654-4c8f-9295-17cb8872dfa9\" (UID: \"8631f4e7-5654-4c8f-9295-17cb8872dfa9\") "
Sep 10 19:34:35.266657 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:35.266627    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8631f4e7-5654-4c8f-9295-17cb8872dfa9-kube-api-access-87z6n" (OuterVolumeSpecName: "kube-api-access-87z6n") pod "8631f4e7-5654-4c8f-9295-17cb8872dfa9" (UID: "8631f4e7-5654-4c8f-9295-17cb8872dfa9"). InnerVolumeSpecName "kube-api-access-87z6n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:34:35.365812 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:35.365753    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-87z6n\" (UniqueName: \"kubernetes.io/projected/8631f4e7-5654-4c8f-9295-17cb8872dfa9-kube-api-access-87z6n\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:34:35.384708 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:35.384683    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8631f4e7-5654-4c8f-9295-17cb8872dfa9" path="/var/lib/kubelet/pods/8631f4e7-5654-4c8f-9295-17cb8872dfa9/volumes"
Sep 10 19:34:35.914792 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:35.914759    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:35.967695 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:35.967664    2575 generic.go:356] "Generic (PLEG): container finished" podID="8631f4e7-5654-4c8f-9295-17cb8872dfa9" containerID="2250eefae4862d5c5a1a5b25ebdab44daf0085e48c46d5e71c9b84b618f271b4" exitCode=137
Sep 10 19:34:35.968061 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:35.967721    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7455-77fa9d"
Sep 10 19:34:35.968061 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:35.967759    2575 scope.go:117] "RemoveContainer" containerID="2250eefae4862d5c5a1a5b25ebdab44daf0085e48c46d5e71c9b84b618f271b4"
Sep 10 19:34:35.976645 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:35.976630    2575 scope.go:117] "RemoveContainer" containerID="2250eefae4862d5c5a1a5b25ebdab44daf0085e48c46d5e71c9b84b618f271b4"
Sep 10 19:34:35.976901 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:34:35.976880    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2250eefae4862d5c5a1a5b25ebdab44daf0085e48c46d5e71c9b84b618f271b4\": container with ID starting with 2250eefae4862d5c5a1a5b25ebdab44daf0085e48c46d5e71c9b84b618f271b4 not found: ID does not exist" containerID="2250eefae4862d5c5a1a5b25ebdab44daf0085e48c46d5e71c9b84b618f271b4"
Sep 10 19:34:35.976963 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:35.976907    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2250eefae4862d5c5a1a5b25ebdab44daf0085e48c46d5e71c9b84b618f271b4"} err="failed to get container status \"2250eefae4862d5c5a1a5b25ebdab44daf0085e48c46d5e71c9b84b618f271b4\": rpc error: code = NotFound desc = could not find container \"2250eefae4862d5c5a1a5b25ebdab44daf0085e48c46d5e71c9b84b618f271b4\": container with ID starting with 2250eefae4862d5c5a1a5b25ebdab44daf0085e48c46d5e71c9b84b618f271b4 not found: ID does not exist"
Sep 10 19:34:36.007953 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:36.007933    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:37.263249 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:37.263219    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:37.364595 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:37.364564    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:37.467440 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:37.467404    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:37.941246 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:37.941214    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:38.035020 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:38.034992    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:38.224756 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:38.224723    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-c615f98e-768d58678f-klqfn"]
Sep 10 19:34:38.229794 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:38.229772    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-c615f98e-768d58678f-klqfn"
Sep 10 19:34:38.233660 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:38.233633    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-c615f98e-dockercfg-7g9j8\""
Sep 10 19:34:38.234117 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:38.234101    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-c615f98e\""
Sep 10 19:34:38.244219 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:38.244199    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-c615f98e-768d58678f-klqfn"]
Sep 10 19:34:38.390061 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:38.390035    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n4xwm\" (UniqueName: \"kubernetes.io/projected/fa64f873-f7ab-45e4-bae5-45ff78ab2c01-kube-api-access-n4xwm\") pod \"payload-processing-e2e-mt-c615f98e-768d58678f-klqfn\" (UID: \"fa64f873-f7ab-45e4-bae5-45ff78ab2c01\") " pod="openshift-ingress/payload-processing-e2e-mt-c615f98e-768d58678f-klqfn"
Sep 10 19:34:38.390385 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:38.390073    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fa64f873-f7ab-45e4-bae5-45ff78ab2c01-plugins-config-volume\") pod \"payload-processing-e2e-mt-c615f98e-768d58678f-klqfn\" (UID: \"fa64f873-f7ab-45e4-bae5-45ff78ab2c01\") " pod="openshift-ingress/payload-processing-e2e-mt-c615f98e-768d58678f-klqfn"
Sep 10 19:34:38.490728 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:38.490674    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fa64f873-f7ab-45e4-bae5-45ff78ab2c01-plugins-config-volume\") pod \"payload-processing-e2e-mt-c615f98e-768d58678f-klqfn\" (UID: \"fa64f873-f7ab-45e4-bae5-45ff78ab2c01\") " pod="openshift-ingress/payload-processing-e2e-mt-c615f98e-768d58678f-klqfn"
Sep 10 19:34:38.490824 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:38.490758    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n4xwm\" (UniqueName: \"kubernetes.io/projected/fa64f873-f7ab-45e4-bae5-45ff78ab2c01-kube-api-access-n4xwm\") pod \"payload-processing-e2e-mt-c615f98e-768d58678f-klqfn\" (UID: \"fa64f873-f7ab-45e4-bae5-45ff78ab2c01\") " pod="openshift-ingress/payload-processing-e2e-mt-c615f98e-768d58678f-klqfn"
Sep 10 19:34:38.491314 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:38.491298    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fa64f873-f7ab-45e4-bae5-45ff78ab2c01-plugins-config-volume\") pod \"payload-processing-e2e-mt-c615f98e-768d58678f-klqfn\" (UID: \"fa64f873-f7ab-45e4-bae5-45ff78ab2c01\") " pod="openshift-ingress/payload-processing-e2e-mt-c615f98e-768d58678f-klqfn"
Sep 10 19:34:38.499721 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:38.499703    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n4xwm\" (UniqueName: \"kubernetes.io/projected/fa64f873-f7ab-45e4-bae5-45ff78ab2c01-kube-api-access-n4xwm\") pod \"payload-processing-e2e-mt-c615f98e-768d58678f-klqfn\" (UID: \"fa64f873-f7ab-45e4-bae5-45ff78ab2c01\") " pod="openshift-ingress/payload-processing-e2e-mt-c615f98e-768d58678f-klqfn"
Sep 10 19:34:38.540112 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:38.540090    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-c615f98e-768d58678f-klqfn"
Sep 10 19:34:38.813323 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:38.813299    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-c615f98e-768d58678f-klqfn"]
Sep 10 19:34:38.815449 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:34:38.815411    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa64f873_f7ab_45e4_bae5_45ff78ab2c01.slice/crio-218de8c6b9ea9c830814e3c1b695df6e46663fdf467321e210031caffa1a26cb WatchSource:0}: Error finding container 218de8c6b9ea9c830814e3c1b695df6e46663fdf467321e210031caffa1a26cb: Status 404 returned error can't find the container with id 218de8c6b9ea9c830814e3c1b695df6e46663fdf467321e210031caffa1a26cb
Sep 10 19:34:38.987108 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:38.987075    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-c615f98e-768d58678f-klqfn" event={"ID":"fa64f873-f7ab-45e4-bae5-45ff78ab2c01","Type":"ContainerStarted","Data":"157f5254fa8926095d6f89342669681b8a98fee2609f4aa9ef0fdb29e7003ff2"}
Sep 10 19:34:38.987250 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:38.987132    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-c615f98e-768d58678f-klqfn" event={"ID":"fa64f873-f7ab-45e4-bae5-45ff78ab2c01","Type":"ContainerStarted","Data":"218de8c6b9ea9c830814e3c1b695df6e46663fdf467321e210031caffa1a26cb"}
Sep 10 19:34:38.987250 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:38.987187    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-c615f98e-768d58678f-klqfn"
Sep 10 19:34:39.005604 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:39.005559    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-c615f98e-768d58678f-klqfn" podStartSLOduration=1.005545438 podStartE2EDuration="1.005545438s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:34:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:34:39.003327161 +0000 UTC m=+1150.063154830" watchObservedRunningTime="2026-09-10 19:34:39.005545438 +0000 UTC m=+1150.065373106"
Sep 10 19:34:42.546893 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:34:42.546862    2575 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod17693f58_c507_4301_89c1_07487aa965ac.slice/crio-ee1c8aec864b9f411e6e343dd1a15de90a630c88a3eafb111f57b798a74c6ea0\": RecentStats: unable to find data in memory cache]"
Sep 10 19:34:43.959957 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:43.959910    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qr7rl"]
Sep 10 19:34:49.281947 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:34:49.281908    2575 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod17693f58_c507_4301_89c1_07487aa965ac.slice/crio-ee1c8aec864b9f411e6e343dd1a15de90a630c88a3eafb111f57b798a74c6ea0\": RecentStats: unable to find data in memory cache]"
Sep 10 19:34:49.994171 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:49.994137    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-c615f98e-768d58678f-klqfn"
Sep 10 19:34:52.558739 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:34:52.558659    2575 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod17693f58_c507_4301_89c1_07487aa965ac.slice/crio-ee1c8aec864b9f411e6e343dd1a15de90a630c88a3eafb111f57b798a74c6ea0\": RecentStats: unable to find data in memory cache]"
Sep 10 19:34:54.079141 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:54.079111    2575 generic.go:356] "Generic (PLEG): container finished" podID="5f2b470f-b238-4321-b5de-ff03dcfd1b79" containerID="530ebf86ed555c60404d66090baefdafb1afb57b44f627d8431ad33adc8de4bf" exitCode=0
Sep 10 19:34:54.079542 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:54.079191    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-c615f98e-openshift-default-7fbbb5c8d-dggrv" event={"ID":"5f2b470f-b238-4321-b5de-ff03dcfd1b79","Type":"ContainerDied","Data":"530ebf86ed555c60404d66090baefdafb1afb57b44f627d8431ad33adc8de4bf"}
Sep 10 19:34:54.079584 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:54.079553    2575 scope.go:117] "RemoveContainer" containerID="530ebf86ed555c60404d66090baefdafb1afb57b44f627d8431ad33adc8de4bf"
Sep 10 19:34:54.080683 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:54.080653    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 19:34:54.080745 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:54.080724    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 19:34:54.080786 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:54.080755    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 19:34:55.086442 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:55.086403    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-c615f98e-openshift-default-7fbbb5c8d-dggrv" event={"ID":"5f2b470f-b238-4321-b5de-ff03dcfd1b79","Type":"ContainerStarted","Data":"ebf25600876dd5c720229eb68bc2c3c305a02e43e950a9f8984d2914cbdd421e"}
Sep 10 19:34:55.786812 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:55.786771    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-c615f98e-openshift-default-7fbbb5c8d-dggrv"
Sep 10 19:34:55.787042 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:55.787018    2575 patch_prober.go:28] interesting pod/e2e-mt-c615f98e-openshift-default-7fbbb5c8d-dggrv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Sep 10 19:34:55.787132 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:55.787073    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-c615f98e-openshift-default-7fbbb5c8d-dggrv" podUID="5f2b470f-b238-4321-b5de-ff03dcfd1b79" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Sep 10 19:34:56.786719 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:56.786681    2575 patch_prober.go:28] interesting pod/e2e-mt-c615f98e-openshift-default-7fbbb5c8d-dggrv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Sep 10 19:34:56.787102 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:56.786735    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-c615f98e-openshift-default-7fbbb5c8d-dggrv" podUID="5f2b470f-b238-4321-b5de-ff03dcfd1b79" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Sep 10 19:34:57.040409 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:57.040298    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:57.786024 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:57.785990    2575 patch_prober.go:28] interesting pod/e2e-mt-c615f98e-openshift-default-7fbbb5c8d-dggrv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Sep 10 19:34:57.786202 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:57.786121    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-c615f98e-openshift-default-7fbbb5c8d-dggrv" podUID="5f2b470f-b238-4321-b5de-ff03dcfd1b79" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Sep 10 19:34:58.786457 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:58.786418    2575 patch_prober.go:28] interesting pod/e2e-mt-c615f98e-openshift-default-7fbbb5c8d-dggrv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Sep 10 19:34:58.786835 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:58.786480    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-c615f98e-openshift-default-7fbbb5c8d-dggrv" podUID="5f2b470f-b238-4321-b5de-ff03dcfd1b79" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Sep 10 19:34:59.291540 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:59.291493    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:59.786812 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:59.786775    2575 patch_prober.go:28] interesting pod/e2e-mt-c615f98e-openshift-default-7fbbb5c8d-dggrv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Sep 10 19:34:59.787179 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:34:59.786844    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-c615f98e-openshift-default-7fbbb5c8d-dggrv" podUID="5f2b470f-b238-4321-b5de-ff03dcfd1b79" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Sep 10 19:35:00.251422 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:35:00.251385    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:35:00.790441 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:35:00.790411    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-c615f98e-openshift-default-7fbbb5c8d-dggrv"
Sep 10 19:35:00.790823 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:35:00.790661    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-c615f98e-openshift-default-7fbbb5c8d-dggrv"
Sep 10 19:35:00.791336 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:35:00.791317    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-c615f98e-openshift-default-7fbbb5c8d-dggrv"
Sep 10 19:35:00.840550 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:35:00.840521    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:35:02.052970 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:35:02.052932    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qr7rl"]
Sep 10 19:35:02.625010 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:35:02.624966    2575 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod17693f58_c507_4301_89c1_07487aa965ac.slice/crio-ee1c8aec864b9f411e6e343dd1a15de90a630c88a3eafb111f57b798a74c6ea0\": RecentStats: unable to find data in memory cache]"
Sep 10 19:35:04.230667 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:35:04.230629    2575 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod17693f58_c507_4301_89c1_07487aa965ac.slice/crio-ee1c8aec864b9f411e6e343dd1a15de90a630c88a3eafb111f57b798a74c6ea0\": RecentStats: unable to find data in memory cache]"
Sep 10 19:35:08.443009 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:35:08.442973    2575 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod17693f58_c507_4301_89c1_07487aa965ac.slice/crio-ee1c8aec864b9f411e6e343dd1a15de90a630c88a3eafb111f57b798a74c6ea0\": RecentStats: unable to find data in memory cache]"
Sep 10 19:35:08.443595 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:35:08.442976    2575 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod17693f58_c507_4301_89c1_07487aa965ac.slice/crio-ee1c8aec864b9f411e6e343dd1a15de90a630c88a3eafb111f57b798a74c6ea0\": RecentStats: unable to find data in memory cache]"
Sep 10 19:35:12.642835 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:35:12.642802    2575 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod17693f58_c507_4301_89c1_07487aa965ac.slice/crio-ee1c8aec864b9f411e6e343dd1a15de90a630c88a3eafb111f57b798a74c6ea0\": RecentStats: unable to find data in memory cache]"
Sep 10 19:35:17.506755 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:35:17.506724    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-c615f98e/e2e-lifecycle-model-c615f98e-kserve-865f4b64f5-5w28w"]
Sep 10 19:35:17.511989 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:35:17.511968    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-c615f98e/e2e-lifecycle-model-c615f98e-kserve-865f4b64f5-5w28w"
Sep 10 19:35:17.519792 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:35:17.519771    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-c615f98e\"/\"kube-root-ca.crt\""
Sep 10 19:35:17.522007 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:35:17.521987    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-c615f98e\"/\"e2e-lifecycle-model-c615f98e-kserve-self-signed-certs\""
Sep 10 19:35:17.522141 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:35:17.522086    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-c615f98e\"/\"openshift-service-ca.crt\""
Sep 10 19:35:17.522189 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:35:17.522167    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-c615f98e\"/\"default-dockercfg-tfknv\""
Sep 10 19:35:17.531447 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:35:17.531426    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-c615f98e/e2e-lifecycle-model-c615f98e-kserve-865f4b64f5-5w28w"]
Sep 10 19:35:17.596929 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:35:17.596902    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:35:17.632082 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:35:17.632057    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bc70833f-d7d9-444d-8756-4f872b7fafe6-home\") pod \"e2e-lifecycle-model-c615f98e-kserve-865f4b64f5-5w28w\" (UID: \"bc70833f-d7d9-444d-8756-4f872b7fafe6\") " pod="ai-tenant-e2e-mt-c615f98e/e2e-lifecycle-model-c615f98e-kserve-865f4b64f5-5w28w"
Sep 10 19:35:17.632189 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:35:17.632091    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bc70833f-d7d9-444d-8756-4f872b7fafe6-model-cache\") pod \"e2e-lifecycle-model-c615f98e-kserve-865f4b64f5-5w28w\" (UID: \"bc70833f-d7d9-444d-8756-4f872b7fafe6\") " pod="ai-tenant-e2e-mt-c615f98e/e2e-lifecycle-model-c615f98e-kserve-865f4b64f5-5w28w"
Sep 10 19:35:17.632189 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:35:17.632115    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bc70833f-d7d9-444d-8756-4f872b7fafe6-dshm\") pod \"e2e-lifecycle-model-c615f98e-kserve-865f4b64f5-5w28w\" (UID: \"bc70833f-d7d9-444d-8756-4f872b7fafe6\") " pod="ai-tenant-e2e-mt-c615f98e/e2e-lifecycle-model-c615f98e-kserve-865f4b64f5-5w28w"
Sep 10 19:35:17.632274 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:35:17.632179    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-82g7f\" (UniqueName: \"kubernetes.io/projected/bc70833f-d7d9-444d-8756-4f872b7fafe6-kube-api-access-82g7f\") pod \"e2e-lifecycle-model-c615f98e-kserve-865f4b64f5-5w28w\" (UID: \"bc70833f-d7d9-444d-8756-4f872b7fafe6\") " pod="ai-tenant-e2e-mt-c615f98e/e2e-lifecycle-model-c615f98e-kserve-865f4b64f5-5w28w"
Sep 10 19:35:17.632312 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:35:17.632272    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bc70833f-d7d9-444d-8756-4f872b7fafe6-tmp-dir\") pod \"e2e-lifecycle-model-c615f98e-kserve-865f4b64f5-5w28w\" (UID: \"bc70833f-d7d9-444d-8756-4f872b7fafe6\") " pod="ai-tenant-e2e-mt-c615f98e/e2e-lifecycle-model-c615f98e-kserve-865f4b64f5-5w28w"
Sep 10 19:35:17.632384 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:35:17.632326    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bc70833f-d7d9-444d-8756-4f872b7fafe6-tls-certs\") pod \"e2e-lifecycle-model-c615f98e-kserve-865f4b64f5-5w28w\" (UID: \"bc70833f-d7d9-444d-8756-4f872b7fafe6\") " pod="ai-tenant-e2e-mt-c615f98e/e2e-lifecycle-model-c615f98e-kserve-865f4b64f5-5w28w"
Sep 10 19:35:17.733571 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:35:17.733533    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bc70833f-d7d9-444d-8756-4f872b7fafe6-tmp-dir\") pod \"e2e-lifecycle-model-c615f98e-kserve-865f4b64f5-5w28w\" (UID: \"bc70833f-d7d9-444d-8756-4f872b7fafe6\") " pod="ai-tenant-e2e-mt-c615f98e/e2e-lifecycle-model-c615f98e-kserve-865f4b64f5-5w28w"
Sep 10 19:35:17.733765 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:35:17.733602    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bc70833f-d7d9-444d-8756-4f872b7fafe6-tls-certs\") pod \"e2e-lifecycle-model-c615f98e-kserve-865f4b64f5-5w28w\" (UID: \"bc70833f-d7d9-444d-8756-4f872b7fafe6\") " pod="ai-tenant-e2e-mt-c615f98e/e2e-lifecycle-model-c615f98e-kserve-865f4b64f5-5w28w"
Sep 10 19:35:17.733765 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:35:17.733663    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bc70833f-d7d9-444d-8756-4f872b7fafe6-home\") pod \"e2e-lifecycle-model-c615f98e-kserve-865f4b64f5-5w28w\" (UID: \"bc70833f-d7d9-444d-8756-4f872b7fafe6\") " pod="ai-tenant-e2e-mt-c615f98e/e2e-lifecycle-model-c615f98e-kserve-865f4b64f5-5w28w"
Sep 10 19:35:17.733765 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:35:17.733685    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bc70833f-d7d9-444d-8756-4f872b7fafe6-model-cache\") pod \"e2e-lifecycle-model-c615f98e-kserve-865f4b64f5-5w28w\" (UID: \"bc70833f-d7d9-444d-8756-4f872b7fafe6\") " pod="ai-tenant-e2e-mt-c615f98e/e2e-lifecycle-model-c615f98e-kserve-865f4b64f5-5w28w"
Sep 10 19:35:17.733765 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:35:17.733713    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bc70833f-d7d9-444d-8756-4f872b7fafe6-dshm\") pod \"e2e-lifecycle-model-c615f98e-kserve-865f4b64f5-5w28w\" (UID: \"bc70833f-d7d9-444d-8756-4f872b7fafe6\") " pod="ai-tenant-e2e-mt-c615f98e/e2e-lifecycle-model-c615f98e-kserve-865f4b64f5-5w28w"
Sep 10 19:35:17.733765 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:35:17.733741    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-82g7f\" (UniqueName: \"kubernetes.io/projected/bc70833f-d7d9-444d-8756-4f872b7fafe6-kube-api-access-82g7f\") pod \"e2e-lifecycle-model-c615f98e-kserve-865f4b64f5-5w28w\" (UID: \"bc70833f-d7d9-444d-8756-4f872b7fafe6\") " pod="ai-tenant-e2e-mt-c615f98e/e2e-lifecycle-model-c615f98e-kserve-865f4b64f5-5w28w"
Sep 10 19:35:17.734049 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:35:17.734025    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bc70833f-d7d9-444d-8756-4f872b7fafe6-tmp-dir\") pod \"e2e-lifecycle-model-c615f98e-kserve-865f4b64f5-5w28w\" (UID: \"bc70833f-d7d9-444d-8756-4f872b7fafe6\") " pod="ai-tenant-e2e-mt-c615f98e/e2e-lifecycle-model-c615f98e-kserve-865f4b64f5-5w28w"
Sep 10 19:35:17.734099 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:35:17.734063    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bc70833f-d7d9-444d-8756-4f872b7fafe6-model-cache\") pod \"e2e-lifecycle-model-c615f98e-kserve-865f4b64f5-5w28w\" (UID: \"bc70833f-d7d9-444d-8756-4f872b7fafe6\") " pod="ai-tenant-e2e-mt-c615f98e/e2e-lifecycle-model-c615f98e-kserve-865f4b64f5-5w28w"
Sep 10 19:35:17.734099 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:35:17.734085    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bc70833f-d7d9-444d-8756-4f872b7fafe6-home\") pod \"e2e-lifecycle-model-c615f98e-kserve-865f4b64f5-5w28w\" (UID: \"bc70833f-d7d9-444d-8756-4f872b7fafe6\") " pod="ai-tenant-e2e-mt-c615f98e/e2e-lifecycle-model-c615f98e-kserve-865f4b64f5-5w28w"
Sep 10 19:35:17.736012 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:35:17.735992    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bc70833f-d7d9-444d-8756-4f872b7fafe6-dshm\") pod \"e2e-lifecycle-model-c615f98e-kserve-865f4b64f5-5w28w\" (UID: \"bc70833f-d7d9-444d-8756-4f872b7fafe6\") " pod="ai-tenant-e2e-mt-c615f98e/e2e-lifecycle-model-c615f98e-kserve-865f4b64f5-5w28w"
Sep 10 19:35:17.736402 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:35:17.736383    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bc70833f-d7d9-444d-8756-4f872b7fafe6-tls-certs\") pod \"e2e-lifecycle-model-c615f98e-kserve-865f4b64f5-5w28w\" (UID: \"bc70833f-d7d9-444d-8756-4f872b7fafe6\") " pod="ai-tenant-e2e-mt-c615f98e/e2e-lifecycle-model-c615f98e-kserve-865f4b64f5-5w28w"
Sep 10 19:35:17.749455 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:35:17.749434    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-82g7f\" (UniqueName: \"kubernetes.io/projected/bc70833f-d7d9-444d-8756-4f872b7fafe6-kube-api-access-82g7f\") pod \"e2e-lifecycle-model-c615f98e-kserve-865f4b64f5-5w28w\" (UID: \"bc70833f-d7d9-444d-8756-4f872b7fafe6\") " pod="ai-tenant-e2e-mt-c615f98e/e2e-lifecycle-model-c615f98e-kserve-865f4b64f5-5w28w"
Sep 10 19:35:17.821456 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:35:17.821383    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-c615f98e/e2e-lifecycle-model-c615f98e-kserve-865f4b64f5-5w28w"
Sep 10 19:35:18.112598 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:35:18.112560    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbc70833f_d7d9_444d_8756_4f872b7fafe6.slice/crio-b8ad264efcb0bfd81f3c3b0d078bde5be88eaa2bb9b1b2457ca3e2166664772a WatchSource:0}: Error finding container b8ad264efcb0bfd81f3c3b0d078bde5be88eaa2bb9b1b2457ca3e2166664772a: Status 404 returned error can't find the container with id b8ad264efcb0bfd81f3c3b0d078bde5be88eaa2bb9b1b2457ca3e2166664772a
Sep 10 19:35:18.112715 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:35:18.112663    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-c615f98e/e2e-lifecycle-model-c615f98e-kserve-865f4b64f5-5w28w"]
Sep 10 19:35:18.207314 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:35:18.207281    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-c615f98e/e2e-lifecycle-model-c615f98e-kserve-865f4b64f5-5w28w" event={"ID":"bc70833f-d7d9-444d-8756-4f872b7fafe6","Type":"ContainerStarted","Data":"c2f53636e52657eed04272577e1706ebb8b5fb204cd61d3b1b60ac63b67243c5"}
Sep 10 19:35:18.207454 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:35:18.207330    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-c615f98e/e2e-lifecycle-model-c615f98e-kserve-865f4b64f5-5w28w" event={"ID":"bc70833f-d7d9-444d-8756-4f872b7fafe6","Type":"ContainerStarted","Data":"b8ad264efcb0bfd81f3c3b0d078bde5be88eaa2bb9b1b2457ca3e2166664772a"}
Sep 10 19:35:18.229006 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:35:18.228957    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-c615f98e/e2e-lifecycle-model-c615f98e-kserve-865f4b64f5-5w28w" podStartSLOduration=1.228940992 podStartE2EDuration="1.228940992s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:35:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:35:18.22775507 +0000 UTC m=+1189.287582742" watchObservedRunningTime="2026-09-10 19:35:18.228940992 +0000 UTC m=+1189.288768659"
Sep 10 19:35:19.272577 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:35:19.272545    2575 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod17693f58_c507_4301_89c1_07487aa965ac.slice/crio-ee1c8aec864b9f411e6e343dd1a15de90a630c88a3eafb111f57b798a74c6ea0\": RecentStats: unable to find data in memory cache]"
Sep 10 19:35:22.652508 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:35:22.652475    2575 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod17693f58_c507_4301_89c1_07487aa965ac.slice/crio-ee1c8aec864b9f411e6e343dd1a15de90a630c88a3eafb111f57b798a74c6ea0\": RecentStats: unable to find data in memory cache]"
Sep 10 19:35:22.768417 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:35:22.768367    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qr7rl"]
Sep 10 19:35:24.350185 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:35:24.350144    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:35:24.450809 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:35:24.450778    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:35:27.822334 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:35:27.822300    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-c615f98e/e2e-lifecycle-model-c615f98e-kserve-865f4b64f5-5w28w"
Sep 10 19:35:27.822781 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:35:27.822564    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-c615f98e/e2e-lifecycle-model-c615f98e-kserve-865f4b64f5-5w28w"
Sep 10 19:35:27.834854 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:35:27.834832    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-c615f98e/e2e-lifecycle-model-c615f98e-kserve-865f4b64f5-5w28w"
Sep 10 19:35:28.273789 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:35:28.273763    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-c615f98e/e2e-lifecycle-model-c615f98e-kserve-865f4b64f5-5w28w"
Sep 10 19:35:29.407902 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:35:29.407876    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rrxfj_c7511dfe-c588-4be1-bd16-1dce34740809/ovn-acl-logging/0.log"
Sep 10 19:35:29.409651 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:35:29.409630    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rrxfj_c7511dfe-c588-4be1-bd16-1dce34740809/ovn-acl-logging/0.log"
Sep 10 19:35:36.765007 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:35:36.764972    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qr7rl"]
Sep 10 19:35:55.694432 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:35:55.694397    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qr7rl"]
Sep 10 19:35:55.937320 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:35:55.937287    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:35:55.985911 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:35:55.985881    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-c615f98e-768d58678f-klqfn"]
Sep 10 19:35:55.986151 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:35:55.986123    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-c615f98e-768d58678f-klqfn" podUID="fa64f873-f7ab-45e4-bae5-45ff78ab2c01" containerName="payload-processing" containerID="cri-o://157f5254fa8926095d6f89342669681b8a98fee2609f4aa9ef0fdb29e7003ff2" gracePeriod=30
Sep 10 19:35:56.072552 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:35:56.072513    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:35:56.178000 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:35:56.177971    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:35:56.241899 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:35:56.241844    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-c615f98e-768d58678f-klqfn"
Sep 10 19:35:56.263137 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:35:56.263112    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fa64f873-f7ab-45e4-bae5-45ff78ab2c01-plugins-config-volume\") pod \"fa64f873-f7ab-45e4-bae5-45ff78ab2c01\" (UID: \"fa64f873-f7ab-45e4-bae5-45ff78ab2c01\") "
Sep 10 19:35:56.263245 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:35:56.263172    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n4xwm\" (UniqueName: \"kubernetes.io/projected/fa64f873-f7ab-45e4-bae5-45ff78ab2c01-kube-api-access-n4xwm\") pod \"fa64f873-f7ab-45e4-bae5-45ff78ab2c01\" (UID: \"fa64f873-f7ab-45e4-bae5-45ff78ab2c01\") "
Sep 10 19:35:56.263463 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:35:56.263439    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fa64f873-f7ab-45e4-bae5-45ff78ab2c01-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "fa64f873-f7ab-45e4-bae5-45ff78ab2c01" (UID: "fa64f873-f7ab-45e4-bae5-45ff78ab2c01"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:35:56.265056 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:35:56.265033    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fa64f873-f7ab-45e4-bae5-45ff78ab2c01-kube-api-access-n4xwm" (OuterVolumeSpecName: "kube-api-access-n4xwm") pod "fa64f873-f7ab-45e4-bae5-45ff78ab2c01" (UID: "fa64f873-f7ab-45e4-bae5-45ff78ab2c01"). InnerVolumeSpecName "kube-api-access-n4xwm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:35:56.279725 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:35:56.279694    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:35:56.364905 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:35:56.364874    2575 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fa64f873-f7ab-45e4-bae5-45ff78ab2c01-plugins-config-volume\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:35:56.364905 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:35:56.364906    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n4xwm\" (UniqueName: \"kubernetes.io/projected/fa64f873-f7ab-45e4-bae5-45ff78ab2c01-kube-api-access-n4xwm\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:35:56.391759 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:35:56.391723    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:35:56.423230 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:35:56.423196    2575 generic.go:356] "Generic (PLEG): container finished" podID="fa64f873-f7ab-45e4-bae5-45ff78ab2c01" containerID="157f5254fa8926095d6f89342669681b8a98fee2609f4aa9ef0fdb29e7003ff2" exitCode=0
Sep 10 19:35:56.423383 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:35:56.423249    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-c615f98e-768d58678f-klqfn" event={"ID":"fa64f873-f7ab-45e4-bae5-45ff78ab2c01","Type":"ContainerDied","Data":"157f5254fa8926095d6f89342669681b8a98fee2609f4aa9ef0fdb29e7003ff2"}
Sep 10 19:35:56.423383 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:35:56.423288    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-c615f98e-768d58678f-klqfn"
Sep 10 19:35:56.423383 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:35:56.423314    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-c615f98e-768d58678f-klqfn" event={"ID":"fa64f873-f7ab-45e4-bae5-45ff78ab2c01","Type":"ContainerDied","Data":"218de8c6b9ea9c830814e3c1b695df6e46663fdf467321e210031caffa1a26cb"}
Sep 10 19:35:56.423383 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:35:56.423364    2575 scope.go:117] "RemoveContainer" containerID="157f5254fa8926095d6f89342669681b8a98fee2609f4aa9ef0fdb29e7003ff2"
Sep 10 19:35:56.432706 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:35:56.432690    2575 scope.go:117] "RemoveContainer" containerID="157f5254fa8926095d6f89342669681b8a98fee2609f4aa9ef0fdb29e7003ff2"
Sep 10 19:35:56.432970 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:35:56.432950    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"157f5254fa8926095d6f89342669681b8a98fee2609f4aa9ef0fdb29e7003ff2\": container with ID starting with 157f5254fa8926095d6f89342669681b8a98fee2609f4aa9ef0fdb29e7003ff2 not found: ID does not exist" containerID="157f5254fa8926095d6f89342669681b8a98fee2609f4aa9ef0fdb29e7003ff2"
Sep 10 19:35:56.433011 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:35:56.432981    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"157f5254fa8926095d6f89342669681b8a98fee2609f4aa9ef0fdb29e7003ff2"} err="failed to get container status \"157f5254fa8926095d6f89342669681b8a98fee2609f4aa9ef0fdb29e7003ff2\": rpc error: code = NotFound desc = could not find container \"157f5254fa8926095d6f89342669681b8a98fee2609f4aa9ef0fdb29e7003ff2\": container with ID starting with 157f5254fa8926095d6f89342669681b8a98fee2609f4aa9ef0fdb29e7003ff2 not found: ID does not exist"
Sep 10 19:35:56.452607 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:35:56.452586    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-c615f98e-768d58678f-klqfn"]
Sep 10 19:35:56.458733 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:35:56.458713    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-c615f98e-768d58678f-klqfn"]
Sep 10 19:35:57.385504 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:35:57.385472    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fa64f873-f7ab-45e4-bae5-45ff78ab2c01" path="/var/lib/kubelet/pods/fa64f873-f7ab-45e4-bae5-45ff78ab2c01/volumes"
Sep 10 19:35:59.513537 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:35:59.513506    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:36:04.449795 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:36:04.449760    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-c615f98e/e2e-lifecycle-model-c615f98e-kserve-865f4b64f5-5w28w"]
Sep 10 19:36:04.544162 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:36:04.544130    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:36:13.963087 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:36:13.963057    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qr7rl"]
Sep 10 19:36:19.469540 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:36:19.469481    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-c615f98e/e2e-lifecycle-model-c615f98e-kserve-865f4b64f5-5w28w" podUID="bc70833f-d7d9-444d-8756-4f872b7fafe6" containerName="main" containerID="cri-o://c2f53636e52657eed04272577e1706ebb8b5fb204cd61d3b1b60ac63b67243c5" gracePeriod=45
Sep 10 19:36:19.730202 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:36:19.730140    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-c615f98e/e2e-lifecycle-model-c615f98e-kserve-865f4b64f5-5w28w"
Sep 10 19:36:19.865739 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:36:19.865711    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bc70833f-d7d9-444d-8756-4f872b7fafe6-model-cache\") pod \"bc70833f-d7d9-444d-8756-4f872b7fafe6\" (UID: \"bc70833f-d7d9-444d-8756-4f872b7fafe6\") "
Sep 10 19:36:19.865922 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:36:19.865753    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bc70833f-d7d9-444d-8756-4f872b7fafe6-home\") pod \"bc70833f-d7d9-444d-8756-4f872b7fafe6\" (UID: \"bc70833f-d7d9-444d-8756-4f872b7fafe6\") "
Sep 10 19:36:19.865922 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:36:19.865845    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bc70833f-d7d9-444d-8756-4f872b7fafe6-dshm\") pod \"bc70833f-d7d9-444d-8756-4f872b7fafe6\" (UID: \"bc70833f-d7d9-444d-8756-4f872b7fafe6\") "
Sep 10 19:36:19.865922 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:36:19.865888    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bc70833f-d7d9-444d-8756-4f872b7fafe6-tls-certs\") pod \"bc70833f-d7d9-444d-8756-4f872b7fafe6\" (UID: \"bc70833f-d7d9-444d-8756-4f872b7fafe6\") "
Sep 10 19:36:19.866082 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:36:19.865952    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bc70833f-d7d9-444d-8756-4f872b7fafe6-tmp-dir\") pod \"bc70833f-d7d9-444d-8756-4f872b7fafe6\" (UID: \"bc70833f-d7d9-444d-8756-4f872b7fafe6\") "
Sep 10 19:36:19.866082 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:36:19.865955    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bc70833f-d7d9-444d-8756-4f872b7fafe6-model-cache" (OuterVolumeSpecName: "model-cache") pod "bc70833f-d7d9-444d-8756-4f872b7fafe6" (UID: "bc70833f-d7d9-444d-8756-4f872b7fafe6"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:36:19.866082 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:36:19.865988    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-82g7f\" (UniqueName: \"kubernetes.io/projected/bc70833f-d7d9-444d-8756-4f872b7fafe6-kube-api-access-82g7f\") pod \"bc70833f-d7d9-444d-8756-4f872b7fafe6\" (UID: \"bc70833f-d7d9-444d-8756-4f872b7fafe6\") "
Sep 10 19:36:19.866082 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:36:19.866047    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bc70833f-d7d9-444d-8756-4f872b7fafe6-home" (OuterVolumeSpecName: "home") pod "bc70833f-d7d9-444d-8756-4f872b7fafe6" (UID: "bc70833f-d7d9-444d-8756-4f872b7fafe6"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:36:19.866258 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:36:19.866156    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bc70833f-d7d9-444d-8756-4f872b7fafe6-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "bc70833f-d7d9-444d-8756-4f872b7fafe6" (UID: "bc70833f-d7d9-444d-8756-4f872b7fafe6"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:36:19.866327 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:36:19.866276    2575 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bc70833f-d7d9-444d-8756-4f872b7fafe6-tmp-dir\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:36:19.866327 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:36:19.866292    2575 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bc70833f-d7d9-444d-8756-4f872b7fafe6-model-cache\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:36:19.866327 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:36:19.866300    2575 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bc70833f-d7d9-444d-8756-4f872b7fafe6-home\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:36:19.867945 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:36:19.867926    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bc70833f-d7d9-444d-8756-4f872b7fafe6-dshm" (OuterVolumeSpecName: "dshm") pod "bc70833f-d7d9-444d-8756-4f872b7fafe6" (UID: "bc70833f-d7d9-444d-8756-4f872b7fafe6"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:36:19.868049 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:36:19.868029    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bc70833f-d7d9-444d-8756-4f872b7fafe6-kube-api-access-82g7f" (OuterVolumeSpecName: "kube-api-access-82g7f") pod "bc70833f-d7d9-444d-8756-4f872b7fafe6" (UID: "bc70833f-d7d9-444d-8756-4f872b7fafe6"). InnerVolumeSpecName "kube-api-access-82g7f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:36:19.868120 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:36:19.868098    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bc70833f-d7d9-444d-8756-4f872b7fafe6-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "bc70833f-d7d9-444d-8756-4f872b7fafe6" (UID: "bc70833f-d7d9-444d-8756-4f872b7fafe6"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:36:19.967558 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:36:19.967524    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-82g7f\" (UniqueName: \"kubernetes.io/projected/bc70833f-d7d9-444d-8756-4f872b7fafe6-kube-api-access-82g7f\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:36:19.967558 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:36:19.967554    2575 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bc70833f-d7d9-444d-8756-4f872b7fafe6-dshm\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:36:19.967558 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:36:19.967563    2575 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bc70833f-d7d9-444d-8756-4f872b7fafe6-tls-certs\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:36:20.545801 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:36:20.545767    2575 generic.go:356] "Generic (PLEG): container finished" podID="bc70833f-d7d9-444d-8756-4f872b7fafe6" containerID="c2f53636e52657eed04272577e1706ebb8b5fb204cd61d3b1b60ac63b67243c5" exitCode=0
Sep 10 19:36:20.546182 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:36:20.545813    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-c615f98e/e2e-lifecycle-model-c615f98e-kserve-865f4b64f5-5w28w"
Sep 10 19:36:20.546182 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:36:20.545853    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-c615f98e/e2e-lifecycle-model-c615f98e-kserve-865f4b64f5-5w28w" event={"ID":"bc70833f-d7d9-444d-8756-4f872b7fafe6","Type":"ContainerDied","Data":"c2f53636e52657eed04272577e1706ebb8b5fb204cd61d3b1b60ac63b67243c5"}
Sep 10 19:36:20.546182 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:36:20.545898    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-c615f98e/e2e-lifecycle-model-c615f98e-kserve-865f4b64f5-5w28w" event={"ID":"bc70833f-d7d9-444d-8756-4f872b7fafe6","Type":"ContainerDied","Data":"b8ad264efcb0bfd81f3c3b0d078bde5be88eaa2bb9b1b2457ca3e2166664772a"}
Sep 10 19:36:20.546182 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:36:20.545923    2575 scope.go:117] "RemoveContainer" containerID="c2f53636e52657eed04272577e1706ebb8b5fb204cd61d3b1b60ac63b67243c5"
Sep 10 19:36:20.559687 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:36:20.559668    2575 scope.go:117] "RemoveContainer" containerID="c2f53636e52657eed04272577e1706ebb8b5fb204cd61d3b1b60ac63b67243c5"
Sep 10 19:36:20.559967 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:36:20.559943    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c2f53636e52657eed04272577e1706ebb8b5fb204cd61d3b1b60ac63b67243c5\": container with ID starting with c2f53636e52657eed04272577e1706ebb8b5fb204cd61d3b1b60ac63b67243c5 not found: ID does not exist" containerID="c2f53636e52657eed04272577e1706ebb8b5fb204cd61d3b1b60ac63b67243c5"
Sep 10 19:36:20.560014 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:36:20.559975    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c2f53636e52657eed04272577e1706ebb8b5fb204cd61d3b1b60ac63b67243c5"} err="failed to get container status \"c2f53636e52657eed04272577e1706ebb8b5fb204cd61d3b1b60ac63b67243c5\": rpc error: code = NotFound desc = could not find container \"c2f53636e52657eed04272577e1706ebb8b5fb204cd61d3b1b60ac63b67243c5\": container with ID starting with c2f53636e52657eed04272577e1706ebb8b5fb204cd61d3b1b60ac63b67243c5 not found: ID does not exist"
Sep 10 19:36:20.569104 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:36:20.569080    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-c615f98e/e2e-lifecycle-model-c615f98e-kserve-865f4b64f5-5w28w"]
Sep 10 19:36:20.573028 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:36:20.572998    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-c615f98e/e2e-lifecycle-model-c615f98e-kserve-865f4b64f5-5w28w"]
Sep 10 19:36:21.385558 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:36:21.385527    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bc70833f-d7d9-444d-8756-4f872b7fafe6" path="/var/lib/kubelet/pods/bc70833f-d7d9-444d-8756-4f872b7fafe6/volumes"
Sep 10 19:36:29.518837 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:36:29.518773    2575 scope.go:117] "RemoveContainer" containerID="7b818d6616a80bd7e8420a4f0e4d4ebdb53901b20afc28e20d4eff88643e7dbe"
Sep 10 19:36:29.526868 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:36:29.526847    2575 scope.go:117] "RemoveContainer" containerID="d7462fdf9ddaa712f5d708d8b6ea8a5d1649f8cad4b8847054f957f877dbef4a"
Sep 10 19:36:30.252386 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:36:30.252355    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qr7rl"]
Sep 10 19:36:41.638130 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:36:41.638096    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:36:42.788631 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:36:42.788590    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:36:45.848665 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:36:45.848633    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qr7rl"]
Sep 10 19:36:47.441270 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:36:47.441211    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:36:53.306721 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:36:53.306683    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:36:53.408301 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:36:53.408275    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:36:53.546292 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:36:53.546255    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-c615f98e-openshift-default-7fbbb5c8d-dggrv"]
Sep 10 19:36:53.546499 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:36:53.546473    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-c615f98e-openshift-default-7fbbb5c8d-dggrv" podUID="5f2b470f-b238-4321-b5de-ff03dcfd1b79" containerName="istio-proxy" containerID="cri-o://ebf25600876dd5c720229eb68bc2c3c305a02e43e950a9f8984d2914cbdd421e" gracePeriod=30
Sep 10 19:36:53.568998 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:36:53.568931    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:36:54.775789 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:36:54.775753    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:36:54.875352 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:36:54.875286    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:36:54.975788 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:36:54.975763    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:36:55.539730 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:36:55.539702    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:36:55.633922 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:36:55.633897    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:36:55.850645 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:36:55.850566    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:36:56.923792 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:36:56.923763    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:36:58.743874 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:36:58.743838    2575 generic.go:356] "Generic (PLEG): container finished" podID="5f2b470f-b238-4321-b5de-ff03dcfd1b79" containerID="ebf25600876dd5c720229eb68bc2c3c305a02e43e950a9f8984d2914cbdd421e" exitCode=0
Sep 10 19:36:58.744313 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:36:58.743915    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-c615f98e-openshift-default-7fbbb5c8d-dggrv" event={"ID":"5f2b470f-b238-4321-b5de-ff03dcfd1b79","Type":"ContainerDied","Data":"ebf25600876dd5c720229eb68bc2c3c305a02e43e950a9f8984d2914cbdd421e"}
Sep 10 19:36:58.744313 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:36:58.743979    2575 scope.go:117] "RemoveContainer" containerID="530ebf86ed555c60404d66090baefdafb1afb57b44f627d8431ad33adc8de4bf"
Sep 10 19:36:58.802349 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:36:58.802322    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-c615f98e-openshift-default-7fbbb5c8d-dggrv"
Sep 10 19:36:58.886455 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:36:58.886378    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5f2b470f-b238-4321-b5de-ff03dcfd1b79-istiod-ca-cert\") pod \"5f2b470f-b238-4321-b5de-ff03dcfd1b79\" (UID: \"5f2b470f-b238-4321-b5de-ff03dcfd1b79\") "
Sep 10 19:36:58.886455 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:36:58.886416    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bw7s7\" (UniqueName: \"kubernetes.io/projected/5f2b470f-b238-4321-b5de-ff03dcfd1b79-kube-api-access-bw7s7\") pod \"5f2b470f-b238-4321-b5de-ff03dcfd1b79\" (UID: \"5f2b470f-b238-4321-b5de-ff03dcfd1b79\") "
Sep 10 19:36:58.886455 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:36:58.886438    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5f2b470f-b238-4321-b5de-ff03dcfd1b79-workload-certs\") pod \"5f2b470f-b238-4321-b5de-ff03dcfd1b79\" (UID: \"5f2b470f-b238-4321-b5de-ff03dcfd1b79\") "
Sep 10 19:36:58.886739 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:36:58.886615    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5f2b470f-b238-4321-b5de-ff03dcfd1b79-istio-token\") pod \"5f2b470f-b238-4321-b5de-ff03dcfd1b79\" (UID: \"5f2b470f-b238-4321-b5de-ff03dcfd1b79\") "
Sep 10 19:36:58.886739 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:36:58.886650    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f2b470f-b238-4321-b5de-ff03dcfd1b79-credential-socket\") pod \"5f2b470f-b238-4321-b5de-ff03dcfd1b79\" (UID: \"5f2b470f-b238-4321-b5de-ff03dcfd1b79\") "
Sep 10 19:36:58.886739 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:36:58.886654    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f2b470f-b238-4321-b5de-ff03dcfd1b79-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "5f2b470f-b238-4321-b5de-ff03dcfd1b79" (UID: "5f2b470f-b238-4321-b5de-ff03dcfd1b79"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:36:58.886739 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:36:58.886676    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5f2b470f-b238-4321-b5de-ff03dcfd1b79-istio-podinfo\") pod \"5f2b470f-b238-4321-b5de-ff03dcfd1b79\" (UID: \"5f2b470f-b238-4321-b5de-ff03dcfd1b79\") "
Sep 10 19:36:58.886954 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:36:58.886811    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5f2b470f-b238-4321-b5de-ff03dcfd1b79-istio-data\") pod \"5f2b470f-b238-4321-b5de-ff03dcfd1b79\" (UID: \"5f2b470f-b238-4321-b5de-ff03dcfd1b79\") "
Sep 10 19:36:58.886954 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:36:58.886869    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f2b470f-b238-4321-b5de-ff03dcfd1b79-workload-socket\") pod \"5f2b470f-b238-4321-b5de-ff03dcfd1b79\" (UID: \"5f2b470f-b238-4321-b5de-ff03dcfd1b79\") "
Sep 10 19:36:58.886954 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:36:58.886886    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f2b470f-b238-4321-b5de-ff03dcfd1b79-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "5f2b470f-b238-4321-b5de-ff03dcfd1b79" (UID: "5f2b470f-b238-4321-b5de-ff03dcfd1b79"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:36:58.886954 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:36:58.886897    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5f2b470f-b238-4321-b5de-ff03dcfd1b79-istio-envoy\") pod \"5f2b470f-b238-4321-b5de-ff03dcfd1b79\" (UID: \"5f2b470f-b238-4321-b5de-ff03dcfd1b79\") "
Sep 10 19:36:58.886954 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:36:58.886926    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5f2b470f-b238-4321-b5de-ff03dcfd1b79-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "5f2b470f-b238-4321-b5de-ff03dcfd1b79" (UID: "5f2b470f-b238-4321-b5de-ff03dcfd1b79"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:36:58.887216 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:36:58.887040    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f2b470f-b238-4321-b5de-ff03dcfd1b79-istio-data" (OuterVolumeSpecName: "istio-data") pod "5f2b470f-b238-4321-b5de-ff03dcfd1b79" (UID: "5f2b470f-b238-4321-b5de-ff03dcfd1b79"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:36:58.887216 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:36:58.887085    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f2b470f-b238-4321-b5de-ff03dcfd1b79-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "5f2b470f-b238-4321-b5de-ff03dcfd1b79" (UID: "5f2b470f-b238-4321-b5de-ff03dcfd1b79"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:36:58.887370 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:36:58.887264    2575 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5f2b470f-b238-4321-b5de-ff03dcfd1b79-workload-certs\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:36:58.887370 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:36:58.887283    2575 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f2b470f-b238-4321-b5de-ff03dcfd1b79-credential-socket\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:36:58.887370 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:36:58.887297    2575 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5f2b470f-b238-4321-b5de-ff03dcfd1b79-istio-data\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:36:58.887370 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:36:58.887311    2575 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f2b470f-b238-4321-b5de-ff03dcfd1b79-workload-socket\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:36:58.887370 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:36:58.887325    2575 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5f2b470f-b238-4321-b5de-ff03dcfd1b79-istiod-ca-cert\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:36:58.888731 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:36:58.888705    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5f2b470f-b238-4321-b5de-ff03dcfd1b79-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "5f2b470f-b238-4321-b5de-ff03dcfd1b79" (UID: "5f2b470f-b238-4321-b5de-ff03dcfd1b79"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 19:36:58.888846 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:36:58.888791    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f2b470f-b238-4321-b5de-ff03dcfd1b79-kube-api-access-bw7s7" (OuterVolumeSpecName: "kube-api-access-bw7s7") pod "5f2b470f-b238-4321-b5de-ff03dcfd1b79" (UID: "5f2b470f-b238-4321-b5de-ff03dcfd1b79"). InnerVolumeSpecName "kube-api-access-bw7s7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:36:58.888846 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:36:58.888812    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f2b470f-b238-4321-b5de-ff03dcfd1b79-istio-token" (OuterVolumeSpecName: "istio-token") pod "5f2b470f-b238-4321-b5de-ff03dcfd1b79" (UID: "5f2b470f-b238-4321-b5de-ff03dcfd1b79"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:36:58.888938 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:36:58.888927    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f2b470f-b238-4321-b5de-ff03dcfd1b79-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "5f2b470f-b238-4321-b5de-ff03dcfd1b79" (UID: "5f2b470f-b238-4321-b5de-ff03dcfd1b79"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:36:58.988071 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:36:58.988046    2575 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5f2b470f-b238-4321-b5de-ff03dcfd1b79-istio-envoy\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:36:58.988071 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:36:58.988069    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bw7s7\" (UniqueName: \"kubernetes.io/projected/5f2b470f-b238-4321-b5de-ff03dcfd1b79-kube-api-access-bw7s7\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:36:58.988214 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:36:58.988078    2575 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5f2b470f-b238-4321-b5de-ff03dcfd1b79-istio-token\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:36:58.988214 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:36:58.988088    2575 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5f2b470f-b238-4321-b5de-ff03dcfd1b79-istio-podinfo\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:36:59.749957 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:36:59.749934    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-c615f98e-openshift-default-7fbbb5c8d-dggrv"
Sep 10 19:36:59.749957 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:36:59.749932    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-c615f98e-openshift-default-7fbbb5c8d-dggrv" event={"ID":"5f2b470f-b238-4321-b5de-ff03dcfd1b79","Type":"ContainerDied","Data":"54ae713484fabff893f1a9f25195226668c02dda3f644fb7fdbc6b15b6fc11a8"}
Sep 10 19:36:59.750475 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:36:59.749999    2575 scope.go:117] "RemoveContainer" containerID="ebf25600876dd5c720229eb68bc2c3c305a02e43e950a9f8984d2914cbdd421e"
Sep 10 19:36:59.766582 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:36:59.766558    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-c615f98e-openshift-default-7fbbb5c8d-dggrv"]
Sep 10 19:36:59.772776 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:36:59.772757    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-c615f98e-openshift-default-7fbbb5c8d-dggrv"]
Sep 10 19:37:01.385518 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:01.385481    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5f2b470f-b238-4321-b5de-ff03dcfd1b79" path="/var/lib/kubelet/pods/5f2b470f-b238-4321-b5de-ff03dcfd1b79/volumes"
Sep 10 19:37:01.551649 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:01.551616    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qr7rl"]
Sep 10 19:37:02.934297 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:02.934259    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:37:03.034497 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:03.034463    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:37:03.573734 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:03.573692    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:37:09.219446 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:09.219410    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7458-59410d"]
Sep 10 19:37:09.219819 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:09.219801    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bc70833f-d7d9-444d-8756-4f872b7fafe6" containerName="main"
Sep 10 19:37:09.219819 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:09.219813    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="bc70833f-d7d9-444d-8756-4f872b7fafe6" containerName="main"
Sep 10 19:37:09.219894 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:09.219834    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f2b470f-b238-4321-b5de-ff03dcfd1b79" containerName="istio-proxy"
Sep 10 19:37:09.219894 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:09.219840    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f2b470f-b238-4321-b5de-ff03dcfd1b79" containerName="istio-proxy"
Sep 10 19:37:09.219894 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:09.219849    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fa64f873-f7ab-45e4-bae5-45ff78ab2c01" containerName="payload-processing"
Sep 10 19:37:09.219894 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:09.219856    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="fa64f873-f7ab-45e4-bae5-45ff78ab2c01" containerName="payload-processing"
Sep 10 19:37:09.220015 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:09.219920    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="fa64f873-f7ab-45e4-bae5-45ff78ab2c01" containerName="payload-processing"
Sep 10 19:37:09.220015 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:09.219931    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f2b470f-b238-4321-b5de-ff03dcfd1b79" containerName="istio-proxy"
Sep 10 19:37:09.220015 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:09.219939    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="bc70833f-d7d9-444d-8756-4f872b7fafe6" containerName="main"
Sep 10 19:37:09.224637 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:09.224618    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7458-59410d"
Sep 10 19:37:09.226773 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:09.226752    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-8vfd9\""
Sep 10 19:37:09.231643 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:09.231622    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7458-59410d"]
Sep 10 19:37:09.275949 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:09.275928    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tx58h\" (UniqueName: \"kubernetes.io/projected/44f59ab4-b3d7-49ed-a78d-0c5968d4d88e-kube-api-access-tx58h\") pod \"test-curl-7458-59410d\" (UID: \"44f59ab4-b3d7-49ed-a78d-0c5968d4d88e\") " pod="opendatahub/test-curl-7458-59410d"
Sep 10 19:37:09.377322 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:09.377296    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tx58h\" (UniqueName: \"kubernetes.io/projected/44f59ab4-b3d7-49ed-a78d-0c5968d4d88e-kube-api-access-tx58h\") pod \"test-curl-7458-59410d\" (UID: \"44f59ab4-b3d7-49ed-a78d-0c5968d4d88e\") " pod="opendatahub/test-curl-7458-59410d"
Sep 10 19:37:09.385466 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:09.385439    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tx58h\" (UniqueName: \"kubernetes.io/projected/44f59ab4-b3d7-49ed-a78d-0c5968d4d88e-kube-api-access-tx58h\") pod \"test-curl-7458-59410d\" (UID: \"44f59ab4-b3d7-49ed-a78d-0c5968d4d88e\") " pod="opendatahub/test-curl-7458-59410d"
Sep 10 19:37:09.535071 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:09.535020    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7458-59410d"
Sep 10 19:37:09.694079 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:09.694045    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:37:09.800975 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:09.800893    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:37:09.806711 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:09.806691    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7458-59410d"]
Sep 10 19:37:09.808601 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:37:09.808574    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod44f59ab4_b3d7_49ed_a78d_0c5968d4d88e.slice/crio-046b322e2944e10dd08cbb119dd812dec7499a4e3ac18bd0d7f1757806e5305c WatchSource:0}: Error finding container 046b322e2944e10dd08cbb119dd812dec7499a4e3ac18bd0d7f1757806e5305c: Status 404 returned error can't find the container with id 046b322e2944e10dd08cbb119dd812dec7499a4e3ac18bd0d7f1757806e5305c
Sep 10 19:37:09.894055 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:09.894032    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:37:10.805992 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:10.805958    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7458-59410d" event={"ID":"44f59ab4-b3d7-49ed-a78d-0c5968d4d88e","Type":"ContainerStarted","Data":"36abcff198738a56c6f30921670ccfc24bf5f046759fdf35891114d9eff5183d"}
Sep 10 19:37:10.805992 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:10.805998    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7458-59410d" event={"ID":"44f59ab4-b3d7-49ed-a78d-0c5968d4d88e","Type":"ContainerStarted","Data":"046b322e2944e10dd08cbb119dd812dec7499a4e3ac18bd0d7f1757806e5305c"}
Sep 10 19:37:10.820884 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:10.820840    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7458-59410d" podStartSLOduration=1.82082681 podStartE2EDuration="1.82082681s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:37:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:37:10.818791681 +0000 UTC m=+1301.878619349" watchObservedRunningTime="2026-09-10 19:37:10.82082681 +0000 UTC m=+1301.880654477"
Sep 10 19:37:11.387023 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:11.386991    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7458-59410d"]
Sep 10 19:37:11.391430 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:11.391403    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7458-59410d"]
Sep 10 19:37:11.812277 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:37:11.812240    2575 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7458-59410d" container="test-curl-7458-59410d"
Sep 10 19:37:12.711350 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:12.711298    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:37:12.818086 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:12.818041    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7458-59410d" podUID="44f59ab4-b3d7-49ed-a78d-0c5968d4d88e" containerName="test-curl-7458-59410d" containerID="cri-o://36abcff198738a56c6f30921670ccfc24bf5f046759fdf35891114d9eff5183d" gracePeriod=2
Sep 10 19:37:12.819785 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:37:12.819758    2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7458-59410d\" is forbidden: User \"system:node:ip-10-0-129-114.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-129-114.ec2.internal' and this object" podUID="44f59ab4-b3d7-49ed-a78d-0c5968d4d88e" pod="opendatahub/test-curl-7458-59410d"
Sep 10 19:37:13.586239 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:13.586205    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:37:13.846535 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:13.846459    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qr7rl"]
Sep 10 19:37:14.980353 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:14.980322    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7458-59410d"
Sep 10 19:37:15.129492 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:15.129434    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tx58h\" (UniqueName: \"kubernetes.io/projected/44f59ab4-b3d7-49ed-a78d-0c5968d4d88e-kube-api-access-tx58h\") pod \"44f59ab4-b3d7-49ed-a78d-0c5968d4d88e\" (UID: \"44f59ab4-b3d7-49ed-a78d-0c5968d4d88e\") "
Sep 10 19:37:15.131428 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:15.131401    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/44f59ab4-b3d7-49ed-a78d-0c5968d4d88e-kube-api-access-tx58h" (OuterVolumeSpecName: "kube-api-access-tx58h") pod "44f59ab4-b3d7-49ed-a78d-0c5968d4d88e" (UID: "44f59ab4-b3d7-49ed-a78d-0c5968d4d88e"). InnerVolumeSpecName "kube-api-access-tx58h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:37:15.230549 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:15.230527    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tx58h\" (UniqueName: \"kubernetes.io/projected/44f59ab4-b3d7-49ed-a78d-0c5968d4d88e-kube-api-access-tx58h\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:37:15.384557 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:15.384485    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="44f59ab4-b3d7-49ed-a78d-0c5968d4d88e" path="/var/lib/kubelet/pods/44f59ab4-b3d7-49ed-a78d-0c5968d4d88e/volumes"
Sep 10 19:37:15.835737 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:15.835699    2575 generic.go:356] "Generic (PLEG): container finished" podID="44f59ab4-b3d7-49ed-a78d-0c5968d4d88e" containerID="36abcff198738a56c6f30921670ccfc24bf5f046759fdf35891114d9eff5183d" exitCode=137
Sep 10 19:37:15.835879 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:15.835742    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7458-59410d"
Sep 10 19:37:15.835879 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:15.835790    2575 scope.go:117] "RemoveContainer" containerID="36abcff198738a56c6f30921670ccfc24bf5f046759fdf35891114d9eff5183d"
Sep 10 19:37:15.844150 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:15.844131    2575 scope.go:117] "RemoveContainer" containerID="36abcff198738a56c6f30921670ccfc24bf5f046759fdf35891114d9eff5183d"
Sep 10 19:37:15.844449 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:37:15.844427    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"36abcff198738a56c6f30921670ccfc24bf5f046759fdf35891114d9eff5183d\": container with ID starting with 36abcff198738a56c6f30921670ccfc24bf5f046759fdf35891114d9eff5183d not found: ID does not exist" containerID="36abcff198738a56c6f30921670ccfc24bf5f046759fdf35891114d9eff5183d"
Sep 10 19:37:15.844520 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:15.844456    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"36abcff198738a56c6f30921670ccfc24bf5f046759fdf35891114d9eff5183d"} err="failed to get container status \"36abcff198738a56c6f30921670ccfc24bf5f046759fdf35891114d9eff5183d\": rpc error: code = NotFound desc = could not find container \"36abcff198738a56c6f30921670ccfc24bf5f046759fdf35891114d9eff5183d\": container with ID starting with 36abcff198738a56c6f30921670ccfc24bf5f046759fdf35891114d9eff5183d not found: ID does not exist"
Sep 10 19:37:20.363172 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:20.363139    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:37:20.584178 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:20.584145    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:37:24.343768 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:24.343733    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qr7rl"]
Sep 10 19:37:35.385755 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:35.385724    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-14a8d1/sub-test-model-14a8d1-kserve-96c54d5cc-dstjk"]
Sep 10 19:37:35.386134 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:35.386082    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f2b470f-b238-4321-b5de-ff03dcfd1b79" containerName="istio-proxy"
Sep 10 19:37:35.386134 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:35.386092    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f2b470f-b238-4321-b5de-ff03dcfd1b79" containerName="istio-proxy"
Sep 10 19:37:35.386134 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:35.386103    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44f59ab4-b3d7-49ed-a78d-0c5968d4d88e" containerName="test-curl-7458-59410d"
Sep 10 19:37:35.386134 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:35.386108    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="44f59ab4-b3d7-49ed-a78d-0c5968d4d88e" containerName="test-curl-7458-59410d"
Sep 10 19:37:35.386266 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:35.386174    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f2b470f-b238-4321-b5de-ff03dcfd1b79" containerName="istio-proxy"
Sep 10 19:37:35.386266 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:35.386187    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="44f59ab4-b3d7-49ed-a78d-0c5968d4d88e" containerName="test-curl-7458-59410d"
Sep 10 19:37:35.390494 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:35.390477    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-14a8d1/sub-test-model-14a8d1-kserve-96c54d5cc-dstjk"
Sep 10 19:37:35.392629 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:35.392602    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-14a8d1\"/\"sub-test-model-14a8d1-kserve-self-signed-certs\""
Sep 10 19:37:35.392765 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:35.392632    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-14a8d1\"/\"openshift-service-ca.crt\""
Sep 10 19:37:35.392765 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:35.392633    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-14a8d1\"/\"kube-root-ca.crt\""
Sep 10 19:37:35.392765 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:35.392603    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-14a8d1\"/\"default-dockercfg-s9krw\""
Sep 10 19:37:35.396745 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:35.396724    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-14a8d1/sub-test-model-14a8d1-kserve-96c54d5cc-dstjk"]
Sep 10 19:37:35.483774 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:35.483738    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:37:35.501825 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:35.501798    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e870f493-ce00-4586-930a-098ede28e8ba-tmp-dir\") pod \"sub-test-model-14a8d1-kserve-96c54d5cc-dstjk\" (UID: \"e870f493-ce00-4586-930a-098ede28e8ba\") " pod="ai-tenant-e2e-shared-a-w3-14a8d1/sub-test-model-14a8d1-kserve-96c54d5cc-dstjk"
Sep 10 19:37:35.501941 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:35.501847    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hm76b\" (UniqueName: \"kubernetes.io/projected/e870f493-ce00-4586-930a-098ede28e8ba-kube-api-access-hm76b\") pod \"sub-test-model-14a8d1-kserve-96c54d5cc-dstjk\" (UID: \"e870f493-ce00-4586-930a-098ede28e8ba\") " pod="ai-tenant-e2e-shared-a-w3-14a8d1/sub-test-model-14a8d1-kserve-96c54d5cc-dstjk"
Sep 10 19:37:35.501941 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:35.501875    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e870f493-ce00-4586-930a-098ede28e8ba-tls-certs\") pod \"sub-test-model-14a8d1-kserve-96c54d5cc-dstjk\" (UID: \"e870f493-ce00-4586-930a-098ede28e8ba\") " pod="ai-tenant-e2e-shared-a-w3-14a8d1/sub-test-model-14a8d1-kserve-96c54d5cc-dstjk"
Sep 10 19:37:35.502031 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:35.501968    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e870f493-ce00-4586-930a-098ede28e8ba-home\") pod \"sub-test-model-14a8d1-kserve-96c54d5cc-dstjk\" (UID: \"e870f493-ce00-4586-930a-098ede28e8ba\") " pod="ai-tenant-e2e-shared-a-w3-14a8d1/sub-test-model-14a8d1-kserve-96c54d5cc-dstjk"
Sep 10 19:37:35.502031 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:35.502001    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e870f493-ce00-4586-930a-098ede28e8ba-dshm\") pod \"sub-test-model-14a8d1-kserve-96c54d5cc-dstjk\" (UID: \"e870f493-ce00-4586-930a-098ede28e8ba\") " pod="ai-tenant-e2e-shared-a-w3-14a8d1/sub-test-model-14a8d1-kserve-96c54d5cc-dstjk"
Sep 10 19:37:35.502128 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:35.502038    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e870f493-ce00-4586-930a-098ede28e8ba-model-cache\") pod \"sub-test-model-14a8d1-kserve-96c54d5cc-dstjk\" (UID: \"e870f493-ce00-4586-930a-098ede28e8ba\") " pod="ai-tenant-e2e-shared-a-w3-14a8d1/sub-test-model-14a8d1-kserve-96c54d5cc-dstjk"
Sep 10 19:37:35.602993 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:35.602963    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e870f493-ce00-4586-930a-098ede28e8ba-model-cache\") pod \"sub-test-model-14a8d1-kserve-96c54d5cc-dstjk\" (UID: \"e870f493-ce00-4586-930a-098ede28e8ba\") " pod="ai-tenant-e2e-shared-a-w3-14a8d1/sub-test-model-14a8d1-kserve-96c54d5cc-dstjk"
Sep 10 19:37:35.603125 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:35.603001    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e870f493-ce00-4586-930a-098ede28e8ba-tmp-dir\") pod \"sub-test-model-14a8d1-kserve-96c54d5cc-dstjk\" (UID: \"e870f493-ce00-4586-930a-098ede28e8ba\") " pod="ai-tenant-e2e-shared-a-w3-14a8d1/sub-test-model-14a8d1-kserve-96c54d5cc-dstjk"
Sep 10 19:37:35.603125 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:35.603048    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hm76b\" (UniqueName: \"kubernetes.io/projected/e870f493-ce00-4586-930a-098ede28e8ba-kube-api-access-hm76b\") pod \"sub-test-model-14a8d1-kserve-96c54d5cc-dstjk\" (UID: \"e870f493-ce00-4586-930a-098ede28e8ba\") " pod="ai-tenant-e2e-shared-a-w3-14a8d1/sub-test-model-14a8d1-kserve-96c54d5cc-dstjk"
Sep 10 19:37:35.603125 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:35.603073    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e870f493-ce00-4586-930a-098ede28e8ba-tls-certs\") pod \"sub-test-model-14a8d1-kserve-96c54d5cc-dstjk\" (UID: \"e870f493-ce00-4586-930a-098ede28e8ba\") " pod="ai-tenant-e2e-shared-a-w3-14a8d1/sub-test-model-14a8d1-kserve-96c54d5cc-dstjk"
Sep 10 19:37:35.603311 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:35.603292    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e870f493-ce00-4586-930a-098ede28e8ba-home\") pod \"sub-test-model-14a8d1-kserve-96c54d5cc-dstjk\" (UID: \"e870f493-ce00-4586-930a-098ede28e8ba\") " pod="ai-tenant-e2e-shared-a-w3-14a8d1/sub-test-model-14a8d1-kserve-96c54d5cc-dstjk"
Sep 10 19:37:35.603389 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:35.603336    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e870f493-ce00-4586-930a-098ede28e8ba-dshm\") pod \"sub-test-model-14a8d1-kserve-96c54d5cc-dstjk\" (UID: \"e870f493-ce00-4586-930a-098ede28e8ba\") " pod="ai-tenant-e2e-shared-a-w3-14a8d1/sub-test-model-14a8d1-kserve-96c54d5cc-dstjk"
Sep 10 19:37:35.603453 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:35.603414    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e870f493-ce00-4586-930a-098ede28e8ba-model-cache\") pod \"sub-test-model-14a8d1-kserve-96c54d5cc-dstjk\" (UID: \"e870f493-ce00-4586-930a-098ede28e8ba\") " pod="ai-tenant-e2e-shared-a-w3-14a8d1/sub-test-model-14a8d1-kserve-96c54d5cc-dstjk"
Sep 10 19:37:35.603453 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:35.603442    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e870f493-ce00-4586-930a-098ede28e8ba-tmp-dir\") pod \"sub-test-model-14a8d1-kserve-96c54d5cc-dstjk\" (UID: \"e870f493-ce00-4586-930a-098ede28e8ba\") " pod="ai-tenant-e2e-shared-a-w3-14a8d1/sub-test-model-14a8d1-kserve-96c54d5cc-dstjk"
Sep 10 19:37:35.603638 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:35.603619    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e870f493-ce00-4586-930a-098ede28e8ba-home\") pod \"sub-test-model-14a8d1-kserve-96c54d5cc-dstjk\" (UID: \"e870f493-ce00-4586-930a-098ede28e8ba\") " pod="ai-tenant-e2e-shared-a-w3-14a8d1/sub-test-model-14a8d1-kserve-96c54d5cc-dstjk"
Sep 10 19:37:35.605422 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:35.605392    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e870f493-ce00-4586-930a-098ede28e8ba-dshm\") pod \"sub-test-model-14a8d1-kserve-96c54d5cc-dstjk\" (UID: \"e870f493-ce00-4586-930a-098ede28e8ba\") " pod="ai-tenant-e2e-shared-a-w3-14a8d1/sub-test-model-14a8d1-kserve-96c54d5cc-dstjk"
Sep 10 19:37:35.605537 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:35.605522    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e870f493-ce00-4586-930a-098ede28e8ba-tls-certs\") pod \"sub-test-model-14a8d1-kserve-96c54d5cc-dstjk\" (UID: \"e870f493-ce00-4586-930a-098ede28e8ba\") " pod="ai-tenant-e2e-shared-a-w3-14a8d1/sub-test-model-14a8d1-kserve-96c54d5cc-dstjk"
Sep 10 19:37:35.612280 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:35.612260    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hm76b\" (UniqueName: \"kubernetes.io/projected/e870f493-ce00-4586-930a-098ede28e8ba-kube-api-access-hm76b\") pod \"sub-test-model-14a8d1-kserve-96c54d5cc-dstjk\" (UID: \"e870f493-ce00-4586-930a-098ede28e8ba\") " pod="ai-tenant-e2e-shared-a-w3-14a8d1/sub-test-model-14a8d1-kserve-96c54d5cc-dstjk"
Sep 10 19:37:35.701525 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:35.701503    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-14a8d1/sub-test-model-14a8d1-kserve-96c54d5cc-dstjk"
Sep 10 19:37:35.977318 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:35.977251    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-14a8d1/sub-test-model-14a8d1-kserve-96c54d5cc-dstjk"]
Sep 10 19:37:35.980684 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:37:35.980650    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode870f493_ce00_4586_930a_098ede28e8ba.slice/crio-9ebd9659c7d3a86e5a8206ba8d2f201a7f2e3105f67392ec1ffd62468746727e WatchSource:0}: Error finding container 9ebd9659c7d3a86e5a8206ba8d2f201a7f2e3105f67392ec1ffd62468746727e: Status 404 returned error can't find the container with id 9ebd9659c7d3a86e5a8206ba8d2f201a7f2e3105f67392ec1ffd62468746727e
Sep 10 19:37:36.937755 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:36.937717    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-14a8d1/sub-test-model-14a8d1-kserve-96c54d5cc-dstjk" event={"ID":"e870f493-ce00-4586-930a-098ede28e8ba","Type":"ContainerStarted","Data":"940d1acd090ae25e25190cb807900465b45abf2810e804274b15283e193963ce"}
Sep 10 19:37:36.938113 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:36.937767    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-14a8d1/sub-test-model-14a8d1-kserve-96c54d5cc-dstjk" event={"ID":"e870f493-ce00-4586-930a-098ede28e8ba","Type":"ContainerStarted","Data":"9ebd9659c7d3a86e5a8206ba8d2f201a7f2e3105f67392ec1ffd62468746727e"}
Sep 10 19:37:36.955280 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:36.955232    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-14a8d1/sub-test-model-14a8d1-kserve-96c54d5cc-dstjk" podStartSLOduration=1.955214432 podStartE2EDuration="1.955214432s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:37:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:37:36.953535956 +0000 UTC m=+1328.013363624" watchObservedRunningTime="2026-09-10 19:37:36.955214432 +0000 UTC m=+1328.015042097"
Sep 10 19:37:39.156462 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:39.156427    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qr7rl"]
Sep 10 19:37:41.655450 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:41.655418    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:37:45.701624 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:45.701585    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-14a8d1/sub-test-model-14a8d1-kserve-96c54d5cc-dstjk"
Sep 10 19:37:45.701624 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:45.701628    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-14a8d1/sub-test-model-14a8d1-kserve-96c54d5cc-dstjk"
Sep 10 19:37:45.713818 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:45.713796    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-14a8d1/sub-test-model-14a8d1-kserve-96c54d5cc-dstjk"
Sep 10 19:37:45.994027 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:45.993945    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-14a8d1/sub-test-model-14a8d1-kserve-96c54d5cc-dstjk"
Sep 10 19:37:47.584214 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:47.584174    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:37:47.764266 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:47.764225    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:37:48.067538 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:48.067498    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:37:48.400430 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:48.400314    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-439275-openshift-default-666b8ddd88-4jpmd"]
Sep 10 19:37:48.405227 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:48.405203    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-439275-openshift-default-666b8ddd88-4jpmd"
Sep 10 19:37:48.407214 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:48.407192    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-439275-openshift-default-dockercfg-qh9p6\""
Sep 10 19:37:48.414816 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:48.414794    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-439275-openshift-default-666b8ddd88-4jpmd"]
Sep 10 19:37:48.427647 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:48.427621    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:37:48.506163 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:48.506140    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c8cf5d22-c0dd-4995-8d48-8a993dbae7cc-istio-envoy\") pod \"e2e-api-a-439275-openshift-default-666b8ddd88-4jpmd\" (UID: \"c8cf5d22-c0dd-4995-8d48-8a993dbae7cc\") " pod="openshift-ingress/e2e-api-a-439275-openshift-default-666b8ddd88-4jpmd"
Sep 10 19:37:48.506310 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:48.506167    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8cf5d22-c0dd-4995-8d48-8a993dbae7cc-credential-socket\") pod \"e2e-api-a-439275-openshift-default-666b8ddd88-4jpmd\" (UID: \"c8cf5d22-c0dd-4995-8d48-8a993dbae7cc\") " pod="openshift-ingress/e2e-api-a-439275-openshift-default-666b8ddd88-4jpmd"
Sep 10 19:37:48.506310 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:48.506190    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c8cf5d22-c0dd-4995-8d48-8a993dbae7cc-istio-token\") pod \"e2e-api-a-439275-openshift-default-666b8ddd88-4jpmd\" (UID: \"c8cf5d22-c0dd-4995-8d48-8a993dbae7cc\") " pod="openshift-ingress/e2e-api-a-439275-openshift-default-666b8ddd88-4jpmd"
Sep 10 19:37:48.506310 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:48.506243    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c4fqb\" (UniqueName: \"kubernetes.io/projected/c8cf5d22-c0dd-4995-8d48-8a993dbae7cc-kube-api-access-c4fqb\") pod \"e2e-api-a-439275-openshift-default-666b8ddd88-4jpmd\" (UID: \"c8cf5d22-c0dd-4995-8d48-8a993dbae7cc\") " pod="openshift-ingress/e2e-api-a-439275-openshift-default-666b8ddd88-4jpmd"
Sep 10 19:37:48.506310 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:48.506284    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c8cf5d22-c0dd-4995-8d48-8a993dbae7cc-istio-podinfo\") pod \"e2e-api-a-439275-openshift-default-666b8ddd88-4jpmd\" (UID: \"c8cf5d22-c0dd-4995-8d48-8a993dbae7cc\") " pod="openshift-ingress/e2e-api-a-439275-openshift-default-666b8ddd88-4jpmd"
Sep 10 19:37:48.506533 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:48.506365    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8cf5d22-c0dd-4995-8d48-8a993dbae7cc-workload-socket\") pod \"e2e-api-a-439275-openshift-default-666b8ddd88-4jpmd\" (UID: \"c8cf5d22-c0dd-4995-8d48-8a993dbae7cc\") " pod="openshift-ingress/e2e-api-a-439275-openshift-default-666b8ddd88-4jpmd"
Sep 10 19:37:48.506533 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:48.506399    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c8cf5d22-c0dd-4995-8d48-8a993dbae7cc-istiod-ca-cert\") pod \"e2e-api-a-439275-openshift-default-666b8ddd88-4jpmd\" (UID: \"c8cf5d22-c0dd-4995-8d48-8a993dbae7cc\") " pod="openshift-ingress/e2e-api-a-439275-openshift-default-666b8ddd88-4jpmd"
Sep 10 19:37:48.506533 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:48.506440    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c8cf5d22-c0dd-4995-8d48-8a993dbae7cc-workload-certs\") pod \"e2e-api-a-439275-openshift-default-666b8ddd88-4jpmd\" (UID: \"c8cf5d22-c0dd-4995-8d48-8a993dbae7cc\") " pod="openshift-ingress/e2e-api-a-439275-openshift-default-666b8ddd88-4jpmd"
Sep 10 19:37:48.506533 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:48.506478    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c8cf5d22-c0dd-4995-8d48-8a993dbae7cc-istio-data\") pod \"e2e-api-a-439275-openshift-default-666b8ddd88-4jpmd\" (UID: \"c8cf5d22-c0dd-4995-8d48-8a993dbae7cc\") " pod="openshift-ingress/e2e-api-a-439275-openshift-default-666b8ddd88-4jpmd"
Sep 10 19:37:48.607515 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:48.607484    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8cf5d22-c0dd-4995-8d48-8a993dbae7cc-workload-socket\") pod \"e2e-api-a-439275-openshift-default-666b8ddd88-4jpmd\" (UID: \"c8cf5d22-c0dd-4995-8d48-8a993dbae7cc\") " pod="openshift-ingress/e2e-api-a-439275-openshift-default-666b8ddd88-4jpmd"
Sep 10 19:37:48.607515 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:48.607520    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c8cf5d22-c0dd-4995-8d48-8a993dbae7cc-istiod-ca-cert\") pod \"e2e-api-a-439275-openshift-default-666b8ddd88-4jpmd\" (UID: \"c8cf5d22-c0dd-4995-8d48-8a993dbae7cc\") " pod="openshift-ingress/e2e-api-a-439275-openshift-default-666b8ddd88-4jpmd"
Sep 10 19:37:48.607999 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:48.607543    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c8cf5d22-c0dd-4995-8d48-8a993dbae7cc-workload-certs\") pod \"e2e-api-a-439275-openshift-default-666b8ddd88-4jpmd\" (UID: \"c8cf5d22-c0dd-4995-8d48-8a993dbae7cc\") " pod="openshift-ingress/e2e-api-a-439275-openshift-default-666b8ddd88-4jpmd"
Sep 10 19:37:48.607999 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:48.607569    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c8cf5d22-c0dd-4995-8d48-8a993dbae7cc-istio-data\") pod \"e2e-api-a-439275-openshift-default-666b8ddd88-4jpmd\" (UID: \"c8cf5d22-c0dd-4995-8d48-8a993dbae7cc\") " pod="openshift-ingress/e2e-api-a-439275-openshift-default-666b8ddd88-4jpmd"
Sep 10 19:37:48.607999 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:48.607624    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c8cf5d22-c0dd-4995-8d48-8a993dbae7cc-istio-envoy\") pod \"e2e-api-a-439275-openshift-default-666b8ddd88-4jpmd\" (UID: \"c8cf5d22-c0dd-4995-8d48-8a993dbae7cc\") " pod="openshift-ingress/e2e-api-a-439275-openshift-default-666b8ddd88-4jpmd"
Sep 10 19:37:48.607999 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:48.607639    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8cf5d22-c0dd-4995-8d48-8a993dbae7cc-credential-socket\") pod \"e2e-api-a-439275-openshift-default-666b8ddd88-4jpmd\" (UID: \"c8cf5d22-c0dd-4995-8d48-8a993dbae7cc\") " pod="openshift-ingress/e2e-api-a-439275-openshift-default-666b8ddd88-4jpmd"
Sep 10 19:37:48.607999 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:48.607668    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c8cf5d22-c0dd-4995-8d48-8a993dbae7cc-istio-token\") pod \"e2e-api-a-439275-openshift-default-666b8ddd88-4jpmd\" (UID: \"c8cf5d22-c0dd-4995-8d48-8a993dbae7cc\") " pod="openshift-ingress/e2e-api-a-439275-openshift-default-666b8ddd88-4jpmd"
Sep 10 19:37:48.607999 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:48.607695    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c4fqb\" (UniqueName: \"kubernetes.io/projected/c8cf5d22-c0dd-4995-8d48-8a993dbae7cc-kube-api-access-c4fqb\") pod \"e2e-api-a-439275-openshift-default-666b8ddd88-4jpmd\" (UID: \"c8cf5d22-c0dd-4995-8d48-8a993dbae7cc\") " pod="openshift-ingress/e2e-api-a-439275-openshift-default-666b8ddd88-4jpmd"
Sep 10 19:37:48.607999 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:48.607720    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c8cf5d22-c0dd-4995-8d48-8a993dbae7cc-istio-podinfo\") pod \"e2e-api-a-439275-openshift-default-666b8ddd88-4jpmd\" (UID: \"c8cf5d22-c0dd-4995-8d48-8a993dbae7cc\") " pod="openshift-ingress/e2e-api-a-439275-openshift-default-666b8ddd88-4jpmd"
Sep 10 19:37:48.607999 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:48.607910    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8cf5d22-c0dd-4995-8d48-8a993dbae7cc-workload-socket\") pod \"e2e-api-a-439275-openshift-default-666b8ddd88-4jpmd\" (UID: \"c8cf5d22-c0dd-4995-8d48-8a993dbae7cc\") " pod="openshift-ingress/e2e-api-a-439275-openshift-default-666b8ddd88-4jpmd"
Sep 10 19:37:48.607999 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:48.607930    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c8cf5d22-c0dd-4995-8d48-8a993dbae7cc-workload-certs\") pod \"e2e-api-a-439275-openshift-default-666b8ddd88-4jpmd\" (UID: \"c8cf5d22-c0dd-4995-8d48-8a993dbae7cc\") " pod="openshift-ingress/e2e-api-a-439275-openshift-default-666b8ddd88-4jpmd"
Sep 10 19:37:48.608496 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:48.608040    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c8cf5d22-c0dd-4995-8d48-8a993dbae7cc-istio-data\") pod \"e2e-api-a-439275-openshift-default-666b8ddd88-4jpmd\" (UID: \"c8cf5d22-c0dd-4995-8d48-8a993dbae7cc\") " pod="openshift-ingress/e2e-api-a-439275-openshift-default-666b8ddd88-4jpmd"
Sep 10 19:37:48.608496 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:48.608254    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8cf5d22-c0dd-4995-8d48-8a993dbae7cc-credential-socket\") pod \"e2e-api-a-439275-openshift-default-666b8ddd88-4jpmd\" (UID: \"c8cf5d22-c0dd-4995-8d48-8a993dbae7cc\") " pod="openshift-ingress/e2e-api-a-439275-openshift-default-666b8ddd88-4jpmd"
Sep 10 19:37:48.608496 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:48.608368    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c8cf5d22-c0dd-4995-8d48-8a993dbae7cc-istiod-ca-cert\") pod \"e2e-api-a-439275-openshift-default-666b8ddd88-4jpmd\" (UID: \"c8cf5d22-c0dd-4995-8d48-8a993dbae7cc\") " pod="openshift-ingress/e2e-api-a-439275-openshift-default-666b8ddd88-4jpmd"
Sep 10 19:37:48.609984 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:48.609964    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c8cf5d22-c0dd-4995-8d48-8a993dbae7cc-istio-envoy\") pod \"e2e-api-a-439275-openshift-default-666b8ddd88-4jpmd\" (UID: \"c8cf5d22-c0dd-4995-8d48-8a993dbae7cc\") " pod="openshift-ingress/e2e-api-a-439275-openshift-default-666b8ddd88-4jpmd"
Sep 10 19:37:48.610222 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:48.610206    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c8cf5d22-c0dd-4995-8d48-8a993dbae7cc-istio-podinfo\") pod \"e2e-api-a-439275-openshift-default-666b8ddd88-4jpmd\" (UID: \"c8cf5d22-c0dd-4995-8d48-8a993dbae7cc\") " pod="openshift-ingress/e2e-api-a-439275-openshift-default-666b8ddd88-4jpmd"
Sep 10 19:37:48.615392 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:48.615372    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c8cf5d22-c0dd-4995-8d48-8a993dbae7cc-istio-token\") pod \"e2e-api-a-439275-openshift-default-666b8ddd88-4jpmd\" (UID: \"c8cf5d22-c0dd-4995-8d48-8a993dbae7cc\") " pod="openshift-ingress/e2e-api-a-439275-openshift-default-666b8ddd88-4jpmd"
Sep 10 19:37:48.615603 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:48.615587    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c4fqb\" (UniqueName: \"kubernetes.io/projected/c8cf5d22-c0dd-4995-8d48-8a993dbae7cc-kube-api-access-c4fqb\") pod \"e2e-api-a-439275-openshift-default-666b8ddd88-4jpmd\" (UID: \"c8cf5d22-c0dd-4995-8d48-8a993dbae7cc\") " pod="openshift-ingress/e2e-api-a-439275-openshift-default-666b8ddd88-4jpmd"
Sep 10 19:37:48.717855 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:48.717828    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-439275-openshift-default-666b8ddd88-4jpmd"
Sep 10 19:37:48.927614 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:48.927562    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:37:48.993836 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:48.993757    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-439275-openshift-default-666b8ddd88-4jpmd"]
Sep 10 19:37:48.997726 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:37:48.997699    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8cf5d22_c0dd_4995_8d48_8a993dbae7cc.slice/crio-fe5fb3379a2fc8675169878ee8e8df45d46271941b0db8f5b5fd2f674dd4c82c WatchSource:0}: Error finding container fe5fb3379a2fc8675169878ee8e8df45d46271941b0db8f5b5fd2f674dd4c82c: Status 404 returned error can't find the container with id fe5fb3379a2fc8675169878ee8e8df45d46271941b0db8f5b5fd2f674dd4c82c
Sep 10 19:37:49.000018 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:48.999961    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 19:37:49.000112 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:49.000051    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 19:37:49.000112 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:49.000083    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 19:37:50.005684 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:50.005645    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-439275-openshift-default-666b8ddd88-4jpmd" event={"ID":"c8cf5d22-c0dd-4995-8d48-8a993dbae7cc","Type":"ContainerStarted","Data":"014f2fd7664bafe6b6a5acb7dc75a227a7797da3226373cc1c64c74b39cb14bf"}
Sep 10 19:37:50.006037 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:50.005743    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-439275-openshift-default-666b8ddd88-4jpmd" event={"ID":"c8cf5d22-c0dd-4995-8d48-8a993dbae7cc","Type":"ContainerStarted","Data":"fe5fb3379a2fc8675169878ee8e8df45d46271941b0db8f5b5fd2f674dd4c82c"}
Sep 10 19:37:50.030751 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:50.030705    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-439275-openshift-default-666b8ddd88-4jpmd" podStartSLOduration=2.030690112 podStartE2EDuration="2.030690112s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:37:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:37:50.029061937 +0000 UTC m=+1341.088889606" watchObservedRunningTime="2026-09-10 19:37:50.030690112 +0000 UTC m=+1341.090517780"
Sep 10 19:37:50.718720 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:50.718686    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-439275-openshift-default-666b8ddd88-4jpmd"
Sep 10 19:37:50.723867 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:50.723841    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-439275-openshift-default-666b8ddd88-4jpmd"
Sep 10 19:37:51.010760 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:51.010672    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-439275-openshift-default-666b8ddd88-4jpmd"
Sep 10 19:37:51.011465 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:51.011446    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-439275-openshift-default-666b8ddd88-4jpmd"
Sep 10 19:37:52.232740 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:52.232653    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:37:52.367858 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:52.367828    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:37:52.456255 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:52.456223    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:37:53.663195 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:53.663169    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:37:53.763025 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:53.762989    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:37:53.863387 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:53.863332    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:37:54.171135 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:54.171098    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:37:54.273711 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:54.273675    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:37:59.052076 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:37:59.052039    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qr7rl"]
Sep 10 19:38:08.395465 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:38:08.395423    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:38:10.109983 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:38:10.109949    2575 generic.go:356] "Generic (PLEG): container finished" podID="c8cf5d22-c0dd-4995-8d48-8a993dbae7cc" containerID="014f2fd7664bafe6b6a5acb7dc75a227a7797da3226373cc1c64c74b39cb14bf" exitCode=0
Sep 10 19:38:10.110409 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:38:10.110019    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-439275-openshift-default-666b8ddd88-4jpmd" event={"ID":"c8cf5d22-c0dd-4995-8d48-8a993dbae7cc","Type":"ContainerDied","Data":"014f2fd7664bafe6b6a5acb7dc75a227a7797da3226373cc1c64c74b39cb14bf"}
Sep 10 19:38:10.110409 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:38:10.110398    2575 scope.go:117] "RemoveContainer" containerID="014f2fd7664bafe6b6a5acb7dc75a227a7797da3226373cc1c64c74b39cb14bf"
Sep 10 19:38:10.111416 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:38:10.111385    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 19:38:10.111521 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:38:10.111450    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 19:38:10.111521 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:38:10.111480    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 19:38:11.118212 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:38:11.118168    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-439275-openshift-default-666b8ddd88-4jpmd" event={"ID":"c8cf5d22-c0dd-4995-8d48-8a993dbae7cc","Type":"ContainerStarted","Data":"7e4432911bb373b6de8b1890ec2f9b181167ca584e79dbfcf5d38a80005304c6"}
Sep 10 19:38:11.718618 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:38:11.718580    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-439275-openshift-default-666b8ddd88-4jpmd"
Sep 10 19:38:11.718823 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:38:11.718701    2575 patch_prober.go:28] interesting pod/e2e-api-a-439275-openshift-default-666b8ddd88-4jpmd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Sep 10 19:38:11.718823 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:38:11.718744    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-439275-openshift-default-666b8ddd88-4jpmd" podUID="c8cf5d22-c0dd-4995-8d48-8a993dbae7cc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Sep 10 19:38:12.718883 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:38:12.718842    2575 patch_prober.go:28] interesting pod/e2e-api-a-439275-openshift-default-666b8ddd88-4jpmd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Sep 10 19:38:12.719466 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:38:12.719432    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-439275-openshift-default-666b8ddd88-4jpmd" podUID="c8cf5d22-c0dd-4995-8d48-8a993dbae7cc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Sep 10 19:38:13.718302 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:38:13.718264    2575 patch_prober.go:28] interesting pod/e2e-api-a-439275-openshift-default-666b8ddd88-4jpmd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Sep 10 19:38:13.718634 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:38:13.718603    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-439275-openshift-default-666b8ddd88-4jpmd" podUID="c8cf5d22-c0dd-4995-8d48-8a993dbae7cc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Sep 10 19:38:14.622143 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:38:14.622104    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:38:14.718531 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:38:14.718496    2575 patch_prober.go:28] interesting pod/e2e-api-a-439275-openshift-default-666b8ddd88-4jpmd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Sep 10 19:38:14.718808 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:38:14.718780    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-439275-openshift-default-666b8ddd88-4jpmd" podUID="c8cf5d22-c0dd-4995-8d48-8a993dbae7cc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Sep 10 19:38:15.719066 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:38:15.719027    2575 patch_prober.go:28] interesting pod/e2e-api-a-439275-openshift-default-666b8ddd88-4jpmd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Sep 10 19:38:15.719678 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:38:15.719646    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-439275-openshift-default-666b8ddd88-4jpmd" podUID="c8cf5d22-c0dd-4995-8d48-8a993dbae7cc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Sep 10 19:38:16.723532 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:38:16.723504    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-439275-openshift-default-666b8ddd88-4jpmd"
Sep 10 19:38:16.723966 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:38:16.723785    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-439275-openshift-default-666b8ddd88-4jpmd"
Sep 10 19:38:16.724521 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:38:16.724504    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-439275-openshift-default-666b8ddd88-4jpmd"
Sep 10 19:38:17.399654 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:38:17.399609    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:38:17.819962 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:38:17.819930    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:38:18.087647 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:38:18.087577    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-cbac1f-openshift-default-5d4cdbcfc8-w88f7"]
Sep 10 19:38:18.091469 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:38:18.091450    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-cbac1f-openshift-default-5d4cdbcfc8-w88f7"
Sep 10 19:38:18.093309 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:38:18.093280    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-cbac1f-openshift-default-dockercfg-wqgg6\""
Sep 10 19:38:18.101598 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:38:18.101561    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ec61c77-6b68-471d-b746-17b5936f07a2-credential-socket\") pod \"e2e-api-b-cbac1f-openshift-default-5d4cdbcfc8-w88f7\" (UID: \"1ec61c77-6b68-471d-b746-17b5936f07a2\") " pod="openshift-ingress/e2e-api-b-cbac1f-openshift-default-5d4cdbcfc8-w88f7"
Sep 10 19:38:18.101725 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:38:18.101622    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1ec61c77-6b68-471d-b746-17b5936f07a2-istio-envoy\") pod \"e2e-api-b-cbac1f-openshift-default-5d4cdbcfc8-w88f7\" (UID: \"1ec61c77-6b68-471d-b746-17b5936f07a2\") " pod="openshift-ingress/e2e-api-b-cbac1f-openshift-default-5d4cdbcfc8-w88f7"
Sep 10 19:38:18.101725 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:38:18.101653    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1ec61c77-6b68-471d-b746-17b5936f07a2-istio-data\") pod \"e2e-api-b-cbac1f-openshift-default-5d4cdbcfc8-w88f7\" (UID: \"1ec61c77-6b68-471d-b746-17b5936f07a2\") " pod="openshift-ingress/e2e-api-b-cbac1f-openshift-default-5d4cdbcfc8-w88f7"
Sep 10 19:38:18.101725 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:38:18.101687    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ec61c77-6b68-471d-b746-17b5936f07a2-workload-socket\") pod \"e2e-api-b-cbac1f-openshift-default-5d4cdbcfc8-w88f7\" (UID: \"1ec61c77-6b68-471d-b746-17b5936f07a2\") " pod="openshift-ingress/e2e-api-b-cbac1f-openshift-default-5d4cdbcfc8-w88f7"
Sep 10 19:38:18.101725 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:38:18.101714    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z5z8v\" (UniqueName: \"kubernetes.io/projected/1ec61c77-6b68-471d-b746-17b5936f07a2-kube-api-access-z5z8v\") pod \"e2e-api-b-cbac1f-openshift-default-5d4cdbcfc8-w88f7\" (UID: \"1ec61c77-6b68-471d-b746-17b5936f07a2\") " pod="openshift-ingress/e2e-api-b-cbac1f-openshift-default-5d4cdbcfc8-w88f7"
Sep 10 19:38:18.101952 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:38:18.101888    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1ec61c77-6b68-471d-b746-17b5936f07a2-istio-token\") pod \"e2e-api-b-cbac1f-openshift-default-5d4cdbcfc8-w88f7\" (UID: \"1ec61c77-6b68-471d-b746-17b5936f07a2\") " pod="openshift-ingress/e2e-api-b-cbac1f-openshift-default-5d4cdbcfc8-w88f7"
Sep 10 19:38:18.101952 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:38:18.101924    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1ec61c77-6b68-471d-b746-17b5936f07a2-istiod-ca-cert\") pod \"e2e-api-b-cbac1f-openshift-default-5d4cdbcfc8-w88f7\" (UID: \"1ec61c77-6b68-471d-b746-17b5936f07a2\") " pod="openshift-ingress/e2e-api-b-cbac1f-openshift-default-5d4cdbcfc8-w88f7"
Sep 10 19:38:18.102060 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:38:18.101955    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1ec61c77-6b68-471d-b746-17b5936f07a2-workload-certs\") pod \"e2e-api-b-cbac1f-openshift-default-5d4cdbcfc8-w88f7\" (UID: \"1ec61c77-6b68-471d-b746-17b5936f07a2\") " pod="openshift-ingress/e2e-api-b-cbac1f-openshift-default-5d4cdbcfc8-w88f7"
Sep 10 19:38:18.102060 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:38:18.101992    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1ec61c77-6b68-471d-b746-17b5936f07a2-istio-podinfo\") pod \"e2e-api-b-cbac1f-openshift-default-5d4cdbcfc8-w88f7\" (UID: \"1ec61c77-6b68-471d-b746-17b5936f07a2\") " pod="openshift-ingress/e2e-api-b-cbac1f-openshift-default-5d4cdbcfc8-w88f7"
Sep 10 19:38:18.102895 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:38:18.102877    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-cbac1f-openshift-default-5d4cdbcfc8-w88f7"]
Sep 10 19:38:18.114637 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:38:18.114612    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:38:18.203481 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:38:18.203267    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1ec61c77-6b68-471d-b746-17b5936f07a2-istio-token\") pod \"e2e-api-b-cbac1f-openshift-default-5d4cdbcfc8-w88f7\" (UID: \"1ec61c77-6b68-471d-b746-17b5936f07a2\") " pod="openshift-ingress/e2e-api-b-cbac1f-openshift-default-5d4cdbcfc8-w88f7"
Sep 10 19:38:18.203481 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:38:18.203318    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1ec61c77-6b68-471d-b746-17b5936f07a2-istiod-ca-cert\") pod \"e2e-api-b-cbac1f-openshift-default-5d4cdbcfc8-w88f7\" (UID: \"1ec61c77-6b68-471d-b746-17b5936f07a2\") " pod="openshift-ingress/e2e-api-b-cbac1f-openshift-default-5d4cdbcfc8-w88f7"
Sep 10 19:38:18.203481 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:38:18.203378    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1ec61c77-6b68-471d-b746-17b5936f07a2-workload-certs\") pod \"e2e-api-b-cbac1f-openshift-default-5d4cdbcfc8-w88f7\" (UID: \"1ec61c77-6b68-471d-b746-17b5936f07a2\") " pod="openshift-ingress/e2e-api-b-cbac1f-openshift-default-5d4cdbcfc8-w88f7"
Sep 10 19:38:18.203481 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:38:18.203415    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1ec61c77-6b68-471d-b746-17b5936f07a2-istio-podinfo\") pod \"e2e-api-b-cbac1f-openshift-default-5d4cdbcfc8-w88f7\" (UID: \"1ec61c77-6b68-471d-b746-17b5936f07a2\") " pod="openshift-ingress/e2e-api-b-cbac1f-openshift-default-5d4cdbcfc8-w88f7"
Sep 10 19:38:18.203481 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:38:18.203460    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ec61c77-6b68-471d-b746-17b5936f07a2-credential-socket\") pod \"e2e-api-b-cbac1f-openshift-default-5d4cdbcfc8-w88f7\" (UID: \"1ec61c77-6b68-471d-b746-17b5936f07a2\") " pod="openshift-ingress/e2e-api-b-cbac1f-openshift-default-5d4cdbcfc8-w88f7"
Sep 10 19:38:18.203481 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:38:18.203483    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1ec61c77-6b68-471d-b746-17b5936f07a2-istio-envoy\") pod \"e2e-api-b-cbac1f-openshift-default-5d4cdbcfc8-w88f7\" (UID: \"1ec61c77-6b68-471d-b746-17b5936f07a2\") " pod="openshift-ingress/e2e-api-b-cbac1f-openshift-default-5d4cdbcfc8-w88f7"
Sep 10 19:38:18.203981 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:38:18.203506    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1ec61c77-6b68-471d-b746-17b5936f07a2-istio-data\") pod \"e2e-api-b-cbac1f-openshift-default-5d4cdbcfc8-w88f7\" (UID: \"1ec61c77-6b68-471d-b746-17b5936f07a2\") " pod="openshift-ingress/e2e-api-b-cbac1f-openshift-default-5d4cdbcfc8-w88f7"
Sep 10 19:38:18.203981 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:38:18.203558    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ec61c77-6b68-471d-b746-17b5936f07a2-workload-socket\") pod \"e2e-api-b-cbac1f-openshift-default-5d4cdbcfc8-w88f7\" (UID: \"1ec61c77-6b68-471d-b746-17b5936f07a2\") " pod="openshift-ingress/e2e-api-b-cbac1f-openshift-default-5d4cdbcfc8-w88f7"
Sep 10 19:38:18.203981 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:38:18.203637    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z5z8v\" (UniqueName: \"kubernetes.io/projected/1ec61c77-6b68-471d-b746-17b5936f07a2-kube-api-access-z5z8v\") pod \"e2e-api-b-cbac1f-openshift-default-5d4cdbcfc8-w88f7\" (UID: \"1ec61c77-6b68-471d-b746-17b5936f07a2\") " pod="openshift-ingress/e2e-api-b-cbac1f-openshift-default-5d4cdbcfc8-w88f7"
Sep 10 19:38:18.205406 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:38:18.205371    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1ec61c77-6b68-471d-b746-17b5936f07a2-istio-data\") pod \"e2e-api-b-cbac1f-openshift-default-5d4cdbcfc8-w88f7\" (UID: \"1ec61c77-6b68-471d-b746-17b5936f07a2\") " pod="openshift-ingress/e2e-api-b-cbac1f-openshift-default-5d4cdbcfc8-w88f7"
Sep 10 19:38:18.205849 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:38:18.205618    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ec61c77-6b68-471d-b746-17b5936f07a2-credential-socket\") pod \"e2e-api-b-cbac1f-openshift-default-5d4cdbcfc8-w88f7\" (UID: \"1ec61c77-6b68-471d-b746-17b5936f07a2\") " pod="openshift-ingress/e2e-api-b-cbac1f-openshift-default-5d4cdbcfc8-w88f7"
Sep 10 19:38:18.205996 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:38:18.205786    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ec61c77-6b68-471d-b746-17b5936f07a2-workload-socket\") pod \"e2e-api-b-cbac1f-openshift-default-5d4cdbcfc8-w88f7\" (UID: \"1ec61c77-6b68-471d-b746-17b5936f07a2\") " pod="openshift-ingress/e2e-api-b-cbac1f-openshift-default-5d4cdbcfc8-w88f7"
Sep 10 19:38:18.206145 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:38:18.206119    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1ec61c77-6b68-471d-b746-17b5936f07a2-workload-certs\") pod \"e2e-api-b-cbac1f-openshift-default-5d4cdbcfc8-w88f7\" (UID: \"1ec61c77-6b68-471d-b746-17b5936f07a2\") " pod="openshift-ingress/e2e-api-b-cbac1f-openshift-default-5d4cdbcfc8-w88f7"
Sep 10 19:38:18.206467 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:38:18.206437    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1ec61c77-6b68-471d-b746-17b5936f07a2-istiod-ca-cert\") pod \"e2e-api-b-cbac1f-openshift-default-5d4cdbcfc8-w88f7\" (UID: \"1ec61c77-6b68-471d-b746-17b5936f07a2\") " pod="openshift-ingress/e2e-api-b-cbac1f-openshift-default-5d4cdbcfc8-w88f7"
Sep 10 19:38:18.206831 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:38:18.206814    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1ec61c77-6b68-471d-b746-17b5936f07a2-istio-envoy\") pod \"e2e-api-b-cbac1f-openshift-default-5d4cdbcfc8-w88f7\" (UID: \"1ec61c77-6b68-471d-b746-17b5936f07a2\") " pod="openshift-ingress/e2e-api-b-cbac1f-openshift-default-5d4cdbcfc8-w88f7"
Sep 10 19:38:18.208110 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:38:18.208090    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1ec61c77-6b68-471d-b746-17b5936f07a2-istio-podinfo\") pod \"e2e-api-b-cbac1f-openshift-default-5d4cdbcfc8-w88f7\" (UID: \"1ec61c77-6b68-471d-b746-17b5936f07a2\") " pod="openshift-ingress/e2e-api-b-cbac1f-openshift-default-5d4cdbcfc8-w88f7"
Sep 10 19:38:18.213207 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:38:18.213188    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1ec61c77-6b68-471d-b746-17b5936f07a2-istio-token\") pod \"e2e-api-b-cbac1f-openshift-default-5d4cdbcfc8-w88f7\" (UID: \"1ec61c77-6b68-471d-b746-17b5936f07a2\") " pod="openshift-ingress/e2e-api-b-cbac1f-openshift-default-5d4cdbcfc8-w88f7"
Sep 10 19:38:18.213615 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:38:18.213595    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z5z8v\" (UniqueName: \"kubernetes.io/projected/1ec61c77-6b68-471d-b746-17b5936f07a2-kube-api-access-z5z8v\") pod \"e2e-api-b-cbac1f-openshift-default-5d4cdbcfc8-w88f7\" (UID: \"1ec61c77-6b68-471d-b746-17b5936f07a2\") " pod="openshift-ingress/e2e-api-b-cbac1f-openshift-default-5d4cdbcfc8-w88f7"
Sep 10 19:38:18.402298 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:38:18.402223    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-cbac1f-openshift-default-5d4cdbcfc8-w88f7"
Sep 10 19:38:18.701306 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:38:18.701282    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-cbac1f-openshift-default-5d4cdbcfc8-w88f7"]
Sep 10 19:38:18.703833 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:38:18.703800    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:38:18.704179 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:38:18.704154    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ec61c77_6b68_471d_b746_17b5936f07a2.slice/crio-60cc834613d2fce5f835f351e4401c45afa974f93a5cb2e86db83889457e849e WatchSource:0}: Error finding container 60cc834613d2fce5f835f351e4401c45afa974f93a5cb2e86db83889457e849e: Status 404 returned error can't find the container with id 60cc834613d2fce5f835f351e4401c45afa974f93a5cb2e86db83889457e849e
Sep 10 19:38:18.707665 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:38:18.707619    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 19:38:18.707910 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:38:18.707711    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 19:38:18.707910 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:38:18.707750    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 19:38:19.169404 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:38:19.169287    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-cbac1f-openshift-default-5d4cdbcfc8-w88f7" event={"ID":"1ec61c77-6b68-471d-b746-17b5936f07a2","Type":"ContainerStarted","Data":"6bf0e90ccbb8d50c78bb1e478880396dace7cb6f801dc00cccd6957f58f2b7d0"}
Sep 10 19:38:19.169404 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:38:19.169363    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-cbac1f-openshift-default-5d4cdbcfc8-w88f7" event={"ID":"1ec61c77-6b68-471d-b746-17b5936f07a2","Type":"ContainerStarted","Data":"60cc834613d2fce5f835f351e4401c45afa974f93a5cb2e86db83889457e849e"}
Sep 10 19:38:19.189787 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:38:19.189745    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-cbac1f-openshift-default-5d4cdbcfc8-w88f7" podStartSLOduration=1.189729525 podStartE2EDuration="1.189729525s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:38:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:38:19.1878487 +0000 UTC m=+1370.247676368" watchObservedRunningTime="2026-09-10 19:38:19.189729525 +0000 UTC m=+1370.249557192"
Sep 10 19:38:19.403255 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:38:19.403222    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-cbac1f-openshift-default-5d4cdbcfc8-w88f7"
Sep 10 19:38:19.408661 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:38:19.408636    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-cbac1f-openshift-default-5d4cdbcfc8-w88f7"
Sep 10 19:38:20.174472 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:38:20.174443    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-cbac1f-openshift-default-5d4cdbcfc8-w88f7"
Sep 10 19:38:20.175226 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:38:20.175205    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-cbac1f-openshift-default-5d4cdbcfc8-w88f7"
Sep 10 19:38:21.107355 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:38:21.107281    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:38:21.207088 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:38:21.207054    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:38:21.788412 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:38:21.788354    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:38:22.470812 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:38:22.470779    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:38:22.585748 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:38:22.585691    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:38:22.790147 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:38:22.790061    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:38:22.888701 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:38:22.888675    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:38:22.987551 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:38:22.987511    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:38:29.046686 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:38:29.046653    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qr7rl"]
Sep 10 19:38:36.563734 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:38:36.563695    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:38:39.284725 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:38:39.284695    2575 generic.go:356] "Generic (PLEG): container finished" podID="1ec61c77-6b68-471d-b746-17b5936f07a2" containerID="6bf0e90ccbb8d50c78bb1e478880396dace7cb6f801dc00cccd6957f58f2b7d0" exitCode=0
Sep 10 19:38:39.285048 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:38:39.284766    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-cbac1f-openshift-default-5d4cdbcfc8-w88f7" event={"ID":"1ec61c77-6b68-471d-b746-17b5936f07a2","Type":"ContainerDied","Data":"6bf0e90ccbb8d50c78bb1e478880396dace7cb6f801dc00cccd6957f58f2b7d0"}
Sep 10 19:38:39.285154 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:38:39.285141    2575 scope.go:117] "RemoveContainer" containerID="6bf0e90ccbb8d50c78bb1e478880396dace7cb6f801dc00cccd6957f58f2b7d0"
Sep 10 19:38:39.286236 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:38:39.286204    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 19:38:39.286331 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:38:39.286292    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 19:38:39.286417 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:38:39.286366    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 19:38:40.292326 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:38:40.292291    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-cbac1f-openshift-default-5d4cdbcfc8-w88f7" event={"ID":"1ec61c77-6b68-471d-b746-17b5936f07a2","Type":"ContainerStarted","Data":"f1014fe814fc875c057d2413038d06b2cc5a740916c9a9aca03aac6de33c82b8"}
Sep 10 19:38:40.403173 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:38:40.403136    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-cbac1f-openshift-default-5d4cdbcfc8-w88f7"
Sep 10 19:38:40.403446 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:38:40.403422    2575 patch_prober.go:28] interesting pod/e2e-api-b-cbac1f-openshift-default-5d4cdbcfc8-w88f7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body=
Sep 10 19:38:40.403560 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:38:40.403476    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-cbac1f-openshift-default-5d4cdbcfc8-w88f7" podUID="1ec61c77-6b68-471d-b746-17b5936f07a2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused"
Sep 10 19:38:41.403361 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:38:41.403308    2575 patch_prober.go:28] interesting pod/e2e-api-b-cbac1f-openshift-default-5d4cdbcfc8-w88f7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body=
Sep 10 19:38:41.403740 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:38:41.403381    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-cbac1f-openshift-default-5d4cdbcfc8-w88f7" podUID="1ec61c77-6b68-471d-b746-17b5936f07a2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused"
Sep 10 19:38:42.403580 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:38:42.403543    2575 patch_prober.go:28] interesting pod/e2e-api-b-cbac1f-openshift-default-5d4cdbcfc8-w88f7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body=
Sep 10 19:38:42.403953 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:38:42.403607    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-cbac1f-openshift-default-5d4cdbcfc8-w88f7" podUID="1ec61c77-6b68-471d-b746-17b5936f07a2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused"
Sep 10 19:38:43.305045 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:38:43.305006    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:38:43.403595 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:38:43.403562    2575 patch_prober.go:28] interesting pod/e2e-api-b-cbac1f-openshift-default-5d4cdbcfc8-w88f7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body=
Sep 10 19:38:43.403954 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:38:43.403612    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-cbac1f-openshift-default-5d4cdbcfc8-w88f7" podUID="1ec61c77-6b68-471d-b746-17b5936f07a2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused"
Sep 10 19:38:44.403052 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:38:44.403020    2575 patch_prober.go:28] interesting pod/e2e-api-b-cbac1f-openshift-default-5d4cdbcfc8-w88f7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body=
Sep 10 19:38:44.403220 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:38:44.403076    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-cbac1f-openshift-default-5d4cdbcfc8-w88f7" podUID="1ec61c77-6b68-471d-b746-17b5936f07a2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused"
Sep 10 19:38:45.403279 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:38:45.403242    2575 patch_prober.go:28] interesting pod/e2e-api-b-cbac1f-openshift-default-5d4cdbcfc8-w88f7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body=
Sep 10 19:38:45.403689 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:38:45.403320    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-cbac1f-openshift-default-5d4cdbcfc8-w88f7" podUID="1ec61c77-6b68-471d-b746-17b5936f07a2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused"
Sep 10 19:38:46.066301 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:38:46.066259    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:38:46.406857 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:38:46.406783    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-cbac1f-openshift-default-5d4cdbcfc8-w88f7"
Sep 10 19:38:46.407207 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:38:46.407182    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-cbac1f-openshift-default-5d4cdbcfc8-w88f7"
Sep 10 19:38:46.408017 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:38:46.407995    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-cbac1f-openshift-default-5d4cdbcfc8-w88f7"
Sep 10 19:38:48.897437 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:38:48.897386    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-439275-fhk88"]
Sep 10 19:38:48.900968 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:38:48.900952    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-439275-fhk88"
Sep 10 19:38:48.902827 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:38:48.902807    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 19:38:48.902954 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:38:48.902927    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wkgt8\""
Sep 10 19:38:48.908151 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:38:48.908123    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-439275-fhk88"]
Sep 10 19:38:49.008828 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:38:49.008799    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/01f7464f-eec6-4b58-977a-3eb79d75ada4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-439275-fhk88\" (UID: \"01f7464f-eec6-4b58-977a-3eb79d75ada4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-439275-fhk88"
Sep 10 19:38:49.039795 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:38:49.039772    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:38:49.109937 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:38:49.109896    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/01f7464f-eec6-4b58-977a-3eb79d75ada4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-439275-fhk88\" (UID: \"01f7464f-eec6-4b58-977a-3eb79d75ada4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-439275-fhk88"
Sep 10 19:38:49.111004 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:38:49.110934    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/01f7464f-eec6-4b58-977a-3eb79d75ada4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-439275-fhk88\" (UID: \"01f7464f-eec6-4b58-977a-3eb79d75ada4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-439275-fhk88"
Sep 10 19:38:49.212126 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:38:49.212104    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-439275-fhk88"
Sep 10 19:38:49.486847 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:38:49.486825    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-439275-fhk88"]
Sep 10 19:38:49.488991 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:38:49.488965    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01f7464f_eec6_4b58_977a_3eb79d75ada4.slice/crio-dd954eff586966e1be9f83bdc6d3b394a2caea97ea77e4040b2e1e3a6ff58d2f WatchSource:0}: Error finding container dd954eff586966e1be9f83bdc6d3b394a2caea97ea77e4040b2e1e3a6ff58d2f: Status 404 returned error can't find the container with id dd954eff586966e1be9f83bdc6d3b394a2caea97ea77e4040b2e1e3a6ff58d2f
Sep 10 19:38:50.381250 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:38:50.381175    2575 generic.go:356] "Generic (PLEG): container finished" podID="01f7464f-eec6-4b58-977a-3eb79d75ada4" containerID="0e1957cdf272f7971ca408cd9289c3b2f468a42eb846306ccf7dffb11436d228" exitCode=0
Sep 10 19:38:50.381250 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:38:50.381235    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-439275-fhk88" event={"ID":"01f7464f-eec6-4b58-977a-3eb79d75ada4","Type":"ContainerDied","Data":"0e1957cdf272f7971ca408cd9289c3b2f468a42eb846306ccf7dffb11436d228"}
Sep 10 19:38:50.381658 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:38:50.381280    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-439275-fhk88" event={"ID":"01f7464f-eec6-4b58-977a-3eb79d75ada4","Type":"ContainerStarted","Data":"dd954eff586966e1be9f83bdc6d3b394a2caea97ea77e4040b2e1e3a6ff58d2f"}
Sep 10 19:38:51.516047 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:38:51.516026    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-439275-fhk88"
Sep 10 19:38:51.633381 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:38:51.633353    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/01f7464f-eec6-4b58-977a-3eb79d75ada4-maas-api-service-ca\") pod \"01f7464f-eec6-4b58-977a-3eb79d75ada4\" (UID: \"01f7464f-eec6-4b58-977a-3eb79d75ada4\") "
Sep 10 19:38:51.633624 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:38:51.633602    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/01f7464f-eec6-4b58-977a-3eb79d75ada4-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "01f7464f-eec6-4b58-977a-3eb79d75ada4" (UID: "01f7464f-eec6-4b58-977a-3eb79d75ada4"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:38:51.734133 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:38:51.734106    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/01f7464f-eec6-4b58-977a-3eb79d75ada4-maas-api-service-ca\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:38:51.783809 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:38:51.783763    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:38:52.393901 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:38:52.393866    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-439275-fhk88" event={"ID":"01f7464f-eec6-4b58-977a-3eb79d75ada4","Type":"ContainerDied","Data":"dd954eff586966e1be9f83bdc6d3b394a2caea97ea77e4040b2e1e3a6ff58d2f"}
Sep 10 19:38:52.394085 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:38:52.393915    2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dd954eff586966e1be9f83bdc6d3b394a2caea97ea77e4040b2e1e3a6ff58d2f"
Sep 10 19:38:52.394085 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:38:52.393881    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-439275-fhk88"
Sep 10 19:38:58.856710 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:38:58.856675    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qr7rl"]
Sep 10 19:38:59.140052 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:38:59.139964    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:39:04.906299 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:04.906263    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:39:04.979205 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:04.979172    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:39:05.042263 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:05.042222    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:39:05.124104 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:05.124078    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-439275-fhk88"]
Sep 10 19:39:05.126794 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:05.126759    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:39:05.127211 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:05.127193    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-439275-fhk88"]
Sep 10 19:39:05.186533 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:05.186461    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:39:05.385100 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:05.385060    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="01f7464f-eec6-4b58-977a-3eb79d75ada4" path="/var/lib/kubelet/pods/01f7464f-eec6-4b58-977a-3eb79d75ada4/volumes"
Sep 10 19:39:05.520352 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:05.520311    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:39:07.960680 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:07.960648    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:39:08.061558 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:08.061529    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:39:11.466752 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:11.466675    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:39:11.568265 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:11.568226    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:39:11.701201 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:11.701171    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-439275-openshift-default-666b8ddd88-4jpmd"]
Sep 10 19:39:11.701422 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:11.701379    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-439275-openshift-default-666b8ddd88-4jpmd" podUID="c8cf5d22-c0dd-4995-8d48-8a993dbae7cc" containerName="istio-proxy" containerID="cri-o://7e4432911bb373b6de8b1890ec2f9b181167ca584e79dbfcf5d38a80005304c6" gracePeriod=30
Sep 10 19:39:11.716537 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:11.716501    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:39:12.106687 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:12.106656    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-cbac1f-g9svd"]
Sep 10 19:39:12.107119 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:12.107105    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="01f7464f-eec6-4b58-977a-3eb79d75ada4" containerName="revoke-keys"
Sep 10 19:39:12.107166 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:12.107120    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="01f7464f-eec6-4b58-977a-3eb79d75ada4" containerName="revoke-keys"
Sep 10 19:39:12.107216 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:12.107207    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="01f7464f-eec6-4b58-977a-3eb79d75ada4" containerName="revoke-keys"
Sep 10 19:39:12.110497 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:12.110481    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-cbac1f-g9svd"
Sep 10 19:39:12.112892 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:12.112872    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wkgt8\""
Sep 10 19:39:12.113293 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:12.113273    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 19:39:12.124661 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:12.124639    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-cbac1f-g9svd"]
Sep 10 19:39:12.197166 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:12.197144    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/feb35367-1430-477e-875d-425c8a6fbda9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-cbac1f-g9svd\" (UID: \"feb35367-1430-477e-875d-425c8a6fbda9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-cbac1f-g9svd"
Sep 10 19:39:12.297908 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:12.297872    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/feb35367-1430-477e-875d-425c8a6fbda9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-cbac1f-g9svd\" (UID: \"feb35367-1430-477e-875d-425c8a6fbda9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-cbac1f-g9svd"
Sep 10 19:39:12.298469 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:12.298450    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/feb35367-1430-477e-875d-425c8a6fbda9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-cbac1f-g9svd\" (UID: \"feb35367-1430-477e-875d-425c8a6fbda9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-cbac1f-g9svd"
Sep 10 19:39:12.345972 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:12.345942    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:39:12.421303 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:12.421237    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-cbac1f-g9svd"
Sep 10 19:39:12.691171 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:12.691146    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-cbac1f-g9svd"]
Sep 10 19:39:12.693874 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:39:12.693848    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfeb35367_1430_477e_875d_425c8a6fbda9.slice/crio-ca317134f4d235dbb277b78368234ccd4389c5c46c63a97543804f5c715edb93 WatchSource:0}: Error finding container ca317134f4d235dbb277b78368234ccd4389c5c46c63a97543804f5c715edb93: Status 404 returned error can't find the container with id ca317134f4d235dbb277b78368234ccd4389c5c46c63a97543804f5c715edb93
Sep 10 19:39:13.499607 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:13.499575    2575 generic.go:356] "Generic (PLEG): container finished" podID="feb35367-1430-477e-875d-425c8a6fbda9" containerID="07f9e96ce4d039833adfab7c4a4ffb1d5d65d7116c88f870aba23bcdb005736d" exitCode=0
Sep 10 19:39:13.499720 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:13.499642    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-cbac1f-g9svd" event={"ID":"feb35367-1430-477e-875d-425c8a6fbda9","Type":"ContainerDied","Data":"07f9e96ce4d039833adfab7c4a4ffb1d5d65d7116c88f870aba23bcdb005736d"}
Sep 10 19:39:13.499720 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:13.499681    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-cbac1f-g9svd" event={"ID":"feb35367-1430-477e-875d-425c8a6fbda9","Type":"ContainerStarted","Data":"ca317134f4d235dbb277b78368234ccd4389c5c46c63a97543804f5c715edb93"}
Sep 10 19:39:14.637161 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:14.637139    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-cbac1f-g9svd"
Sep 10 19:39:14.720928 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:14.720903    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/feb35367-1430-477e-875d-425c8a6fbda9-maas-api-service-ca\") pod \"feb35367-1430-477e-875d-425c8a6fbda9\" (UID: \"feb35367-1430-477e-875d-425c8a6fbda9\") "
Sep 10 19:39:14.721231 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:14.721212    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/feb35367-1430-477e-875d-425c8a6fbda9-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "feb35367-1430-477e-875d-425c8a6fbda9" (UID: "feb35367-1430-477e-875d-425c8a6fbda9"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:39:14.822564 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:14.822505    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/feb35367-1430-477e-875d-425c8a6fbda9-maas-api-service-ca\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:39:15.511800 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:15.511768    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-cbac1f-g9svd"
Sep 10 19:39:15.511940 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:15.511824    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-cbac1f-g9svd" event={"ID":"feb35367-1430-477e-875d-425c8a6fbda9","Type":"ContainerDied","Data":"ca317134f4d235dbb277b78368234ccd4389c5c46c63a97543804f5c715edb93"}
Sep 10 19:39:15.511940 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:15.511869    2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ca317134f4d235dbb277b78368234ccd4389c5c46c63a97543804f5c715edb93"
Sep 10 19:39:16.648888 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:16.648853    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qr7rl"]
Sep 10 19:39:16.725159 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:16.725133    2575 patch_prober.go:28] interesting pod/e2e-api-a-439275-openshift-default-666b8ddd88-4jpmd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Sep 10 19:39:16.725265 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:16.725192    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-439275-openshift-default-666b8ddd88-4jpmd" podUID="c8cf5d22-c0dd-4995-8d48-8a993dbae7cc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Sep 10 19:39:16.969219 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:16.969191    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-439275-openshift-default-666b8ddd88-4jpmd"
Sep 10 19:39:17.043751 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:17.043723    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c8cf5d22-c0dd-4995-8d48-8a993dbae7cc-istio-token\") pod \"c8cf5d22-c0dd-4995-8d48-8a993dbae7cc\" (UID: \"c8cf5d22-c0dd-4995-8d48-8a993dbae7cc\") "
Sep 10 19:39:17.043868 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:17.043774    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c8cf5d22-c0dd-4995-8d48-8a993dbae7cc-istio-podinfo\") pod \"c8cf5d22-c0dd-4995-8d48-8a993dbae7cc\" (UID: \"c8cf5d22-c0dd-4995-8d48-8a993dbae7cc\") "
Sep 10 19:39:17.043868 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:17.043805    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8cf5d22-c0dd-4995-8d48-8a993dbae7cc-workload-socket\") pod \"c8cf5d22-c0dd-4995-8d48-8a993dbae7cc\" (UID: \"c8cf5d22-c0dd-4995-8d48-8a993dbae7cc\") "
Sep 10 19:39:17.043868 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:17.043844    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8cf5d22-c0dd-4995-8d48-8a993dbae7cc-credential-socket\") pod \"c8cf5d22-c0dd-4995-8d48-8a993dbae7cc\" (UID: \"c8cf5d22-c0dd-4995-8d48-8a993dbae7cc\") "
Sep 10 19:39:17.044042 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:17.043877    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c8cf5d22-c0dd-4995-8d48-8a993dbae7cc-istiod-ca-cert\") pod \"c8cf5d22-c0dd-4995-8d48-8a993dbae7cc\" (UID: \"c8cf5d22-c0dd-4995-8d48-8a993dbae7cc\") "
Sep 10 19:39:17.044042 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:17.043912    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c8cf5d22-c0dd-4995-8d48-8a993dbae7cc-istio-data\") pod \"c8cf5d22-c0dd-4995-8d48-8a993dbae7cc\" (UID: \"c8cf5d22-c0dd-4995-8d48-8a993dbae7cc\") "
Sep 10 19:39:17.044042 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:17.043983    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c8cf5d22-c0dd-4995-8d48-8a993dbae7cc-istio-envoy\") pod \"c8cf5d22-c0dd-4995-8d48-8a993dbae7cc\" (UID: \"c8cf5d22-c0dd-4995-8d48-8a993dbae7cc\") "
Sep 10 19:39:17.044042 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:17.044010    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c4fqb\" (UniqueName: \"kubernetes.io/projected/c8cf5d22-c0dd-4995-8d48-8a993dbae7cc-kube-api-access-c4fqb\") pod \"c8cf5d22-c0dd-4995-8d48-8a993dbae7cc\" (UID: \"c8cf5d22-c0dd-4995-8d48-8a993dbae7cc\") "
Sep 10 19:39:17.044251 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:17.044046    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c8cf5d22-c0dd-4995-8d48-8a993dbae7cc-workload-certs\") pod \"c8cf5d22-c0dd-4995-8d48-8a993dbae7cc\" (UID: \"c8cf5d22-c0dd-4995-8d48-8a993dbae7cc\") "
Sep 10 19:39:17.044251 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:17.044131    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c8cf5d22-c0dd-4995-8d48-8a993dbae7cc-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c8cf5d22-c0dd-4995-8d48-8a993dbae7cc" (UID: "c8cf5d22-c0dd-4995-8d48-8a993dbae7cc"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:39:17.044251 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:17.044170    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c8cf5d22-c0dd-4995-8d48-8a993dbae7cc-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c8cf5d22-c0dd-4995-8d48-8a993dbae7cc" (UID: "c8cf5d22-c0dd-4995-8d48-8a993dbae7cc"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:39:17.044447 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:17.044396    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c8cf5d22-c0dd-4995-8d48-8a993dbae7cc-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c8cf5d22-c0dd-4995-8d48-8a993dbae7cc" (UID: "c8cf5d22-c0dd-4995-8d48-8a993dbae7cc"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:39:17.044447 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:17.044408    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c8cf5d22-c0dd-4995-8d48-8a993dbae7cc-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c8cf5d22-c0dd-4995-8d48-8a993dbae7cc" (UID: "c8cf5d22-c0dd-4995-8d48-8a993dbae7cc"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:39:17.044447 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:17.044420    2575 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8cf5d22-c0dd-4995-8d48-8a993dbae7cc-workload-socket\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:39:17.044447 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:17.044441    2575 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8cf5d22-c0dd-4995-8d48-8a993dbae7cc-credential-socket\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:39:17.044657 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:17.044452    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c8cf5d22-c0dd-4995-8d48-8a993dbae7cc-istio-data" (OuterVolumeSpecName: "istio-data") pod "c8cf5d22-c0dd-4995-8d48-8a993dbae7cc" (UID: "c8cf5d22-c0dd-4995-8d48-8a993dbae7cc"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:39:17.046051 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:17.046032    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c8cf5d22-c0dd-4995-8d48-8a993dbae7cc-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c8cf5d22-c0dd-4995-8d48-8a993dbae7cc" (UID: "c8cf5d22-c0dd-4995-8d48-8a993dbae7cc"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 19:39:17.046240 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:17.046207    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c8cf5d22-c0dd-4995-8d48-8a993dbae7cc-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c8cf5d22-c0dd-4995-8d48-8a993dbae7cc" (UID: "c8cf5d22-c0dd-4995-8d48-8a993dbae7cc"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:39:17.046383 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:17.046317    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c8cf5d22-c0dd-4995-8d48-8a993dbae7cc-istio-token" (OuterVolumeSpecName: "istio-token") pod "c8cf5d22-c0dd-4995-8d48-8a993dbae7cc" (UID: "c8cf5d22-c0dd-4995-8d48-8a993dbae7cc"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:39:17.046457 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:17.046410    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c8cf5d22-c0dd-4995-8d48-8a993dbae7cc-kube-api-access-c4fqb" (OuterVolumeSpecName: "kube-api-access-c4fqb") pod "c8cf5d22-c0dd-4995-8d48-8a993dbae7cc" (UID: "c8cf5d22-c0dd-4995-8d48-8a993dbae7cc"). InnerVolumeSpecName "kube-api-access-c4fqb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:39:17.145845 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:17.145819    2575 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c8cf5d22-c0dd-4995-8d48-8a993dbae7cc-istio-token\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:39:17.145845 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:17.145840    2575 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c8cf5d22-c0dd-4995-8d48-8a993dbae7cc-istio-podinfo\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:39:17.145996 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:17.145850    2575 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c8cf5d22-c0dd-4995-8d48-8a993dbae7cc-istiod-ca-cert\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:39:17.145996 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:17.145859    2575 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c8cf5d22-c0dd-4995-8d48-8a993dbae7cc-istio-data\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:39:17.145996 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:17.145866    2575 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c8cf5d22-c0dd-4995-8d48-8a993dbae7cc-istio-envoy\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:39:17.145996 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:17.145875    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c4fqb\" (UniqueName: \"kubernetes.io/projected/c8cf5d22-c0dd-4995-8d48-8a993dbae7cc-kube-api-access-c4fqb\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:39:17.145996 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:17.145883    2575 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c8cf5d22-c0dd-4995-8d48-8a993dbae7cc-workload-certs\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:39:17.526017 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:17.525987    2575 generic.go:356] "Generic (PLEG): container finished" podID="c8cf5d22-c0dd-4995-8d48-8a993dbae7cc" containerID="7e4432911bb373b6de8b1890ec2f9b181167ca584e79dbfcf5d38a80005304c6" exitCode=0
Sep 10 19:39:17.526196 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:17.526042    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-439275-openshift-default-666b8ddd88-4jpmd"
Sep 10 19:39:17.526196 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:17.526071    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-439275-openshift-default-666b8ddd88-4jpmd" event={"ID":"c8cf5d22-c0dd-4995-8d48-8a993dbae7cc","Type":"ContainerDied","Data":"7e4432911bb373b6de8b1890ec2f9b181167ca584e79dbfcf5d38a80005304c6"}
Sep 10 19:39:17.526196 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:17.526140    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-439275-openshift-default-666b8ddd88-4jpmd" event={"ID":"c8cf5d22-c0dd-4995-8d48-8a993dbae7cc","Type":"ContainerDied","Data":"fe5fb3379a2fc8675169878ee8e8df45d46271941b0db8f5b5fd2f674dd4c82c"}
Sep 10 19:39:17.526196 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:17.526173    2575 scope.go:117] "RemoveContainer" containerID="7e4432911bb373b6de8b1890ec2f9b181167ca584e79dbfcf5d38a80005304c6"
Sep 10 19:39:17.535606 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:17.535578    2575 scope.go:117] "RemoveContainer" containerID="014f2fd7664bafe6b6a5acb7dc75a227a7797da3226373cc1c64c74b39cb14bf"
Sep 10 19:39:17.542547 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:17.542526    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-439275-openshift-default-666b8ddd88-4jpmd"]
Sep 10 19:39:17.544410 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:17.544303    2575 scope.go:117] "RemoveContainer" containerID="7e4432911bb373b6de8b1890ec2f9b181167ca584e79dbfcf5d38a80005304c6"
Sep 10 19:39:17.544774 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:39:17.544740    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7e4432911bb373b6de8b1890ec2f9b181167ca584e79dbfcf5d38a80005304c6\": container with ID starting with 7e4432911bb373b6de8b1890ec2f9b181167ca584e79dbfcf5d38a80005304c6 not found: ID does not exist" containerID="7e4432911bb373b6de8b1890ec2f9b181167ca584e79dbfcf5d38a80005304c6"
Sep 10 19:39:17.544892 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:17.544783    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7e4432911bb373b6de8b1890ec2f9b181167ca584e79dbfcf5d38a80005304c6"} err="failed to get container status \"7e4432911bb373b6de8b1890ec2f9b181167ca584e79dbfcf5d38a80005304c6\": rpc error: code = NotFound desc = could not find container \"7e4432911bb373b6de8b1890ec2f9b181167ca584e79dbfcf5d38a80005304c6\": container with ID starting with 7e4432911bb373b6de8b1890ec2f9b181167ca584e79dbfcf5d38a80005304c6 not found: ID does not exist"
Sep 10 19:39:17.544892 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:17.544807    2575 scope.go:117] "RemoveContainer" containerID="014f2fd7664bafe6b6a5acb7dc75a227a7797da3226373cc1c64c74b39cb14bf"
Sep 10 19:39:17.545139 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:39:17.545110    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"014f2fd7664bafe6b6a5acb7dc75a227a7797da3226373cc1c64c74b39cb14bf\": container with ID starting with 014f2fd7664bafe6b6a5acb7dc75a227a7797da3226373cc1c64c74b39cb14bf not found: ID does not exist" containerID="014f2fd7664bafe6b6a5acb7dc75a227a7797da3226373cc1c64c74b39cb14bf"
Sep 10 19:39:17.545237 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:17.545144    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"014f2fd7664bafe6b6a5acb7dc75a227a7797da3226373cc1c64c74b39cb14bf"} err="failed to get container status \"014f2fd7664bafe6b6a5acb7dc75a227a7797da3226373cc1c64c74b39cb14bf\": rpc error: code = NotFound desc = could not find container \"014f2fd7664bafe6b6a5acb7dc75a227a7797da3226373cc1c64c74b39cb14bf\": container with ID starting with 014f2fd7664bafe6b6a5acb7dc75a227a7797da3226373cc1c64c74b39cb14bf not found: ID does not exist"
Sep 10 19:39:17.547544 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:17.547522    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-439275-openshift-default-666b8ddd88-4jpmd"]
Sep 10 19:39:18.942212 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:18.942182    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7458-5640fb"]
Sep 10 19:39:18.943290 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:18.943250    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="feb35367-1430-477e-875d-425c8a6fbda9" containerName="revoke-keys"
Sep 10 19:39:18.943290 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:18.943278    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="feb35367-1430-477e-875d-425c8a6fbda9" containerName="revoke-keys"
Sep 10 19:39:18.943479 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:18.943304    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c8cf5d22-c0dd-4995-8d48-8a993dbae7cc" containerName="istio-proxy"
Sep 10 19:39:18.943479 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:18.943312    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="c8cf5d22-c0dd-4995-8d48-8a993dbae7cc" containerName="istio-proxy"
Sep 10 19:39:18.943479 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:18.943441    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="c8cf5d22-c0dd-4995-8d48-8a993dbae7cc" containerName="istio-proxy"
Sep 10 19:39:18.943479 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:18.943452    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="feb35367-1430-477e-875d-425c8a6fbda9" containerName="revoke-keys"
Sep 10 19:39:18.943479 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:18.943464    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="c8cf5d22-c0dd-4995-8d48-8a993dbae7cc" containerName="istio-proxy"
Sep 10 19:39:18.948235 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:18.948217    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7458-5640fb"
Sep 10 19:39:18.950321 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:18.950298    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-8vfd9\""
Sep 10 19:39:18.951638 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:18.951614    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7458-5640fb"]
Sep 10 19:39:19.061832 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:19.061800    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xrrp2\" (UniqueName: \"kubernetes.io/projected/0c270113-780e-47be-a503-d207b2c6eff6-kube-api-access-xrrp2\") pod \"test-curl-7458-5640fb\" (UID: \"0c270113-780e-47be-a503-d207b2c6eff6\") " pod="opendatahub/test-curl-7458-5640fb"
Sep 10 19:39:19.163190 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:19.163165    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xrrp2\" (UniqueName: \"kubernetes.io/projected/0c270113-780e-47be-a503-d207b2c6eff6-kube-api-access-xrrp2\") pod \"test-curl-7458-5640fb\" (UID: \"0c270113-780e-47be-a503-d207b2c6eff6\") " pod="opendatahub/test-curl-7458-5640fb"
Sep 10 19:39:19.171548 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:19.171525    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xrrp2\" (UniqueName: \"kubernetes.io/projected/0c270113-780e-47be-a503-d207b2c6eff6-kube-api-access-xrrp2\") pod \"test-curl-7458-5640fb\" (UID: \"0c270113-780e-47be-a503-d207b2c6eff6\") " pod="opendatahub/test-curl-7458-5640fb"
Sep 10 19:39:19.258867 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:19.258848    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7458-5640fb"
Sep 10 19:39:19.385090 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:19.385057    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c8cf5d22-c0dd-4995-8d48-8a993dbae7cc" path="/var/lib/kubelet/pods/c8cf5d22-c0dd-4995-8d48-8a993dbae7cc/volumes"
Sep 10 19:39:19.535132 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:19.535110    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7458-5640fb"]
Sep 10 19:39:19.537264 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:39:19.537238    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0c270113_780e_47be_a503_d207b2c6eff6.slice/crio-ea3b7f5d703c967d615b957702c2c29da75a396ef467cba9b7156d04ba2467ee WatchSource:0}: Error finding container ea3b7f5d703c967d615b957702c2c29da75a396ef467cba9b7156d04ba2467ee: Status 404 returned error can't find the container with id ea3b7f5d703c967d615b957702c2c29da75a396ef467cba9b7156d04ba2467ee
Sep 10 19:39:19.933473 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:19.933375    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:39:20.544709 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:20.544677    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7458-5640fb" event={"ID":"0c270113-780e-47be-a503-d207b2c6eff6","Type":"ContainerStarted","Data":"61b1ff989cae679cf86a75503e01b62bb198332925779f0fe5548e882c6aba52"}
Sep 10 19:39:20.545139 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:20.544717    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7458-5640fb" event={"ID":"0c270113-780e-47be-a503-d207b2c6eff6","Type":"ContainerStarted","Data":"ea3b7f5d703c967d615b957702c2c29da75a396ef467cba9b7156d04ba2467ee"}
Sep 10 19:39:20.561178 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:20.561132    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7458-5640fb" podStartSLOduration=2.561116434 podStartE2EDuration="2.561116434s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:39:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:39:20.559426161 +0000 UTC m=+1431.619253830" watchObservedRunningTime="2026-09-10 19:39:20.561116434 +0000 UTC m=+1431.620944101"
Sep 10 19:39:21.109252 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:21.109219    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7458-5640fb"]
Sep 10 19:39:21.113120 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:21.113092    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7458-5640fb"]
Sep 10 19:39:21.229783 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:21.229756    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7458-71aaac"]
Sep 10 19:39:21.230155 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:21.230142    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0c270113-780e-47be-a503-d207b2c6eff6" containerName="test-curl-7458-5640fb"
Sep 10 19:39:21.230203 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:21.230156    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="0c270113-780e-47be-a503-d207b2c6eff6" containerName="test-curl-7458-5640fb"
Sep 10 19:39:21.230203 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:21.230184    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c8cf5d22-c0dd-4995-8d48-8a993dbae7cc" containerName="istio-proxy"
Sep 10 19:39:21.230203 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:21.230189    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="c8cf5d22-c0dd-4995-8d48-8a993dbae7cc" containerName="istio-proxy"
Sep 10 19:39:21.230296 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:21.230254    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="0c270113-780e-47be-a503-d207b2c6eff6" containerName="test-curl-7458-5640fb"
Sep 10 19:39:21.233023 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:21.233006    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7458-71aaac"
Sep 10 19:39:21.238585 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:21.238559    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7458-71aaac"]
Sep 10 19:39:21.281526 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:21.281501    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qqgwm\" (UniqueName: \"kubernetes.io/projected/29a4fded-ae24-4eb9-a6e7-aecfddb1093f-kube-api-access-qqgwm\") pod \"test-curl-7458-71aaac\" (UID: \"29a4fded-ae24-4eb9-a6e7-aecfddb1093f\") " pod="opendatahub/test-curl-7458-71aaac"
Sep 10 19:39:21.381843 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:21.381820    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qqgwm\" (UniqueName: \"kubernetes.io/projected/29a4fded-ae24-4eb9-a6e7-aecfddb1093f-kube-api-access-qqgwm\") pod \"test-curl-7458-71aaac\" (UID: \"29a4fded-ae24-4eb9-a6e7-aecfddb1093f\") " pod="opendatahub/test-curl-7458-71aaac"
Sep 10 19:39:21.389791 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:21.389772    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qqgwm\" (UniqueName: \"kubernetes.io/projected/29a4fded-ae24-4eb9-a6e7-aecfddb1093f-kube-api-access-qqgwm\") pod \"test-curl-7458-71aaac\" (UID: \"29a4fded-ae24-4eb9-a6e7-aecfddb1093f\") " pod="opendatahub/test-curl-7458-71aaac"
Sep 10 19:39:21.544129 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:21.544062    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7458-71aaac"
Sep 10 19:39:21.549838 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:39:21.549819    2575 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7458-5640fb" container="test-curl-7458-5640fb"
Sep 10 19:39:21.813829 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:21.813592    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7458-71aaac"]
Sep 10 19:39:21.816245 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:39:21.816218    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod29a4fded_ae24_4eb9_a6e7_aecfddb1093f.slice/crio-38fc84090835bb5215d5e34827e597cfad0ab636cf0c9bc133d8f4655f488301 WatchSource:0}: Error finding container 38fc84090835bb5215d5e34827e597cfad0ab636cf0c9bc133d8f4655f488301: Status 404 returned error can't find the container with id 38fc84090835bb5215d5e34827e597cfad0ab636cf0c9bc133d8f4655f488301
Sep 10 19:39:22.236372 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:22.236326    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:39:22.556367 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:22.556271    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7458-71aaac" event={"ID":"29a4fded-ae24-4eb9-a6e7-aecfddb1093f","Type":"ContainerStarted","Data":"999c9e765858792406605f2391b5f28434a4555ea942303466a7f2f1c7387546"}
Sep 10 19:39:22.556367 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:22.556312    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7458-71aaac" event={"ID":"29a4fded-ae24-4eb9-a6e7-aecfddb1093f","Type":"ContainerStarted","Data":"38fc84090835bb5215d5e34827e597cfad0ab636cf0c9bc133d8f4655f488301"}
Sep 10 19:39:22.556766 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:22.556404    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7458-5640fb" podUID="0c270113-780e-47be-a503-d207b2c6eff6" containerName="test-curl-7458-5640fb" containerID="cri-o://61b1ff989cae679cf86a75503e01b62bb198332925779f0fe5548e882c6aba52" gracePeriod=2
Sep 10 19:39:22.570635 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:22.570595    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7458-71aaac" podStartSLOduration=1.570582602 podStartE2EDuration="1.570582602s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:39:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:39:22.569425099 +0000 UTC m=+1433.629252751" watchObservedRunningTime="2026-09-10 19:39:22.570582602 +0000 UTC m=+1433.630410269"
Sep 10 19:39:23.126493 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:23.126463    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7458-71aaac"]
Sep 10 19:39:23.133666 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:23.133632    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7458-71aaac"]
Sep 10 19:39:23.567702 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:39:23.567668    2575 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7458-71aaac" container="test-curl-7458-71aaac"
Sep 10 19:39:24.572599 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:24.572567    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7458-71aaac" podUID="29a4fded-ae24-4eb9-a6e7-aecfddb1093f" containerName="test-curl-7458-71aaac" containerID="cri-o://999c9e765858792406605f2391b5f28434a4555ea942303466a7f2f1c7387546" gracePeriod=2
Sep 10 19:39:24.741822 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:24.741798    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7458-5640fb"
Sep 10 19:39:24.811792 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:24.811740    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xrrp2\" (UniqueName: \"kubernetes.io/projected/0c270113-780e-47be-a503-d207b2c6eff6-kube-api-access-xrrp2\") pod \"0c270113-780e-47be-a503-d207b2c6eff6\" (UID: \"0c270113-780e-47be-a503-d207b2c6eff6\") "
Sep 10 19:39:24.813687 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:24.813658    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0c270113-780e-47be-a503-d207b2c6eff6-kube-api-access-xrrp2" (OuterVolumeSpecName: "kube-api-access-xrrp2") pod "0c270113-780e-47be-a503-d207b2c6eff6" (UID: "0c270113-780e-47be-a503-d207b2c6eff6"). InnerVolumeSpecName "kube-api-access-xrrp2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:39:24.912626 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:24.912602    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xrrp2\" (UniqueName: \"kubernetes.io/projected/0c270113-780e-47be-a503-d207b2c6eff6-kube-api-access-xrrp2\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:39:25.075860 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:25.075796    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-14a8d1/sub-test-model-14a8d1-kserve-96c54d5cc-dstjk"]
Sep 10 19:39:25.097744 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:25.097716    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:39:25.385834 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:25.385757    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0c270113-780e-47be-a503-d207b2c6eff6" path="/var/lib/kubelet/pods/0c270113-780e-47be-a503-d207b2c6eff6/volumes"
Sep 10 19:39:25.578796 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:25.578767    2575 generic.go:356] "Generic (PLEG): container finished" podID="0c270113-780e-47be-a503-d207b2c6eff6" containerID="61b1ff989cae679cf86a75503e01b62bb198332925779f0fe5548e882c6aba52" exitCode=137
Sep 10 19:39:25.579200 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:25.578808    2575 scope.go:117] "RemoveContainer" containerID="61b1ff989cae679cf86a75503e01b62bb198332925779f0fe5548e882c6aba52"
Sep 10 19:39:25.579200 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:25.578811    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7458-5640fb"
Sep 10 19:39:25.587406 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:25.587386    2575 scope.go:117] "RemoveContainer" containerID="61b1ff989cae679cf86a75503e01b62bb198332925779f0fe5548e882c6aba52"
Sep 10 19:39:25.587682 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:39:25.587665    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"61b1ff989cae679cf86a75503e01b62bb198332925779f0fe5548e882c6aba52\": container with ID starting with 61b1ff989cae679cf86a75503e01b62bb198332925779f0fe5548e882c6aba52 not found: ID does not exist" containerID="61b1ff989cae679cf86a75503e01b62bb198332925779f0fe5548e882c6aba52"
Sep 10 19:39:25.587730 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:25.587695    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"61b1ff989cae679cf86a75503e01b62bb198332925779f0fe5548e882c6aba52"} err="failed to get container status \"61b1ff989cae679cf86a75503e01b62bb198332925779f0fe5548e882c6aba52\": rpc error: code = NotFound desc = could not find container \"61b1ff989cae679cf86a75503e01b62bb198332925779f0fe5548e882c6aba52\": container with ID starting with 61b1ff989cae679cf86a75503e01b62bb198332925779f0fe5548e882c6aba52 not found: ID does not exist"
Sep 10 19:39:25.985009 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:25.984976    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:39:26.749501 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:26.749478    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7458-71aaac"
Sep 10 19:39:26.828455 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:26.828393    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qqgwm\" (UniqueName: \"kubernetes.io/projected/29a4fded-ae24-4eb9-a6e7-aecfddb1093f-kube-api-access-qqgwm\") pod \"29a4fded-ae24-4eb9-a6e7-aecfddb1093f\" (UID: \"29a4fded-ae24-4eb9-a6e7-aecfddb1093f\") "
Sep 10 19:39:26.830238 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:26.830212    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/29a4fded-ae24-4eb9-a6e7-aecfddb1093f-kube-api-access-qqgwm" (OuterVolumeSpecName: "kube-api-access-qqgwm") pod "29a4fded-ae24-4eb9-a6e7-aecfddb1093f" (UID: "29a4fded-ae24-4eb9-a6e7-aecfddb1093f"). InnerVolumeSpecName "kube-api-access-qqgwm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:39:26.929654 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:26.929629    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qqgwm\" (UniqueName: \"kubernetes.io/projected/29a4fded-ae24-4eb9-a6e7-aecfddb1093f-kube-api-access-qqgwm\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:39:27.385541 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:27.385509    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="29a4fded-ae24-4eb9-a6e7-aecfddb1093f" path="/var/lib/kubelet/pods/29a4fded-ae24-4eb9-a6e7-aecfddb1093f/volumes"
Sep 10 19:39:27.592826 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:27.592789    2575 generic.go:356] "Generic (PLEG): container finished" podID="29a4fded-ae24-4eb9-a6e7-aecfddb1093f" containerID="999c9e765858792406605f2391b5f28434a4555ea942303466a7f2f1c7387546" exitCode=137
Sep 10 19:39:27.592826 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:27.592830    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7458-71aaac"
Sep 10 19:39:27.593047 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:27.592882    2575 scope.go:117] "RemoveContainer" containerID="999c9e765858792406605f2391b5f28434a4555ea942303466a7f2f1c7387546"
Sep 10 19:39:27.601391 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:27.601374    2575 scope.go:117] "RemoveContainer" containerID="999c9e765858792406605f2391b5f28434a4555ea942303466a7f2f1c7387546"
Sep 10 19:39:27.601671 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:39:27.601649    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"999c9e765858792406605f2391b5f28434a4555ea942303466a7f2f1c7387546\": container with ID starting with 999c9e765858792406605f2391b5f28434a4555ea942303466a7f2f1c7387546 not found: ID does not exist" containerID="999c9e765858792406605f2391b5f28434a4555ea942303466a7f2f1c7387546"
Sep 10 19:39:27.601718 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:27.601678    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"999c9e765858792406605f2391b5f28434a4555ea942303466a7f2f1c7387546"} err="failed to get container status \"999c9e765858792406605f2391b5f28434a4555ea942303466a7f2f1c7387546\": rpc error: code = NotFound desc = could not find container \"999c9e765858792406605f2391b5f28434a4555ea942303466a7f2f1c7387546\": container with ID starting with 999c9e765858792406605f2391b5f28434a4555ea942303466a7f2f1c7387546 not found: ID does not exist"
Sep 10 19:39:27.911725 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:27.911692    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:39:27.992258 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:27.992225    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:39:28.050330 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:28.050289    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:39:28.132857 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:28.132824    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-cbac1f-g9svd"]
Sep 10 19:39:28.135284 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:28.135252    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:39:28.135458 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:28.135437    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-cbac1f-g9svd"]
Sep 10 19:39:28.189160 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:28.189099    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:39:28.464152 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:28.464120    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:39:29.389855 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:29.389821    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="feb35367-1430-477e-875d-425c8a6fbda9" path="/var/lib/kubelet/pods/feb35367-1430-477e-875d-425c8a6fbda9/volumes"
Sep 10 19:39:29.579466 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:29.579437    2575 scope.go:117] "RemoveContainer" containerID="a6fae3a63dfb6b9d778315f028570bb9eb1e58ebd2971502d48db7d9b3017709"
Sep 10 19:39:34.316028 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:34.315997    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:39:34.530649 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:34.530618    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:39:34.630843 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:34.630753    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:39:34.766097 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:34.766066    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-cbac1f-openshift-default-5d4cdbcfc8-w88f7"]
Sep 10 19:39:34.766284 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:34.766265    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-cbac1f-openshift-default-5d4cdbcfc8-w88f7" podUID="1ec61c77-6b68-471d-b746-17b5936f07a2" containerName="istio-proxy" containerID="cri-o://f1014fe814fc875c057d2413038d06b2cc5a740916c9a9aca03aac6de33c82b8" gracePeriod=30
Sep 10 19:39:34.795053 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:34.795025    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:39:35.348508 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:35.348473    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:39:35.951221 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:35.951181    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qr7rl"]
Sep 10 19:39:40.015974 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:40.015954    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-cbac1f-openshift-default-5d4cdbcfc8-w88f7"
Sep 10 19:39:40.095637 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:40.095542    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-14a8d1/sub-test-model-14a8d1-kserve-96c54d5cc-dstjk" podUID="e870f493-ce00-4586-930a-098ede28e8ba" containerName="main" containerID="cri-o://940d1acd090ae25e25190cb807900465b45abf2810e804274b15283e193963ce" gracePeriod=45
Sep 10 19:39:40.138847 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:40.138821    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ec61c77-6b68-471d-b746-17b5936f07a2-credential-socket\") pod \"1ec61c77-6b68-471d-b746-17b5936f07a2\" (UID: \"1ec61c77-6b68-471d-b746-17b5936f07a2\") "
Sep 10 19:39:40.138970 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:40.138886    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1ec61c77-6b68-471d-b746-17b5936f07a2-istio-envoy\") pod \"1ec61c77-6b68-471d-b746-17b5936f07a2\" (UID: \"1ec61c77-6b68-471d-b746-17b5936f07a2\") "
Sep 10 19:39:40.139031 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:40.138976    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1ec61c77-6b68-471d-b746-17b5936f07a2-istio-token\") pod \"1ec61c77-6b68-471d-b746-17b5936f07a2\" (UID: \"1ec61c77-6b68-471d-b746-17b5936f07a2\") "
Sep 10 19:39:40.139031 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:40.139002    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1ec61c77-6b68-471d-b746-17b5936f07a2-workload-certs\") pod \"1ec61c77-6b68-471d-b746-17b5936f07a2\" (UID: \"1ec61c77-6b68-471d-b746-17b5936f07a2\") "
Sep 10 19:39:40.139127 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:40.139029    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ec61c77-6b68-471d-b746-17b5936f07a2-workload-socket\") pod \"1ec61c77-6b68-471d-b746-17b5936f07a2\" (UID: \"1ec61c77-6b68-471d-b746-17b5936f07a2\") "
Sep 10 19:39:40.139127 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:40.139065    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ec61c77-6b68-471d-b746-17b5936f07a2-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1ec61c77-6b68-471d-b746-17b5936f07a2" (UID: "1ec61c77-6b68-471d-b746-17b5936f07a2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:39:40.139127 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:40.139118    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1ec61c77-6b68-471d-b746-17b5936f07a2-istio-podinfo\") pod \"1ec61c77-6b68-471d-b746-17b5936f07a2\" (UID: \"1ec61c77-6b68-471d-b746-17b5936f07a2\") "
Sep 10 19:39:40.139282 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:40.139152    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1ec61c77-6b68-471d-b746-17b5936f07a2-istio-data\") pod \"1ec61c77-6b68-471d-b746-17b5936f07a2\" (UID: \"1ec61c77-6b68-471d-b746-17b5936f07a2\") "
Sep 10 19:39:40.139282 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:40.139191    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z5z8v\" (UniqueName: \"kubernetes.io/projected/1ec61c77-6b68-471d-b746-17b5936f07a2-kube-api-access-z5z8v\") pod \"1ec61c77-6b68-471d-b746-17b5936f07a2\" (UID: \"1ec61c77-6b68-471d-b746-17b5936f07a2\") "
Sep 10 19:39:40.139282 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:40.139245    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1ec61c77-6b68-471d-b746-17b5936f07a2-istiod-ca-cert\") pod \"1ec61c77-6b68-471d-b746-17b5936f07a2\" (UID: \"1ec61c77-6b68-471d-b746-17b5936f07a2\") "
Sep 10 19:39:40.139282 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:40.139266    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ec61c77-6b68-471d-b746-17b5936f07a2-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1ec61c77-6b68-471d-b746-17b5936f07a2" (UID: "1ec61c77-6b68-471d-b746-17b5936f07a2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:39:40.139505 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:40.139281    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ec61c77-6b68-471d-b746-17b5936f07a2-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1ec61c77-6b68-471d-b746-17b5936f07a2" (UID: "1ec61c77-6b68-471d-b746-17b5936f07a2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:39:40.139505 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:40.139440    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ec61c77-6b68-471d-b746-17b5936f07a2-istio-data" (OuterVolumeSpecName: "istio-data") pod "1ec61c77-6b68-471d-b746-17b5936f07a2" (UID: "1ec61c77-6b68-471d-b746-17b5936f07a2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:39:40.139634 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:40.139608    2575 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ec61c77-6b68-471d-b746-17b5936f07a2-credential-socket\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:39:40.139694 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:40.139646    2575 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1ec61c77-6b68-471d-b746-17b5936f07a2-workload-certs\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:39:40.139694 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:40.139649    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1ec61c77-6b68-471d-b746-17b5936f07a2-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1ec61c77-6b68-471d-b746-17b5936f07a2" (UID: "1ec61c77-6b68-471d-b746-17b5936f07a2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:39:40.139694 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:40.139658    2575 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ec61c77-6b68-471d-b746-17b5936f07a2-workload-socket\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:39:40.139694 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:40.139688    2575 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1ec61c77-6b68-471d-b746-17b5936f07a2-istio-data\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:39:40.141141 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:40.141113    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1ec61c77-6b68-471d-b746-17b5936f07a2-istio-token" (OuterVolumeSpecName: "istio-token") pod "1ec61c77-6b68-471d-b746-17b5936f07a2" (UID: "1ec61c77-6b68-471d-b746-17b5936f07a2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:39:40.141240 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:40.141177    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1ec61c77-6b68-471d-b746-17b5936f07a2-kube-api-access-z5z8v" (OuterVolumeSpecName: "kube-api-access-z5z8v") pod "1ec61c77-6b68-471d-b746-17b5936f07a2" (UID: "1ec61c77-6b68-471d-b746-17b5936f07a2"). InnerVolumeSpecName "kube-api-access-z5z8v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:39:40.141327 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:40.141310    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ec61c77-6b68-471d-b746-17b5936f07a2-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1ec61c77-6b68-471d-b746-17b5936f07a2" (UID: "1ec61c77-6b68-471d-b746-17b5936f07a2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:39:40.141474 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:40.141455    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1ec61c77-6b68-471d-b746-17b5936f07a2-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1ec61c77-6b68-471d-b746-17b5936f07a2" (UID: "1ec61c77-6b68-471d-b746-17b5936f07a2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 19:39:40.241156 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:40.241127    2575 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1ec61c77-6b68-471d-b746-17b5936f07a2-istio-token\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:39:40.241156 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:40.241152    2575 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1ec61c77-6b68-471d-b746-17b5936f07a2-istio-podinfo\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:39:40.241156 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:40.241162    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z5z8v\" (UniqueName: \"kubernetes.io/projected/1ec61c77-6b68-471d-b746-17b5936f07a2-kube-api-access-z5z8v\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:39:40.241450 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:40.241172    2575 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1ec61c77-6b68-471d-b746-17b5936f07a2-istiod-ca-cert\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:39:40.241450 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:40.241181    2575 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1ec61c77-6b68-471d-b746-17b5936f07a2-istio-envoy\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:39:40.326469 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:40.326438    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-14a8d1/sub-test-model-14a8d1-kserve-96c54d5cc-dstjk"
Sep 10 19:39:40.442908 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:40.442839    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e870f493-ce00-4586-930a-098ede28e8ba-tls-certs\") pod \"e870f493-ce00-4586-930a-098ede28e8ba\" (UID: \"e870f493-ce00-4586-930a-098ede28e8ba\") "
Sep 10 19:39:40.442908 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:40.442869    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e870f493-ce00-4586-930a-098ede28e8ba-dshm\") pod \"e870f493-ce00-4586-930a-098ede28e8ba\" (UID: \"e870f493-ce00-4586-930a-098ede28e8ba\") "
Sep 10 19:39:40.443084 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:40.443009    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e870f493-ce00-4586-930a-098ede28e8ba-home\") pod \"e870f493-ce00-4586-930a-098ede28e8ba\" (UID: \"e870f493-ce00-4586-930a-098ede28e8ba\") "
Sep 10 19:39:40.443084 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:40.443054    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hm76b\" (UniqueName: \"kubernetes.io/projected/e870f493-ce00-4586-930a-098ede28e8ba-kube-api-access-hm76b\") pod \"e870f493-ce00-4586-930a-098ede28e8ba\" (UID: \"e870f493-ce00-4586-930a-098ede28e8ba\") "
Sep 10 19:39:40.443162 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:40.443119    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e870f493-ce00-4586-930a-098ede28e8ba-model-cache\") pod \"e870f493-ce00-4586-930a-098ede28e8ba\" (UID: \"e870f493-ce00-4586-930a-098ede28e8ba\") "
Sep 10 19:39:40.443212 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:40.443180    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e870f493-ce00-4586-930a-098ede28e8ba-tmp-dir\") pod \"e870f493-ce00-4586-930a-098ede28e8ba\" (UID: \"e870f493-ce00-4586-930a-098ede28e8ba\") "
Sep 10 19:39:40.443283 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:40.443260    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e870f493-ce00-4586-930a-098ede28e8ba-home" (OuterVolumeSpecName: "home") pod "e870f493-ce00-4586-930a-098ede28e8ba" (UID: "e870f493-ce00-4586-930a-098ede28e8ba"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:39:40.443486 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:40.443464    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e870f493-ce00-4586-930a-098ede28e8ba-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e870f493-ce00-4586-930a-098ede28e8ba" (UID: "e870f493-ce00-4586-930a-098ede28e8ba"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:39:40.443601 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:40.443481    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e870f493-ce00-4586-930a-098ede28e8ba-model-cache" (OuterVolumeSpecName: "model-cache") pod "e870f493-ce00-4586-930a-098ede28e8ba" (UID: "e870f493-ce00-4586-930a-098ede28e8ba"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:39:40.443645 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:40.443601    2575 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e870f493-ce00-4586-930a-098ede28e8ba-model-cache\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:39:40.443645 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:40.443612    2575 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e870f493-ce00-4586-930a-098ede28e8ba-tmp-dir\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:39:40.443645 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:40.443620    2575 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e870f493-ce00-4586-930a-098ede28e8ba-home\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:39:40.445146 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:40.445131    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e870f493-ce00-4586-930a-098ede28e8ba-kube-api-access-hm76b" (OuterVolumeSpecName: "kube-api-access-hm76b") pod "e870f493-ce00-4586-930a-098ede28e8ba" (UID: "e870f493-ce00-4586-930a-098ede28e8ba"). InnerVolumeSpecName "kube-api-access-hm76b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:39:40.445367 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:40.445352    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e870f493-ce00-4586-930a-098ede28e8ba-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e870f493-ce00-4586-930a-098ede28e8ba" (UID: "e870f493-ce00-4586-930a-098ede28e8ba"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:39:40.445429 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:40.445379    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e870f493-ce00-4586-930a-098ede28e8ba-dshm" (OuterVolumeSpecName: "dshm") pod "e870f493-ce00-4586-930a-098ede28e8ba" (UID: "e870f493-ce00-4586-930a-098ede28e8ba"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:39:40.544655 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:40.544625    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hm76b\" (UniqueName: \"kubernetes.io/projected/e870f493-ce00-4586-930a-098ede28e8ba-kube-api-access-hm76b\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:39:40.544655 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:40.544649    2575 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e870f493-ce00-4586-930a-098ede28e8ba-tls-certs\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:39:40.544809 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:40.544662    2575 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e870f493-ce00-4586-930a-098ede28e8ba-dshm\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:39:40.660787 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:40.660758    2575 generic.go:356] "Generic (PLEG): container finished" podID="e870f493-ce00-4586-930a-098ede28e8ba" containerID="940d1acd090ae25e25190cb807900465b45abf2810e804274b15283e193963ce" exitCode=0
Sep 10 19:39:40.660923 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:40.660801    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-14a8d1/sub-test-model-14a8d1-kserve-96c54d5cc-dstjk"
Sep 10 19:39:40.660923 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:40.660843    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-14a8d1/sub-test-model-14a8d1-kserve-96c54d5cc-dstjk" event={"ID":"e870f493-ce00-4586-930a-098ede28e8ba","Type":"ContainerDied","Data":"940d1acd090ae25e25190cb807900465b45abf2810e804274b15283e193963ce"}
Sep 10 19:39:40.660923 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:40.660892    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-14a8d1/sub-test-model-14a8d1-kserve-96c54d5cc-dstjk" event={"ID":"e870f493-ce00-4586-930a-098ede28e8ba","Type":"ContainerDied","Data":"9ebd9659c7d3a86e5a8206ba8d2f201a7f2e3105f67392ec1ffd62468746727e"}
Sep 10 19:39:40.660923 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:40.660914    2575 scope.go:117] "RemoveContainer" containerID="940d1acd090ae25e25190cb807900465b45abf2810e804274b15283e193963ce"
Sep 10 19:39:40.662657 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:40.662632    2575 generic.go:356] "Generic (PLEG): container finished" podID="1ec61c77-6b68-471d-b746-17b5936f07a2" containerID="f1014fe814fc875c057d2413038d06b2cc5a740916c9a9aca03aac6de33c82b8" exitCode=0
Sep 10 19:39:40.662779 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:40.662669    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-cbac1f-openshift-default-5d4cdbcfc8-w88f7"
Sep 10 19:39:40.662779 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:40.662693    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-cbac1f-openshift-default-5d4cdbcfc8-w88f7" event={"ID":"1ec61c77-6b68-471d-b746-17b5936f07a2","Type":"ContainerDied","Data":"f1014fe814fc875c057d2413038d06b2cc5a740916c9a9aca03aac6de33c82b8"}
Sep 10 19:39:40.662779 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:40.662742    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-cbac1f-openshift-default-5d4cdbcfc8-w88f7" event={"ID":"1ec61c77-6b68-471d-b746-17b5936f07a2","Type":"ContainerDied","Data":"60cc834613d2fce5f835f351e4401c45afa974f93a5cb2e86db83889457e849e"}
Sep 10 19:39:40.670687 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:40.670672    2575 scope.go:117] "RemoveContainer" containerID="940d1acd090ae25e25190cb807900465b45abf2810e804274b15283e193963ce"
Sep 10 19:39:40.670966 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:39:40.670940    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"940d1acd090ae25e25190cb807900465b45abf2810e804274b15283e193963ce\": container with ID starting with 940d1acd090ae25e25190cb807900465b45abf2810e804274b15283e193963ce not found: ID does not exist" containerID="940d1acd090ae25e25190cb807900465b45abf2810e804274b15283e193963ce"
Sep 10 19:39:40.671034 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:40.670978    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"940d1acd090ae25e25190cb807900465b45abf2810e804274b15283e193963ce"} err="failed to get container status \"940d1acd090ae25e25190cb807900465b45abf2810e804274b15283e193963ce\": rpc error: code = NotFound desc = could not find container \"940d1acd090ae25e25190cb807900465b45abf2810e804274b15283e193963ce\": container with ID starting with 940d1acd090ae25e25190cb807900465b45abf2810e804274b15283e193963ce not found: ID does not exist"
Sep 10 19:39:40.671034 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:40.671002    2575 scope.go:117] "RemoveContainer" containerID="f1014fe814fc875c057d2413038d06b2cc5a740916c9a9aca03aac6de33c82b8"
Sep 10 19:39:40.679227 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:40.679211    2575 scope.go:117] "RemoveContainer" containerID="6bf0e90ccbb8d50c78bb1e478880396dace7cb6f801dc00cccd6957f58f2b7d0"
Sep 10 19:39:40.682423 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:40.682335    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-14a8d1/sub-test-model-14a8d1-kserve-96c54d5cc-dstjk"]
Sep 10 19:39:40.687797 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:40.687776    2575 scope.go:117] "RemoveContainer" containerID="f1014fe814fc875c057d2413038d06b2cc5a740916c9a9aca03aac6de33c82b8"
Sep 10 19:39:40.688064 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:39:40.688038    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f1014fe814fc875c057d2413038d06b2cc5a740916c9a9aca03aac6de33c82b8\": container with ID starting with f1014fe814fc875c057d2413038d06b2cc5a740916c9a9aca03aac6de33c82b8 not found: ID does not exist" containerID="f1014fe814fc875c057d2413038d06b2cc5a740916c9a9aca03aac6de33c82b8"
Sep 10 19:39:40.688137 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:40.688070    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f1014fe814fc875c057d2413038d06b2cc5a740916c9a9aca03aac6de33c82b8"} err="failed to get container status \"f1014fe814fc875c057d2413038d06b2cc5a740916c9a9aca03aac6de33c82b8\": rpc error: code = NotFound desc = could not find container \"f1014fe814fc875c057d2413038d06b2cc5a740916c9a9aca03aac6de33c82b8\": container with ID starting with f1014fe814fc875c057d2413038d06b2cc5a740916c9a9aca03aac6de33c82b8 not found: ID does not exist"
Sep 10 19:39:40.688137 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:40.688085    2575 scope.go:117] "RemoveContainer" containerID="6bf0e90ccbb8d50c78bb1e478880396dace7cb6f801dc00cccd6957f58f2b7d0"
Sep 10 19:39:40.688385 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:39:40.688371    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6bf0e90ccbb8d50c78bb1e478880396dace7cb6f801dc00cccd6957f58f2b7d0\": container with ID starting with 6bf0e90ccbb8d50c78bb1e478880396dace7cb6f801dc00cccd6957f58f2b7d0 not found: ID does not exist" containerID="6bf0e90ccbb8d50c78bb1e478880396dace7cb6f801dc00cccd6957f58f2b7d0"
Sep 10 19:39:40.688439 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:40.688389    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6bf0e90ccbb8d50c78bb1e478880396dace7cb6f801dc00cccd6957f58f2b7d0"} err="failed to get container status \"6bf0e90ccbb8d50c78bb1e478880396dace7cb6f801dc00cccd6957f58f2b7d0\": rpc error: code = NotFound desc = could not find container \"6bf0e90ccbb8d50c78bb1e478880396dace7cb6f801dc00cccd6957f58f2b7d0\": container with ID starting with 6bf0e90ccbb8d50c78bb1e478880396dace7cb6f801dc00cccd6957f58f2b7d0 not found: ID does not exist"
Sep 10 19:39:40.688439 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:40.688408    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-14a8d1/sub-test-model-14a8d1-kserve-96c54d5cc-dstjk"]
Sep 10 19:39:40.698969 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:40.698945    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-cbac1f-openshift-default-5d4cdbcfc8-w88f7"]
Sep 10 19:39:40.702980 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:40.702961    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-cbac1f-openshift-default-5d4cdbcfc8-w88f7"]
Sep 10 19:39:41.384640 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:41.384606    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1ec61c77-6b68-471d-b746-17b5936f07a2" path="/var/lib/kubelet/pods/1ec61c77-6b68-471d-b746-17b5936f07a2/volumes"
Sep 10 19:39:41.385040 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:41.385026    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e870f493-ce00-4586-930a-098ede28e8ba" path="/var/lib/kubelet/pods/e870f493-ce00-4586-930a-098ede28e8ba/volumes"
Sep 10 19:39:45.691910 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:45.691869    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:39:45.726785 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:45.726753    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-8dd4a3-6ff9bb8949-mbb6m"]
Sep 10 19:39:45.727051 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:45.726999    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-8dd4a3-6ff9bb8949-mbb6m" podUID="fdff22b9-2e1f-4e1e-b7af-d929036e4e82" containerName="payload-processing" containerID="cri-o://48c1f7e157fa199d0a3d3bfbd9f84dfae051c0cf7a2975448530d8892f3ad1eb" gracePeriod=30
Sep 10 19:39:45.735375 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:45.735335    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-8dd4a3-75545f76bb-pmdbk"]
Sep 10 19:39:45.735982 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:45.735568    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-8dd4a3-75545f76bb-pmdbk" podUID="0dc7c183-7da2-4dc6-8fea-b317f4f2553c" containerName="payload-pre-processing" containerID="cri-o://5fab4b74e272fd32fa8daef3323c92208073af19230b0426fe50197aa6c714c0" gracePeriod=30
Sep 10 19:39:45.774715 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:45.774680    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:39:45.914909 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:45.914832    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:39:46.003065 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:46.003046    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-8dd4a3-6ff9bb8949-mbb6m"
Sep 10 19:39:46.016737 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:46.016712    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-8dd4a3-75545f76bb-pmdbk"
Sep 10 19:39:46.037098 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:46.037068    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:39:46.086960 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:46.086936    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fdff22b9-2e1f-4e1e-b7af-d929036e4e82-plugins-config-volume\") pod \"fdff22b9-2e1f-4e1e-b7af-d929036e4e82\" (UID: \"fdff22b9-2e1f-4e1e-b7af-d929036e4e82\") "
Sep 10 19:39:46.087060 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:46.087011    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0dc7c183-7da2-4dc6-8fea-b317f4f2553c-plugins-config-volume\") pod \"0dc7c183-7da2-4dc6-8fea-b317f4f2553c\" (UID: \"0dc7c183-7da2-4dc6-8fea-b317f4f2553c\") "
Sep 10 19:39:46.087119 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:46.087105    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-smqcm\" (UniqueName: \"kubernetes.io/projected/0dc7c183-7da2-4dc6-8fea-b317f4f2553c-kube-api-access-smqcm\") pod \"0dc7c183-7da2-4dc6-8fea-b317f4f2553c\" (UID: \"0dc7c183-7da2-4dc6-8fea-b317f4f2553c\") "
Sep 10 19:39:46.087166 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:46.087145    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hqcvf\" (UniqueName: \"kubernetes.io/projected/fdff22b9-2e1f-4e1e-b7af-d929036e4e82-kube-api-access-hqcvf\") pod \"fdff22b9-2e1f-4e1e-b7af-d929036e4e82\" (UID: \"fdff22b9-2e1f-4e1e-b7af-d929036e4e82\") "
Sep 10 19:39:46.087333 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:46.087312    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fdff22b9-2e1f-4e1e-b7af-d929036e4e82-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "fdff22b9-2e1f-4e1e-b7af-d929036e4e82" (UID: "fdff22b9-2e1f-4e1e-b7af-d929036e4e82"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:39:46.087470 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:46.087451    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0dc7c183-7da2-4dc6-8fea-b317f4f2553c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "0dc7c183-7da2-4dc6-8fea-b317f4f2553c" (UID: "0dc7c183-7da2-4dc6-8fea-b317f4f2553c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:39:46.087550 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:46.087524    2575 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fdff22b9-2e1f-4e1e-b7af-d929036e4e82-plugins-config-volume\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:39:46.089064 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:46.089044    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0dc7c183-7da2-4dc6-8fea-b317f4f2553c-kube-api-access-smqcm" (OuterVolumeSpecName: "kube-api-access-smqcm") pod "0dc7c183-7da2-4dc6-8fea-b317f4f2553c" (UID: "0dc7c183-7da2-4dc6-8fea-b317f4f2553c"). InnerVolumeSpecName "kube-api-access-smqcm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:39:46.089146 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:46.089126    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fdff22b9-2e1f-4e1e-b7af-d929036e4e82-kube-api-access-hqcvf" (OuterVolumeSpecName: "kube-api-access-hqcvf") pod "fdff22b9-2e1f-4e1e-b7af-d929036e4e82" (UID: "fdff22b9-2e1f-4e1e-b7af-d929036e4e82"). InnerVolumeSpecName "kube-api-access-hqcvf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:39:46.187978 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:46.187953    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-smqcm\" (UniqueName: \"kubernetes.io/projected/0dc7c183-7da2-4dc6-8fea-b317f4f2553c-kube-api-access-smqcm\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:39:46.187978 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:46.187975    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hqcvf\" (UniqueName: \"kubernetes.io/projected/fdff22b9-2e1f-4e1e-b7af-d929036e4e82-kube-api-access-hqcvf\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:39:46.188123 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:46.187985    2575 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0dc7c183-7da2-4dc6-8fea-b317f4f2553c-plugins-config-volume\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:39:46.263540 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:46.263467    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:39:46.345148 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:46.345120    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qr7rl"]
Sep 10 19:39:46.450830 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:46.450789    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:39:46.699546 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:46.699516    2575 generic.go:356] "Generic (PLEG): container finished" podID="0dc7c183-7da2-4dc6-8fea-b317f4f2553c" containerID="5fab4b74e272fd32fa8daef3323c92208073af19230b0426fe50197aa6c714c0" exitCode=0
Sep 10 19:39:46.699941 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:46.699559    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-8dd4a3-75545f76bb-pmdbk"
Sep 10 19:39:46.699941 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:46.699600    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-8dd4a3-75545f76bb-pmdbk" event={"ID":"0dc7c183-7da2-4dc6-8fea-b317f4f2553c","Type":"ContainerDied","Data":"5fab4b74e272fd32fa8daef3323c92208073af19230b0426fe50197aa6c714c0"}
Sep 10 19:39:46.699941 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:46.699644    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-8dd4a3-75545f76bb-pmdbk" event={"ID":"0dc7c183-7da2-4dc6-8fea-b317f4f2553c","Type":"ContainerDied","Data":"5d59161382697312e9b6117c2d660373d42a31d34a88a551029102783acded93"}
Sep 10 19:39:46.699941 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:46.699670    2575 scope.go:117] "RemoveContainer" containerID="5fab4b74e272fd32fa8daef3323c92208073af19230b0426fe50197aa6c714c0"
Sep 10 19:39:46.701050 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:46.701025    2575 generic.go:356] "Generic (PLEG): container finished" podID="fdff22b9-2e1f-4e1e-b7af-d929036e4e82" containerID="48c1f7e157fa199d0a3d3bfbd9f84dfae051c0cf7a2975448530d8892f3ad1eb" exitCode=0
Sep 10 19:39:46.701192 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:46.701076    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-8dd4a3-6ff9bb8949-mbb6m"
Sep 10 19:39:46.701192 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:46.701107    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-8dd4a3-6ff9bb8949-mbb6m" event={"ID":"fdff22b9-2e1f-4e1e-b7af-d929036e4e82","Type":"ContainerDied","Data":"48c1f7e157fa199d0a3d3bfbd9f84dfae051c0cf7a2975448530d8892f3ad1eb"}
Sep 10 19:39:46.701192 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:46.701169    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-8dd4a3-6ff9bb8949-mbb6m" event={"ID":"fdff22b9-2e1f-4e1e-b7af-d929036e4e82","Type":"ContainerDied","Data":"a8f6271e895438fcc56f823d6c00bd3b313a810c29ce11ef97d7fbbdde9a1f1c"}
Sep 10 19:39:46.714042 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:46.713964    2575 scope.go:117] "RemoveContainer" containerID="5fab4b74e272fd32fa8daef3323c92208073af19230b0426fe50197aa6c714c0"
Sep 10 19:39:46.714723 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:39:46.714693    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5fab4b74e272fd32fa8daef3323c92208073af19230b0426fe50197aa6c714c0\": container with ID starting with 5fab4b74e272fd32fa8daef3323c92208073af19230b0426fe50197aa6c714c0 not found: ID does not exist" containerID="5fab4b74e272fd32fa8daef3323c92208073af19230b0426fe50197aa6c714c0"
Sep 10 19:39:46.714823 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:46.714725    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5fab4b74e272fd32fa8daef3323c92208073af19230b0426fe50197aa6c714c0"} err="failed to get container status \"5fab4b74e272fd32fa8daef3323c92208073af19230b0426fe50197aa6c714c0\": rpc error: code = NotFound desc = could not find container \"5fab4b74e272fd32fa8daef3323c92208073af19230b0426fe50197aa6c714c0\": container with ID starting with 5fab4b74e272fd32fa8daef3323c92208073af19230b0426fe50197aa6c714c0 not found: ID does not exist"
Sep 10 19:39:46.714823 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:46.714740    2575 scope.go:117] "RemoveContainer" containerID="48c1f7e157fa199d0a3d3bfbd9f84dfae051c0cf7a2975448530d8892f3ad1eb"
Sep 10 19:39:46.723171 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:46.723154    2575 scope.go:117] "RemoveContainer" containerID="48c1f7e157fa199d0a3d3bfbd9f84dfae051c0cf7a2975448530d8892f3ad1eb"
Sep 10 19:39:46.723448 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:39:46.723432    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"48c1f7e157fa199d0a3d3bfbd9f84dfae051c0cf7a2975448530d8892f3ad1eb\": container with ID starting with 48c1f7e157fa199d0a3d3bfbd9f84dfae051c0cf7a2975448530d8892f3ad1eb not found: ID does not exist" containerID="48c1f7e157fa199d0a3d3bfbd9f84dfae051c0cf7a2975448530d8892f3ad1eb"
Sep 10 19:39:46.723487 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:46.723455    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"48c1f7e157fa199d0a3d3bfbd9f84dfae051c0cf7a2975448530d8892f3ad1eb"} err="failed to get container status \"48c1f7e157fa199d0a3d3bfbd9f84dfae051c0cf7a2975448530d8892f3ad1eb\": rpc error: code = NotFound desc = could not find container \"48c1f7e157fa199d0a3d3bfbd9f84dfae051c0cf7a2975448530d8892f3ad1eb\": container with ID starting with 48c1f7e157fa199d0a3d3bfbd9f84dfae051c0cf7a2975448530d8892f3ad1eb not found: ID does not exist"
Sep 10 19:39:46.729386 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:46.729334    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-8dd4a3-6ff9bb8949-mbb6m"]
Sep 10 19:39:46.730962 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:46.730937    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-8dd4a3-6ff9bb8949-mbb6m"]
Sep 10 19:39:46.740656 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:46.740636    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-8dd4a3-75545f76bb-pmdbk"]
Sep 10 19:39:46.744257 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:46.744236    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-8dd4a3-75545f76bb-pmdbk"]
Sep 10 19:39:47.385037 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:47.385007    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0dc7c183-7da2-4dc6-8fea-b317f4f2553c" path="/var/lib/kubelet/pods/0dc7c183-7da2-4dc6-8fea-b317f4f2553c/volumes"
Sep 10 19:39:47.385434 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:47.385418    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fdff22b9-2e1f-4e1e-b7af-d929036e4e82" path="/var/lib/kubelet/pods/fdff22b9-2e1f-4e1e-b7af-d929036e4e82/volumes"
Sep 10 19:39:52.473292 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:52.473245    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:39:52.573663 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:52.573630    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:39:52.714558 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:52.714528    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-8dd4a3-openshift-default-856bd4c5f8-24xmz"]
Sep 10 19:39:52.714791 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:52.714768    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w2-8dd4a3-openshift-default-856bd4c5f8-24xmz" podUID="60e22eaa-6ec6-4298-bfcd-3f42bcce4117" containerName="istio-proxy" containerID="cri-o://dc8ab877844a2e08a88f9dc563d21091d8c1eea9ef2954eb2240e8bfad587ae4" gracePeriod=30
Sep 10 19:39:52.735839 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:52.735764    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:39:53.076813 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:53.076737    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-4bc8b3-kfzjc"]
Sep 10 19:39:53.077119 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:53.077106    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0dc7c183-7da2-4dc6-8fea-b317f4f2553c" containerName="payload-pre-processing"
Sep 10 19:39:53.077163 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:53.077121    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="0dc7c183-7da2-4dc6-8fea-b317f4f2553c" containerName="payload-pre-processing"
Sep 10 19:39:53.077163 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:53.077130    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fdff22b9-2e1f-4e1e-b7af-d929036e4e82" containerName="payload-processing"
Sep 10 19:39:53.077163 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:53.077136    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="fdff22b9-2e1f-4e1e-b7af-d929036e4e82" containerName="payload-processing"
Sep 10 19:39:53.077163 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:53.077143    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1ec61c77-6b68-471d-b746-17b5936f07a2" containerName="istio-proxy"
Sep 10 19:39:53.077163 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:53.077149    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ec61c77-6b68-471d-b746-17b5936f07a2" containerName="istio-proxy"
Sep 10 19:39:53.077163 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:53.077164    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="29a4fded-ae24-4eb9-a6e7-aecfddb1093f" containerName="test-curl-7458-71aaac"
Sep 10 19:39:53.077378 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:53.077170    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="29a4fded-ae24-4eb9-a6e7-aecfddb1093f" containerName="test-curl-7458-71aaac"
Sep 10 19:39:53.077378 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:53.077178    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1ec61c77-6b68-471d-b746-17b5936f07a2" containerName="istio-proxy"
Sep 10 19:39:53.077378 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:53.077183    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ec61c77-6b68-471d-b746-17b5936f07a2" containerName="istio-proxy"
Sep 10 19:39:53.077378 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:53.077194    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e870f493-ce00-4586-930a-098ede28e8ba" containerName="main"
Sep 10 19:39:53.077378 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:53.077199    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="e870f493-ce00-4586-930a-098ede28e8ba" containerName="main"
Sep 10 19:39:53.077378 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:53.077251    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="fdff22b9-2e1f-4e1e-b7af-d929036e4e82" containerName="payload-processing"
Sep 10 19:39:53.077378 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:53.077259    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="1ec61c77-6b68-471d-b746-17b5936f07a2" containerName="istio-proxy"
Sep 10 19:39:53.077378 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:53.077265    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="1ec61c77-6b68-471d-b746-17b5936f07a2" containerName="istio-proxy"
Sep 10 19:39:53.077378 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:53.077272    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="29a4fded-ae24-4eb9-a6e7-aecfddb1093f" containerName="test-curl-7458-71aaac"
Sep 10 19:39:53.077378 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:53.077278    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="e870f493-ce00-4586-930a-098ede28e8ba" containerName="main"
Sep 10 19:39:53.077378 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:53.077284    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="0dc7c183-7da2-4dc6-8fea-b317f4f2553c" containerName="payload-pre-processing"
Sep 10 19:39:53.081947 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:53.081932    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-4bc8b3-kfzjc"
Sep 10 19:39:53.083967 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:53.083945    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 19:39:53.084078 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:53.084012    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wkgt8\""
Sep 10 19:39:53.087387 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:53.087362    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-4bc8b3-kfzjc"]
Sep 10 19:39:53.199252 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:53.199220    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:39:53.247137 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:53.247111    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/42573895-5f18-43e5-a07a-a41bee8be062-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-4bc8b3-kfzjc\" (UID: \"42573895-5f18-43e5-a07a-a41bee8be062\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-4bc8b3-kfzjc"
Sep 10 19:39:53.348185 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:53.348111    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/42573895-5f18-43e5-a07a-a41bee8be062-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-4bc8b3-kfzjc\" (UID: \"42573895-5f18-43e5-a07a-a41bee8be062\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-4bc8b3-kfzjc"
Sep 10 19:39:53.348715 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:53.348698    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/42573895-5f18-43e5-a07a-a41bee8be062-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-4bc8b3-kfzjc\" (UID: \"42573895-5f18-43e5-a07a-a41bee8be062\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-4bc8b3-kfzjc"
Sep 10 19:39:53.394688 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:53.394666    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-4bc8b3-kfzjc"
Sep 10 19:39:53.666291 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:53.666217    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-4bc8b3-kfzjc"]
Sep 10 19:39:53.669782 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:39:53.669754    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42573895_5f18_43e5_a07a_a41bee8be062.slice/crio-4107c117ba6b045a7411f261858264edf0c3c5538af0e0150b812bd0f32d72e0 WatchSource:0}: Error finding container 4107c117ba6b045a7411f261858264edf0c3c5538af0e0150b812bd0f32d72e0: Status 404 returned error can't find the container with id 4107c117ba6b045a7411f261858264edf0c3c5538af0e0150b812bd0f32d72e0
Sep 10 19:39:53.738061 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:53.738035    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-4bc8b3-kfzjc" event={"ID":"42573895-5f18-43e5-a07a-a41bee8be062","Type":"ContainerStarted","Data":"4107c117ba6b045a7411f261858264edf0c3c5538af0e0150b812bd0f32d72e0"}
Sep 10 19:39:53.891217 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:53.891186    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:39:53.991895 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:53.991871    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:39:54.744802 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:54.744772    2575 generic.go:356] "Generic (PLEG): container finished" podID="42573895-5f18-43e5-a07a-a41bee8be062" containerID="43ef681f23bb902808eba910d465eefdbc7e9ed88a090e4d06f330823ffb7050" exitCode=0
Sep 10 19:39:54.745146 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:54.744825    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-4bc8b3-kfzjc" event={"ID":"42573895-5f18-43e5-a07a-a41bee8be062","Type":"ContainerDied","Data":"43ef681f23bb902808eba910d465eefdbc7e9ed88a090e4d06f330823ffb7050"}
Sep 10 19:39:55.878298 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:55.878268    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-4bc8b3-kfzjc"
Sep 10 19:39:55.970464 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:55.970434    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/42573895-5f18-43e5-a07a-a41bee8be062-maas-api-service-ca\") pod \"42573895-5f18-43e5-a07a-a41bee8be062\" (UID: \"42573895-5f18-43e5-a07a-a41bee8be062\") "
Sep 10 19:39:55.970804 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:55.970785    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/42573895-5f18-43e5-a07a-a41bee8be062-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "42573895-5f18-43e5-a07a-a41bee8be062" (UID: "42573895-5f18-43e5-a07a-a41bee8be062"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:39:56.071979 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:56.071920    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/42573895-5f18-43e5-a07a-a41bee8be062-maas-api-service-ca\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:39:56.249210 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:56.249182    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qr7rl"]
Sep 10 19:39:56.755464 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:56.755427    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-4bc8b3-kfzjc" event={"ID":"42573895-5f18-43e5-a07a-a41bee8be062","Type":"ContainerDied","Data":"4107c117ba6b045a7411f261858264edf0c3c5538af0e0150b812bd0f32d72e0"}
Sep 10 19:39:56.755464 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:56.755449    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-4bc8b3-kfzjc"
Sep 10 19:39:56.755664 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:56.755475    2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4107c117ba6b045a7411f261858264edf0c3c5538af0e0150b812bd0f32d72e0"
Sep 10 19:39:57.966254 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:57.966224    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-8dd4a3-openshift-default-856bd4c5f8-24xmz"
Sep 10 19:39:58.090177 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:58.090104    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/60e22eaa-6ec6-4298-bfcd-3f42bcce4117-istio-podinfo\") pod \"60e22eaa-6ec6-4298-bfcd-3f42bcce4117\" (UID: \"60e22eaa-6ec6-4298-bfcd-3f42bcce4117\") "
Sep 10 19:39:58.090177 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:58.090141    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/60e22eaa-6ec6-4298-bfcd-3f42bcce4117-istio-envoy\") pod \"60e22eaa-6ec6-4298-bfcd-3f42bcce4117\" (UID: \"60e22eaa-6ec6-4298-bfcd-3f42bcce4117\") "
Sep 10 19:39:58.090177 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:58.090163    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/60e22eaa-6ec6-4298-bfcd-3f42bcce4117-istio-data\") pod \"60e22eaa-6ec6-4298-bfcd-3f42bcce4117\" (UID: \"60e22eaa-6ec6-4298-bfcd-3f42bcce4117\") "
Sep 10 19:39:58.090494 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:58.090210    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/60e22eaa-6ec6-4298-bfcd-3f42bcce4117-workload-socket\") pod \"60e22eaa-6ec6-4298-bfcd-3f42bcce4117\" (UID: \"60e22eaa-6ec6-4298-bfcd-3f42bcce4117\") "
Sep 10 19:39:58.090494 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:58.090232    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/60e22eaa-6ec6-4298-bfcd-3f42bcce4117-istio-token\") pod \"60e22eaa-6ec6-4298-bfcd-3f42bcce4117\" (UID: \"60e22eaa-6ec6-4298-bfcd-3f42bcce4117\") "
Sep 10 19:39:58.090494 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:58.090251    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/60e22eaa-6ec6-4298-bfcd-3f42bcce4117-workload-certs\") pod \"60e22eaa-6ec6-4298-bfcd-3f42bcce4117\" (UID: \"60e22eaa-6ec6-4298-bfcd-3f42bcce4117\") "
Sep 10 19:39:58.090494 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:58.090275    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/60e22eaa-6ec6-4298-bfcd-3f42bcce4117-credential-socket\") pod \"60e22eaa-6ec6-4298-bfcd-3f42bcce4117\" (UID: \"60e22eaa-6ec6-4298-bfcd-3f42bcce4117\") "
Sep 10 19:39:58.090494 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:58.090307    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/60e22eaa-6ec6-4298-bfcd-3f42bcce4117-istiod-ca-cert\") pod \"60e22eaa-6ec6-4298-bfcd-3f42bcce4117\" (UID: \"60e22eaa-6ec6-4298-bfcd-3f42bcce4117\") "
Sep 10 19:39:58.090494 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:58.090375    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dkvqw\" (UniqueName: \"kubernetes.io/projected/60e22eaa-6ec6-4298-bfcd-3f42bcce4117-kube-api-access-dkvqw\") pod \"60e22eaa-6ec6-4298-bfcd-3f42bcce4117\" (UID: \"60e22eaa-6ec6-4298-bfcd-3f42bcce4117\") "
Sep 10 19:39:58.090804 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:58.090548    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60e22eaa-6ec6-4298-bfcd-3f42bcce4117-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "60e22eaa-6ec6-4298-bfcd-3f42bcce4117" (UID: "60e22eaa-6ec6-4298-bfcd-3f42bcce4117"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:39:58.090804 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:58.090675    2575 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/60e22eaa-6ec6-4298-bfcd-3f42bcce4117-workload-certs\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:39:58.090804 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:58.090560    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60e22eaa-6ec6-4298-bfcd-3f42bcce4117-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "60e22eaa-6ec6-4298-bfcd-3f42bcce4117" (UID: "60e22eaa-6ec6-4298-bfcd-3f42bcce4117"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:39:58.090804 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:58.090731    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60e22eaa-6ec6-4298-bfcd-3f42bcce4117-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "60e22eaa-6ec6-4298-bfcd-3f42bcce4117" (UID: "60e22eaa-6ec6-4298-bfcd-3f42bcce4117"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:39:58.091062 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:58.091015    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60e22eaa-6ec6-4298-bfcd-3f42bcce4117-istio-data" (OuterVolumeSpecName: "istio-data") pod "60e22eaa-6ec6-4298-bfcd-3f42bcce4117" (UID: "60e22eaa-6ec6-4298-bfcd-3f42bcce4117"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:39:58.091062 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:58.091053    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/60e22eaa-6ec6-4298-bfcd-3f42bcce4117-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "60e22eaa-6ec6-4298-bfcd-3f42bcce4117" (UID: "60e22eaa-6ec6-4298-bfcd-3f42bcce4117"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:39:58.092527 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:58.092499    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60e22eaa-6ec6-4298-bfcd-3f42bcce4117-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "60e22eaa-6ec6-4298-bfcd-3f42bcce4117" (UID: "60e22eaa-6ec6-4298-bfcd-3f42bcce4117"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:39:58.092527 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:58.092509    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/60e22eaa-6ec6-4298-bfcd-3f42bcce4117-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "60e22eaa-6ec6-4298-bfcd-3f42bcce4117" (UID: "60e22eaa-6ec6-4298-bfcd-3f42bcce4117"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 19:39:58.092669 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:58.092528    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/60e22eaa-6ec6-4298-bfcd-3f42bcce4117-kube-api-access-dkvqw" (OuterVolumeSpecName: "kube-api-access-dkvqw") pod "60e22eaa-6ec6-4298-bfcd-3f42bcce4117" (UID: "60e22eaa-6ec6-4298-bfcd-3f42bcce4117"). InnerVolumeSpecName "kube-api-access-dkvqw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:39:58.092787 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:58.092765    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/60e22eaa-6ec6-4298-bfcd-3f42bcce4117-istio-token" (OuterVolumeSpecName: "istio-token") pod "60e22eaa-6ec6-4298-bfcd-3f42bcce4117" (UID: "60e22eaa-6ec6-4298-bfcd-3f42bcce4117"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:39:58.191639 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:58.191612    2575 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/60e22eaa-6ec6-4298-bfcd-3f42bcce4117-credential-socket\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:39:58.191639 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:58.191635    2575 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/60e22eaa-6ec6-4298-bfcd-3f42bcce4117-istiod-ca-cert\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:39:58.191802 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:58.191645    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dkvqw\" (UniqueName: \"kubernetes.io/projected/60e22eaa-6ec6-4298-bfcd-3f42bcce4117-kube-api-access-dkvqw\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:39:58.191802 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:58.191655    2575 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/60e22eaa-6ec6-4298-bfcd-3f42bcce4117-istio-podinfo\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:39:58.191802 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:58.191664    2575 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/60e22eaa-6ec6-4298-bfcd-3f42bcce4117-istio-envoy\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:39:58.191802 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:58.191672    2575 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/60e22eaa-6ec6-4298-bfcd-3f42bcce4117-istio-data\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:39:58.191802 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:58.191679    2575 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/60e22eaa-6ec6-4298-bfcd-3f42bcce4117-workload-socket\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:39:58.191802 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:58.191687    2575 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/60e22eaa-6ec6-4298-bfcd-3f42bcce4117-istio-token\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:39:58.767986 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:58.767949    2575 generic.go:356] "Generic (PLEG): container finished" podID="60e22eaa-6ec6-4298-bfcd-3f42bcce4117" containerID="dc8ab877844a2e08a88f9dc563d21091d8c1eea9ef2954eb2240e8bfad587ae4" exitCode=0
Sep 10 19:39:58.768137 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:58.768005    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-8dd4a3-openshift-default-856bd4c5f8-24xmz" event={"ID":"60e22eaa-6ec6-4298-bfcd-3f42bcce4117","Type":"ContainerDied","Data":"dc8ab877844a2e08a88f9dc563d21091d8c1eea9ef2954eb2240e8bfad587ae4"}
Sep 10 19:39:58.768137 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:58.768009    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-8dd4a3-openshift-default-856bd4c5f8-24xmz"
Sep 10 19:39:58.768137 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:58.768052    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-8dd4a3-openshift-default-856bd4c5f8-24xmz" event={"ID":"60e22eaa-6ec6-4298-bfcd-3f42bcce4117","Type":"ContainerDied","Data":"ca82e3759cfb7f4fb4837b165f1eecefd06edb56ead600e3f0164fd4b9369cfd"}
Sep 10 19:39:58.768137 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:58.768077    2575 scope.go:117] "RemoveContainer" containerID="dc8ab877844a2e08a88f9dc563d21091d8c1eea9ef2954eb2240e8bfad587ae4"
Sep 10 19:39:58.777474 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:58.777414    2575 scope.go:117] "RemoveContainer" containerID="ff6dc3b967c2f1c6a58b4a055e5eb5867f5073af8ebbff11d723ff2ee70f10ab"
Sep 10 19:39:58.785083 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:58.785066    2575 scope.go:117] "RemoveContainer" containerID="dc8ab877844a2e08a88f9dc563d21091d8c1eea9ef2954eb2240e8bfad587ae4"
Sep 10 19:39:58.785328 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:39:58.785309    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dc8ab877844a2e08a88f9dc563d21091d8c1eea9ef2954eb2240e8bfad587ae4\": container with ID starting with dc8ab877844a2e08a88f9dc563d21091d8c1eea9ef2954eb2240e8bfad587ae4 not found: ID does not exist" containerID="dc8ab877844a2e08a88f9dc563d21091d8c1eea9ef2954eb2240e8bfad587ae4"
Sep 10 19:39:58.785390 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:58.785336    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dc8ab877844a2e08a88f9dc563d21091d8c1eea9ef2954eb2240e8bfad587ae4"} err="failed to get container status \"dc8ab877844a2e08a88f9dc563d21091d8c1eea9ef2954eb2240e8bfad587ae4\": rpc error: code = NotFound desc = could not find container \"dc8ab877844a2e08a88f9dc563d21091d8c1eea9ef2954eb2240e8bfad587ae4\": container with ID starting with dc8ab877844a2e08a88f9dc563d21091d8c1eea9ef2954eb2240e8bfad587ae4 not found: ID does not exist"
Sep 10 19:39:58.785390 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:58.785367    2575 scope.go:117] "RemoveContainer" containerID="ff6dc3b967c2f1c6a58b4a055e5eb5867f5073af8ebbff11d723ff2ee70f10ab"
Sep 10 19:39:58.785607 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:39:58.785590    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ff6dc3b967c2f1c6a58b4a055e5eb5867f5073af8ebbff11d723ff2ee70f10ab\": container with ID starting with ff6dc3b967c2f1c6a58b4a055e5eb5867f5073af8ebbff11d723ff2ee70f10ab not found: ID does not exist" containerID="ff6dc3b967c2f1c6a58b4a055e5eb5867f5073af8ebbff11d723ff2ee70f10ab"
Sep 10 19:39:58.785651 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:58.785613    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ff6dc3b967c2f1c6a58b4a055e5eb5867f5073af8ebbff11d723ff2ee70f10ab"} err="failed to get container status \"ff6dc3b967c2f1c6a58b4a055e5eb5867f5073af8ebbff11d723ff2ee70f10ab\": rpc error: code = NotFound desc = could not find container \"ff6dc3b967c2f1c6a58b4a055e5eb5867f5073af8ebbff11d723ff2ee70f10ab\": container with ID starting with ff6dc3b967c2f1c6a58b4a055e5eb5867f5073af8ebbff11d723ff2ee70f10ab not found: ID does not exist"
Sep 10 19:39:58.792165 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:58.792143    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-8dd4a3-openshift-default-856bd4c5f8-24xmz"]
Sep 10 19:39:58.798104 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:58.798078    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-8dd4a3-openshift-default-856bd4c5f8-24xmz"]
Sep 10 19:39:59.385948 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:39:59.385916    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="60e22eaa-6ec6-4298-bfcd-3f42bcce4117" path="/var/lib/kubelet/pods/60e22eaa-6ec6-4298-bfcd-3f42bcce4117/volumes"
Sep 10 19:40:00.188855 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:00.188826    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:40:03.315805 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:03.315768    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:40:04.129657 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:04.129608    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:40:04.229472 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:04.229438    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:40:04.328906 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:04.328874    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:40:04.413042 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:04.412972    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-4bc8b3-kfzjc"]
Sep 10 19:40:04.415691 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:04.415664    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:40:04.416473 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:04.416449    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-4bc8b3-kfzjc"]
Sep 10 19:40:04.483475 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:04.483449    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:40:04.760107 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:04.760078    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:40:05.384941 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:05.384911    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="42573895-5f18-43e5-a07a-a41bee8be062" path="/var/lib/kubelet/pods/42573895-5f18-43e5-a07a-a41bee8be062/volumes"
Sep 10 19:40:07.290440 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:07.290409    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-cab4ee/rate-test-model-cab4ee-kserve-6956f6d775-x2wqm"]
Sep 10 19:40:07.290786 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:07.290778    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="60e22eaa-6ec6-4298-bfcd-3f42bcce4117" containerName="istio-proxy"
Sep 10 19:40:07.290828 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:07.290790    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="60e22eaa-6ec6-4298-bfcd-3f42bcce4117" containerName="istio-proxy"
Sep 10 19:40:07.290828 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:07.290803    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="42573895-5f18-43e5-a07a-a41bee8be062" containerName="revoke-keys"
Sep 10 19:40:07.290828 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:07.290808    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="42573895-5f18-43e5-a07a-a41bee8be062" containerName="revoke-keys"
Sep 10 19:40:07.290922 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:07.290865    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="60e22eaa-6ec6-4298-bfcd-3f42bcce4117" containerName="istio-proxy"
Sep 10 19:40:07.290922 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:07.290876    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="60e22eaa-6ec6-4298-bfcd-3f42bcce4117" containerName="istio-proxy"
Sep 10 19:40:07.290922 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:07.290882    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="42573895-5f18-43e5-a07a-a41bee8be062" containerName="revoke-keys"
Sep 10 19:40:07.293765 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:07.293749    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-cab4ee/rate-test-model-cab4ee-kserve-6956f6d775-x2wqm"
Sep 10 19:40:07.295835 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:07.295804    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-cab4ee\"/\"openshift-service-ca.crt\""
Sep 10 19:40:07.296372 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:07.296325    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-cab4ee\"/\"kube-root-ca.crt\""
Sep 10 19:40:07.296487 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:07.296326    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-cab4ee\"/\"default-dockercfg-z99gm\""
Sep 10 19:40:07.296487 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:07.296364    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-cab4ee\"/\"rate-test-model-cab4ee-kserve-self-signed-certs\""
Sep 10 19:40:07.304841 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:07.304817    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-cab4ee/rate-test-model-cab4ee-kserve-6956f6d775-x2wqm"]
Sep 10 19:40:07.315875 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:07.315851    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:40:07.368273 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:07.368247    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/50950b90-c83f-4b1f-a7a0-37eee0da5237-dshm\") pod \"rate-test-model-cab4ee-kserve-6956f6d775-x2wqm\" (UID: \"50950b90-c83f-4b1f-a7a0-37eee0da5237\") " pod="ai-tenant-e2e-shared-b-w3-cab4ee/rate-test-model-cab4ee-kserve-6956f6d775-x2wqm"
Sep 10 19:40:07.368418 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:07.368285    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/50950b90-c83f-4b1f-a7a0-37eee0da5237-tmp-dir\") pod \"rate-test-model-cab4ee-kserve-6956f6d775-x2wqm\" (UID: \"50950b90-c83f-4b1f-a7a0-37eee0da5237\") " pod="ai-tenant-e2e-shared-b-w3-cab4ee/rate-test-model-cab4ee-kserve-6956f6d775-x2wqm"
Sep 10 19:40:07.368418 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:07.368351    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dw7qk\" (UniqueName: \"kubernetes.io/projected/50950b90-c83f-4b1f-a7a0-37eee0da5237-kube-api-access-dw7qk\") pod \"rate-test-model-cab4ee-kserve-6956f6d775-x2wqm\" (UID: \"50950b90-c83f-4b1f-a7a0-37eee0da5237\") " pod="ai-tenant-e2e-shared-b-w3-cab4ee/rate-test-model-cab4ee-kserve-6956f6d775-x2wqm"
Sep 10 19:40:07.368496 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:07.368422    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/50950b90-c83f-4b1f-a7a0-37eee0da5237-home\") pod \"rate-test-model-cab4ee-kserve-6956f6d775-x2wqm\" (UID: \"50950b90-c83f-4b1f-a7a0-37eee0da5237\") " pod="ai-tenant-e2e-shared-b-w3-cab4ee/rate-test-model-cab4ee-kserve-6956f6d775-x2wqm"
Sep 10 19:40:07.368496 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:07.368446    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/50950b90-c83f-4b1f-a7a0-37eee0da5237-tls-certs\") pod \"rate-test-model-cab4ee-kserve-6956f6d775-x2wqm\" (UID: \"50950b90-c83f-4b1f-a7a0-37eee0da5237\") " pod="ai-tenant-e2e-shared-b-w3-cab4ee/rate-test-model-cab4ee-kserve-6956f6d775-x2wqm"
Sep 10 19:40:07.368496 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:07.368481    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/50950b90-c83f-4b1f-a7a0-37eee0da5237-model-cache\") pod \"rate-test-model-cab4ee-kserve-6956f6d775-x2wqm\" (UID: \"50950b90-c83f-4b1f-a7a0-37eee0da5237\") " pod="ai-tenant-e2e-shared-b-w3-cab4ee/rate-test-model-cab4ee-kserve-6956f6d775-x2wqm"
Sep 10 19:40:07.468828 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:07.468803    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dw7qk\" (UniqueName: \"kubernetes.io/projected/50950b90-c83f-4b1f-a7a0-37eee0da5237-kube-api-access-dw7qk\") pod \"rate-test-model-cab4ee-kserve-6956f6d775-x2wqm\" (UID: \"50950b90-c83f-4b1f-a7a0-37eee0da5237\") " pod="ai-tenant-e2e-shared-b-w3-cab4ee/rate-test-model-cab4ee-kserve-6956f6d775-x2wqm"
Sep 10 19:40:07.468970 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:07.468853    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/50950b90-c83f-4b1f-a7a0-37eee0da5237-home\") pod \"rate-test-model-cab4ee-kserve-6956f6d775-x2wqm\" (UID: \"50950b90-c83f-4b1f-a7a0-37eee0da5237\") " pod="ai-tenant-e2e-shared-b-w3-cab4ee/rate-test-model-cab4ee-kserve-6956f6d775-x2wqm"
Sep 10 19:40:07.469012 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:07.468964    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/50950b90-c83f-4b1f-a7a0-37eee0da5237-tls-certs\") pod \"rate-test-model-cab4ee-kserve-6956f6d775-x2wqm\" (UID: \"50950b90-c83f-4b1f-a7a0-37eee0da5237\") " pod="ai-tenant-e2e-shared-b-w3-cab4ee/rate-test-model-cab4ee-kserve-6956f6d775-x2wqm"
Sep 10 19:40:07.469051 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:07.469019    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/50950b90-c83f-4b1f-a7a0-37eee0da5237-model-cache\") pod \"rate-test-model-cab4ee-kserve-6956f6d775-x2wqm\" (UID: \"50950b90-c83f-4b1f-a7a0-37eee0da5237\") " pod="ai-tenant-e2e-shared-b-w3-cab4ee/rate-test-model-cab4ee-kserve-6956f6d775-x2wqm"
Sep 10 19:40:07.469101 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:07.469061    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/50950b90-c83f-4b1f-a7a0-37eee0da5237-dshm\") pod \"rate-test-model-cab4ee-kserve-6956f6d775-x2wqm\" (UID: \"50950b90-c83f-4b1f-a7a0-37eee0da5237\") " pod="ai-tenant-e2e-shared-b-w3-cab4ee/rate-test-model-cab4ee-kserve-6956f6d775-x2wqm"
Sep 10 19:40:07.469143 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:07.469100    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/50950b90-c83f-4b1f-a7a0-37eee0da5237-tmp-dir\") pod \"rate-test-model-cab4ee-kserve-6956f6d775-x2wqm\" (UID: \"50950b90-c83f-4b1f-a7a0-37eee0da5237\") " pod="ai-tenant-e2e-shared-b-w3-cab4ee/rate-test-model-cab4ee-kserve-6956f6d775-x2wqm"
Sep 10 19:40:07.469186 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:07.469165    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/50950b90-c83f-4b1f-a7a0-37eee0da5237-home\") pod \"rate-test-model-cab4ee-kserve-6956f6d775-x2wqm\" (UID: \"50950b90-c83f-4b1f-a7a0-37eee0da5237\") " pod="ai-tenant-e2e-shared-b-w3-cab4ee/rate-test-model-cab4ee-kserve-6956f6d775-x2wqm"
Sep 10 19:40:07.469386 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:07.469369    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/50950b90-c83f-4b1f-a7a0-37eee0da5237-model-cache\") pod \"rate-test-model-cab4ee-kserve-6956f6d775-x2wqm\" (UID: \"50950b90-c83f-4b1f-a7a0-37eee0da5237\") " pod="ai-tenant-e2e-shared-b-w3-cab4ee/rate-test-model-cab4ee-kserve-6956f6d775-x2wqm"
Sep 10 19:40:07.469469 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:07.469453    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/50950b90-c83f-4b1f-a7a0-37eee0da5237-tmp-dir\") pod \"rate-test-model-cab4ee-kserve-6956f6d775-x2wqm\" (UID: \"50950b90-c83f-4b1f-a7a0-37eee0da5237\") " pod="ai-tenant-e2e-shared-b-w3-cab4ee/rate-test-model-cab4ee-kserve-6956f6d775-x2wqm"
Sep 10 19:40:07.471211 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:07.471190    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/50950b90-c83f-4b1f-a7a0-37eee0da5237-dshm\") pod \"rate-test-model-cab4ee-kserve-6956f6d775-x2wqm\" (UID: \"50950b90-c83f-4b1f-a7a0-37eee0da5237\") " pod="ai-tenant-e2e-shared-b-w3-cab4ee/rate-test-model-cab4ee-kserve-6956f6d775-x2wqm"
Sep 10 19:40:07.471558 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:07.471540    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/50950b90-c83f-4b1f-a7a0-37eee0da5237-tls-certs\") pod \"rate-test-model-cab4ee-kserve-6956f6d775-x2wqm\" (UID: \"50950b90-c83f-4b1f-a7a0-37eee0da5237\") " pod="ai-tenant-e2e-shared-b-w3-cab4ee/rate-test-model-cab4ee-kserve-6956f6d775-x2wqm"
Sep 10 19:40:07.476740 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:07.476721    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dw7qk\" (UniqueName: \"kubernetes.io/projected/50950b90-c83f-4b1f-a7a0-37eee0da5237-kube-api-access-dw7qk\") pod \"rate-test-model-cab4ee-kserve-6956f6d775-x2wqm\" (UID: \"50950b90-c83f-4b1f-a7a0-37eee0da5237\") " pod="ai-tenant-e2e-shared-b-w3-cab4ee/rate-test-model-cab4ee-kserve-6956f6d775-x2wqm"
Sep 10 19:40:07.602653 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:07.602589    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-cab4ee/rate-test-model-cab4ee-kserve-6956f6d775-x2wqm"
Sep 10 19:40:07.873188 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:07.873106    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-cab4ee/rate-test-model-cab4ee-kserve-6956f6d775-x2wqm"]
Sep 10 19:40:07.876353 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:40:07.876306    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod50950b90_c83f_4b1f_a7a0_37eee0da5237.slice/crio-8268f36ef2c3c20c6151d18c8fb7763e4be080851f16fb13517ecf9e7477edbd WatchSource:0}: Error finding container 8268f36ef2c3c20c6151d18c8fb7763e4be080851f16fb13517ecf9e7477edbd: Status 404 returned error can't find the container with id 8268f36ef2c3c20c6151d18c8fb7763e4be080851f16fb13517ecf9e7477edbd
Sep 10 19:40:07.955910 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:07.955888    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qr7rl"]
Sep 10 19:40:08.823996 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:08.823962    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-cab4ee/rate-test-model-cab4ee-kserve-6956f6d775-x2wqm" event={"ID":"50950b90-c83f-4b1f-a7a0-37eee0da5237","Type":"ContainerStarted","Data":"0047e020760019ef69205236c6070874851dacaf01d2ed6fb38fd3cb5af99b1f"}
Sep 10 19:40:08.823996 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:08.824005    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-cab4ee/rate-test-model-cab4ee-kserve-6956f6d775-x2wqm" event={"ID":"50950b90-c83f-4b1f-a7a0-37eee0da5237","Type":"ContainerStarted","Data":"8268f36ef2c3c20c6151d18c8fb7763e4be080851f16fb13517ecf9e7477edbd"}
Sep 10 19:40:08.841805 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:08.841667    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-cab4ee/rate-test-model-cab4ee-kserve-6956f6d775-x2wqm" podStartSLOduration=1.841635867 podStartE2EDuration="1.841635867s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:40:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:40:08.840696312 +0000 UTC m=+1479.900523990" watchObservedRunningTime="2026-09-10 19:40:08.841635867 +0000 UTC m=+1479.901463536"
Sep 10 19:40:10.661218 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:10.661184    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:40:10.762964 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:10.762925    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:40:10.949146 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:10.949108    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:40:17.603053 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:17.603018    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-cab4ee/rate-test-model-cab4ee-kserve-6956f6d775-x2wqm"
Sep 10 19:40:17.603479 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:17.603105    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-cab4ee/rate-test-model-cab4ee-kserve-6956f6d775-x2wqm"
Sep 10 19:40:17.615757 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:17.615733    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-cab4ee/rate-test-model-cab4ee-kserve-6956f6d775-x2wqm"
Sep 10 19:40:17.881095 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:17.881014    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-cab4ee/rate-test-model-cab4ee-kserve-6956f6d775-x2wqm"
Sep 10 19:40:19.654362 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:19.654322    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qr7rl"]
Sep 10 19:40:26.276820 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:26.276786    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:40:29.439683 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:29.439657    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rrxfj_c7511dfe-c588-4be1-bd16-1dce34740809/ovn-acl-logging/0.log"
Sep 10 19:40:29.442418 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:29.442398    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rrxfj_c7511dfe-c588-4be1-bd16-1dce34740809/ovn-acl-logging/0.log"
Sep 10 19:40:33.054860 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:33.054827    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qr7rl"]
Sep 10 19:40:39.719677 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:39.719605    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:40:39.782316 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:39.782284    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:40:42.750285 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:42.750252    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qr7rl"]
Sep 10 19:40:49.107659 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:49.107627    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:40:49.414243 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:49.414147    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:40:50.300276 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:50.300246    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-cab4ee/rate-test-model-cab4ee-kserve-6956f6d775-x2wqm"]
Sep 10 19:40:50.326173 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:50.326142    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:40:50.770974 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:50.770939    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:40:51.079177 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:51.079103    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-df863b-openshift-default-84d4c7f644-8wtmd"]
Sep 10 19:40:51.079510 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:51.079496    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="60e22eaa-6ec6-4298-bfcd-3f42bcce4117" containerName="istio-proxy"
Sep 10 19:40:51.079510 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:51.079511    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="60e22eaa-6ec6-4298-bfcd-3f42bcce4117" containerName="istio-proxy"
Sep 10 19:40:51.084294 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:51.084274    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-df863b-openshift-default-84d4c7f644-8wtmd"
Sep 10 19:40:51.086405 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:51.086386    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-df863b-openshift-default-dockercfg-dzts6\""
Sep 10 19:40:51.100797 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:51.100774    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-df863b-openshift-default-84d4c7f644-8wtmd"]
Sep 10 19:40:51.115038 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:51.115011    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:40:51.218936 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:51.218908    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/82bd2e04-378d-4f6f-92b5-205d1890d4cb-workload-socket\") pod \"e2e-ipp-a-df863b-openshift-default-84d4c7f644-8wtmd\" (UID: \"82bd2e04-378d-4f6f-92b5-205d1890d4cb\") " pod="openshift-ingress/e2e-ipp-a-df863b-openshift-default-84d4c7f644-8wtmd"
Sep 10 19:40:51.218936 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:51.218942    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cknpb\" (UniqueName: \"kubernetes.io/projected/82bd2e04-378d-4f6f-92b5-205d1890d4cb-kube-api-access-cknpb\") pod \"e2e-ipp-a-df863b-openshift-default-84d4c7f644-8wtmd\" (UID: \"82bd2e04-378d-4f6f-92b5-205d1890d4cb\") " pod="openshift-ingress/e2e-ipp-a-df863b-openshift-default-84d4c7f644-8wtmd"
Sep 10 19:40:51.219112 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:51.219013    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/82bd2e04-378d-4f6f-92b5-205d1890d4cb-istio-token\") pod \"e2e-ipp-a-df863b-openshift-default-84d4c7f644-8wtmd\" (UID: \"82bd2e04-378d-4f6f-92b5-205d1890d4cb\") " pod="openshift-ingress/e2e-ipp-a-df863b-openshift-default-84d4c7f644-8wtmd"
Sep 10 19:40:51.219112 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:51.219067    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/82bd2e04-378d-4f6f-92b5-205d1890d4cb-credential-socket\") pod \"e2e-ipp-a-df863b-openshift-default-84d4c7f644-8wtmd\" (UID: \"82bd2e04-378d-4f6f-92b5-205d1890d4cb\") " pod="openshift-ingress/e2e-ipp-a-df863b-openshift-default-84d4c7f644-8wtmd"
Sep 10 19:40:51.219112 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:51.219094    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/82bd2e04-378d-4f6f-92b5-205d1890d4cb-istio-envoy\") pod \"e2e-ipp-a-df863b-openshift-default-84d4c7f644-8wtmd\" (UID: \"82bd2e04-378d-4f6f-92b5-205d1890d4cb\") " pod="openshift-ingress/e2e-ipp-a-df863b-openshift-default-84d4c7f644-8wtmd"
Sep 10 19:40:51.219112 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:51.219112    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/82bd2e04-378d-4f6f-92b5-205d1890d4cb-istiod-ca-cert\") pod \"e2e-ipp-a-df863b-openshift-default-84d4c7f644-8wtmd\" (UID: \"82bd2e04-378d-4f6f-92b5-205d1890d4cb\") " pod="openshift-ingress/e2e-ipp-a-df863b-openshift-default-84d4c7f644-8wtmd"
Sep 10 19:40:51.219239 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:51.219147    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/82bd2e04-378d-4f6f-92b5-205d1890d4cb-workload-certs\") pod \"e2e-ipp-a-df863b-openshift-default-84d4c7f644-8wtmd\" (UID: \"82bd2e04-378d-4f6f-92b5-205d1890d4cb\") " pod="openshift-ingress/e2e-ipp-a-df863b-openshift-default-84d4c7f644-8wtmd"
Sep 10 19:40:51.219239 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:51.219164    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/82bd2e04-378d-4f6f-92b5-205d1890d4cb-istio-podinfo\") pod \"e2e-ipp-a-df863b-openshift-default-84d4c7f644-8wtmd\" (UID: \"82bd2e04-378d-4f6f-92b5-205d1890d4cb\") " pod="openshift-ingress/e2e-ipp-a-df863b-openshift-default-84d4c7f644-8wtmd"
Sep 10 19:40:51.219239 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:51.219189    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/82bd2e04-378d-4f6f-92b5-205d1890d4cb-istio-data\") pod \"e2e-ipp-a-df863b-openshift-default-84d4c7f644-8wtmd\" (UID: \"82bd2e04-378d-4f6f-92b5-205d1890d4cb\") " pod="openshift-ingress/e2e-ipp-a-df863b-openshift-default-84d4c7f644-8wtmd"
Sep 10 19:40:51.319798 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:51.319764    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/82bd2e04-378d-4f6f-92b5-205d1890d4cb-workload-socket\") pod \"e2e-ipp-a-df863b-openshift-default-84d4c7f644-8wtmd\" (UID: \"82bd2e04-378d-4f6f-92b5-205d1890d4cb\") " pod="openshift-ingress/e2e-ipp-a-df863b-openshift-default-84d4c7f644-8wtmd"
Sep 10 19:40:51.319798 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:51.319794    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cknpb\" (UniqueName: \"kubernetes.io/projected/82bd2e04-378d-4f6f-92b5-205d1890d4cb-kube-api-access-cknpb\") pod \"e2e-ipp-a-df863b-openshift-default-84d4c7f644-8wtmd\" (UID: \"82bd2e04-378d-4f6f-92b5-205d1890d4cb\") " pod="openshift-ingress/e2e-ipp-a-df863b-openshift-default-84d4c7f644-8wtmd"
Sep 10 19:40:51.320271 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:51.319823    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/82bd2e04-378d-4f6f-92b5-205d1890d4cb-istio-token\") pod \"e2e-ipp-a-df863b-openshift-default-84d4c7f644-8wtmd\" (UID: \"82bd2e04-378d-4f6f-92b5-205d1890d4cb\") " pod="openshift-ingress/e2e-ipp-a-df863b-openshift-default-84d4c7f644-8wtmd"
Sep 10 19:40:51.320271 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:51.319852    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/82bd2e04-378d-4f6f-92b5-205d1890d4cb-credential-socket\") pod \"e2e-ipp-a-df863b-openshift-default-84d4c7f644-8wtmd\" (UID: \"82bd2e04-378d-4f6f-92b5-205d1890d4cb\") " pod="openshift-ingress/e2e-ipp-a-df863b-openshift-default-84d4c7f644-8wtmd"
Sep 10 19:40:51.320271 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:51.319969    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/82bd2e04-378d-4f6f-92b5-205d1890d4cb-istio-envoy\") pod \"e2e-ipp-a-df863b-openshift-default-84d4c7f644-8wtmd\" (UID: \"82bd2e04-378d-4f6f-92b5-205d1890d4cb\") " pod="openshift-ingress/e2e-ipp-a-df863b-openshift-default-84d4c7f644-8wtmd"
Sep 10 19:40:51.320271 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:51.320008    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/82bd2e04-378d-4f6f-92b5-205d1890d4cb-istiod-ca-cert\") pod \"e2e-ipp-a-df863b-openshift-default-84d4c7f644-8wtmd\" (UID: \"82bd2e04-378d-4f6f-92b5-205d1890d4cb\") " pod="openshift-ingress/e2e-ipp-a-df863b-openshift-default-84d4c7f644-8wtmd"
Sep 10 19:40:51.320271 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:51.320057    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/82bd2e04-378d-4f6f-92b5-205d1890d4cb-workload-certs\") pod \"e2e-ipp-a-df863b-openshift-default-84d4c7f644-8wtmd\" (UID: \"82bd2e04-378d-4f6f-92b5-205d1890d4cb\") " pod="openshift-ingress/e2e-ipp-a-df863b-openshift-default-84d4c7f644-8wtmd"
Sep 10 19:40:51.320271 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:51.320095    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/82bd2e04-378d-4f6f-92b5-205d1890d4cb-istio-podinfo\") pod \"e2e-ipp-a-df863b-openshift-default-84d4c7f644-8wtmd\" (UID: \"82bd2e04-378d-4f6f-92b5-205d1890d4cb\") " pod="openshift-ingress/e2e-ipp-a-df863b-openshift-default-84d4c7f644-8wtmd"
Sep 10 19:40:51.320271 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:51.320137    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/82bd2e04-378d-4f6f-92b5-205d1890d4cb-istio-data\") pod \"e2e-ipp-a-df863b-openshift-default-84d4c7f644-8wtmd\" (UID: \"82bd2e04-378d-4f6f-92b5-205d1890d4cb\") " pod="openshift-ingress/e2e-ipp-a-df863b-openshift-default-84d4c7f644-8wtmd"
Sep 10 19:40:51.320271 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:51.320243    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/82bd2e04-378d-4f6f-92b5-205d1890d4cb-workload-socket\") pod \"e2e-ipp-a-df863b-openshift-default-84d4c7f644-8wtmd\" (UID: \"82bd2e04-378d-4f6f-92b5-205d1890d4cb\") " pod="openshift-ingress/e2e-ipp-a-df863b-openshift-default-84d4c7f644-8wtmd"
Sep 10 19:40:51.320787 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:51.320415    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/82bd2e04-378d-4f6f-92b5-205d1890d4cb-workload-certs\") pod \"e2e-ipp-a-df863b-openshift-default-84d4c7f644-8wtmd\" (UID: \"82bd2e04-378d-4f6f-92b5-205d1890d4cb\") " pod="openshift-ingress/e2e-ipp-a-df863b-openshift-default-84d4c7f644-8wtmd"
Sep 10 19:40:51.320787 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:51.320444    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/82bd2e04-378d-4f6f-92b5-205d1890d4cb-credential-socket\") pod \"e2e-ipp-a-df863b-openshift-default-84d4c7f644-8wtmd\" (UID: \"82bd2e04-378d-4f6f-92b5-205d1890d4cb\") " pod="openshift-ingress/e2e-ipp-a-df863b-openshift-default-84d4c7f644-8wtmd"
Sep 10 19:40:51.320787 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:51.320630    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/82bd2e04-378d-4f6f-92b5-205d1890d4cb-istio-data\") pod \"e2e-ipp-a-df863b-openshift-default-84d4c7f644-8wtmd\" (UID: \"82bd2e04-378d-4f6f-92b5-205d1890d4cb\") " pod="openshift-ingress/e2e-ipp-a-df863b-openshift-default-84d4c7f644-8wtmd"
Sep 10 19:40:51.320787 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:51.320778    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/82bd2e04-378d-4f6f-92b5-205d1890d4cb-istiod-ca-cert\") pod \"e2e-ipp-a-df863b-openshift-default-84d4c7f644-8wtmd\" (UID: \"82bd2e04-378d-4f6f-92b5-205d1890d4cb\") " pod="openshift-ingress/e2e-ipp-a-df863b-openshift-default-84d4c7f644-8wtmd"
Sep 10 19:40:51.322184 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:51.322158    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/82bd2e04-378d-4f6f-92b5-205d1890d4cb-istio-envoy\") pod \"e2e-ipp-a-df863b-openshift-default-84d4c7f644-8wtmd\" (UID: \"82bd2e04-378d-4f6f-92b5-205d1890d4cb\") " pod="openshift-ingress/e2e-ipp-a-df863b-openshift-default-84d4c7f644-8wtmd"
Sep 10 19:40:51.322594 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:51.322568    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/82bd2e04-378d-4f6f-92b5-205d1890d4cb-istio-podinfo\") pod \"e2e-ipp-a-df863b-openshift-default-84d4c7f644-8wtmd\" (UID: \"82bd2e04-378d-4f6f-92b5-205d1890d4cb\") " pod="openshift-ingress/e2e-ipp-a-df863b-openshift-default-84d4c7f644-8wtmd"
Sep 10 19:40:51.331022 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:51.330960    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cknpb\" (UniqueName: \"kubernetes.io/projected/82bd2e04-378d-4f6f-92b5-205d1890d4cb-kube-api-access-cknpb\") pod \"e2e-ipp-a-df863b-openshift-default-84d4c7f644-8wtmd\" (UID: \"82bd2e04-378d-4f6f-92b5-205d1890d4cb\") " pod="openshift-ingress/e2e-ipp-a-df863b-openshift-default-84d4c7f644-8wtmd"
Sep 10 19:40:51.331234 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:51.331217    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/82bd2e04-378d-4f6f-92b5-205d1890d4cb-istio-token\") pod \"e2e-ipp-a-df863b-openshift-default-84d4c7f644-8wtmd\" (UID: \"82bd2e04-378d-4f6f-92b5-205d1890d4cb\") " pod="openshift-ingress/e2e-ipp-a-df863b-openshift-default-84d4c7f644-8wtmd"
Sep 10 19:40:51.340133 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:51.340112    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:40:51.394720 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:51.394700    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-df863b-openshift-default-84d4c7f644-8wtmd"
Sep 10 19:40:51.675586 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:51.675557    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-df863b-openshift-default-84d4c7f644-8wtmd"]
Sep 10 19:40:51.677613 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:40:51.677585    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82bd2e04_378d_4f6f_92b5_205d1890d4cb.slice/crio-a76f754534f10187f3fa89e1ef262e32f80f5efb95874be94d6d1747777733d4 WatchSource:0}: Error finding container a76f754534f10187f3fa89e1ef262e32f80f5efb95874be94d6d1747777733d4: Status 404 returned error can't find the container with id a76f754534f10187f3fa89e1ef262e32f80f5efb95874be94d6d1747777733d4
Sep 10 19:40:51.679727 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:51.679693    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 19:40:51.679825 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:51.679767    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 19:40:51.679825 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:51.679811    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 19:40:52.044799 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:52.044763    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-df863b-openshift-default-84d4c7f644-8wtmd" event={"ID":"82bd2e04-378d-4f6f-92b5-205d1890d4cb","Type":"ContainerStarted","Data":"baa5ab60567a0e9b5556f7adc0499b692434d418a00abd7fdfb64a9108937c6e"}
Sep 10 19:40:52.044970 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:52.044826    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-df863b-openshift-default-84d4c7f644-8wtmd" event={"ID":"82bd2e04-378d-4f6f-92b5-205d1890d4cb","Type":"ContainerStarted","Data":"a76f754534f10187f3fa89e1ef262e32f80f5efb95874be94d6d1747777733d4"}
Sep 10 19:40:52.074430 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:52.074371    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-df863b-openshift-default-84d4c7f644-8wtmd" podStartSLOduration=1.074356358 podStartE2EDuration="1.074356358s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:40:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:40:52.074272035 +0000 UTC m=+1523.134099703" watchObservedRunningTime="2026-09-10 19:40:52.074356358 +0000 UTC m=+1523.134184016"
Sep 10 19:40:52.395317 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:52.395245    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-df863b-openshift-default-84d4c7f644-8wtmd"
Sep 10 19:40:52.399984 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:52.399963    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-df863b-openshift-default-84d4c7f644-8wtmd"
Sep 10 19:40:52.904230 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:52.904195    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qr7rl"]
Sep 10 19:40:53.049823 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:53.049794    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-df863b-openshift-default-84d4c7f644-8wtmd"
Sep 10 19:40:53.050716 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:53.050697    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-df863b-openshift-default-84d4c7f644-8wtmd"
Sep 10 19:40:54.031378 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:54.031303    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:40:54.126245 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:54.126204    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:40:55.221282 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:55.221249    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:40:55.321663 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:55.321627    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:40:55.461521 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:55.461481    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:40:55.902012 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:55.901972    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:40:56.001688 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:56.001649    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:40:56.078981 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:56.078949    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-df863b-79948f674b-gfn6r"]
Sep 10 19:40:56.082678 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:56.082661    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-df863b-79948f674b-gfn6r"
Sep 10 19:40:56.084700 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:56.084675    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-df863b\""
Sep 10 19:40:56.084826 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:56.084708    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-df863b-dockercfg-rfxpx\""
Sep 10 19:40:56.093434 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:56.093415    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-df863b-79948f674b-gfn6r"]
Sep 10 19:40:56.163322 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:56.163242    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bd267769-63da-4a57-b2bc-0d1e0d797c19-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-df863b-79948f674b-gfn6r\" (UID: \"bd267769-63da-4a57-b2bc-0d1e0d797c19\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-df863b-79948f674b-gfn6r"
Sep 10 19:40:56.163322 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:56.163279    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dlwxv\" (UniqueName: \"kubernetes.io/projected/bd267769-63da-4a57-b2bc-0d1e0d797c19-kube-api-access-dlwxv\") pod \"payload-pre-processing-e2e-ipp-a-df863b-79948f674b-gfn6r\" (UID: \"bd267769-63da-4a57-b2bc-0d1e0d797c19\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-df863b-79948f674b-gfn6r"
Sep 10 19:40:56.264666 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:56.264632    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bd267769-63da-4a57-b2bc-0d1e0d797c19-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-df863b-79948f674b-gfn6r\" (UID: \"bd267769-63da-4a57-b2bc-0d1e0d797c19\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-df863b-79948f674b-gfn6r"
Sep 10 19:40:56.264666 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:56.264668    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dlwxv\" (UniqueName: \"kubernetes.io/projected/bd267769-63da-4a57-b2bc-0d1e0d797c19-kube-api-access-dlwxv\") pod \"payload-pre-processing-e2e-ipp-a-df863b-79948f674b-gfn6r\" (UID: \"bd267769-63da-4a57-b2bc-0d1e0d797c19\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-df863b-79948f674b-gfn6r"
Sep 10 19:40:56.265299 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:56.265278    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bd267769-63da-4a57-b2bc-0d1e0d797c19-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-df863b-79948f674b-gfn6r\" (UID: \"bd267769-63da-4a57-b2bc-0d1e0d797c19\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-df863b-79948f674b-gfn6r"
Sep 10 19:40:56.272207 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:56.272185    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dlwxv\" (UniqueName: \"kubernetes.io/projected/bd267769-63da-4a57-b2bc-0d1e0d797c19-kube-api-access-dlwxv\") pod \"payload-pre-processing-e2e-ipp-a-df863b-79948f674b-gfn6r\" (UID: \"bd267769-63da-4a57-b2bc-0d1e0d797c19\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-df863b-79948f674b-gfn6r"
Sep 10 19:40:56.393916 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:56.393882    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-df863b-79948f674b-gfn6r"
Sep 10 19:40:56.665756 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:56.665732    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-df863b-79948f674b-gfn6r"]
Sep 10 19:40:56.667306 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:40:56.667276    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd267769_63da_4a57_b2bc_0d1e0d797c19.slice/crio-0adc7137604e770a3a368fb8721bf08393aef9aaf70c653a4276d2b836b0c3a4 WatchSource:0}: Error finding container 0adc7137604e770a3a368fb8721bf08393aef9aaf70c653a4276d2b836b0c3a4: Status 404 returned error can't find the container with id 0adc7137604e770a3a368fb8721bf08393aef9aaf70c653a4276d2b836b0c3a4
Sep 10 19:40:57.075932 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:57.075828    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-df863b-79948f674b-gfn6r" event={"ID":"bd267769-63da-4a57-b2bc-0d1e0d797c19","Type":"ContainerStarted","Data":"62d28232e25c882caa5ae9e2b74229ae065d19d05df15e353ba4ddf4acf064ce"}
Sep 10 19:40:57.076104 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:57.075957    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-df863b-79948f674b-gfn6r" event={"ID":"bd267769-63da-4a57-b2bc-0d1e0d797c19","Type":"ContainerStarted","Data":"0adc7137604e770a3a368fb8721bf08393aef9aaf70c653a4276d2b836b0c3a4"}
Sep 10 19:40:57.076104 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:57.076002    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-df863b-79948f674b-gfn6r"
Sep 10 19:40:57.097074 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:40:57.097028    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-df863b-79948f674b-gfn6r" podStartSLOduration=1.097013282 podStartE2EDuration="1.097013282s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:40:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:40:57.094922425 +0000 UTC m=+1528.154750092" watchObservedRunningTime="2026-09-10 19:40:57.097013282 +0000 UTC m=+1528.156840999"
Sep 10 19:41:05.325149 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:41:05.325066    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-cab4ee/rate-test-model-cab4ee-kserve-6956f6d775-x2wqm" podUID="50950b90-c83f-4b1f-a7a0-37eee0da5237" containerName="main" containerID="cri-o://0047e020760019ef69205236c6070874851dacaf01d2ed6fb38fd3cb5af99b1f" gracePeriod=45
Sep 10 19:41:05.596066 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:41:05.596043    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-cab4ee/rate-test-model-cab4ee-kserve-6956f6d775-x2wqm"
Sep 10 19:41:05.751474 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:41:05.751441    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qr7rl"]
Sep 10 19:41:05.759445 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:41:05.759427    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dw7qk\" (UniqueName: \"kubernetes.io/projected/50950b90-c83f-4b1f-a7a0-37eee0da5237-kube-api-access-dw7qk\") pod \"50950b90-c83f-4b1f-a7a0-37eee0da5237\" (UID: \"50950b90-c83f-4b1f-a7a0-37eee0da5237\") "
Sep 10 19:41:05.759509 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:41:05.759478    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/50950b90-c83f-4b1f-a7a0-37eee0da5237-home\") pod \"50950b90-c83f-4b1f-a7a0-37eee0da5237\" (UID: \"50950b90-c83f-4b1f-a7a0-37eee0da5237\") "
Sep 10 19:41:05.759551 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:41:05.759521    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/50950b90-c83f-4b1f-a7a0-37eee0da5237-model-cache\") pod \"50950b90-c83f-4b1f-a7a0-37eee0da5237\" (UID: \"50950b90-c83f-4b1f-a7a0-37eee0da5237\") "
Sep 10 19:41:05.759599 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:41:05.759589    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/50950b90-c83f-4b1f-a7a0-37eee0da5237-tmp-dir\") pod \"50950b90-c83f-4b1f-a7a0-37eee0da5237\" (UID: \"50950b90-c83f-4b1f-a7a0-37eee0da5237\") "
Sep 10 19:41:05.759635 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:41:05.759616    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/50950b90-c83f-4b1f-a7a0-37eee0da5237-dshm\") pod \"50950b90-c83f-4b1f-a7a0-37eee0da5237\" (UID: \"50950b90-c83f-4b1f-a7a0-37eee0da5237\") "
Sep 10 19:41:05.759635 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:41:05.759631    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/50950b90-c83f-4b1f-a7a0-37eee0da5237-tls-certs\") pod \"50950b90-c83f-4b1f-a7a0-37eee0da5237\" (UID: \"50950b90-c83f-4b1f-a7a0-37eee0da5237\") "
Sep 10 19:41:05.759740 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:41:05.759715    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50950b90-c83f-4b1f-a7a0-37eee0da5237-home" (OuterVolumeSpecName: "home") pod "50950b90-c83f-4b1f-a7a0-37eee0da5237" (UID: "50950b90-c83f-4b1f-a7a0-37eee0da5237"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:41:05.759826 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:41:05.759802    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50950b90-c83f-4b1f-a7a0-37eee0da5237-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "50950b90-c83f-4b1f-a7a0-37eee0da5237" (UID: "50950b90-c83f-4b1f-a7a0-37eee0da5237"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:41:05.759888 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:41:05.759874    2575 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/50950b90-c83f-4b1f-a7a0-37eee0da5237-home\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:41:05.759939 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:41:05.759879    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50950b90-c83f-4b1f-a7a0-37eee0da5237-model-cache" (OuterVolumeSpecName: "model-cache") pod "50950b90-c83f-4b1f-a7a0-37eee0da5237" (UID: "50950b90-c83f-4b1f-a7a0-37eee0da5237"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:41:05.761754 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:41:05.761723    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/50950b90-c83f-4b1f-a7a0-37eee0da5237-kube-api-access-dw7qk" (OuterVolumeSpecName: "kube-api-access-dw7qk") pod "50950b90-c83f-4b1f-a7a0-37eee0da5237" (UID: "50950b90-c83f-4b1f-a7a0-37eee0da5237"). InnerVolumeSpecName "kube-api-access-dw7qk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:41:05.761917 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:41:05.761896    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50950b90-c83f-4b1f-a7a0-37eee0da5237-dshm" (OuterVolumeSpecName: "dshm") pod "50950b90-c83f-4b1f-a7a0-37eee0da5237" (UID: "50950b90-c83f-4b1f-a7a0-37eee0da5237"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:41:05.761917 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:41:05.761908    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/50950b90-c83f-4b1f-a7a0-37eee0da5237-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "50950b90-c83f-4b1f-a7a0-37eee0da5237" (UID: "50950b90-c83f-4b1f-a7a0-37eee0da5237"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:41:05.860767 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:41:05.860635    2575 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/50950b90-c83f-4b1f-a7a0-37eee0da5237-model-cache\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:41:05.860767 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:41:05.860659    2575 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/50950b90-c83f-4b1f-a7a0-37eee0da5237-tmp-dir\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:41:05.860767 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:41:05.860667    2575 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/50950b90-c83f-4b1f-a7a0-37eee0da5237-dshm\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:41:05.860767 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:41:05.860675    2575 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/50950b90-c83f-4b1f-a7a0-37eee0da5237-tls-certs\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:41:05.860767 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:41:05.860683    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dw7qk\" (UniqueName: \"kubernetes.io/projected/50950b90-c83f-4b1f-a7a0-37eee0da5237-kube-api-access-dw7qk\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:41:06.125255 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:41:06.125158    2575 generic.go:356] "Generic (PLEG): container finished" podID="50950b90-c83f-4b1f-a7a0-37eee0da5237" containerID="0047e020760019ef69205236c6070874851dacaf01d2ed6fb38fd3cb5af99b1f" exitCode=0
Sep 10 19:41:06.125655 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:41:06.125487    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-cab4ee/rate-test-model-cab4ee-kserve-6956f6d775-x2wqm" event={"ID":"50950b90-c83f-4b1f-a7a0-37eee0da5237","Type":"ContainerDied","Data":"0047e020760019ef69205236c6070874851dacaf01d2ed6fb38fd3cb5af99b1f"}
Sep 10 19:41:06.125868 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:41:06.125805    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-cab4ee/rate-test-model-cab4ee-kserve-6956f6d775-x2wqm" event={"ID":"50950b90-c83f-4b1f-a7a0-37eee0da5237","Type":"ContainerDied","Data":"8268f36ef2c3c20c6151d18c8fb7763e4be080851f16fb13517ecf9e7477edbd"}
Sep 10 19:41:06.126006 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:41:06.125561    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-cab4ee/rate-test-model-cab4ee-kserve-6956f6d775-x2wqm"
Sep 10 19:41:06.126321 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:41:06.125848    2575 scope.go:117] "RemoveContainer" containerID="0047e020760019ef69205236c6070874851dacaf01d2ed6fb38fd3cb5af99b1f"
Sep 10 19:41:06.149643 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:41:06.149626    2575 scope.go:117] "RemoveContainer" containerID="0047e020760019ef69205236c6070874851dacaf01d2ed6fb38fd3cb5af99b1f"
Sep 10 19:41:06.149964 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:41:06.149946    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0047e020760019ef69205236c6070874851dacaf01d2ed6fb38fd3cb5af99b1f\": container with ID starting with 0047e020760019ef69205236c6070874851dacaf01d2ed6fb38fd3cb5af99b1f not found: ID does not exist" containerID="0047e020760019ef69205236c6070874851dacaf01d2ed6fb38fd3cb5af99b1f"
Sep 10 19:41:06.150037 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:41:06.149971    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0047e020760019ef69205236c6070874851dacaf01d2ed6fb38fd3cb5af99b1f"} err="failed to get container status \"0047e020760019ef69205236c6070874851dacaf01d2ed6fb38fd3cb5af99b1f\": rpc error: code = NotFound desc = could not find container \"0047e020760019ef69205236c6070874851dacaf01d2ed6fb38fd3cb5af99b1f\": container with ID starting with 0047e020760019ef69205236c6070874851dacaf01d2ed6fb38fd3cb5af99b1f not found: ID does not exist"
Sep 10 19:41:06.156489 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:41:06.156469    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-cab4ee/rate-test-model-cab4ee-kserve-6956f6d775-x2wqm"]
Sep 10 19:41:06.160864 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:41:06.160843    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-cab4ee/rate-test-model-cab4ee-kserve-6956f6d775-x2wqm"]
Sep 10 19:41:07.385395 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:41:07.385363    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="50950b90-c83f-4b1f-a7a0-37eee0da5237" path="/var/lib/kubelet/pods/50950b90-c83f-4b1f-a7a0-37eee0da5237/volumes"
Sep 10 19:41:08.084416 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:41:08.084386    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-df863b-79948f674b-gfn6r"
Sep 10 19:41:09.091473 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:41:09.091433    2575 patch_prober.go:28] interesting pod/e2e-ipp-a-df863b-openshift-default-84d4c7f644-8wtmd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 10 19:41:09.091849 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:41:09.091496    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-a-df863b-openshift-default-84d4c7f644-8wtmd" podUID="82bd2e04-378d-4f6f-92b5-205d1890d4cb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 10 19:41:10.161921 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:41:10.161832    2575 generic.go:356] "Generic (PLEG): container finished" podID="82bd2e04-378d-4f6f-92b5-205d1890d4cb" containerID="baa5ab60567a0e9b5556f7adc0499b692434d418a00abd7fdfb64a9108937c6e" exitCode=0
Sep 10 19:41:10.161921 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:41:10.161905    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-df863b-openshift-default-84d4c7f644-8wtmd" event={"ID":"82bd2e04-378d-4f6f-92b5-205d1890d4cb","Type":"ContainerDied","Data":"baa5ab60567a0e9b5556f7adc0499b692434d418a00abd7fdfb64a9108937c6e"}
Sep 10 19:41:10.162311 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:41:10.162278    2575 scope.go:117] "RemoveContainer" containerID="baa5ab60567a0e9b5556f7adc0499b692434d418a00abd7fdfb64a9108937c6e"
Sep 10 19:41:10.163315 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:41:10.163287    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 19:41:10.163377 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:41:10.163368    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 19:41:10.163422 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:41:10.163400    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 19:41:11.169092 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:41:11.169050    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-df863b-openshift-default-84d4c7f644-8wtmd" event={"ID":"82bd2e04-378d-4f6f-92b5-205d1890d4cb","Type":"ContainerStarted","Data":"f7354979cffd8bb07284327dabe028563962e1be7798909eccc51a4e6baf9442"}
Sep 10 19:41:11.395734 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:41:11.395697    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-df863b-openshift-default-84d4c7f644-8wtmd"
Sep 10 19:41:11.395941 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:41:11.395887    2575 patch_prober.go:28] interesting pod/e2e-ipp-a-df863b-openshift-default-84d4c7f644-8wtmd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body=
Sep 10 19:41:11.395941 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:41:11.395929    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-df863b-openshift-default-84d4c7f644-8wtmd" podUID="82bd2e04-378d-4f6f-92b5-205d1890d4cb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused"
Sep 10 19:41:12.395898 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:41:12.395862    2575 patch_prober.go:28] interesting pod/e2e-ipp-a-df863b-openshift-default-84d4c7f644-8wtmd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body=
Sep 10 19:41:12.396279 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:41:12.395915    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-df863b-openshift-default-84d4c7f644-8wtmd" podUID="82bd2e04-378d-4f6f-92b5-205d1890d4cb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused"
Sep 10 19:41:13.395599 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:41:13.395566    2575 patch_prober.go:28] interesting pod/e2e-ipp-a-df863b-openshift-default-84d4c7f644-8wtmd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body=
Sep 10 19:41:13.395823 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:41:13.395624    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-df863b-openshift-default-84d4c7f644-8wtmd" podUID="82bd2e04-378d-4f6f-92b5-205d1890d4cb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused"
Sep 10 19:41:14.395384 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:41:14.395325    2575 patch_prober.go:28] interesting pod/e2e-ipp-a-df863b-openshift-default-84d4c7f644-8wtmd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body=
Sep 10 19:41:14.395747 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:41:14.395406    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-df863b-openshift-default-84d4c7f644-8wtmd" podUID="82bd2e04-378d-4f6f-92b5-205d1890d4cb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused"
Sep 10 19:41:15.395388 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:41:15.395331    2575 patch_prober.go:28] interesting pod/e2e-ipp-a-df863b-openshift-default-84d4c7f644-8wtmd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body=
Sep 10 19:41:15.395839 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:41:15.395413    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-df863b-openshift-default-84d4c7f644-8wtmd" podUID="82bd2e04-378d-4f6f-92b5-205d1890d4cb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused"
Sep 10 19:41:16.398318 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:41:16.398289    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-df863b-openshift-default-84d4c7f644-8wtmd"
Sep 10 19:41:16.398803 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:41:16.398643    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-df863b-openshift-default-84d4c7f644-8wtmd"
Sep 10 19:41:16.399173 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:41:16.399155    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-df863b-openshift-default-84d4c7f644-8wtmd"
Sep 10 19:41:18.996716 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:41:18.996682    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:41:19.658254 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:41:19.658217    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:41:19.925605 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:41:19.925506    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-3765cf-openshift-default-5994cd68fb-42qbw"]
Sep 10 19:41:19.926135 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:41:19.926118    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="50950b90-c83f-4b1f-a7a0-37eee0da5237" containerName="main"
Sep 10 19:41:19.926135 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:41:19.926137    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="50950b90-c83f-4b1f-a7a0-37eee0da5237" containerName="main"
Sep 10 19:41:19.926323 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:41:19.926258    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="50950b90-c83f-4b1f-a7a0-37eee0da5237" containerName="main"
Sep 10 19:41:19.930259 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:41:19.930239    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-3765cf-openshift-default-5994cd68fb-42qbw"
Sep 10 19:41:19.932448 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:41:19.932425    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-3765cf-openshift-default-dockercfg-77msj\""
Sep 10 19:41:19.939966 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:41:19.939945    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-3765cf-openshift-default-5994cd68fb-42qbw"]
Sep 10 19:41:19.975748 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:41:19.975709    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:41:19.984824 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:41:19.984803    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9597ee5-e04a-417f-8b4e-274e52bdf9e7-workload-socket\") pod \"e2e-ipp-b-3765cf-openshift-default-5994cd68fb-42qbw\" (UID: \"b9597ee5-e04a-417f-8b4e-274e52bdf9e7\") " pod="openshift-ingress/e2e-ipp-b-3765cf-openshift-default-5994cd68fb-42qbw"
Sep 10 19:41:19.984905 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:41:19.984841    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b9597ee5-e04a-417f-8b4e-274e52bdf9e7-istio-token\") pod \"e2e-ipp-b-3765cf-openshift-default-5994cd68fb-42qbw\" (UID: \"b9597ee5-e04a-417f-8b4e-274e52bdf9e7\") " pod="openshift-ingress/e2e-ipp-b-3765cf-openshift-default-5994cd68fb-42qbw"
Sep 10 19:41:19.984905 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:41:19.984894    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b9597ee5-e04a-417f-8b4e-274e52bdf9e7-workload-certs\") pod \"e2e-ipp-b-3765cf-openshift-default-5994cd68fb-42qbw\" (UID: \"b9597ee5-e04a-417f-8b4e-274e52bdf9e7\") " pod="openshift-ingress/e2e-ipp-b-3765cf-openshift-default-5994cd68fb-42qbw"
Sep 10 19:41:19.984977 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:41:19.984915    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b9597ee5-e04a-417f-8b4e-274e52bdf9e7-istiod-ca-cert\") pod \"e2e-ipp-b-3765cf-openshift-default-5994cd68fb-42qbw\" (UID: \"b9597ee5-e04a-417f-8b4e-274e52bdf9e7\") " pod="openshift-ingress/e2e-ipp-b-3765cf-openshift-default-5994cd68fb-42qbw"
Sep 10 19:41:19.984977 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:41:19.984939    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9597ee5-e04a-417f-8b4e-274e52bdf9e7-credential-socket\") pod \"e2e-ipp-b-3765cf-openshift-default-5994cd68fb-42qbw\" (UID: \"b9597ee5-e04a-417f-8b4e-274e52bdf9e7\") " pod="openshift-ingress/e2e-ipp-b-3765cf-openshift-default-5994cd68fb-42qbw"
Sep 10 19:41:19.985045 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:41:19.984988    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b9597ee5-e04a-417f-8b4e-274e52bdf9e7-istio-data\") pod \"e2e-ipp-b-3765cf-openshift-default-5994cd68fb-42qbw\" (UID: \"b9597ee5-e04a-417f-8b4e-274e52bdf9e7\") " pod="openshift-ingress/e2e-ipp-b-3765cf-openshift-default-5994cd68fb-42qbw"
Sep 10 19:41:19.985045 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:41:19.985012    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gjnbl\" (UniqueName: \"kubernetes.io/projected/b9597ee5-e04a-417f-8b4e-274e52bdf9e7-kube-api-access-gjnbl\") pod \"e2e-ipp-b-3765cf-openshift-default-5994cd68fb-42qbw\" (UID: \"b9597ee5-e04a-417f-8b4e-274e52bdf9e7\") " pod="openshift-ingress/e2e-ipp-b-3765cf-openshift-default-5994cd68fb-42qbw"
Sep 10 19:41:19.985110 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:41:19.985047    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b9597ee5-e04a-417f-8b4e-274e52bdf9e7-istio-envoy\") pod \"e2e-ipp-b-3765cf-openshift-default-5994cd68fb-42qbw\" (UID: \"b9597ee5-e04a-417f-8b4e-274e52bdf9e7\") " pod="openshift-ingress/e2e-ipp-b-3765cf-openshift-default-5994cd68fb-42qbw"
Sep 10 19:41:19.985110 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:41:19.985069    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b9597ee5-e04a-417f-8b4e-274e52bdf9e7-istio-podinfo\") pod \"e2e-ipp-b-3765cf-openshift-default-5994cd68fb-42qbw\" (UID: \"b9597ee5-e04a-417f-8b4e-274e52bdf9e7\") " pod="openshift-ingress/e2e-ipp-b-3765cf-openshift-default-5994cd68fb-42qbw"
Sep 10 19:41:20.086185 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:41:20.086151    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9597ee5-e04a-417f-8b4e-274e52bdf9e7-workload-socket\") pod \"e2e-ipp-b-3765cf-openshift-default-5994cd68fb-42qbw\" (UID: \"b9597ee5-e04a-417f-8b4e-274e52bdf9e7\") " pod="openshift-ingress/e2e-ipp-b-3765cf-openshift-default-5994cd68fb-42qbw"
Sep 10 19:41:20.086611 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:41:20.086195    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b9597ee5-e04a-417f-8b4e-274e52bdf9e7-istio-token\") pod \"e2e-ipp-b-3765cf-openshift-default-5994cd68fb-42qbw\" (UID: \"b9597ee5-e04a-417f-8b4e-274e52bdf9e7\") " pod="openshift-ingress/e2e-ipp-b-3765cf-openshift-default-5994cd68fb-42qbw"
Sep 10 19:41:20.086611 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:41:20.086230    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b9597ee5-e04a-417f-8b4e-274e52bdf9e7-workload-certs\") pod \"e2e-ipp-b-3765cf-openshift-default-5994cd68fb-42qbw\" (UID: \"b9597ee5-e04a-417f-8b4e-274e52bdf9e7\") " pod="openshift-ingress/e2e-ipp-b-3765cf-openshift-default-5994cd68fb-42qbw"
Sep 10 19:41:20.086611 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:41:20.086252    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b9597ee5-e04a-417f-8b4e-274e52bdf9e7-istiod-ca-cert\") pod \"e2e-ipp-b-3765cf-openshift-default-5994cd68fb-42qbw\" (UID: \"b9597ee5-e04a-417f-8b4e-274e52bdf9e7\") " pod="openshift-ingress/e2e-ipp-b-3765cf-openshift-default-5994cd68fb-42qbw"
Sep 10 19:41:20.086611 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:41:20.086276    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9597ee5-e04a-417f-8b4e-274e52bdf9e7-credential-socket\") pod \"e2e-ipp-b-3765cf-openshift-default-5994cd68fb-42qbw\" (UID: \"b9597ee5-e04a-417f-8b4e-274e52bdf9e7\") " pod="openshift-ingress/e2e-ipp-b-3765cf-openshift-default-5994cd68fb-42qbw"
Sep 10 19:41:20.086611 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:41:20.086301    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b9597ee5-e04a-417f-8b4e-274e52bdf9e7-istio-data\") pod \"e2e-ipp-b-3765cf-openshift-default-5994cd68fb-42qbw\" (UID: \"b9597ee5-e04a-417f-8b4e-274e52bdf9e7\") " pod="openshift-ingress/e2e-ipp-b-3765cf-openshift-default-5994cd68fb-42qbw"
Sep 10 19:41:20.086611 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:41:20.086320    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gjnbl\" (UniqueName: \"kubernetes.io/projected/b9597ee5-e04a-417f-8b4e-274e52bdf9e7-kube-api-access-gjnbl\") pod \"e2e-ipp-b-3765cf-openshift-default-5994cd68fb-42qbw\" (UID: \"b9597ee5-e04a-417f-8b4e-274e52bdf9e7\") " pod="openshift-ingress/e2e-ipp-b-3765cf-openshift-default-5994cd68fb-42qbw"
Sep 10 19:41:20.086611 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:41:20.086375    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b9597ee5-e04a-417f-8b4e-274e52bdf9e7-istio-envoy\") pod \"e2e-ipp-b-3765cf-openshift-default-5994cd68fb-42qbw\" (UID: \"b9597ee5-e04a-417f-8b4e-274e52bdf9e7\") " pod="openshift-ingress/e2e-ipp-b-3765cf-openshift-default-5994cd68fb-42qbw"
Sep 10 19:41:20.086611 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:41:20.086405    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b9597ee5-e04a-417f-8b4e-274e52bdf9e7-istio-podinfo\") pod \"e2e-ipp-b-3765cf-openshift-default-5994cd68fb-42qbw\" (UID: \"b9597ee5-e04a-417f-8b4e-274e52bdf9e7\") " pod="openshift-ingress/e2e-ipp-b-3765cf-openshift-default-5994cd68fb-42qbw"
Sep 10 19:41:20.087001 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:41:20.086630    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9597ee5-e04a-417f-8b4e-274e52bdf9e7-workload-socket\") pod \"e2e-ipp-b-3765cf-openshift-default-5994cd68fb-42qbw\" (UID: \"b9597ee5-e04a-417f-8b4e-274e52bdf9e7\") " pod="openshift-ingress/e2e-ipp-b-3765cf-openshift-default-5994cd68fb-42qbw"
Sep 10 19:41:20.087001 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:41:20.086752    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9597ee5-e04a-417f-8b4e-274e52bdf9e7-credential-socket\") pod \"e2e-ipp-b-3765cf-openshift-default-5994cd68fb-42qbw\" (UID: \"b9597ee5-e04a-417f-8b4e-274e52bdf9e7\") " pod="openshift-ingress/e2e-ipp-b-3765cf-openshift-default-5994cd68fb-42qbw"
Sep 10 19:41:20.087001 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:41:20.086836    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b9597ee5-e04a-417f-8b4e-274e52bdf9e7-istio-data\") pod \"e2e-ipp-b-3765cf-openshift-default-5994cd68fb-42qbw\" (UID: \"b9597ee5-e04a-417f-8b4e-274e52bdf9e7\") " pod="openshift-ingress/e2e-ipp-b-3765cf-openshift-default-5994cd68fb-42qbw"
Sep 10 19:41:20.087001 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:41:20.086888    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b9597ee5-e04a-417f-8b4e-274e52bdf9e7-workload-certs\") pod \"e2e-ipp-b-3765cf-openshift-default-5994cd68fb-42qbw\" (UID: \"b9597ee5-e04a-417f-8b4e-274e52bdf9e7\") " pod="openshift-ingress/e2e-ipp-b-3765cf-openshift-default-5994cd68fb-42qbw"
Sep 10 19:41:20.087001 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:41:20.086955    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b9597ee5-e04a-417f-8b4e-274e52bdf9e7-istiod-ca-cert\") pod \"e2e-ipp-b-3765cf-openshift-default-5994cd68fb-42qbw\" (UID: \"b9597ee5-e04a-417f-8b4e-274e52bdf9e7\") " pod="openshift-ingress/e2e-ipp-b-3765cf-openshift-default-5994cd68fb-42qbw"
Sep 10 19:41:20.088749 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:41:20.088730    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b9597ee5-e04a-417f-8b4e-274e52bdf9e7-istio-envoy\") pod \"e2e-ipp-b-3765cf-openshift-default-5994cd68fb-42qbw\" (UID: \"b9597ee5-e04a-417f-8b4e-274e52bdf9e7\") " pod="openshift-ingress/e2e-ipp-b-3765cf-openshift-default-5994cd68fb-42qbw"
Sep 10 19:41:20.088962 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:41:20.088942    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b9597ee5-e04a-417f-8b4e-274e52bdf9e7-istio-podinfo\") pod \"e2e-ipp-b-3765cf-openshift-default-5994cd68fb-42qbw\" (UID: \"b9597ee5-e04a-417f-8b4e-274e52bdf9e7\") " pod="openshift-ingress/e2e-ipp-b-3765cf-openshift-default-5994cd68fb-42qbw"
Sep 10 19:41:20.094047 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:41:20.094025    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b9597ee5-e04a-417f-8b4e-274e52bdf9e7-istio-token\") pod \"e2e-ipp-b-3765cf-openshift-default-5994cd68fb-42qbw\" (UID: \"b9597ee5-e04a-417f-8b4e-274e52bdf9e7\") " pod="openshift-ingress/e2e-ipp-b-3765cf-openshift-default-5994cd68fb-42qbw"
Sep 10 19:41:20.094502 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:41:20.094482    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gjnbl\" (UniqueName: \"kubernetes.io/projected/b9597ee5-e04a-417f-8b4e-274e52bdf9e7-kube-api-access-gjnbl\") pod \"e2e-ipp-b-3765cf-openshift-default-5994cd68fb-42qbw\" (UID: \"b9597ee5-e04a-417f-8b4e-274e52bdf9e7\") " pod="openshift-ingress/e2e-ipp-b-3765cf-openshift-default-5994cd68fb-42qbw"
Sep 10 19:41:20.194385 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:41:20.194286    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:41:20.243068 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:41:20.243039    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-3765cf-openshift-default-5994cd68fb-42qbw"
Sep 10 19:41:20.520200 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:41:20.520169    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-3765cf-openshift-default-5994cd68fb-42qbw"]
Sep 10 19:41:20.522636 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:41:20.522606    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9597ee5_e04a_417f_8b4e_274e52bdf9e7.slice/crio-179aedcd93e79bd5fa546e4f27ea8bb4758d4b1333c1b0f68c9a39ca53a9f453 WatchSource:0}: Error finding container 179aedcd93e79bd5fa546e4f27ea8bb4758d4b1333c1b0f68c9a39ca53a9f453: Status 404 returned error can't find the container with id 179aedcd93e79bd5fa546e4f27ea8bb4758d4b1333c1b0f68c9a39ca53a9f453
Sep 10 19:41:20.524971 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:41:20.524937    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 19:41:20.525059 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:41:20.525021    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 19:41:20.525059 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:41:20.525049    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 19:41:21.224388 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:41:21.224334    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-3765cf-openshift-default-5994cd68fb-42qbw" event={"ID":"b9597ee5-e04a-417f-8b4e-274e52bdf9e7","Type":"ContainerStarted","Data":"a831a3a99298950772b341033e782633264ec38f344a2ecebdf397edc77c2818"}
Sep 10 19:41:21.224729 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:41:21.224411    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-3765cf-openshift-default-5994cd68fb-42qbw" event={"ID":"b9597ee5-e04a-417f-8b4e-274e52bdf9e7","Type":"ContainerStarted","Data":"179aedcd93e79bd5fa546e4f27ea8bb4758d4b1333c1b0f68c9a39ca53a9f453"}
Sep 10 19:41:21.243315 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:41:21.243288    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-3765cf-openshift-default-5994cd68fb-42qbw"
Sep 10 19:41:21.247939 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:41:21.247894    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-3765cf-openshift-default-5994cd68fb-42qbw" podStartSLOduration=2.247879291 podStartE2EDuration="2.247879291s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:41:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:41:21.246638468 +0000 UTC m=+1552.306466136" watchObservedRunningTime="2026-09-10 19:41:21.247879291 +0000 UTC m=+1552.307706955"
Sep 10 19:41:21.248643 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:41:21.248624    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-3765cf-openshift-default-5994cd68fb-42qbw"
Sep 10 19:41:22.229096 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:41:22.229057    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-3765cf-openshift-default-5994cd68fb-42qbw"
Sep 10 19:41:22.230165 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:41:22.230143    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-3765cf-openshift-default-5994cd68fb-42qbw"
Sep 10 19:41:23.129605 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:41:23.129577    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:41:23.230876 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:41:23.230835    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:41:24.353572 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:41:24.353540    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:41:24.453684 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:41:24.453645    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:41:24.555084 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:41:24.555050    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:41:24.891068 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:41:24.891037    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:41:24.992597 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:41:24.992563    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:41:31.158392 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:41:31.158352    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qr7rl"]
Sep 10 19:41:41.325802 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:41:41.325766    2575 generic.go:356] "Generic (PLEG): container finished" podID="b9597ee5-e04a-417f-8b4e-274e52bdf9e7" containerID="a831a3a99298950772b341033e782633264ec38f344a2ecebdf397edc77c2818" exitCode=0
Sep 10 19:41:41.326210 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:41:41.325843    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-3765cf-openshift-default-5994cd68fb-42qbw" event={"ID":"b9597ee5-e04a-417f-8b4e-274e52bdf9e7","Type":"ContainerDied","Data":"a831a3a99298950772b341033e782633264ec38f344a2ecebdf397edc77c2818"}
Sep 10 19:41:41.326210 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:41:41.326192    2575 scope.go:117] "RemoveContainer" containerID="a831a3a99298950772b341033e782633264ec38f344a2ecebdf397edc77c2818"
Sep 10 19:41:41.327226 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:41:41.327198    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 19:41:41.327283 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:41:41.327263    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 19:41:41.327324 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:41:41.327298    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 19:41:42.337240 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:41:42.337196    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-3765cf-openshift-default-5994cd68fb-42qbw" event={"ID":"b9597ee5-e04a-417f-8b4e-274e52bdf9e7","Type":"ContainerStarted","Data":"fec752f2c798928f6cfb53eff7079ab86b22117555e2936216ed037858b33e29"}
Sep 10 19:41:43.243243 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:41:43.243208    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-3765cf-openshift-default-5994cd68fb-42qbw"
Sep 10 19:41:43.243472 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:41:43.243446    2575 patch_prober.go:28] interesting pod/e2e-ipp-b-3765cf-openshift-default-5994cd68fb-42qbw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Sep 10 19:41:43.243549 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:41:43.243503    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-3765cf-openshift-default-5994cd68fb-42qbw" podUID="b9597ee5-e04a-417f-8b4e-274e52bdf9e7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Sep 10 19:41:44.243424 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:41:44.243386    2575 patch_prober.go:28] interesting pod/e2e-ipp-b-3765cf-openshift-default-5994cd68fb-42qbw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Sep 10 19:41:44.243905 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:41:44.243455    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-3765cf-openshift-default-5994cd68fb-42qbw" podUID="b9597ee5-e04a-417f-8b4e-274e52bdf9e7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Sep 10 19:41:45.244356 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:41:45.244309    2575 patch_prober.go:28] interesting pod/e2e-ipp-b-3765cf-openshift-default-5994cd68fb-42qbw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Sep 10 19:41:45.244744 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:41:45.244389    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-3765cf-openshift-default-5994cd68fb-42qbw" podUID="b9597ee5-e04a-417f-8b4e-274e52bdf9e7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Sep 10 19:41:46.243611 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:41:46.243570    2575 patch_prober.go:28] interesting pod/e2e-ipp-b-3765cf-openshift-default-5994cd68fb-42qbw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Sep 10 19:41:46.243781 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:41:46.243638    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-3765cf-openshift-default-5994cd68fb-42qbw" podUID="b9597ee5-e04a-417f-8b4e-274e52bdf9e7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Sep 10 19:41:47.244076 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:41:47.244037    2575 patch_prober.go:28] interesting pod/e2e-ipp-b-3765cf-openshift-default-5994cd68fb-42qbw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Sep 10 19:41:47.244557 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:41:47.244105    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-3765cf-openshift-default-5994cd68fb-42qbw" podUID="b9597ee5-e04a-417f-8b4e-274e52bdf9e7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Sep 10 19:41:48.140116 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:41:48.140085    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:41:48.247795 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:41:48.247764    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-3765cf-openshift-default-5994cd68fb-42qbw"
Sep 10 19:41:48.248175 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:41:48.248117    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-3765cf-openshift-default-5994cd68fb-42qbw"
Sep 10 19:41:48.248920 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:41:48.248904    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-3765cf-openshift-default-5994cd68fb-42qbw"
Sep 10 19:42:03.302034 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:42:03.301995    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:42:06.961700 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:42:06.961662    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qr7rl"]
Sep 10 19:42:11.485086 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:42:11.485056    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:42:11.586815 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:42:11.586773    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:42:22.555982 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:42:22.555889    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qr7rl"]
Sep 10 19:42:28.940827 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:42:28.940793    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:42:29.963855 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:42:29.963827    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:42:30.472480 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:42:30.472439    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-8a4013-openshift-default-c7f86645d-vq2l4"]
Sep 10 19:42:30.476072 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:42:30.476050    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-8a4013-openshift-default-c7f86645d-vq2l4"
Sep 10 19:42:30.478355 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:42:30.478312    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-cleanup-8a4013-openshift-default-dockercfg-5dfcl\""
Sep 10 19:42:30.491187 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:42:30.491165    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-8a4013-openshift-default-c7f86645d-vq2l4"]
Sep 10 19:42:30.516908 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:42:30.516870    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:42:30.626446 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:42:30.626404    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ea113d18-69c9-465d-affe-eae633782b80-istio-podinfo\") pod \"e2e-ipp-cleanup-8a4013-openshift-default-c7f86645d-vq2l4\" (UID: \"ea113d18-69c9-465d-affe-eae633782b80\") " pod="openshift-ingress/e2e-ipp-cleanup-8a4013-openshift-default-c7f86645d-vq2l4"
Sep 10 19:42:30.626911 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:42:30.626456    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ea113d18-69c9-465d-affe-eae633782b80-istiod-ca-cert\") pod \"e2e-ipp-cleanup-8a4013-openshift-default-c7f86645d-vq2l4\" (UID: \"ea113d18-69c9-465d-affe-eae633782b80\") " pod="openshift-ingress/e2e-ipp-cleanup-8a4013-openshift-default-c7f86645d-vq2l4"
Sep 10 19:42:30.626911 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:42:30.626484    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2294h\" (UniqueName: \"kubernetes.io/projected/ea113d18-69c9-465d-affe-eae633782b80-kube-api-access-2294h\") pod \"e2e-ipp-cleanup-8a4013-openshift-default-c7f86645d-vq2l4\" (UID: \"ea113d18-69c9-465d-affe-eae633782b80\") " pod="openshift-ingress/e2e-ipp-cleanup-8a4013-openshift-default-c7f86645d-vq2l4"
Sep 10 19:42:30.626911 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:42:30.626605    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ea113d18-69c9-465d-affe-eae633782b80-istio-data\") pod \"e2e-ipp-cleanup-8a4013-openshift-default-c7f86645d-vq2l4\" (UID: \"ea113d18-69c9-465d-affe-eae633782b80\") " pod="openshift-ingress/e2e-ipp-cleanup-8a4013-openshift-default-c7f86645d-vq2l4"
Sep 10 19:42:30.626911 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:42:30.626630    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea113d18-69c9-465d-affe-eae633782b80-credential-socket\") pod \"e2e-ipp-cleanup-8a4013-openshift-default-c7f86645d-vq2l4\" (UID: \"ea113d18-69c9-465d-affe-eae633782b80\") " pod="openshift-ingress/e2e-ipp-cleanup-8a4013-openshift-default-c7f86645d-vq2l4"
Sep 10 19:42:30.626911 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:42:30.626661    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ea113d18-69c9-465d-affe-eae633782b80-workload-certs\") pod \"e2e-ipp-cleanup-8a4013-openshift-default-c7f86645d-vq2l4\" (UID: \"ea113d18-69c9-465d-affe-eae633782b80\") " pod="openshift-ingress/e2e-ipp-cleanup-8a4013-openshift-default-c7f86645d-vq2l4"
Sep 10 19:42:30.626911 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:42:30.626687    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ea113d18-69c9-465d-affe-eae633782b80-istio-token\") pod \"e2e-ipp-cleanup-8a4013-openshift-default-c7f86645d-vq2l4\" (UID: \"ea113d18-69c9-465d-affe-eae633782b80\") " pod="openshift-ingress/e2e-ipp-cleanup-8a4013-openshift-default-c7f86645d-vq2l4"
Sep 10 19:42:30.626911 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:42:30.626714    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ea113d18-69c9-465d-affe-eae633782b80-istio-envoy\") pod \"e2e-ipp-cleanup-8a4013-openshift-default-c7f86645d-vq2l4\" (UID: \"ea113d18-69c9-465d-affe-eae633782b80\") " pod="openshift-ingress/e2e-ipp-cleanup-8a4013-openshift-default-c7f86645d-vq2l4"
Sep 10 19:42:30.626911 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:42:30.626819    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea113d18-69c9-465d-affe-eae633782b80-workload-socket\") pod \"e2e-ipp-cleanup-8a4013-openshift-default-c7f86645d-vq2l4\" (UID: \"ea113d18-69c9-465d-affe-eae633782b80\") " pod="openshift-ingress/e2e-ipp-cleanup-8a4013-openshift-default-c7f86645d-vq2l4"
Sep 10 19:42:30.697869 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:42:30.697825    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:42:30.727573 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:42:30.727504    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ea113d18-69c9-465d-affe-eae633782b80-istio-data\") pod \"e2e-ipp-cleanup-8a4013-openshift-default-c7f86645d-vq2l4\" (UID: \"ea113d18-69c9-465d-affe-eae633782b80\") " pod="openshift-ingress/e2e-ipp-cleanup-8a4013-openshift-default-c7f86645d-vq2l4"
Sep 10 19:42:30.727573 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:42:30.727535    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea113d18-69c9-465d-affe-eae633782b80-credential-socket\") pod \"e2e-ipp-cleanup-8a4013-openshift-default-c7f86645d-vq2l4\" (UID: \"ea113d18-69c9-465d-affe-eae633782b80\") " pod="openshift-ingress/e2e-ipp-cleanup-8a4013-openshift-default-c7f86645d-vq2l4"
Sep 10 19:42:30.727573 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:42:30.727566    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ea113d18-69c9-465d-affe-eae633782b80-workload-certs\") pod \"e2e-ipp-cleanup-8a4013-openshift-default-c7f86645d-vq2l4\" (UID: \"ea113d18-69c9-465d-affe-eae633782b80\") " pod="openshift-ingress/e2e-ipp-cleanup-8a4013-openshift-default-c7f86645d-vq2l4"
Sep 10 19:42:30.727809 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:42:30.727596    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ea113d18-69c9-465d-affe-eae633782b80-istio-token\") pod \"e2e-ipp-cleanup-8a4013-openshift-default-c7f86645d-vq2l4\" (UID: \"ea113d18-69c9-465d-affe-eae633782b80\") " pod="openshift-ingress/e2e-ipp-cleanup-8a4013-openshift-default-c7f86645d-vq2l4"
Sep 10 19:42:30.727809 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:42:30.727615    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ea113d18-69c9-465d-affe-eae633782b80-istio-envoy\") pod \"e2e-ipp-cleanup-8a4013-openshift-default-c7f86645d-vq2l4\" (UID: \"ea113d18-69c9-465d-affe-eae633782b80\") " pod="openshift-ingress/e2e-ipp-cleanup-8a4013-openshift-default-c7f86645d-vq2l4"
Sep 10 19:42:30.727809 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:42:30.727683    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea113d18-69c9-465d-affe-eae633782b80-workload-socket\") pod \"e2e-ipp-cleanup-8a4013-openshift-default-c7f86645d-vq2l4\" (UID: \"ea113d18-69c9-465d-affe-eae633782b80\") " pod="openshift-ingress/e2e-ipp-cleanup-8a4013-openshift-default-c7f86645d-vq2l4"
Sep 10 19:42:30.727964 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:42:30.727882    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ea113d18-69c9-465d-affe-eae633782b80-istio-podinfo\") pod \"e2e-ipp-cleanup-8a4013-openshift-default-c7f86645d-vq2l4\" (UID: \"ea113d18-69c9-465d-affe-eae633782b80\") " pod="openshift-ingress/e2e-ipp-cleanup-8a4013-openshift-default-c7f86645d-vq2l4"
Sep 10 19:42:30.727964 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:42:30.727918    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea113d18-69c9-465d-affe-eae633782b80-credential-socket\") pod \"e2e-ipp-cleanup-8a4013-openshift-default-c7f86645d-vq2l4\" (UID: \"ea113d18-69c9-465d-affe-eae633782b80\") " pod="openshift-ingress/e2e-ipp-cleanup-8a4013-openshift-default-c7f86645d-vq2l4"
Sep 10 19:42:30.727964 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:42:30.727936    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ea113d18-69c9-465d-affe-eae633782b80-istiod-ca-cert\") pod \"e2e-ipp-cleanup-8a4013-openshift-default-c7f86645d-vq2l4\" (UID: \"ea113d18-69c9-465d-affe-eae633782b80\") " pod="openshift-ingress/e2e-ipp-cleanup-8a4013-openshift-default-c7f86645d-vq2l4"
Sep 10 19:42:30.728112 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:42:30.727988    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2294h\" (UniqueName: \"kubernetes.io/projected/ea113d18-69c9-465d-affe-eae633782b80-kube-api-access-2294h\") pod \"e2e-ipp-cleanup-8a4013-openshift-default-c7f86645d-vq2l4\" (UID: \"ea113d18-69c9-465d-affe-eae633782b80\") " pod="openshift-ingress/e2e-ipp-cleanup-8a4013-openshift-default-c7f86645d-vq2l4"
Sep 10 19:42:30.728112 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:42:30.727997    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ea113d18-69c9-465d-affe-eae633782b80-istio-data\") pod \"e2e-ipp-cleanup-8a4013-openshift-default-c7f86645d-vq2l4\" (UID: \"ea113d18-69c9-465d-affe-eae633782b80\") " pod="openshift-ingress/e2e-ipp-cleanup-8a4013-openshift-default-c7f86645d-vq2l4"
Sep 10 19:42:30.728273 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:42:30.728253    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea113d18-69c9-465d-affe-eae633782b80-workload-socket\") pod \"e2e-ipp-cleanup-8a4013-openshift-default-c7f86645d-vq2l4\" (UID: \"ea113d18-69c9-465d-affe-eae633782b80\") " pod="openshift-ingress/e2e-ipp-cleanup-8a4013-openshift-default-c7f86645d-vq2l4"
Sep 10 19:42:30.728334 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:42:30.728319    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ea113d18-69c9-465d-affe-eae633782b80-workload-certs\") pod \"e2e-ipp-cleanup-8a4013-openshift-default-c7f86645d-vq2l4\" (UID: \"ea113d18-69c9-465d-affe-eae633782b80\") " pod="openshift-ingress/e2e-ipp-cleanup-8a4013-openshift-default-c7f86645d-vq2l4"
Sep 10 19:42:30.728575 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:42:30.728560    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ea113d18-69c9-465d-affe-eae633782b80-istiod-ca-cert\") pod \"e2e-ipp-cleanup-8a4013-openshift-default-c7f86645d-vq2l4\" (UID: \"ea113d18-69c9-465d-affe-eae633782b80\") " pod="openshift-ingress/e2e-ipp-cleanup-8a4013-openshift-default-c7f86645d-vq2l4"
Sep 10 19:42:30.730075 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:42:30.730055    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ea113d18-69c9-465d-affe-eae633782b80-istio-envoy\") pod \"e2e-ipp-cleanup-8a4013-openshift-default-c7f86645d-vq2l4\" (UID: \"ea113d18-69c9-465d-affe-eae633782b80\") " pod="openshift-ingress/e2e-ipp-cleanup-8a4013-openshift-default-c7f86645d-vq2l4"
Sep 10 19:42:30.730474 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:42:30.730455    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ea113d18-69c9-465d-affe-eae633782b80-istio-podinfo\") pod \"e2e-ipp-cleanup-8a4013-openshift-default-c7f86645d-vq2l4\" (UID: \"ea113d18-69c9-465d-affe-eae633782b80\") " pod="openshift-ingress/e2e-ipp-cleanup-8a4013-openshift-default-c7f86645d-vq2l4"
Sep 10 19:42:30.737278 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:42:30.737248    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ea113d18-69c9-465d-affe-eae633782b80-istio-token\") pod \"e2e-ipp-cleanup-8a4013-openshift-default-c7f86645d-vq2l4\" (UID: \"ea113d18-69c9-465d-affe-eae633782b80\") " pod="openshift-ingress/e2e-ipp-cleanup-8a4013-openshift-default-c7f86645d-vq2l4"
Sep 10 19:42:30.737499 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:42:30.737471    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2294h\" (UniqueName: \"kubernetes.io/projected/ea113d18-69c9-465d-affe-eae633782b80-kube-api-access-2294h\") pod \"e2e-ipp-cleanup-8a4013-openshift-default-c7f86645d-vq2l4\" (UID: \"ea113d18-69c9-465d-affe-eae633782b80\") " pod="openshift-ingress/e2e-ipp-cleanup-8a4013-openshift-default-c7f86645d-vq2l4"
Sep 10 19:42:30.788413 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:42:30.788394    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-8a4013-openshift-default-c7f86645d-vq2l4"
Sep 10 19:42:31.074378 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:42:31.068680    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-8a4013-openshift-default-c7f86645d-vq2l4"]
Sep 10 19:42:31.077227 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:42:31.077171    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 19:42:31.077374 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:42:31.077322    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 19:42:31.077716 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:42:31.077686    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 19:42:31.592977 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:42:31.592943    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-8a4013-openshift-default-c7f86645d-vq2l4" event={"ID":"ea113d18-69c9-465d-affe-eae633782b80","Type":"ContainerStarted","Data":"9a9ad941ed7e5e3a7ab1f6e6c37a44de7bedbbcc301e81ab4205f3269d92a92e"}
Sep 10 19:42:31.593155 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:42:31.592998    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-8a4013-openshift-default-c7f86645d-vq2l4" event={"ID":"ea113d18-69c9-465d-affe-eae633782b80","Type":"ContainerStarted","Data":"4c069daab4ebbee824298645253bc614359d89b913ef0861229da8b0bd93997b"}
Sep 10 19:42:31.619375 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:42:31.619314    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-cleanup-8a4013-openshift-default-c7f86645d-vq2l4" podStartSLOduration=1.619296502 podStartE2EDuration="1.619296502s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:42:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:42:31.618106133 +0000 UTC m=+1622.677933801" watchObservedRunningTime="2026-09-10 19:42:31.619296502 +0000 UTC m=+1622.679124239"
Sep 10 19:42:31.788778 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:42:31.788749    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-8a4013-openshift-default-c7f86645d-vq2l4"
Sep 10 19:42:32.794067 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:42:32.794037    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-8a4013-openshift-default-c7f86645d-vq2l4"
Sep 10 19:42:33.563079 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:42:33.563044    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:42:33.603207 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:42:33.603180    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-8a4013-openshift-default-c7f86645d-vq2l4"
Sep 10 19:42:33.603943 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:42:33.603922    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-8a4013-openshift-default-c7f86645d-vq2l4"
Sep 10 19:42:33.689897 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:42:33.689850    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:42:34.744227 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:42:34.744184    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:42:34.844139 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:42:34.844097    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:42:35.066585 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:42:35.066497    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:42:35.347453 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:42:35.347373    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:42:35.450686 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:42:35.450652    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:42:35.461884 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:42:35.461858    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-8a4013-6fc85c8f4-76xmk"]
Sep 10 19:42:35.465538 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:42:35.465518    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-8a4013-6fc85c8f4-76xmk"
Sep 10 19:42:35.468066 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:42:35.468047    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-8a4013-dockercfg-8vkrp\""
Sep 10 19:42:35.468161 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:42:35.468069    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-8a4013\""
Sep 10 19:42:35.474655 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:42:35.474633    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d8fh2\" (UniqueName: \"kubernetes.io/projected/e5622984-29e9-4daa-ab21-e34bfc896f36-kube-api-access-d8fh2\") pod \"payload-pre-processing-e2e-ipp-cleanup-8a4013-6fc85c8f4-76xmk\" (UID: \"e5622984-29e9-4daa-ab21-e34bfc896f36\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-8a4013-6fc85c8f4-76xmk"
Sep 10 19:42:35.474754 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:42:35.474734    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e5622984-29e9-4daa-ab21-e34bfc896f36-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-8a4013-6fc85c8f4-76xmk\" (UID: \"e5622984-29e9-4daa-ab21-e34bfc896f36\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-8a4013-6fc85c8f4-76xmk"
Sep 10 19:42:35.480918 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:42:35.480897    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-8a4013-6fc85c8f4-76xmk"]
Sep 10 19:42:35.490372 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:42:35.490335    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-8a4013-75776b58fc-qlxdv"]
Sep 10 19:42:35.493785 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:42:35.493761    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8a4013-75776b58fc-qlxdv"
Sep 10 19:42:35.503082 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:42:35.503060    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-8a4013-75776b58fc-qlxdv"]
Sep 10 19:42:35.576155 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:42:35.576121    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d8fh2\" (UniqueName: \"kubernetes.io/projected/e5622984-29e9-4daa-ab21-e34bfc896f36-kube-api-access-d8fh2\") pod \"payload-pre-processing-e2e-ipp-cleanup-8a4013-6fc85c8f4-76xmk\" (UID: \"e5622984-29e9-4daa-ab21-e34bfc896f36\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-8a4013-6fc85c8f4-76xmk"
Sep 10 19:42:35.576366 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:42:35.576193    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e5622984-29e9-4daa-ab21-e34bfc896f36-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-8a4013-6fc85c8f4-76xmk\" (UID: \"e5622984-29e9-4daa-ab21-e34bfc896f36\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-8a4013-6fc85c8f4-76xmk"
Sep 10 19:42:35.576366 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:42:35.576253    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nl9bq\" (UniqueName: \"kubernetes.io/projected/b2aabef0-e8ab-49db-9a3d-ef153ed70fc8-kube-api-access-nl9bq\") pod \"payload-processing-e2e-ipp-cleanup-8a4013-75776b58fc-qlxdv\" (UID: \"b2aabef0-e8ab-49db-9a3d-ef153ed70fc8\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8a4013-75776b58fc-qlxdv"
Sep 10 19:42:35.576366 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:42:35.576309    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b2aabef0-e8ab-49db-9a3d-ef153ed70fc8-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-8a4013-75776b58fc-qlxdv\" (UID: \"b2aabef0-e8ab-49db-9a3d-ef153ed70fc8\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8a4013-75776b58fc-qlxdv"
Sep 10 19:42:35.576983 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:42:35.576960    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e5622984-29e9-4daa-ab21-e34bfc896f36-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-8a4013-6fc85c8f4-76xmk\" (UID: \"e5622984-29e9-4daa-ab21-e34bfc896f36\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-8a4013-6fc85c8f4-76xmk"
Sep 10 19:42:35.585428 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:42:35.585407    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d8fh2\" (UniqueName: \"kubernetes.io/projected/e5622984-29e9-4daa-ab21-e34bfc896f36-kube-api-access-d8fh2\") pod \"payload-pre-processing-e2e-ipp-cleanup-8a4013-6fc85c8f4-76xmk\" (UID: \"e5622984-29e9-4daa-ab21-e34bfc896f36\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-8a4013-6fc85c8f4-76xmk"
Sep 10 19:42:35.677550 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:42:35.677472    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nl9bq\" (UniqueName: \"kubernetes.io/projected/b2aabef0-e8ab-49db-9a3d-ef153ed70fc8-kube-api-access-nl9bq\") pod \"payload-processing-e2e-ipp-cleanup-8a4013-75776b58fc-qlxdv\" (UID: \"b2aabef0-e8ab-49db-9a3d-ef153ed70fc8\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8a4013-75776b58fc-qlxdv"
Sep 10 19:42:35.677550 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:42:35.677544    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b2aabef0-e8ab-49db-9a3d-ef153ed70fc8-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-8a4013-75776b58fc-qlxdv\" (UID: \"b2aabef0-e8ab-49db-9a3d-ef153ed70fc8\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8a4013-75776b58fc-qlxdv"
Sep 10 19:42:35.678190 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:42:35.678171    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b2aabef0-e8ab-49db-9a3d-ef153ed70fc8-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-8a4013-75776b58fc-qlxdv\" (UID: \"b2aabef0-e8ab-49db-9a3d-ef153ed70fc8\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8a4013-75776b58fc-qlxdv"
Sep 10 19:42:35.685577 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:42:35.685546    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nl9bq\" (UniqueName: \"kubernetes.io/projected/b2aabef0-e8ab-49db-9a3d-ef153ed70fc8-kube-api-access-nl9bq\") pod \"payload-processing-e2e-ipp-cleanup-8a4013-75776b58fc-qlxdv\" (UID: \"b2aabef0-e8ab-49db-9a3d-ef153ed70fc8\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8a4013-75776b58fc-qlxdv"
Sep 10 19:42:35.775923 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:42:35.775902    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-8a4013-6fc85c8f4-76xmk"
Sep 10 19:42:35.805176 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:42:35.805151    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8a4013-75776b58fc-qlxdv"
Sep 10 19:42:36.050439 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:42:36.050412    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-8a4013-6fc85c8f4-76xmk"]
Sep 10 19:42:36.051771 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:42:36.051747    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5622984_29e9_4daa_ab21_e34bfc896f36.slice/crio-d0135c678cfd82f4f4036d067cefa1045bc10d03220a3ba1b1dcb2ee320de867 WatchSource:0}: Error finding container d0135c678cfd82f4f4036d067cefa1045bc10d03220a3ba1b1dcb2ee320de867: Status 404 returned error can't find the container with id d0135c678cfd82f4f4036d067cefa1045bc10d03220a3ba1b1dcb2ee320de867
Sep 10 19:42:36.084707 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:42:36.084684    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-8a4013-75776b58fc-qlxdv"]
Sep 10 19:42:36.085967 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:42:36.085947    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2aabef0_e8ab_49db_9a3d_ef153ed70fc8.slice/crio-7599fdf4539e555c2f24b4f5e09e7192b32c016cd90c9567b53735c206892877 WatchSource:0}: Error finding container 7599fdf4539e555c2f24b4f5e09e7192b32c016cd90c9567b53735c206892877: Status 404 returned error can't find the container with id 7599fdf4539e555c2f24b4f5e09e7192b32c016cd90c9567b53735c206892877
Sep 10 19:42:36.621728 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:42:36.621689    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-8a4013-6fc85c8f4-76xmk" event={"ID":"e5622984-29e9-4daa-ab21-e34bfc896f36","Type":"ContainerStarted","Data":"1e69708f500e9ef6a598bf72819020b4ab4f01668ca942f6be22db9d1914ef34"}
Sep 10 19:42:36.621908 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:42:36.621758    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-8a4013-6fc85c8f4-76xmk"
Sep 10 19:42:36.621908 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:42:36.621785    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-8a4013-6fc85c8f4-76xmk" event={"ID":"e5622984-29e9-4daa-ab21-e34bfc896f36","Type":"ContainerStarted","Data":"d0135c678cfd82f4f4036d067cefa1045bc10d03220a3ba1b1dcb2ee320de867"}
Sep 10 19:42:36.623299 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:42:36.623274    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8a4013-75776b58fc-qlxdv" event={"ID":"b2aabef0-e8ab-49db-9a3d-ef153ed70fc8","Type":"ContainerStarted","Data":"ee12ae95b6b5bac5e7e1fbf322845fe79fccd753383ee20d7d513872613af453"}
Sep 10 19:42:36.623449 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:42:36.623313    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8a4013-75776b58fc-qlxdv" event={"ID":"b2aabef0-e8ab-49db-9a3d-ef153ed70fc8","Type":"ContainerStarted","Data":"7599fdf4539e555c2f24b4f5e09e7192b32c016cd90c9567b53735c206892877"}
Sep 10 19:42:36.623449 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:42:36.623335    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8a4013-75776b58fc-qlxdv"
Sep 10 19:42:36.638481 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:42:36.638441    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-8a4013-6fc85c8f4-76xmk" podStartSLOduration=1.638428062 podStartE2EDuration="1.638428062s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:42:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:42:36.636617819 +0000 UTC m=+1627.696445486" watchObservedRunningTime="2026-09-10 19:42:36.638428062 +0000 UTC m=+1627.698255731"
Sep 10 19:42:36.651954 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:42:36.651908    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8a4013-75776b58fc-qlxdv" podStartSLOduration=1.651897152 podStartE2EDuration="1.651897152s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:42:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:42:36.650045886 +0000 UTC m=+1627.709873553" watchObservedRunningTime="2026-09-10 19:42:36.651897152 +0000 UTC m=+1627.711724823"
Sep 10 19:42:41.651652 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:42:41.651618    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qr7rl"]
Sep 10 19:42:47.632179 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:42:47.632144    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-8a4013-6fc85c8f4-76xmk"
Sep 10 19:42:47.633951 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:42:47.633929    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8a4013-75776b58fc-qlxdv"
Sep 10 19:42:49.644522 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:42:49.644482    2575 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-8a4013-openshift-default-c7f86645d-vq2l4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 10 19:42:49.644926 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:42:49.644564    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-cleanup-8a4013-openshift-default-c7f86645d-vq2l4" podUID="ea113d18-69c9-465d-affe-eae633782b80" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 10 19:42:51.707523 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:42:51.707492    2575 generic.go:356] "Generic (PLEG): container finished" podID="ea113d18-69c9-465d-affe-eae633782b80" containerID="9a9ad941ed7e5e3a7ab1f6e6c37a44de7bedbbcc301e81ab4205f3269d92a92e" exitCode=0
Sep 10 19:42:51.707945 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:42:51.707570    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-8a4013-openshift-default-c7f86645d-vq2l4" event={"ID":"ea113d18-69c9-465d-affe-eae633782b80","Type":"ContainerDied","Data":"9a9ad941ed7e5e3a7ab1f6e6c37a44de7bedbbcc301e81ab4205f3269d92a92e"}
Sep 10 19:42:51.707945 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:42:51.707922    2575 scope.go:117] "RemoveContainer" containerID="9a9ad941ed7e5e3a7ab1f6e6c37a44de7bedbbcc301e81ab4205f3269d92a92e"
Sep 10 19:42:51.708959 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:42:51.708932    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 19:42:51.709021 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:42:51.708998    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 19:42:51.709057 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:42:51.709030    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 19:42:52.714722 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:42:52.714687    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-8a4013-openshift-default-c7f86645d-vq2l4" event={"ID":"ea113d18-69c9-465d-affe-eae633782b80","Type":"ContainerStarted","Data":"b33f9214623a0d4d2e28cdc74c2ddf9ce77fb18e21d721b10fbcf46cc10bd6d1"}
Sep 10 19:42:52.789571 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:42:52.789541    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-8a4013-openshift-default-c7f86645d-vq2l4"
Sep 10 19:42:52.789730 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:42:52.789665    2575 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-8a4013-openshift-default-c7f86645d-vq2l4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body=
Sep 10 19:42:52.789730 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:42:52.789708    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-8a4013-openshift-default-c7f86645d-vq2l4" podUID="ea113d18-69c9-465d-affe-eae633782b80" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused"
Sep 10 19:42:53.789407 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:42:53.789370    2575 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-8a4013-openshift-default-c7f86645d-vq2l4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body=
Sep 10 19:42:53.789855 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:42:53.789425    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-8a4013-openshift-default-c7f86645d-vq2l4" podUID="ea113d18-69c9-465d-affe-eae633782b80" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused"
Sep 10 19:42:54.789693 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:42:54.789658    2575 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-8a4013-openshift-default-c7f86645d-vq2l4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body=
Sep 10 19:42:54.790063 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:42:54.789711    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-8a4013-openshift-default-c7f86645d-vq2l4" podUID="ea113d18-69c9-465d-affe-eae633782b80" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused"
Sep 10 19:42:55.789507 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:42:55.789474    2575 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-8a4013-openshift-default-c7f86645d-vq2l4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body=
Sep 10 19:42:55.789690 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:42:55.789523    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-8a4013-openshift-default-c7f86645d-vq2l4" podUID="ea113d18-69c9-465d-affe-eae633782b80" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused"
Sep 10 19:42:56.789302 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:42:56.789261    2575 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-8a4013-openshift-default-c7f86645d-vq2l4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body=
Sep 10 19:42:56.789699 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:42:56.789315    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-8a4013-openshift-default-c7f86645d-vq2l4" podUID="ea113d18-69c9-465d-affe-eae633782b80" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused"
Sep 10 19:42:57.963781 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:42:57.963746    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-8a4013-openshift-default-c7f86645d-vq2l4"
Sep 10 19:42:57.964233 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:42:57.964084    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-8a4013-openshift-default-c7f86645d-vq2l4"
Sep 10 19:42:57.964835 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:42:57.964817    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-8a4013-openshift-default-c7f86645d-vq2l4"
Sep 10 19:42:58.592900 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:42:58.592863    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:43:05.506469 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:05.506429    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:43:05.531930 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:05.531896    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-8a4013-75776b58fc-qlxdv"]
Sep 10 19:43:05.532216 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:05.532174    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8a4013-75776b58fc-qlxdv" podUID="b2aabef0-e8ab-49db-9a3d-ef153ed70fc8" containerName="payload-processing" containerID="cri-o://ee12ae95b6b5bac5e7e1fbf322845fe79fccd753383ee20d7d513872613af453" gracePeriod=30
Sep 10 19:43:05.542980 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:05.542953    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-8a4013-6fc85c8f4-76xmk"]
Sep 10 19:43:05.543212 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:05.543165    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-8a4013-6fc85c8f4-76xmk" podUID="e5622984-29e9-4daa-ab21-e34bfc896f36" containerName="payload-pre-processing" containerID="cri-o://1e69708f500e9ef6a598bf72819020b4ab4f01668ca942f6be22db9d1914ef34" gracePeriod=30
Sep 10 19:43:05.610615 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:05.610577    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:43:05.715093 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:05.715058    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:43:05.795964 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:05.795868    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:43:05.801935 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:05.801909    2575 generic.go:356] "Generic (PLEG): container finished" podID="e5622984-29e9-4daa-ab21-e34bfc896f36" containerID="1e69708f500e9ef6a598bf72819020b4ab4f01668ca942f6be22db9d1914ef34" exitCode=0
Sep 10 19:43:05.802057 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:05.801982    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-8a4013-6fc85c8f4-76xmk" event={"ID":"e5622984-29e9-4daa-ab21-e34bfc896f36","Type":"ContainerDied","Data":"1e69708f500e9ef6a598bf72819020b4ab4f01668ca942f6be22db9d1914ef34"}
Sep 10 19:43:05.802920 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:05.802900    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8a4013-75776b58fc-qlxdv"
Sep 10 19:43:05.803416 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:05.803396    2575 generic.go:356] "Generic (PLEG): container finished" podID="b2aabef0-e8ab-49db-9a3d-ef153ed70fc8" containerID="ee12ae95b6b5bac5e7e1fbf322845fe79fccd753383ee20d7d513872613af453" exitCode=0
Sep 10 19:43:05.803509 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:05.803430    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8a4013-75776b58fc-qlxdv" event={"ID":"b2aabef0-e8ab-49db-9a3d-ef153ed70fc8","Type":"ContainerDied","Data":"ee12ae95b6b5bac5e7e1fbf322845fe79fccd753383ee20d7d513872613af453"}
Sep 10 19:43:05.803509 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:05.803465    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8a4013-75776b58fc-qlxdv" event={"ID":"b2aabef0-e8ab-49db-9a3d-ef153ed70fc8","Type":"ContainerDied","Data":"7599fdf4539e555c2f24b4f5e09e7192b32c016cd90c9567b53735c206892877"}
Sep 10 19:43:05.803509 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:05.803486    2575 scope.go:117] "RemoveContainer" containerID="ee12ae95b6b5bac5e7e1fbf322845fe79fccd753383ee20d7d513872613af453"
Sep 10 19:43:05.812110 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:05.812093    2575 scope.go:117] "RemoveContainer" containerID="ee12ae95b6b5bac5e7e1fbf322845fe79fccd753383ee20d7d513872613af453"
Sep 10 19:43:05.812394 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:43:05.812374    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ee12ae95b6b5bac5e7e1fbf322845fe79fccd753383ee20d7d513872613af453\": container with ID starting with ee12ae95b6b5bac5e7e1fbf322845fe79fccd753383ee20d7d513872613af453 not found: ID does not exist" containerID="ee12ae95b6b5bac5e7e1fbf322845fe79fccd753383ee20d7d513872613af453"
Sep 10 19:43:05.812475 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:05.812405    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee12ae95b6b5bac5e7e1fbf322845fe79fccd753383ee20d7d513872613af453"} err="failed to get container status \"ee12ae95b6b5bac5e7e1fbf322845fe79fccd753383ee20d7d513872613af453\": rpc error: code = NotFound desc = could not find container \"ee12ae95b6b5bac5e7e1fbf322845fe79fccd753383ee20d7d513872613af453\": container with ID starting with ee12ae95b6b5bac5e7e1fbf322845fe79fccd753383ee20d7d513872613af453 not found: ID does not exist"
Sep 10 19:43:05.832727 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:05.832709    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-8a4013-6fc85c8f4-76xmk"
Sep 10 19:43:05.950645 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:05.950617    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nl9bq\" (UniqueName: \"kubernetes.io/projected/b2aabef0-e8ab-49db-9a3d-ef153ed70fc8-kube-api-access-nl9bq\") pod \"b2aabef0-e8ab-49db-9a3d-ef153ed70fc8\" (UID: \"b2aabef0-e8ab-49db-9a3d-ef153ed70fc8\") "
Sep 10 19:43:05.950645 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:05.950651    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d8fh2\" (UniqueName: \"kubernetes.io/projected/e5622984-29e9-4daa-ab21-e34bfc896f36-kube-api-access-d8fh2\") pod \"e5622984-29e9-4daa-ab21-e34bfc896f36\" (UID: \"e5622984-29e9-4daa-ab21-e34bfc896f36\") "
Sep 10 19:43:05.950855 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:05.950690    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e5622984-29e9-4daa-ab21-e34bfc896f36-plugins-config-volume\") pod \"e5622984-29e9-4daa-ab21-e34bfc896f36\" (UID: \"e5622984-29e9-4daa-ab21-e34bfc896f36\") "
Sep 10 19:43:05.950855 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:05.950822    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b2aabef0-e8ab-49db-9a3d-ef153ed70fc8-plugins-config-volume\") pod \"b2aabef0-e8ab-49db-9a3d-ef153ed70fc8\" (UID: \"b2aabef0-e8ab-49db-9a3d-ef153ed70fc8\") "
Sep 10 19:43:05.951164 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:05.951128    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e5622984-29e9-4daa-ab21-e34bfc896f36-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e5622984-29e9-4daa-ab21-e34bfc896f36" (UID: "e5622984-29e9-4daa-ab21-e34bfc896f36"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:43:05.951249 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:05.951168    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b2aabef0-e8ab-49db-9a3d-ef153ed70fc8-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b2aabef0-e8ab-49db-9a3d-ef153ed70fc8" (UID: "b2aabef0-e8ab-49db-9a3d-ef153ed70fc8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:43:05.952817 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:05.952791    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e5622984-29e9-4daa-ab21-e34bfc896f36-kube-api-access-d8fh2" (OuterVolumeSpecName: "kube-api-access-d8fh2") pod "e5622984-29e9-4daa-ab21-e34bfc896f36" (UID: "e5622984-29e9-4daa-ab21-e34bfc896f36"). InnerVolumeSpecName "kube-api-access-d8fh2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:43:05.952915 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:05.952818    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b2aabef0-e8ab-49db-9a3d-ef153ed70fc8-kube-api-access-nl9bq" (OuterVolumeSpecName: "kube-api-access-nl9bq") pod "b2aabef0-e8ab-49db-9a3d-ef153ed70fc8" (UID: "b2aabef0-e8ab-49db-9a3d-ef153ed70fc8"). InnerVolumeSpecName "kube-api-access-nl9bq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:43:06.051749 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:06.051697    2575 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b2aabef0-e8ab-49db-9a3d-ef153ed70fc8-plugins-config-volume\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:43:06.051749 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:06.051717    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nl9bq\" (UniqueName: \"kubernetes.io/projected/b2aabef0-e8ab-49db-9a3d-ef153ed70fc8-kube-api-access-nl9bq\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:43:06.051749 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:06.051728    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d8fh2\" (UniqueName: \"kubernetes.io/projected/e5622984-29e9-4daa-ab21-e34bfc896f36-kube-api-access-d8fh2\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:43:06.051749 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:06.051738    2575 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e5622984-29e9-4daa-ab21-e34bfc896f36-plugins-config-volume\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:43:06.178913 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:06.178889    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:43:06.809845 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:06.809766    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-8a4013-6fc85c8f4-76xmk"
Sep 10 19:43:06.809845 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:06.809801    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-8a4013-6fc85c8f4-76xmk" event={"ID":"e5622984-29e9-4daa-ab21-e34bfc896f36","Type":"ContainerDied","Data":"d0135c678cfd82f4f4036d067cefa1045bc10d03220a3ba1b1dcb2ee320de867"}
Sep 10 19:43:06.810323 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:06.809859    2575 scope.go:117] "RemoveContainer" containerID="1e69708f500e9ef6a598bf72819020b4ab4f01668ca942f6be22db9d1914ef34"
Sep 10 19:43:06.811131 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:06.810933    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8a4013-75776b58fc-qlxdv"
Sep 10 19:43:06.835188 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:06.835164    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-8a4013-6fc85c8f4-76xmk"]
Sep 10 19:43:06.840162 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:06.840142    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-8a4013-6fc85c8f4-76xmk"]
Sep 10 19:43:06.851524 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:06.851505    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-8a4013-75776b58fc-qlxdv"]
Sep 10 19:43:06.856006 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:06.855987    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-8a4013-75776b58fc-qlxdv"]
Sep 10 19:43:07.390238 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:07.390204    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b2aabef0-e8ab-49db-9a3d-ef153ed70fc8" path="/var/lib/kubelet/pods/b2aabef0-e8ab-49db-9a3d-ef153ed70fc8/volumes"
Sep 10 19:43:07.390605 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:07.390589    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e5622984-29e9-4daa-ab21-e34bfc896f36" path="/var/lib/kubelet/pods/e5622984-29e9-4daa-ab21-e34bfc896f36/volumes"
Sep 10 19:43:09.249878 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:09.249839    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:43:09.760371 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:09.760325    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qr7rl"]
Sep 10 19:43:12.706636 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:12.706602    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:43:12.810023 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:12.809988    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:43:12.958124 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:12.958050    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-8a4013-openshift-default-c7f86645d-vq2l4"]
Sep 10 19:43:12.958289 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:12.958264    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-cleanup-8a4013-openshift-default-c7f86645d-vq2l4" podUID="ea113d18-69c9-465d-affe-eae633782b80" containerName="istio-proxy" containerID="cri-o://b33f9214623a0d4d2e28cdc74c2ddf9ce77fb18e21d721b10fbcf46cc10bd6d1" gracePeriod=30
Sep 10 19:43:12.966791 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:12.966759    2575 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-8a4013-openshift-default-c7f86645d-vq2l4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused
Sep 10 19:43:12.966903 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:12.966817    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-cleanup-8a4013-openshift-default-c7f86645d-vq2l4" podUID="ea113d18-69c9-465d-affe-eae633782b80" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 10 19:43:12.991099 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:12.991072    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:43:16.769968 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:16.769936    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-df863b-cjvsb"]
Sep 10 19:43:16.770372 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:16.770357    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e5622984-29e9-4daa-ab21-e34bfc896f36" containerName="payload-pre-processing"
Sep 10 19:43:16.770427 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:16.770374    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="e5622984-29e9-4daa-ab21-e34bfc896f36" containerName="payload-pre-processing"
Sep 10 19:43:16.770427 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:16.770383    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b2aabef0-e8ab-49db-9a3d-ef153ed70fc8" containerName="payload-processing"
Sep 10 19:43:16.770427 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:16.770389    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="b2aabef0-e8ab-49db-9a3d-ef153ed70fc8" containerName="payload-processing"
Sep 10 19:43:16.770535 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:16.770456    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="b2aabef0-e8ab-49db-9a3d-ef153ed70fc8" containerName="payload-processing"
Sep 10 19:43:16.770535 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:16.770465    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="e5622984-29e9-4daa-ab21-e34bfc896f36" containerName="payload-pre-processing"
Sep 10 19:43:16.774834 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:16.774820    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-df863b-cjvsb"
Sep 10 19:43:16.777165 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:16.777128    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 19:43:16.777387 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:16.777369    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wkgt8\""
Sep 10 19:43:16.787698 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:16.787678    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-df863b-cjvsb"]
Sep 10 19:43:16.944519 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:16.944483    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:43:16.947679 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:16.947654    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d298824-d2f0-4b94-84ce-8a3bd2ec63d6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-df863b-cjvsb\" (UID: \"5d298824-d2f0-4b94-84ce-8a3bd2ec63d6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-df863b-cjvsb"
Sep 10 19:43:17.048939 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:17.048865    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d298824-d2f0-4b94-84ce-8a3bd2ec63d6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-df863b-cjvsb\" (UID: \"5d298824-d2f0-4b94-84ce-8a3bd2ec63d6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-df863b-cjvsb"
Sep 10 19:43:17.049519 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:17.049500    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d298824-d2f0-4b94-84ce-8a3bd2ec63d6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-df863b-cjvsb\" (UID: \"5d298824-d2f0-4b94-84ce-8a3bd2ec63d6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-df863b-cjvsb"
Sep 10 19:43:17.084635 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:17.084617    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-df863b-cjvsb"
Sep 10 19:43:17.372955 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:17.370273    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-df863b-cjvsb"]
Sep 10 19:43:17.377182 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:43:17.377153    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d298824_d2f0_4b94_84ce_8a3bd2ec63d6.slice/crio-c20af252223d3936c5b6b56736a5155310838b5533e1c3917cdddf4fd45f0496 WatchSource:0}: Error finding container c20af252223d3936c5b6b56736a5155310838b5533e1c3917cdddf4fd45f0496: Status 404 returned error can't find the container with id c20af252223d3936c5b6b56736a5155310838b5533e1c3917cdddf4fd45f0496
Sep 10 19:43:17.873655 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:17.873623    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-df863b-cjvsb" event={"ID":"5d298824-d2f0-4b94-84ce-8a3bd2ec63d6","Type":"ContainerStarted","Data":"ca0b7caa49e1f96ad05a57d891b6efce93f6049ff4600252e43f6e837fa9a8cc"}
Sep 10 19:43:17.873981 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:17.873677    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-df863b-cjvsb" event={"ID":"5d298824-d2f0-4b94-84ce-8a3bd2ec63d6","Type":"ContainerStarted","Data":"c20af252223d3936c5b6b56736a5155310838b5533e1c3917cdddf4fd45f0496"}
Sep 10 19:43:17.901153 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:17.901092    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-df863b-cjvsb" podStartSLOduration=1.901073379 podStartE2EDuration="1.901073379s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:43:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:43:17.900315429 +0000 UTC m=+1668.960143109" watchObservedRunningTime="2026-09-10 19:43:17.901073379 +0000 UTC m=+1668.960901049"
Sep 10 19:43:18.209196 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:18.209173    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-8a4013-openshift-default-c7f86645d-vq2l4"
Sep 10 19:43:18.361124 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:18.361063    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ea113d18-69c9-465d-affe-eae633782b80-istio-token\") pod \"ea113d18-69c9-465d-affe-eae633782b80\" (UID: \"ea113d18-69c9-465d-affe-eae633782b80\") "
Sep 10 19:43:18.361124 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:18.361104    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea113d18-69c9-465d-affe-eae633782b80-credential-socket\") pod \"ea113d18-69c9-465d-affe-eae633782b80\" (UID: \"ea113d18-69c9-465d-affe-eae633782b80\") "
Sep 10 19:43:18.361278 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:18.361142    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2294h\" (UniqueName: \"kubernetes.io/projected/ea113d18-69c9-465d-affe-eae633782b80-kube-api-access-2294h\") pod \"ea113d18-69c9-465d-affe-eae633782b80\" (UID: \"ea113d18-69c9-465d-affe-eae633782b80\") "
Sep 10 19:43:18.361278 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:18.361172    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ea113d18-69c9-465d-affe-eae633782b80-istio-data\") pod \"ea113d18-69c9-465d-affe-eae633782b80\" (UID: \"ea113d18-69c9-465d-affe-eae633782b80\") "
Sep 10 19:43:18.361278 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:18.361191    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ea113d18-69c9-465d-affe-eae633782b80-istio-envoy\") pod \"ea113d18-69c9-465d-affe-eae633782b80\" (UID: \"ea113d18-69c9-465d-affe-eae633782b80\") "
Sep 10 19:43:18.361278 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:18.361222    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ea113d18-69c9-465d-affe-eae633782b80-istiod-ca-cert\") pod \"ea113d18-69c9-465d-affe-eae633782b80\" (UID: \"ea113d18-69c9-465d-affe-eae633782b80\") "
Sep 10 19:43:18.361278 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:18.361253    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea113d18-69c9-465d-affe-eae633782b80-workload-socket\") pod \"ea113d18-69c9-465d-affe-eae633782b80\" (UID: \"ea113d18-69c9-465d-affe-eae633782b80\") "
Sep 10 19:43:18.361560 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:18.361301    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ea113d18-69c9-465d-affe-eae633782b80-istio-podinfo\") pod \"ea113d18-69c9-465d-affe-eae633782b80\" (UID: \"ea113d18-69c9-465d-affe-eae633782b80\") "
Sep 10 19:43:18.361560 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:18.361328    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ea113d18-69c9-465d-affe-eae633782b80-workload-certs\") pod \"ea113d18-69c9-465d-affe-eae633782b80\" (UID: \"ea113d18-69c9-465d-affe-eae633782b80\") "
Sep 10 19:43:18.361664 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:18.361545    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea113d18-69c9-465d-affe-eae633782b80-istio-data" (OuterVolumeSpecName: "istio-data") pod "ea113d18-69c9-465d-affe-eae633782b80" (UID: "ea113d18-69c9-465d-affe-eae633782b80"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:43:18.361664 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:18.361613    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea113d18-69c9-465d-affe-eae633782b80-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ea113d18-69c9-465d-affe-eae633782b80" (UID: "ea113d18-69c9-465d-affe-eae633782b80"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:43:18.361764 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:18.361661    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea113d18-69c9-465d-affe-eae633782b80-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ea113d18-69c9-465d-affe-eae633782b80" (UID: "ea113d18-69c9-465d-affe-eae633782b80"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:43:18.361764 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:18.361750    2575 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea113d18-69c9-465d-affe-eae633782b80-credential-socket\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:43:18.361862 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:18.361769    2575 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ea113d18-69c9-465d-affe-eae633782b80-istio-data\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:43:18.361862 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:18.361786    2575 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea113d18-69c9-465d-affe-eae633782b80-workload-socket\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:43:18.361862 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:18.361822    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea113d18-69c9-465d-affe-eae633782b80-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ea113d18-69c9-465d-affe-eae633782b80" (UID: "ea113d18-69c9-465d-affe-eae633782b80"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:43:18.361862 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:18.361740    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ea113d18-69c9-465d-affe-eae633782b80-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ea113d18-69c9-465d-affe-eae633782b80" (UID: "ea113d18-69c9-465d-affe-eae633782b80"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:43:18.363322 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:18.363297    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea113d18-69c9-465d-affe-eae633782b80-istio-token" (OuterVolumeSpecName: "istio-token") pod "ea113d18-69c9-465d-affe-eae633782b80" (UID: "ea113d18-69c9-465d-affe-eae633782b80"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:43:18.363462 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:18.363441    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea113d18-69c9-465d-affe-eae633782b80-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ea113d18-69c9-465d-affe-eae633782b80" (UID: "ea113d18-69c9-465d-affe-eae633782b80"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:43:18.363557 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:18.363541    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ea113d18-69c9-465d-affe-eae633782b80-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ea113d18-69c9-465d-affe-eae633782b80" (UID: "ea113d18-69c9-465d-affe-eae633782b80"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 19:43:18.363725 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:18.363708    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea113d18-69c9-465d-affe-eae633782b80-kube-api-access-2294h" (OuterVolumeSpecName: "kube-api-access-2294h") pod "ea113d18-69c9-465d-affe-eae633782b80" (UID: "ea113d18-69c9-465d-affe-eae633782b80"). InnerVolumeSpecName "kube-api-access-2294h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:43:18.462645 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:18.462624    2575 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ea113d18-69c9-465d-affe-eae633782b80-istio-podinfo\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:43:18.462645 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:18.462645    2575 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ea113d18-69c9-465d-affe-eae633782b80-workload-certs\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:43:18.462768 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:18.462657    2575 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ea113d18-69c9-465d-affe-eae633782b80-istio-token\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:43:18.462768 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:18.462669    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2294h\" (UniqueName: \"kubernetes.io/projected/ea113d18-69c9-465d-affe-eae633782b80-kube-api-access-2294h\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:43:18.462768 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:18.462682    2575 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ea113d18-69c9-465d-affe-eae633782b80-istio-envoy\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:43:18.462768 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:18.462695    2575 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ea113d18-69c9-465d-affe-eae633782b80-istiod-ca-cert\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:43:18.880202 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:18.880169    2575 generic.go:356] "Generic (PLEG): container finished" podID="ea113d18-69c9-465d-affe-eae633782b80" containerID="b33f9214623a0d4d2e28cdc74c2ddf9ce77fb18e21d721b10fbcf46cc10bd6d1" exitCode=0
Sep 10 19:43:18.880637 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:18.880222    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-8a4013-openshift-default-c7f86645d-vq2l4"
Sep 10 19:43:18.880637 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:18.880260    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-8a4013-openshift-default-c7f86645d-vq2l4" event={"ID":"ea113d18-69c9-465d-affe-eae633782b80","Type":"ContainerDied","Data":"b33f9214623a0d4d2e28cdc74c2ddf9ce77fb18e21d721b10fbcf46cc10bd6d1"}
Sep 10 19:43:18.880637 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:18.880328    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-8a4013-openshift-default-c7f86645d-vq2l4" event={"ID":"ea113d18-69c9-465d-affe-eae633782b80","Type":"ContainerDied","Data":"4c069daab4ebbee824298645253bc614359d89b913ef0861229da8b0bd93997b"}
Sep 10 19:43:18.880637 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:18.880383    2575 scope.go:117] "RemoveContainer" containerID="b33f9214623a0d4d2e28cdc74c2ddf9ce77fb18e21d721b10fbcf46cc10bd6d1"
Sep 10 19:43:18.882141 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:18.882116    2575 generic.go:356] "Generic (PLEG): container finished" podID="5d298824-d2f0-4b94-84ce-8a3bd2ec63d6" containerID="ca0b7caa49e1f96ad05a57d891b6efce93f6049ff4600252e43f6e837fa9a8cc" exitCode=0
Sep 10 19:43:18.882261 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:18.882216    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-df863b-cjvsb" event={"ID":"5d298824-d2f0-4b94-84ce-8a3bd2ec63d6","Type":"ContainerDied","Data":"ca0b7caa49e1f96ad05a57d891b6efce93f6049ff4600252e43f6e837fa9a8cc"}
Sep 10 19:43:18.890211 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:18.890188    2575 scope.go:117] "RemoveContainer" containerID="9a9ad941ed7e5e3a7ab1f6e6c37a44de7bedbbcc301e81ab4205f3269d92a92e"
Sep 10 19:43:18.898139 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:18.898124    2575 scope.go:117] "RemoveContainer" containerID="b33f9214623a0d4d2e28cdc74c2ddf9ce77fb18e21d721b10fbcf46cc10bd6d1"
Sep 10 19:43:18.898414 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:43:18.898389    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b33f9214623a0d4d2e28cdc74c2ddf9ce77fb18e21d721b10fbcf46cc10bd6d1\": container with ID starting with b33f9214623a0d4d2e28cdc74c2ddf9ce77fb18e21d721b10fbcf46cc10bd6d1 not found: ID does not exist" containerID="b33f9214623a0d4d2e28cdc74c2ddf9ce77fb18e21d721b10fbcf46cc10bd6d1"
Sep 10 19:43:18.898458 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:18.898422    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b33f9214623a0d4d2e28cdc74c2ddf9ce77fb18e21d721b10fbcf46cc10bd6d1"} err="failed to get container status \"b33f9214623a0d4d2e28cdc74c2ddf9ce77fb18e21d721b10fbcf46cc10bd6d1\": rpc error: code = NotFound desc = could not find container \"b33f9214623a0d4d2e28cdc74c2ddf9ce77fb18e21d721b10fbcf46cc10bd6d1\": container with ID starting with b33f9214623a0d4d2e28cdc74c2ddf9ce77fb18e21d721b10fbcf46cc10bd6d1 not found: ID does not exist"
Sep 10 19:43:18.898458 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:18.898438    2575 scope.go:117] "RemoveContainer" containerID="9a9ad941ed7e5e3a7ab1f6e6c37a44de7bedbbcc301e81ab4205f3269d92a92e"
Sep 10 19:43:18.898694 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:43:18.898678    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9a9ad941ed7e5e3a7ab1f6e6c37a44de7bedbbcc301e81ab4205f3269d92a92e\": container with ID starting with 9a9ad941ed7e5e3a7ab1f6e6c37a44de7bedbbcc301e81ab4205f3269d92a92e not found: ID does not exist" containerID="9a9ad941ed7e5e3a7ab1f6e6c37a44de7bedbbcc301e81ab4205f3269d92a92e"
Sep 10 19:43:18.898732 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:18.898699    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9a9ad941ed7e5e3a7ab1f6e6c37a44de7bedbbcc301e81ab4205f3269d92a92e"} err="failed to get container status \"9a9ad941ed7e5e3a7ab1f6e6c37a44de7bedbbcc301e81ab4205f3269d92a92e\": rpc error: code = NotFound desc = could not find container \"9a9ad941ed7e5e3a7ab1f6e6c37a44de7bedbbcc301e81ab4205f3269d92a92e\": container with ID starting with 9a9ad941ed7e5e3a7ab1f6e6c37a44de7bedbbcc301e81ab4205f3269d92a92e not found: ID does not exist"
Sep 10 19:43:18.923472 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:18.923445    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-8a4013-openshift-default-c7f86645d-vq2l4"]
Sep 10 19:43:18.930263 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:18.930242    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-8a4013-openshift-default-c7f86645d-vq2l4"]
Sep 10 19:43:19.385573 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:19.385537    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ea113d18-69c9-465d-affe-eae633782b80" path="/var/lib/kubelet/pods/ea113d18-69c9-465d-affe-eae633782b80/volumes"
Sep 10 19:43:20.020712 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:20.020682    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-df863b-cjvsb"
Sep 10 19:43:20.178058 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:20.177998    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d298824-d2f0-4b94-84ce-8a3bd2ec63d6-maas-api-service-ca\") pod \"5d298824-d2f0-4b94-84ce-8a3bd2ec63d6\" (UID: \"5d298824-d2f0-4b94-84ce-8a3bd2ec63d6\") "
Sep 10 19:43:20.178320 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:20.178300    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5d298824-d2f0-4b94-84ce-8a3bd2ec63d6-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5d298824-d2f0-4b94-84ce-8a3bd2ec63d6" (UID: "5d298824-d2f0-4b94-84ce-8a3bd2ec63d6"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:43:20.279413 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:20.279388    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d298824-d2f0-4b94-84ce-8a3bd2ec63d6-maas-api-service-ca\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:43:20.897737 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:20.897708    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-df863b-cjvsb"
Sep 10 19:43:20.897942 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:20.897751    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-df863b-cjvsb" event={"ID":"5d298824-d2f0-4b94-84ce-8a3bd2ec63d6","Type":"ContainerDied","Data":"c20af252223d3936c5b6b56736a5155310838b5533e1c3917cdddf4fd45f0496"}
Sep 10 19:43:20.897942 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:20.897820    2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c20af252223d3936c5b6b56736a5155310838b5533e1c3917cdddf4fd45f0496"
Sep 10 19:43:30.414594 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:30.414553    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qr7rl"]
Sep 10 19:43:32.498973 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:32.498940    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:43:32.537700 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:32.537669    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-df863b-79948f674b-gfn6r"]
Sep 10 19:43:32.537915 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:32.537889    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-df863b-79948f674b-gfn6r" podUID="bd267769-63da-4a57-b2bc-0d1e0d797c19" containerName="payload-pre-processing" containerID="cri-o://62d28232e25c882caa5ae9e2b74229ae065d19d05df15e353ba4ddf4acf064ce" gracePeriod=30
Sep 10 19:43:32.600052 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:32.600012    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:43:32.703115 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:32.703063    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:43:32.800692 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:32.800669    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-df863b-79948f674b-gfn6r"
Sep 10 19:43:32.802542 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:32.802518    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-df863b-cjvsb"]
Sep 10 19:43:32.802837 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:32.802813    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:43:32.805033 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:32.805010    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-df863b-cjvsb"]
Sep 10 19:43:32.879431 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:32.879386    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:43:32.884939 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:32.884921    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bd267769-63da-4a57-b2bc-0d1e0d797c19-plugins-config-volume\") pod \"bd267769-63da-4a57-b2bc-0d1e0d797c19\" (UID: \"bd267769-63da-4a57-b2bc-0d1e0d797c19\") "
Sep 10 19:43:32.885006 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:32.884962    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dlwxv\" (UniqueName: \"kubernetes.io/projected/bd267769-63da-4a57-b2bc-0d1e0d797c19-kube-api-access-dlwxv\") pod \"bd267769-63da-4a57-b2bc-0d1e0d797c19\" (UID: \"bd267769-63da-4a57-b2bc-0d1e0d797c19\") "
Sep 10 19:43:32.885284 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:32.885263    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bd267769-63da-4a57-b2bc-0d1e0d797c19-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "bd267769-63da-4a57-b2bc-0d1e0d797c19" (UID: "bd267769-63da-4a57-b2bc-0d1e0d797c19"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:43:32.887008 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:32.886988    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd267769-63da-4a57-b2bc-0d1e0d797c19-kube-api-access-dlwxv" (OuterVolumeSpecName: "kube-api-access-dlwxv") pod "bd267769-63da-4a57-b2bc-0d1e0d797c19" (UID: "bd267769-63da-4a57-b2bc-0d1e0d797c19"). InnerVolumeSpecName "kube-api-access-dlwxv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:43:32.960520 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:32.960498    2575 generic.go:356] "Generic (PLEG): container finished" podID="bd267769-63da-4a57-b2bc-0d1e0d797c19" containerID="62d28232e25c882caa5ae9e2b74229ae065d19d05df15e353ba4ddf4acf064ce" exitCode=0
Sep 10 19:43:32.960630 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:32.960538    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-df863b-79948f674b-gfn6r"
Sep 10 19:43:32.960630 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:32.960582    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-df863b-79948f674b-gfn6r" event={"ID":"bd267769-63da-4a57-b2bc-0d1e0d797c19","Type":"ContainerDied","Data":"62d28232e25c882caa5ae9e2b74229ae065d19d05df15e353ba4ddf4acf064ce"}
Sep 10 19:43:32.960748 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:32.960640    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-df863b-79948f674b-gfn6r" event={"ID":"bd267769-63da-4a57-b2bc-0d1e0d797c19","Type":"ContainerDied","Data":"0adc7137604e770a3a368fb8721bf08393aef9aaf70c653a4276d2b836b0c3a4"}
Sep 10 19:43:32.960748 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:32.960671    2575 scope.go:117] "RemoveContainer" containerID="62d28232e25c882caa5ae9e2b74229ae065d19d05df15e353ba4ddf4acf064ce"
Sep 10 19:43:32.970036 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:32.970020    2575 scope.go:117] "RemoveContainer" containerID="62d28232e25c882caa5ae9e2b74229ae065d19d05df15e353ba4ddf4acf064ce"
Sep 10 19:43:32.970298 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:43:32.970280    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"62d28232e25c882caa5ae9e2b74229ae065d19d05df15e353ba4ddf4acf064ce\": container with ID starting with 62d28232e25c882caa5ae9e2b74229ae065d19d05df15e353ba4ddf4acf064ce not found: ID does not exist" containerID="62d28232e25c882caa5ae9e2b74229ae065d19d05df15e353ba4ddf4acf064ce"
Sep 10 19:43:32.970375 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:32.970305    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"62d28232e25c882caa5ae9e2b74229ae065d19d05df15e353ba4ddf4acf064ce"} err="failed to get container status \"62d28232e25c882caa5ae9e2b74229ae065d19d05df15e353ba4ddf4acf064ce\": rpc error: code = NotFound desc = could not find container \"62d28232e25c882caa5ae9e2b74229ae065d19d05df15e353ba4ddf4acf064ce\": container with ID starting with 62d28232e25c882caa5ae9e2b74229ae065d19d05df15e353ba4ddf4acf064ce not found: ID does not exist"
Sep 10 19:43:32.982771 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:32.982751    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-df863b-79948f674b-gfn6r"]
Sep 10 19:43:32.986215 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:32.986190    2575 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bd267769-63da-4a57-b2bc-0d1e0d797c19-plugins-config-volume\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:43:32.986215 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:32.986211    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dlwxv\" (UniqueName: \"kubernetes.io/projected/bd267769-63da-4a57-b2bc-0d1e0d797c19-kube-api-access-dlwxv\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:43:32.986931 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:32.986915    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-df863b-79948f674b-gfn6r"]
Sep 10 19:43:33.166368 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:33.166287    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:43:33.384484 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:33.384460    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5d298824-d2f0-4b94-84ce-8a3bd2ec63d6" path="/var/lib/kubelet/pods/5d298824-d2f0-4b94-84ce-8a3bd2ec63d6/volumes"
Sep 10 19:43:33.384807 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:33.384795    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bd267769-63da-4a57-b2bc-0d1e0d797c19" path="/var/lib/kubelet/pods/bd267769-63da-4a57-b2bc-0d1e0d797c19/volumes"
Sep 10 19:43:37.456707 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:37.456670    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:43:38.248108 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:38.248071    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:43:43.163561 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:43:43.163527    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qr7rl"]
Sep 10 19:44:14.558989 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:14.558952    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:44:15.462717 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:15.462679    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:44:21.414288 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:21.414252    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:44:27.378752 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:27.378711    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:44:27.452780 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:27.452743    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:44:27.658729 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:27.658657    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-df863b-openshift-default-84d4c7f644-8wtmd"]
Sep 10 19:44:27.658885 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:27.658863    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-df863b-openshift-default-84d4c7f644-8wtmd" podUID="82bd2e04-378d-4f6f-92b5-205d1890d4cb" containerName="istio-proxy" containerID="cri-o://f7354979cffd8bb07284327dabe028563962e1be7798909eccc51a4e6baf9442" gracePeriod=30
Sep 10 19:44:27.674490 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:27.674456    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:44:28.030763 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:28.030732    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-3765cf-gcwqt"]
Sep 10 19:44:28.031153 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:28.031139    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d298824-d2f0-4b94-84ce-8a3bd2ec63d6" containerName="revoke-keys"
Sep 10 19:44:28.031198 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:28.031157    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d298824-d2f0-4b94-84ce-8a3bd2ec63d6" containerName="revoke-keys"
Sep 10 19:44:28.031198 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:28.031169    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea113d18-69c9-465d-affe-eae633782b80" containerName="istio-proxy"
Sep 10 19:44:28.031198 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:28.031175    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea113d18-69c9-465d-affe-eae633782b80" containerName="istio-proxy"
Sep 10 19:44:28.031198 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:28.031182    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd267769-63da-4a57-b2bc-0d1e0d797c19" containerName="payload-pre-processing"
Sep 10 19:44:28.031198 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:28.031187    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd267769-63da-4a57-b2bc-0d1e0d797c19" containerName="payload-pre-processing"
Sep 10 19:44:28.031426 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:28.031200    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea113d18-69c9-465d-affe-eae633782b80" containerName="istio-proxy"
Sep 10 19:44:28.031426 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:28.031206    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea113d18-69c9-465d-affe-eae633782b80" containerName="istio-proxy"
Sep 10 19:44:28.031426 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:28.031274    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="ea113d18-69c9-465d-affe-eae633782b80" containerName="istio-proxy"
Sep 10 19:44:28.031426 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:28.031282    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="5d298824-d2f0-4b94-84ce-8a3bd2ec63d6" containerName="revoke-keys"
Sep 10 19:44:28.031426 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:28.031290    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd267769-63da-4a57-b2bc-0d1e0d797c19" containerName="payload-pre-processing"
Sep 10 19:44:28.034517 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:28.034499    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-3765cf-gcwqt"
Sep 10 19:44:28.037205 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:28.037175    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 19:44:28.037314 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:28.037292    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wkgt8\""
Sep 10 19:44:28.045196 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:28.045175    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d3e49bf-d326-453a-98ac-9e14e7fb81c6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-3765cf-gcwqt\" (UID: \"5d3e49bf-d326-453a-98ac-9e14e7fb81c6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-3765cf-gcwqt"
Sep 10 19:44:28.050732 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:28.050713    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-3765cf-gcwqt"]
Sep 10 19:44:28.146325 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:28.146293    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d3e49bf-d326-453a-98ac-9e14e7fb81c6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-3765cf-gcwqt\" (UID: \"5d3e49bf-d326-453a-98ac-9e14e7fb81c6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-3765cf-gcwqt"
Sep 10 19:44:28.146914 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:28.146895    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d3e49bf-d326-453a-98ac-9e14e7fb81c6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-3765cf-gcwqt\" (UID: \"5d3e49bf-d326-453a-98ac-9e14e7fb81c6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-3765cf-gcwqt"
Sep 10 19:44:28.179117 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:28.179084    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:44:28.344438 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:28.344362    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-3765cf-gcwqt"
Sep 10 19:44:28.622731 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:28.622706    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-3765cf-gcwqt"]
Sep 10 19:44:28.624893 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:44:28.624859    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d3e49bf_d326_453a_98ac_9e14e7fb81c6.slice/crio-c741d7d198e2e71a2dcfb2e605958d578bc2edb807e184ba350cb48e60e82a53 WatchSource:0}: Error finding container c741d7d198e2e71a2dcfb2e605958d578bc2edb807e184ba350cb48e60e82a53: Status 404 returned error can't find the container with id c741d7d198e2e71a2dcfb2e605958d578bc2edb807e184ba350cb48e60e82a53
Sep 10 19:44:29.245442 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:29.245406    2575 generic.go:356] "Generic (PLEG): container finished" podID="5d3e49bf-d326-453a-98ac-9e14e7fb81c6" containerID="84fda2327c20a2a97205b16f0c769e7e1e4072598e78cddaa7abdb8486974111" exitCode=0
Sep 10 19:44:29.245561 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:29.245487    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-3765cf-gcwqt" event={"ID":"5d3e49bf-d326-453a-98ac-9e14e7fb81c6","Type":"ContainerDied","Data":"84fda2327c20a2a97205b16f0c769e7e1e4072598e78cddaa7abdb8486974111"}
Sep 10 19:44:29.245561 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:29.245546    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-3765cf-gcwqt" event={"ID":"5d3e49bf-d326-453a-98ac-9e14e7fb81c6","Type":"ContainerStarted","Data":"c741d7d198e2e71a2dcfb2e605958d578bc2edb807e184ba350cb48e60e82a53"}
Sep 10 19:44:30.380785 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:30.380763    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-3765cf-gcwqt"
Sep 10 19:44:30.467837 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:30.467811    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d3e49bf-d326-453a-98ac-9e14e7fb81c6-maas-api-service-ca\") pod \"5d3e49bf-d326-453a-98ac-9e14e7fb81c6\" (UID: \"5d3e49bf-d326-453a-98ac-9e14e7fb81c6\") "
Sep 10 19:44:30.468431 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:30.468150    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5d3e49bf-d326-453a-98ac-9e14e7fb81c6-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5d3e49bf-d326-453a-98ac-9e14e7fb81c6" (UID: "5d3e49bf-d326-453a-98ac-9e14e7fb81c6"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:44:30.468588 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:30.468542    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d3e49bf-d326-453a-98ac-9e14e7fb81c6-maas-api-service-ca\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:44:31.256691 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:31.256663    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-3765cf-gcwqt"
Sep 10 19:44:31.256861 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:31.256693    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-3765cf-gcwqt" event={"ID":"5d3e49bf-d326-453a-98ac-9e14e7fb81c6","Type":"ContainerDied","Data":"c741d7d198e2e71a2dcfb2e605958d578bc2edb807e184ba350cb48e60e82a53"}
Sep 10 19:44:31.256861 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:31.256733    2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c741d7d198e2e71a2dcfb2e605958d578bc2edb807e184ba350cb48e60e82a53"
Sep 10 19:44:31.401355 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:31.401307    2575 patch_prober.go:28] interesting pod/e2e-ipp-a-df863b-openshift-default-84d4c7f644-8wtmd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 10 19:44:31.401696 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:31.401372    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-a-df863b-openshift-default-84d4c7f644-8wtmd" podUID="82bd2e04-378d-4f6f-92b5-205d1890d4cb" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 10 19:44:32.899731 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:32.899704    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-df863b-openshift-default-84d4c7f644-8wtmd"
Sep 10 19:44:32.988576 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:32.988549    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/82bd2e04-378d-4f6f-92b5-205d1890d4cb-istio-envoy\") pod \"82bd2e04-378d-4f6f-92b5-205d1890d4cb\" (UID: \"82bd2e04-378d-4f6f-92b5-205d1890d4cb\") "
Sep 10 19:44:32.988711 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:32.988593    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/82bd2e04-378d-4f6f-92b5-205d1890d4cb-workload-socket\") pod \"82bd2e04-378d-4f6f-92b5-205d1890d4cb\" (UID: \"82bd2e04-378d-4f6f-92b5-205d1890d4cb\") "
Sep 10 19:44:32.988711 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:32.988629    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cknpb\" (UniqueName: \"kubernetes.io/projected/82bd2e04-378d-4f6f-92b5-205d1890d4cb-kube-api-access-cknpb\") pod \"82bd2e04-378d-4f6f-92b5-205d1890d4cb\" (UID: \"82bd2e04-378d-4f6f-92b5-205d1890d4cb\") "
Sep 10 19:44:32.988711 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:32.988649    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/82bd2e04-378d-4f6f-92b5-205d1890d4cb-istiod-ca-cert\") pod \"82bd2e04-378d-4f6f-92b5-205d1890d4cb\" (UID: \"82bd2e04-378d-4f6f-92b5-205d1890d4cb\") "
Sep 10 19:44:32.988711 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:32.988669    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/82bd2e04-378d-4f6f-92b5-205d1890d4cb-istio-token\") pod \"82bd2e04-378d-4f6f-92b5-205d1890d4cb\" (UID: \"82bd2e04-378d-4f6f-92b5-205d1890d4cb\") "
Sep 10 19:44:32.988711 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:32.988694    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/82bd2e04-378d-4f6f-92b5-205d1890d4cb-credential-socket\") pod \"82bd2e04-378d-4f6f-92b5-205d1890d4cb\" (UID: \"82bd2e04-378d-4f6f-92b5-205d1890d4cb\") "
Sep 10 19:44:32.988974 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:32.988763    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/82bd2e04-378d-4f6f-92b5-205d1890d4cb-istio-podinfo\") pod \"82bd2e04-378d-4f6f-92b5-205d1890d4cb\" (UID: \"82bd2e04-378d-4f6f-92b5-205d1890d4cb\") "
Sep 10 19:44:32.988974 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:32.988798    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/82bd2e04-378d-4f6f-92b5-205d1890d4cb-istio-data\") pod \"82bd2e04-378d-4f6f-92b5-205d1890d4cb\" (UID: \"82bd2e04-378d-4f6f-92b5-205d1890d4cb\") "
Sep 10 19:44:32.988974 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:32.988825    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/82bd2e04-378d-4f6f-92b5-205d1890d4cb-workload-certs\") pod \"82bd2e04-378d-4f6f-92b5-205d1890d4cb\" (UID: \"82bd2e04-378d-4f6f-92b5-205d1890d4cb\") "
Sep 10 19:44:32.988974 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:32.988967    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/82bd2e04-378d-4f6f-92b5-205d1890d4cb-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "82bd2e04-378d-4f6f-92b5-205d1890d4cb" (UID: "82bd2e04-378d-4f6f-92b5-205d1890d4cb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:44:32.989188 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:32.989148    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/82bd2e04-378d-4f6f-92b5-205d1890d4cb-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "82bd2e04-378d-4f6f-92b5-205d1890d4cb" (UID: "82bd2e04-378d-4f6f-92b5-205d1890d4cb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:44:32.989188 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:32.989076    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/82bd2e04-378d-4f6f-92b5-205d1890d4cb-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "82bd2e04-378d-4f6f-92b5-205d1890d4cb" (UID: "82bd2e04-378d-4f6f-92b5-205d1890d4cb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:44:32.989188 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:32.989170    2575 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/82bd2e04-378d-4f6f-92b5-205d1890d4cb-workload-socket\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:44:32.989368 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:32.989246    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/82bd2e04-378d-4f6f-92b5-205d1890d4cb-istio-data" (OuterVolumeSpecName: "istio-data") pod "82bd2e04-378d-4f6f-92b5-205d1890d4cb" (UID: "82bd2e04-378d-4f6f-92b5-205d1890d4cb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:44:32.989368 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:32.989308    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/82bd2e04-378d-4f6f-92b5-205d1890d4cb-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "82bd2e04-378d-4f6f-92b5-205d1890d4cb" (UID: "82bd2e04-378d-4f6f-92b5-205d1890d4cb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:44:32.990912 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:32.990890    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/82bd2e04-378d-4f6f-92b5-205d1890d4cb-kube-api-access-cknpb" (OuterVolumeSpecName: "kube-api-access-cknpb") pod "82bd2e04-378d-4f6f-92b5-205d1890d4cb" (UID: "82bd2e04-378d-4f6f-92b5-205d1890d4cb"). InnerVolumeSpecName "kube-api-access-cknpb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:44:32.991020 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:32.990913    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/82bd2e04-378d-4f6f-92b5-205d1890d4cb-istio-token" (OuterVolumeSpecName: "istio-token") pod "82bd2e04-378d-4f6f-92b5-205d1890d4cb" (UID: "82bd2e04-378d-4f6f-92b5-205d1890d4cb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:44:32.991172 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:32.991149    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/82bd2e04-378d-4f6f-92b5-205d1890d4cb-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "82bd2e04-378d-4f6f-92b5-205d1890d4cb" (UID: "82bd2e04-378d-4f6f-92b5-205d1890d4cb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 19:44:32.991172 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:32.991154    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/82bd2e04-378d-4f6f-92b5-205d1890d4cb-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "82bd2e04-378d-4f6f-92b5-205d1890d4cb" (UID: "82bd2e04-378d-4f6f-92b5-205d1890d4cb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:44:33.090471 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:33.090449    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cknpb\" (UniqueName: \"kubernetes.io/projected/82bd2e04-378d-4f6f-92b5-205d1890d4cb-kube-api-access-cknpb\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:44:33.090471 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:33.090469    2575 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/82bd2e04-378d-4f6f-92b5-205d1890d4cb-istiod-ca-cert\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:44:33.090607 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:33.090479    2575 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/82bd2e04-378d-4f6f-92b5-205d1890d4cb-istio-token\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:44:33.090607 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:33.090487    2575 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/82bd2e04-378d-4f6f-92b5-205d1890d4cb-credential-socket\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:44:33.090607 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:33.090495    2575 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/82bd2e04-378d-4f6f-92b5-205d1890d4cb-istio-podinfo\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:44:33.090607 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:33.090503    2575 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/82bd2e04-378d-4f6f-92b5-205d1890d4cb-istio-data\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:44:33.090607 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:33.090511    2575 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/82bd2e04-378d-4f6f-92b5-205d1890d4cb-workload-certs\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:44:33.090607 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:33.090519    2575 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/82bd2e04-378d-4f6f-92b5-205d1890d4cb-istio-envoy\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:44:33.267690 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:33.267621    2575 generic.go:356] "Generic (PLEG): container finished" podID="82bd2e04-378d-4f6f-92b5-205d1890d4cb" containerID="f7354979cffd8bb07284327dabe028563962e1be7798909eccc51a4e6baf9442" exitCode=0
Sep 10 19:44:33.267690 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:33.267677    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-df863b-openshift-default-84d4c7f644-8wtmd"
Sep 10 19:44:33.267882 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:33.267703    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-df863b-openshift-default-84d4c7f644-8wtmd" event={"ID":"82bd2e04-378d-4f6f-92b5-205d1890d4cb","Type":"ContainerDied","Data":"f7354979cffd8bb07284327dabe028563962e1be7798909eccc51a4e6baf9442"}
Sep 10 19:44:33.267882 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:33.267784    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-df863b-openshift-default-84d4c7f644-8wtmd" event={"ID":"82bd2e04-378d-4f6f-92b5-205d1890d4cb","Type":"ContainerDied","Data":"a76f754534f10187f3fa89e1ef262e32f80f5efb95874be94d6d1747777733d4"}
Sep 10 19:44:33.267882 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:33.267813    2575 scope.go:117] "RemoveContainer" containerID="f7354979cffd8bb07284327dabe028563962e1be7798909eccc51a4e6baf9442"
Sep 10 19:44:33.276851 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:33.276837    2575 scope.go:117] "RemoveContainer" containerID="baa5ab60567a0e9b5556f7adc0499b692434d418a00abd7fdfb64a9108937c6e"
Sep 10 19:44:33.284650 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:33.284634    2575 scope.go:117] "RemoveContainer" containerID="f7354979cffd8bb07284327dabe028563962e1be7798909eccc51a4e6baf9442"
Sep 10 19:44:33.284883 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:44:33.284868    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f7354979cffd8bb07284327dabe028563962e1be7798909eccc51a4e6baf9442\": container with ID starting with f7354979cffd8bb07284327dabe028563962e1be7798909eccc51a4e6baf9442 not found: ID does not exist" containerID="f7354979cffd8bb07284327dabe028563962e1be7798909eccc51a4e6baf9442"
Sep 10 19:44:33.284929 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:33.284889    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f7354979cffd8bb07284327dabe028563962e1be7798909eccc51a4e6baf9442"} err="failed to get container status \"f7354979cffd8bb07284327dabe028563962e1be7798909eccc51a4e6baf9442\": rpc error: code = NotFound desc = could not find container \"f7354979cffd8bb07284327dabe028563962e1be7798909eccc51a4e6baf9442\": container with ID starting with f7354979cffd8bb07284327dabe028563962e1be7798909eccc51a4e6baf9442 not found: ID does not exist"
Sep 10 19:44:33.284929 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:33.284903    2575 scope.go:117] "RemoveContainer" containerID="baa5ab60567a0e9b5556f7adc0499b692434d418a00abd7fdfb64a9108937c6e"
Sep 10 19:44:33.285133 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:44:33.285115    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"baa5ab60567a0e9b5556f7adc0499b692434d418a00abd7fdfb64a9108937c6e\": container with ID starting with baa5ab60567a0e9b5556f7adc0499b692434d418a00abd7fdfb64a9108937c6e not found: ID does not exist" containerID="baa5ab60567a0e9b5556f7adc0499b692434d418a00abd7fdfb64a9108937c6e"
Sep 10 19:44:33.285177 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:33.285139    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"baa5ab60567a0e9b5556f7adc0499b692434d418a00abd7fdfb64a9108937c6e"} err="failed to get container status \"baa5ab60567a0e9b5556f7adc0499b692434d418a00abd7fdfb64a9108937c6e\": rpc error: code = NotFound desc = could not find container \"baa5ab60567a0e9b5556f7adc0499b692434d418a00abd7fdfb64a9108937c6e\": container with ID starting with baa5ab60567a0e9b5556f7adc0499b692434d418a00abd7fdfb64a9108937c6e not found: ID does not exist"
Sep 10 19:44:33.288367 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:33.288332    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-df863b-openshift-default-84d4c7f644-8wtmd"]
Sep 10 19:44:33.291985 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:33.291966    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-df863b-openshift-default-84d4c7f644-8wtmd"]
Sep 10 19:44:33.384000 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:33.383972    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="82bd2e04-378d-4f6f-92b5-205d1890d4cb" path="/var/lib/kubelet/pods/82bd2e04-378d-4f6f-92b5-205d1890d4cb/volumes"
Sep 10 19:44:38.579316 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:38.579280    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:44:38.648704 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:38.648673    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:44:38.715962 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:38.715922    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:44:38.792507 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:38.792473    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-3765cf-gcwqt"]
Sep 10 19:44:38.796880 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:38.796841    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:44:38.797095 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:38.797078    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-3765cf-gcwqt"]
Sep 10 19:44:38.860372 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:38.860271    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:44:39.186613 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:39.186527    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:44:39.388945 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:39.388913    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5d3e49bf-d326-453a-98ac-9e14e7fb81c6" path="/var/lib/kubelet/pods/5d3e49bf-d326-453a-98ac-9e14e7fb81c6/volumes"
Sep 10 19:44:41.754254 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:41.754216    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qr7rl"]
Sep 10 19:44:45.084385 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:45.084321    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:44:45.183266 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:45.183232    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:44:45.332482 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:45.332442    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-3765cf-openshift-default-5994cd68fb-42qbw"]
Sep 10 19:44:45.332699 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:45.332674    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-3765cf-openshift-default-5994cd68fb-42qbw" podUID="b9597ee5-e04a-417f-8b4e-274e52bdf9e7" containerName="istio-proxy" containerID="cri-o://fec752f2c798928f6cfb53eff7079ab86b22117555e2936216ed037858b33e29" gracePeriod=30
Sep 10 19:44:45.358518 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:45.358443    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:44:45.730862 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:45.730833    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-14a8d1-lk8pw"]
Sep 10 19:44:45.731213 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:45.731201    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d3e49bf-d326-453a-98ac-9e14e7fb81c6" containerName="revoke-keys"
Sep 10 19:44:45.731213 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:45.731215    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d3e49bf-d326-453a-98ac-9e14e7fb81c6" containerName="revoke-keys"
Sep 10 19:44:45.731374 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:45.731232    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="82bd2e04-378d-4f6f-92b5-205d1890d4cb" containerName="istio-proxy"
Sep 10 19:44:45.731374 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:45.731238    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="82bd2e04-378d-4f6f-92b5-205d1890d4cb" containerName="istio-proxy"
Sep 10 19:44:45.731374 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:45.731245    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="82bd2e04-378d-4f6f-92b5-205d1890d4cb" containerName="istio-proxy"
Sep 10 19:44:45.731374 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:45.731251    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="82bd2e04-378d-4f6f-92b5-205d1890d4cb" containerName="istio-proxy"
Sep 10 19:44:45.731374 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:45.731312    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="5d3e49bf-d326-453a-98ac-9e14e7fb81c6" containerName="revoke-keys"
Sep 10 19:44:45.731374 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:45.731321    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="ea113d18-69c9-465d-affe-eae633782b80" containerName="istio-proxy"
Sep 10 19:44:45.731374 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:45.731329    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="82bd2e04-378d-4f6f-92b5-205d1890d4cb" containerName="istio-proxy"
Sep 10 19:44:45.731374 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:45.731349    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="82bd2e04-378d-4f6f-92b5-205d1890d4cb" containerName="istio-proxy"
Sep 10 19:44:45.735833 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:45.735817    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-14a8d1-lk8pw"
Sep 10 19:44:45.738258 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:45.738227    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wkgt8\""
Sep 10 19:44:45.738498 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:45.738467    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 19:44:45.746526 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:45.746502    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-14a8d1-lk8pw"]
Sep 10 19:44:45.791839 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:45.791809    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bc0e9da2-5aa3-4fce-8a9a-e4804bb73b80-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-14a8d1-lk8pw\" (UID: \"bc0e9da2-5aa3-4fce-8a9a-e4804bb73b80\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-14a8d1-lk8pw"
Sep 10 19:44:45.893045 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:45.893021    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bc0e9da2-5aa3-4fce-8a9a-e4804bb73b80-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-14a8d1-lk8pw\" (UID: \"bc0e9da2-5aa3-4fce-8a9a-e4804bb73b80\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-14a8d1-lk8pw"
Sep 10 19:44:45.893630 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:45.893611    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bc0e9da2-5aa3-4fce-8a9a-e4804bb73b80-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-14a8d1-lk8pw\" (UID: \"bc0e9da2-5aa3-4fce-8a9a-e4804bb73b80\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-14a8d1-lk8pw"
Sep 10 19:44:45.923419 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:45.923394    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:44:46.046879 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:46.046803    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-14a8d1-lk8pw"
Sep 10 19:44:46.316670 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:46.316645    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-14a8d1-lk8pw"]
Sep 10 19:44:46.318808 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:44:46.318776    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc0e9da2_5aa3_4fce_8a9a_e4804bb73b80.slice/crio-3c4fa7ecbb3446768006705d5d30f64b383e1b985681a5fa1a8fc82115a4a849 WatchSource:0}: Error finding container 3c4fa7ecbb3446768006705d5d30f64b383e1b985681a5fa1a8fc82115a4a849: Status 404 returned error can't find the container with id 3c4fa7ecbb3446768006705d5d30f64b383e1b985681a5fa1a8fc82115a4a849
Sep 10 19:44:46.337661 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:46.337634    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-14a8d1-lk8pw" event={"ID":"bc0e9da2-5aa3-4fce-8a9a-e4804bb73b80","Type":"ContainerStarted","Data":"3c4fa7ecbb3446768006705d5d30f64b383e1b985681a5fa1a8fc82115a4a849"}
Sep 10 19:44:47.350464 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:47.350432    2575 generic.go:356] "Generic (PLEG): container finished" podID="bc0e9da2-5aa3-4fce-8a9a-e4804bb73b80" containerID="98312b961cbf0779e60d0853a0f58403205a2ad5cd0f9c9167c2abbe10e45af3" exitCode=0
Sep 10 19:44:47.350813 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:47.350546    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-14a8d1-lk8pw" event={"ID":"bc0e9da2-5aa3-4fce-8a9a-e4804bb73b80","Type":"ContainerDied","Data":"98312b961cbf0779e60d0853a0f58403205a2ad5cd0f9c9167c2abbe10e45af3"}
Sep 10 19:44:48.250386 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:48.250336    2575 patch_prober.go:28] interesting pod/e2e-ipp-b-3765cf-openshift-default-5994cd68fb-42qbw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 10 19:44:48.250553 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:48.250401    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-b-3765cf-openshift-default-5994cd68fb-42qbw" podUID="b9597ee5-e04a-417f-8b4e-274e52bdf9e7" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 10 19:44:48.485295 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:48.485273    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-14a8d1-lk8pw"
Sep 10 19:44:48.615050 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:48.614974    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bc0e9da2-5aa3-4fce-8a9a-e4804bb73b80-maas-api-service-ca\") pod \"bc0e9da2-5aa3-4fce-8a9a-e4804bb73b80\" (UID: \"bc0e9da2-5aa3-4fce-8a9a-e4804bb73b80\") "
Sep 10 19:44:48.615296 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:48.615275    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bc0e9da2-5aa3-4fce-8a9a-e4804bb73b80-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "bc0e9da2-5aa3-4fce-8a9a-e4804bb73b80" (UID: "bc0e9da2-5aa3-4fce-8a9a-e4804bb73b80"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:44:48.716538 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:48.716511    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bc0e9da2-5aa3-4fce-8a9a-e4804bb73b80-maas-api-service-ca\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:44:49.361312 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:49.361285    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-14a8d1-lk8pw"
Sep 10 19:44:49.361512 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:49.361288    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-14a8d1-lk8pw" event={"ID":"bc0e9da2-5aa3-4fce-8a9a-e4804bb73b80","Type":"ContainerDied","Data":"3c4fa7ecbb3446768006705d5d30f64b383e1b985681a5fa1a8fc82115a4a849"}
Sep 10 19:44:49.361512 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:49.361403    2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3c4fa7ecbb3446768006705d5d30f64b383e1b985681a5fa1a8fc82115a4a849"
Sep 10 19:44:50.586269 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:50.586236    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-3765cf-openshift-default-5994cd68fb-42qbw"
Sep 10 19:44:50.631439 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:50.631371    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b9597ee5-e04a-417f-8b4e-274e52bdf9e7-istio-envoy\") pod \"b9597ee5-e04a-417f-8b4e-274e52bdf9e7\" (UID: \"b9597ee5-e04a-417f-8b4e-274e52bdf9e7\") "
Sep 10 19:44:50.631439 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:50.631406    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9597ee5-e04a-417f-8b4e-274e52bdf9e7-credential-socket\") pod \"b9597ee5-e04a-417f-8b4e-274e52bdf9e7\" (UID: \"b9597ee5-e04a-417f-8b4e-274e52bdf9e7\") "
Sep 10 19:44:50.631439 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:50.631426    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gjnbl\" (UniqueName: \"kubernetes.io/projected/b9597ee5-e04a-417f-8b4e-274e52bdf9e7-kube-api-access-gjnbl\") pod \"b9597ee5-e04a-417f-8b4e-274e52bdf9e7\" (UID: \"b9597ee5-e04a-417f-8b4e-274e52bdf9e7\") "
Sep 10 19:44:50.631704 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:50.631451    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b9597ee5-e04a-417f-8b4e-274e52bdf9e7-istio-data\") pod \"b9597ee5-e04a-417f-8b4e-274e52bdf9e7\" (UID: \"b9597ee5-e04a-417f-8b4e-274e52bdf9e7\") "
Sep 10 19:44:50.631704 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:50.631537    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b9597ee5-e04a-417f-8b4e-274e52bdf9e7-istio-token\") pod \"b9597ee5-e04a-417f-8b4e-274e52bdf9e7\" (UID: \"b9597ee5-e04a-417f-8b4e-274e52bdf9e7\") "
Sep 10 19:44:50.631704 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:50.631575    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b9597ee5-e04a-417f-8b4e-274e52bdf9e7-istio-podinfo\") pod \"b9597ee5-e04a-417f-8b4e-274e52bdf9e7\" (UID: \"b9597ee5-e04a-417f-8b4e-274e52bdf9e7\") "
Sep 10 19:44:50.631870 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:50.631710    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b9597ee5-e04a-417f-8b4e-274e52bdf9e7-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b9597ee5-e04a-417f-8b4e-274e52bdf9e7" (UID: "b9597ee5-e04a-417f-8b4e-274e52bdf9e7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:44:50.631870 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:50.631748    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b9597ee5-e04a-417f-8b4e-274e52bdf9e7-istio-data" (OuterVolumeSpecName: "istio-data") pod "b9597ee5-e04a-417f-8b4e-274e52bdf9e7" (UID: "b9597ee5-e04a-417f-8b4e-274e52bdf9e7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:44:50.631989 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:50.631871    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b9597ee5-e04a-417f-8b4e-274e52bdf9e7-istiod-ca-cert\") pod \"b9597ee5-e04a-417f-8b4e-274e52bdf9e7\" (UID: \"b9597ee5-e04a-417f-8b4e-274e52bdf9e7\") "
Sep 10 19:44:50.631989 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:50.631906    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b9597ee5-e04a-417f-8b4e-274e52bdf9e7-workload-certs\") pod \"b9597ee5-e04a-417f-8b4e-274e52bdf9e7\" (UID: \"b9597ee5-e04a-417f-8b4e-274e52bdf9e7\") "
Sep 10 19:44:50.631989 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:50.631942    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9597ee5-e04a-417f-8b4e-274e52bdf9e7-workload-socket\") pod \"b9597ee5-e04a-417f-8b4e-274e52bdf9e7\" (UID: \"b9597ee5-e04a-417f-8b4e-274e52bdf9e7\") "
Sep 10 19:44:50.632213 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:50.632190    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b9597ee5-e04a-417f-8b4e-274e52bdf9e7-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b9597ee5-e04a-417f-8b4e-274e52bdf9e7" (UID: "b9597ee5-e04a-417f-8b4e-274e52bdf9e7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:44:50.632289 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:50.632198    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b9597ee5-e04a-417f-8b4e-274e52bdf9e7-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b9597ee5-e04a-417f-8b4e-274e52bdf9e7" (UID: "b9597ee5-e04a-417f-8b4e-274e52bdf9e7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:44:50.632378 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:50.632356    2575 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b9597ee5-e04a-417f-8b4e-274e52bdf9e7-workload-certs\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:44:50.632446 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:50.632380    2575 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9597ee5-e04a-417f-8b4e-274e52bdf9e7-credential-socket\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:44:50.632446 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:50.632383    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b9597ee5-e04a-417f-8b4e-274e52bdf9e7-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b9597ee5-e04a-417f-8b4e-274e52bdf9e7" (UID: "b9597ee5-e04a-417f-8b4e-274e52bdf9e7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:44:50.632446 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:50.632396    2575 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b9597ee5-e04a-417f-8b4e-274e52bdf9e7-istio-data\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:44:50.633764 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:50.633732    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b9597ee5-e04a-417f-8b4e-274e52bdf9e7-istio-token" (OuterVolumeSpecName: "istio-token") pod "b9597ee5-e04a-417f-8b4e-274e52bdf9e7" (UID: "b9597ee5-e04a-417f-8b4e-274e52bdf9e7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:44:50.633864 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:50.633763    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b9597ee5-e04a-417f-8b4e-274e52bdf9e7-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b9597ee5-e04a-417f-8b4e-274e52bdf9e7" (UID: "b9597ee5-e04a-417f-8b4e-274e52bdf9e7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:44:50.634032 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:50.634017    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b9597ee5-e04a-417f-8b4e-274e52bdf9e7-kube-api-access-gjnbl" (OuterVolumeSpecName: "kube-api-access-gjnbl") pod "b9597ee5-e04a-417f-8b4e-274e52bdf9e7" (UID: "b9597ee5-e04a-417f-8b4e-274e52bdf9e7"). InnerVolumeSpecName "kube-api-access-gjnbl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:44:50.634213 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:50.634199    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b9597ee5-e04a-417f-8b4e-274e52bdf9e7-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b9597ee5-e04a-417f-8b4e-274e52bdf9e7" (UID: "b9597ee5-e04a-417f-8b4e-274e52bdf9e7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 19:44:50.733204 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:50.733159    2575 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b9597ee5-e04a-417f-8b4e-274e52bdf9e7-istio-token\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:44:50.733204 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:50.733203    2575 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b9597ee5-e04a-417f-8b4e-274e52bdf9e7-istio-podinfo\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:44:50.733204 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:50.733215    2575 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b9597ee5-e04a-417f-8b4e-274e52bdf9e7-istiod-ca-cert\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:44:50.733449 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:50.733228    2575 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9597ee5-e04a-417f-8b4e-274e52bdf9e7-workload-socket\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:44:50.733449 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:50.733240    2575 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b9597ee5-e04a-417f-8b4e-274e52bdf9e7-istio-envoy\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:44:50.733449 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:50.733252    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gjnbl\" (UniqueName: \"kubernetes.io/projected/b9597ee5-e04a-417f-8b4e-274e52bdf9e7-kube-api-access-gjnbl\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:44:51.374657 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:51.374622    2575 generic.go:356] "Generic (PLEG): container finished" podID="b9597ee5-e04a-417f-8b4e-274e52bdf9e7" containerID="fec752f2c798928f6cfb53eff7079ab86b22117555e2936216ed037858b33e29" exitCode=0
Sep 10 19:44:51.374861 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:51.374683    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-3765cf-openshift-default-5994cd68fb-42qbw"
Sep 10 19:44:51.374861 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:51.374690    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-3765cf-openshift-default-5994cd68fb-42qbw" event={"ID":"b9597ee5-e04a-417f-8b4e-274e52bdf9e7","Type":"ContainerDied","Data":"fec752f2c798928f6cfb53eff7079ab86b22117555e2936216ed037858b33e29"}
Sep 10 19:44:51.374861 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:51.374731    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-3765cf-openshift-default-5994cd68fb-42qbw" event={"ID":"b9597ee5-e04a-417f-8b4e-274e52bdf9e7","Type":"ContainerDied","Data":"179aedcd93e79bd5fa546e4f27ea8bb4758d4b1333c1b0f68c9a39ca53a9f453"}
Sep 10 19:44:51.374861 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:51.374752    2575 scope.go:117] "RemoveContainer" containerID="fec752f2c798928f6cfb53eff7079ab86b22117555e2936216ed037858b33e29"
Sep 10 19:44:51.384394 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:51.384374    2575 scope.go:117] "RemoveContainer" containerID="a831a3a99298950772b341033e782633264ec38f344a2ecebdf397edc77c2818"
Sep 10 19:44:51.392627 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:51.392610    2575 scope.go:117] "RemoveContainer" containerID="fec752f2c798928f6cfb53eff7079ab86b22117555e2936216ed037858b33e29"
Sep 10 19:44:51.392875 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:44:51.392853    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fec752f2c798928f6cfb53eff7079ab86b22117555e2936216ed037858b33e29\": container with ID starting with fec752f2c798928f6cfb53eff7079ab86b22117555e2936216ed037858b33e29 not found: ID does not exist" containerID="fec752f2c798928f6cfb53eff7079ab86b22117555e2936216ed037858b33e29"
Sep 10 19:44:51.392943 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:51.392887    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fec752f2c798928f6cfb53eff7079ab86b22117555e2936216ed037858b33e29"} err="failed to get container status \"fec752f2c798928f6cfb53eff7079ab86b22117555e2936216ed037858b33e29\": rpc error: code = NotFound desc = could not find container \"fec752f2c798928f6cfb53eff7079ab86b22117555e2936216ed037858b33e29\": container with ID starting with fec752f2c798928f6cfb53eff7079ab86b22117555e2936216ed037858b33e29 not found: ID does not exist"
Sep 10 19:44:51.392943 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:51.392911    2575 scope.go:117] "RemoveContainer" containerID="a831a3a99298950772b341033e782633264ec38f344a2ecebdf397edc77c2818"
Sep 10 19:44:51.393144 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:44:51.393127    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a831a3a99298950772b341033e782633264ec38f344a2ecebdf397edc77c2818\": container with ID starting with a831a3a99298950772b341033e782633264ec38f344a2ecebdf397edc77c2818 not found: ID does not exist" containerID="a831a3a99298950772b341033e782633264ec38f344a2ecebdf397edc77c2818"
Sep 10 19:44:51.393187 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:51.393150    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a831a3a99298950772b341033e782633264ec38f344a2ecebdf397edc77c2818"} err="failed to get container status \"a831a3a99298950772b341033e782633264ec38f344a2ecebdf397edc77c2818\": rpc error: code = NotFound desc = could not find container \"a831a3a99298950772b341033e782633264ec38f344a2ecebdf397edc77c2818\": container with ID starting with a831a3a99298950772b341033e782633264ec38f344a2ecebdf397edc77c2818 not found: ID does not exist"
Sep 10 19:44:51.397071 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:51.397049    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-3765cf-openshift-default-5994cd68fb-42qbw"]
Sep 10 19:44:51.401042 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:51.401019    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-3765cf-openshift-default-5994cd68fb-42qbw"]
Sep 10 19:44:53.384493 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:53.384461    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b9597ee5-e04a-417f-8b4e-274e52bdf9e7" path="/var/lib/kubelet/pods/b9597ee5-e04a-417f-8b4e-274e52bdf9e7/volumes"
Sep 10 19:44:56.367072 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:56.367037    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:44:56.468727 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:56.468693    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:44:56.568976 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:56.568935    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:44:56.616033 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:56.616000    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-14a8d1-lk8pw"]
Sep 10 19:44:56.624224 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:56.624153    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-14a8d1-lk8pw"]
Sep 10 19:44:56.669696 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:56.669667    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:44:56.996430 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:56.996396    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:44:57.384775 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:57.384699    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bc0e9da2-5aa3-4fce-8a9a-e4804bb73b80" path="/var/lib/kubelet/pods/bc0e9da2-5aa3-4fce-8a9a-e4804bb73b80/volumes"
Sep 10 19:44:59.259791 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:44:59.259761    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qr7rl"]
Sep 10 19:45:00.142919 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:45:00.142884    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29817825-j52nd"]
Sep 10 19:45:00.143360 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:45:00.143331    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bc0e9da2-5aa3-4fce-8a9a-e4804bb73b80" containerName="revoke-keys"
Sep 10 19:45:00.143360 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:45:00.143360    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="bc0e9da2-5aa3-4fce-8a9a-e4804bb73b80" containerName="revoke-keys"
Sep 10 19:45:00.143451 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:45:00.143373    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b9597ee5-e04a-417f-8b4e-274e52bdf9e7" containerName="istio-proxy"
Sep 10 19:45:00.143451 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:45:00.143380    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="b9597ee5-e04a-417f-8b4e-274e52bdf9e7" containerName="istio-proxy"
Sep 10 19:45:00.143451 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:45:00.143389    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b9597ee5-e04a-417f-8b4e-274e52bdf9e7" containerName="istio-proxy"
Sep 10 19:45:00.143451 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:45:00.143398    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="b9597ee5-e04a-417f-8b4e-274e52bdf9e7" containerName="istio-proxy"
Sep 10 19:45:00.143574 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:45:00.143459    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="b9597ee5-e04a-417f-8b4e-274e52bdf9e7" containerName="istio-proxy"
Sep 10 19:45:00.143574 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:45:00.143471    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="bc0e9da2-5aa3-4fce-8a9a-e4804bb73b80" containerName="revoke-keys"
Sep 10 19:45:00.148139 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:45:00.148117    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817825-j52nd"
Sep 10 19:45:00.150386 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:45:00.150367    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wkgt8\""
Sep 10 19:45:00.156081 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:45:00.156049    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29817825-j52nd"]
Sep 10 19:45:00.159739 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:45:00.159714    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817825-j52nd"
Sep 10 19:45:00.435428 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:45:00.435395    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29817825-j52nd"]
Sep 10 19:45:00.437666 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:45:00.437639    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfdf4ac3e_3fe7_41c1_aff6_c9040a2a7843.slice/crio-f541a8f1ede23c48055a8880c613ccf7c32663b528cf4d8949a0d29543309b27 WatchSource:0}: Error finding container f541a8f1ede23c48055a8880c613ccf7c32663b528cf4d8949a0d29543309b27: Status 404 returned error can't find the container with id f541a8f1ede23c48055a8880c613ccf7c32663b528cf4d8949a0d29543309b27
Sep 10 19:45:00.511411 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:45:00.511370    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:45:01.137268 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:45:01.137230    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:45:01.432471 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:45:01.432409    2575 generic.go:356] "Generic (PLEG): container finished" podID="fdf4ac3e-3fe7-41c1-aff6-c9040a2a7843" containerID="c36871dd98a22abf060865f0a36d264bb4811c74a913f0cf9cdf2347be05a5fa" exitCode=0
Sep 10 19:45:01.432471 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:45:01.432455    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817825-j52nd" event={"ID":"fdf4ac3e-3fe7-41c1-aff6-c9040a2a7843","Type":"ContainerDied","Data":"c36871dd98a22abf060865f0a36d264bb4811c74a913f0cf9cdf2347be05a5fa"}
Sep 10 19:45:01.432631 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:45:01.432492    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817825-j52nd" event={"ID":"fdf4ac3e-3fe7-41c1-aff6-c9040a2a7843","Type":"ContainerStarted","Data":"f541a8f1ede23c48055a8880c613ccf7c32663b528cf4d8949a0d29543309b27"}
Sep 10 19:45:02.563555 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:45:02.563532    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817825-j52nd"
Sep 10 19:45:03.214189 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:45:03.214150    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:45:03.314830 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:45:03.314794    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:45:03.446736 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:45:03.446708    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817825-j52nd"
Sep 10 19:45:03.446912 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:45:03.446752    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817825-j52nd" event={"ID":"fdf4ac3e-3fe7-41c1-aff6-c9040a2a7843","Type":"ContainerDied","Data":"f541a8f1ede23c48055a8880c613ccf7c32663b528cf4d8949a0d29543309b27"}
Sep 10 19:45:03.446912 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:45:03.446808    2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f541a8f1ede23c48055a8880c613ccf7c32663b528cf4d8949a0d29543309b27"
Sep 10 19:45:03.465786 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:45:03.465712    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:45:03.802393 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:45:03.802308    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-cab4ee-xf825"]
Sep 10 19:45:03.802719 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:45:03.802675    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fdf4ac3e-3fe7-41c1-aff6-c9040a2a7843" containerName="cleanup"
Sep 10 19:45:03.802719 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:45:03.802686    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="fdf4ac3e-3fe7-41c1-aff6-c9040a2a7843" containerName="cleanup"
Sep 10 19:45:03.802792 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:45:03.802774    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="b9597ee5-e04a-417f-8b4e-274e52bdf9e7" containerName="istio-proxy"
Sep 10 19:45:03.802792 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:45:03.802788    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="fdf4ac3e-3fe7-41c1-aff6-c9040a2a7843" containerName="cleanup"
Sep 10 19:45:03.805650 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:45:03.805634    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-cab4ee-xf825"
Sep 10 19:45:03.807543 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:45:03.807521    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 19:45:03.807667 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:45:03.807585    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wkgt8\""
Sep 10 19:45:03.812981 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:45:03.812956    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-cab4ee-xf825"]
Sep 10 19:45:03.941152 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:45:03.941119    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b0062bc-991f-4271-8b57-27ea0938f924-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-cab4ee-xf825\" (UID: \"4b0062bc-991f-4271-8b57-27ea0938f924\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-cab4ee-xf825"
Sep 10 19:45:03.946611 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:45:03.946583    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:45:04.041901 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:45:04.041875    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b0062bc-991f-4271-8b57-27ea0938f924-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-cab4ee-xf825\" (UID: \"4b0062bc-991f-4271-8b57-27ea0938f924\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-cab4ee-xf825"
Sep 10 19:45:04.042505 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:45:04.042485    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b0062bc-991f-4271-8b57-27ea0938f924-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-cab4ee-xf825\" (UID: \"4b0062bc-991f-4271-8b57-27ea0938f924\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-cab4ee-xf825"
Sep 10 19:45:04.118873 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:45:04.118817    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-cab4ee-xf825"
Sep 10 19:45:04.387232 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:45:04.387207    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-cab4ee-xf825"]
Sep 10 19:45:04.389639 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:45:04.389605    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b0062bc_991f_4271_8b57_27ea0938f924.slice/crio-e99a53dcfed600d7535429d9dc052dc46071e1965ffd58c91f737bf60430f652 WatchSource:0}: Error finding container e99a53dcfed600d7535429d9dc052dc46071e1965ffd58c91f737bf60430f652: Status 404 returned error can't find the container with id e99a53dcfed600d7535429d9dc052dc46071e1965ffd58c91f737bf60430f652
Sep 10 19:45:04.452571 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:45:04.452545    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-cab4ee-xf825" event={"ID":"4b0062bc-991f-4271-8b57-27ea0938f924","Type":"ContainerStarted","Data":"e99a53dcfed600d7535429d9dc052dc46071e1965ffd58c91f737bf60430f652"}
Sep 10 19:45:05.458787 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:45:05.458752    2575 generic.go:356] "Generic (PLEG): container finished" podID="4b0062bc-991f-4271-8b57-27ea0938f924" containerID="c2605048c7cf4e60599bd57993645faab5e337690cfc76b9cafc060acd93a7e6" exitCode=0
Sep 10 19:45:05.459266 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:45:05.458797    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-cab4ee-xf825" event={"ID":"4b0062bc-991f-4271-8b57-27ea0938f924","Type":"ContainerDied","Data":"c2605048c7cf4e60599bd57993645faab5e337690cfc76b9cafc060acd93a7e6"}
Sep 10 19:45:06.591950 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:45:06.591923    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-cab4ee-xf825"
Sep 10 19:45:06.660692 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:45:06.660660    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b0062bc-991f-4271-8b57-27ea0938f924-maas-api-service-ca\") pod \"4b0062bc-991f-4271-8b57-27ea0938f924\" (UID: \"4b0062bc-991f-4271-8b57-27ea0938f924\") "
Sep 10 19:45:06.660987 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:45:06.660964    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4b0062bc-991f-4271-8b57-27ea0938f924-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4b0062bc-991f-4271-8b57-27ea0938f924" (UID: "4b0062bc-991f-4271-8b57-27ea0938f924"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:45:06.762401 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:45:06.762316    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b0062bc-991f-4271-8b57-27ea0938f924-maas-api-service-ca\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:45:07.469207 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:45:07.469182    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-cab4ee-xf825"
Sep 10 19:45:07.469366 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:45:07.469216    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-cab4ee-xf825" event={"ID":"4b0062bc-991f-4271-8b57-27ea0938f924","Type":"ContainerDied","Data":"e99a53dcfed600d7535429d9dc052dc46071e1965ffd58c91f737bf60430f652"}
Sep 10 19:45:07.469366 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:45:07.469266    2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e99a53dcfed600d7535429d9dc052dc46071e1965ffd58c91f737bf60430f652"
Sep 10 19:45:14.433671 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:45:14.433627    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:45:14.465478 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:45:14.465448    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-cab4ee-5fc779cb6-f4nbr"]
Sep 10 19:45:14.465702 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:45:14.465676    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-cab4ee-5fc779cb6-f4nbr" podUID="24ce5208-91be-4c34-869b-75b0102f6380" containerName="payload-pre-processing" containerID="cri-o://0f57979f80b92e5e3f3b2950597db4c3290edfeab78ea4411e3b5c810b3fcc05" gracePeriod=30
Sep 10 19:45:14.535694 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:45:14.535667    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:45:14.635614 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:45:14.635574    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:45:14.650199 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:45:14.650133    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-cab4ee-xf825"]
Sep 10 19:45:14.653150 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:45:14.653123    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-cab4ee-xf825"]
Sep 10 19:45:14.732845 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:45:14.732817    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-cab4ee-5fc779cb6-f4nbr"
Sep 10 19:45:14.733176 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:45:14.733155    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:45:14.830188 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:45:14.830151    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/24ce5208-91be-4c34-869b-75b0102f6380-plugins-config-volume\") pod \"24ce5208-91be-4c34-869b-75b0102f6380\" (UID: \"24ce5208-91be-4c34-869b-75b0102f6380\") "
Sep 10 19:45:14.830379 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:45:14.830205    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-79msj\" (UniqueName: \"kubernetes.io/projected/24ce5208-91be-4c34-869b-75b0102f6380-kube-api-access-79msj\") pod \"24ce5208-91be-4c34-869b-75b0102f6380\" (UID: \"24ce5208-91be-4c34-869b-75b0102f6380\") "
Sep 10 19:45:14.830768 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:45:14.830745    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/24ce5208-91be-4c34-869b-75b0102f6380-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "24ce5208-91be-4c34-869b-75b0102f6380" (UID: "24ce5208-91be-4c34-869b-75b0102f6380"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:45:14.832398 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:45:14.832376    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/24ce5208-91be-4c34-869b-75b0102f6380-kube-api-access-79msj" (OuterVolumeSpecName: "kube-api-access-79msj") pod "24ce5208-91be-4c34-869b-75b0102f6380" (UID: "24ce5208-91be-4c34-869b-75b0102f6380"). InnerVolumeSpecName "kube-api-access-79msj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:45:14.930899 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:45:14.930872    2575 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/24ce5208-91be-4c34-869b-75b0102f6380-plugins-config-volume\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:45:14.930899 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:45:14.930896    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-79msj\" (UniqueName: \"kubernetes.io/projected/24ce5208-91be-4c34-869b-75b0102f6380-kube-api-access-79msj\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:45:15.043282 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:45:15.043205    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:45:15.390855 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:45:15.390773    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4b0062bc-991f-4271-8b57-27ea0938f924" path="/var/lib/kubelet/pods/4b0062bc-991f-4271-8b57-27ea0938f924/volumes"
Sep 10 19:45:15.510173 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:45:15.510142    2575 generic.go:356] "Generic (PLEG): container finished" podID="24ce5208-91be-4c34-869b-75b0102f6380" containerID="0f57979f80b92e5e3f3b2950597db4c3290edfeab78ea4411e3b5c810b3fcc05" exitCode=0
Sep 10 19:45:15.510539 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:45:15.510194    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-cab4ee-5fc779cb6-f4nbr"
Sep 10 19:45:15.510539 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:45:15.510201    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-cab4ee-5fc779cb6-f4nbr" event={"ID":"24ce5208-91be-4c34-869b-75b0102f6380","Type":"ContainerDied","Data":"0f57979f80b92e5e3f3b2950597db4c3290edfeab78ea4411e3b5c810b3fcc05"}
Sep 10 19:45:15.510539 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:45:15.510246    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-cab4ee-5fc779cb6-f4nbr" event={"ID":"24ce5208-91be-4c34-869b-75b0102f6380","Type":"ContainerDied","Data":"c3e83d05f06ec7119aa48585024e715fca72aee19cb5d42bfe6975781d7f1397"}
Sep 10 19:45:15.510539 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:45:15.510270    2575 scope.go:117] "RemoveContainer" containerID="0f57979f80b92e5e3f3b2950597db4c3290edfeab78ea4411e3b5c810b3fcc05"
Sep 10 19:45:15.519080 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:45:15.519060    2575 scope.go:117] "RemoveContainer" containerID="0f57979f80b92e5e3f3b2950597db4c3290edfeab78ea4411e3b5c810b3fcc05"
Sep 10 19:45:15.519394 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:45:15.519333    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0f57979f80b92e5e3f3b2950597db4c3290edfeab78ea4411e3b5c810b3fcc05\": container with ID starting with 0f57979f80b92e5e3f3b2950597db4c3290edfeab78ea4411e3b5c810b3fcc05 not found: ID does not exist" containerID="0f57979f80b92e5e3f3b2950597db4c3290edfeab78ea4411e3b5c810b3fcc05"
Sep 10 19:45:15.519493 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:45:15.519403    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0f57979f80b92e5e3f3b2950597db4c3290edfeab78ea4411e3b5c810b3fcc05"} err="failed to get container status \"0f57979f80b92e5e3f3b2950597db4c3290edfeab78ea4411e3b5c810b3fcc05\": rpc error: code = NotFound desc = could not find container \"0f57979f80b92e5e3f3b2950597db4c3290edfeab78ea4411e3b5c810b3fcc05\": container with ID starting with 0f57979f80b92e5e3f3b2950597db4c3290edfeab78ea4411e3b5c810b3fcc05 not found: ID does not exist"
Sep 10 19:45:15.525127 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:45:15.525107    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-cab4ee-5fc779cb6-f4nbr"]
Sep 10 19:45:15.528603 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:45:15.528583    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-cab4ee-5fc779cb6-f4nbr"]
Sep 10 19:45:16.997189 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:45:16.997155    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qr7rl"]
Sep 10 19:45:17.384266 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:45:17.384184    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="24ce5208-91be-4c34-869b-75b0102f6380" path="/var/lib/kubelet/pods/24ce5208-91be-4c34-869b-75b0102f6380/volumes"
Sep 10 19:45:21.282739 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:45:21.282710    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:45:21.381958 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:45:21.381920    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:45:21.536602 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:45:21.536520    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-cab4ee-openshift-default-945f9dd46-2dq66"]
Sep 10 19:45:21.536798 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:45:21.536759    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w3-cab4ee-openshift-default-945f9dd46-2dq66" podUID="61a281ba-e630-4357-bb77-64fe9085aadb" containerName="istio-proxy" containerID="cri-o://ca8403791c2b818e31ce7cf8ca88e3379863d4d8c9f7d2a35546fd0feb811ca2" gracePeriod=30
Sep 10 19:45:21.554741 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:45:21.554715    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:45:25.751542 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:45:25.751507    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qr7rl"]
Sep 10 19:45:26.775630 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:45:26.775605    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-cab4ee-openshift-default-945f9dd46-2dq66"
Sep 10 19:45:26.928650 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:45:26.928571    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/61a281ba-e630-4357-bb77-64fe9085aadb-istio-data\") pod \"61a281ba-e630-4357-bb77-64fe9085aadb\" (UID: \"61a281ba-e630-4357-bb77-64fe9085aadb\") "
Sep 10 19:45:26.928650 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:45:26.928620    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/61a281ba-e630-4357-bb77-64fe9085aadb-workload-socket\") pod \"61a281ba-e630-4357-bb77-64fe9085aadb\" (UID: \"61a281ba-e630-4357-bb77-64fe9085aadb\") "
Sep 10 19:45:26.928862 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:45:26.928663    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/61a281ba-e630-4357-bb77-64fe9085aadb-workload-certs\") pod \"61a281ba-e630-4357-bb77-64fe9085aadb\" (UID: \"61a281ba-e630-4357-bb77-64fe9085aadb\") "
Sep 10 19:45:26.928862 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:45:26.928709    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/61a281ba-e630-4357-bb77-64fe9085aadb-istiod-ca-cert\") pod \"61a281ba-e630-4357-bb77-64fe9085aadb\" (UID: \"61a281ba-e630-4357-bb77-64fe9085aadb\") "
Sep 10 19:45:26.928862 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:45:26.928739    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/61a281ba-e630-4357-bb77-64fe9085aadb-istio-token\") pod \"61a281ba-e630-4357-bb77-64fe9085aadb\" (UID: \"61a281ba-e630-4357-bb77-64fe9085aadb\") "
Sep 10 19:45:26.928862 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:45:26.928774    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/61a281ba-e630-4357-bb77-64fe9085aadb-istio-podinfo\") pod \"61a281ba-e630-4357-bb77-64fe9085aadb\" (UID: \"61a281ba-e630-4357-bb77-64fe9085aadb\") "
Sep 10 19:45:26.928862 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:45:26.928814    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g24cz\" (UniqueName: \"kubernetes.io/projected/61a281ba-e630-4357-bb77-64fe9085aadb-kube-api-access-g24cz\") pod \"61a281ba-e630-4357-bb77-64fe9085aadb\" (UID: \"61a281ba-e630-4357-bb77-64fe9085aadb\") "
Sep 10 19:45:26.928862 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:45:26.928830    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61a281ba-e630-4357-bb77-64fe9085aadb-istio-data" (OuterVolumeSpecName: "istio-data") pod "61a281ba-e630-4357-bb77-64fe9085aadb" (UID: "61a281ba-e630-4357-bb77-64fe9085aadb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:45:26.929161 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:45:26.928845    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/61a281ba-e630-4357-bb77-64fe9085aadb-istio-envoy\") pod \"61a281ba-e630-4357-bb77-64fe9085aadb\" (UID: \"61a281ba-e630-4357-bb77-64fe9085aadb\") "
Sep 10 19:45:26.929161 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:45:26.928904    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61a281ba-e630-4357-bb77-64fe9085aadb-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "61a281ba-e630-4357-bb77-64fe9085aadb" (UID: "61a281ba-e630-4357-bb77-64fe9085aadb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:45:26.929161 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:45:26.928926    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/61a281ba-e630-4357-bb77-64fe9085aadb-credential-socket\") pod \"61a281ba-e630-4357-bb77-64fe9085aadb\" (UID: \"61a281ba-e630-4357-bb77-64fe9085aadb\") "
Sep 10 19:45:26.929161 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:45:26.928937    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61a281ba-e630-4357-bb77-64fe9085aadb-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "61a281ba-e630-4357-bb77-64fe9085aadb" (UID: "61a281ba-e630-4357-bb77-64fe9085aadb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:45:26.929394 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:45:26.929186    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61a281ba-e630-4357-bb77-64fe9085aadb-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "61a281ba-e630-4357-bb77-64fe9085aadb" (UID: "61a281ba-e630-4357-bb77-64fe9085aadb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:45:26.929394 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:45:26.929248    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/61a281ba-e630-4357-bb77-64fe9085aadb-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "61a281ba-e630-4357-bb77-64fe9085aadb" (UID: "61a281ba-e630-4357-bb77-64fe9085aadb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:45:26.929394 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:45:26.929257    2575 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/61a281ba-e630-4357-bb77-64fe9085aadb-istio-data\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:45:26.929394 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:45:26.929292    2575 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/61a281ba-e630-4357-bb77-64fe9085aadb-workload-socket\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:45:26.929394 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:45:26.929306    2575 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/61a281ba-e630-4357-bb77-64fe9085aadb-workload-certs\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:45:26.930994 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:45:26.930962    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/61a281ba-e630-4357-bb77-64fe9085aadb-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "61a281ba-e630-4357-bb77-64fe9085aadb" (UID: "61a281ba-e630-4357-bb77-64fe9085aadb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 19:45:26.930994 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:45:26.930973    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/61a281ba-e630-4357-bb77-64fe9085aadb-kube-api-access-g24cz" (OuterVolumeSpecName: "kube-api-access-g24cz") pod "61a281ba-e630-4357-bb77-64fe9085aadb" (UID: "61a281ba-e630-4357-bb77-64fe9085aadb"). InnerVolumeSpecName "kube-api-access-g24cz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:45:26.931187 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:45:26.931000    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/61a281ba-e630-4357-bb77-64fe9085aadb-istio-token" (OuterVolumeSpecName: "istio-token") pod "61a281ba-e630-4357-bb77-64fe9085aadb" (UID: "61a281ba-e630-4357-bb77-64fe9085aadb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:45:26.931187 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:45:26.931028    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61a281ba-e630-4357-bb77-64fe9085aadb-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "61a281ba-e630-4357-bb77-64fe9085aadb" (UID: "61a281ba-e630-4357-bb77-64fe9085aadb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:45:27.030448 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:45:27.030425    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g24cz\" (UniqueName: \"kubernetes.io/projected/61a281ba-e630-4357-bb77-64fe9085aadb-kube-api-access-g24cz\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:45:27.030448 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:45:27.030445    2575 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/61a281ba-e630-4357-bb77-64fe9085aadb-istio-envoy\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:45:27.030599 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:45:27.030456    2575 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/61a281ba-e630-4357-bb77-64fe9085aadb-credential-socket\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:45:27.030599 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:45:27.030464    2575 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/61a281ba-e630-4357-bb77-64fe9085aadb-istiod-ca-cert\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:45:27.030599 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:45:27.030472    2575 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/61a281ba-e630-4357-bb77-64fe9085aadb-istio-token\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:45:27.030599 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:45:27.030480    2575 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/61a281ba-e630-4357-bb77-64fe9085aadb-istio-podinfo\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:45:27.573294 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:45:27.573263    2575 generic.go:356] "Generic (PLEG): container finished" podID="61a281ba-e630-4357-bb77-64fe9085aadb" containerID="ca8403791c2b818e31ce7cf8ca88e3379863d4d8c9f7d2a35546fd0feb811ca2" exitCode=0
Sep 10 19:45:27.573480 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:45:27.573300    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-cab4ee-openshift-default-945f9dd46-2dq66" event={"ID":"61a281ba-e630-4357-bb77-64fe9085aadb","Type":"ContainerDied","Data":"ca8403791c2b818e31ce7cf8ca88e3379863d4d8c9f7d2a35546fd0feb811ca2"}
Sep 10 19:45:27.573480 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:45:27.573313    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-cab4ee-openshift-default-945f9dd46-2dq66"
Sep 10 19:45:27.573480 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:45:27.573372    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-cab4ee-openshift-default-945f9dd46-2dq66" event={"ID":"61a281ba-e630-4357-bb77-64fe9085aadb","Type":"ContainerDied","Data":"fee7ad8d02f589369c253d4b17d4440a19de4d95f0315687cf5dc4ac97233463"}
Sep 10 19:45:27.573480 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:45:27.573396    2575 scope.go:117] "RemoveContainer" containerID="ca8403791c2b818e31ce7cf8ca88e3379863d4d8c9f7d2a35546fd0feb811ca2"
Sep 10 19:45:27.591144 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:45:27.591118    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-cab4ee-openshift-default-945f9dd46-2dq66"]
Sep 10 19:45:27.593463 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:45:27.593441    2575 scope.go:117] "RemoveContainer" containerID="1b568bf9c31904d27f7b7f5c390a7e8e975d0e46edf68f14dc6ba98556945c46"
Sep 10 19:45:27.594897 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:45:27.594873    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-cab4ee-openshift-default-945f9dd46-2dq66"]
Sep 10 19:45:27.601586 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:45:27.601565    2575 scope.go:117] "RemoveContainer" containerID="ca8403791c2b818e31ce7cf8ca88e3379863d4d8c9f7d2a35546fd0feb811ca2"
Sep 10 19:45:27.601849 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:45:27.601832    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ca8403791c2b818e31ce7cf8ca88e3379863d4d8c9f7d2a35546fd0feb811ca2\": container with ID starting with ca8403791c2b818e31ce7cf8ca88e3379863d4d8c9f7d2a35546fd0feb811ca2 not found: ID does not exist" containerID="ca8403791c2b818e31ce7cf8ca88e3379863d4d8c9f7d2a35546fd0feb811ca2"
Sep 10 19:45:27.601898 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:45:27.601857    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca8403791c2b818e31ce7cf8ca88e3379863d4d8c9f7d2a35546fd0feb811ca2"} err="failed to get container status \"ca8403791c2b818e31ce7cf8ca88e3379863d4d8c9f7d2a35546fd0feb811ca2\": rpc error: code = NotFound desc = could not find container \"ca8403791c2b818e31ce7cf8ca88e3379863d4d8c9f7d2a35546fd0feb811ca2\": container with ID starting with ca8403791c2b818e31ce7cf8ca88e3379863d4d8c9f7d2a35546fd0feb811ca2 not found: ID does not exist"
Sep 10 19:45:27.601898 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:45:27.601873    2575 scope.go:117] "RemoveContainer" containerID="1b568bf9c31904d27f7b7f5c390a7e8e975d0e46edf68f14dc6ba98556945c46"
Sep 10 19:45:27.602122 ip-10-0-129-114 kubenswrapper[2575]: E0910 19:45:27.602104    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1b568bf9c31904d27f7b7f5c390a7e8e975d0e46edf68f14dc6ba98556945c46\": container with ID starting with 1b568bf9c31904d27f7b7f5c390a7e8e975d0e46edf68f14dc6ba98556945c46 not found: ID does not exist" containerID="1b568bf9c31904d27f7b7f5c390a7e8e975d0e46edf68f14dc6ba98556945c46"
Sep 10 19:45:27.602159 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:45:27.602135    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b568bf9c31904d27f7b7f5c390a7e8e975d0e46edf68f14dc6ba98556945c46"} err="failed to get container status \"1b568bf9c31904d27f7b7f5c390a7e8e975d0e46edf68f14dc6ba98556945c46\": rpc error: code = NotFound desc = could not find container \"1b568bf9c31904d27f7b7f5c390a7e8e975d0e46edf68f14dc6ba98556945c46\": container with ID starting with 1b568bf9c31904d27f7b7f5c390a7e8e975d0e46edf68f14dc6ba98556945c46 not found: ID does not exist"
Sep 10 19:45:29.385425 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:45:29.385391    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="61a281ba-e630-4357-bb77-64fe9085aadb" path="/var/lib/kubelet/pods/61a281ba-e630-4357-bb77-64fe9085aadb/volumes"
Sep 10 19:45:29.471498 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:45:29.471471    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rrxfj_c7511dfe-c588-4be1-bd16-1dce34740809/ovn-acl-logging/0.log"
Sep 10 19:45:29.475007 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:45:29.474989    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rrxfj_c7511dfe-c588-4be1-bd16-1dce34740809/ovn-acl-logging/0.log"
Sep 10 19:45:29.690503 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:45:29.690440    2575 scope.go:117] "RemoveContainer" containerID="07f9e96ce4d039833adfab7c4a4ffb1d5d65d7116c88f870aba23bcdb005736d"
Sep 10 19:45:29.698273 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:45:29.698254    2575 scope.go:117] "RemoveContainer" containerID="0e1957cdf272f7971ca408cd9289c3b2f468a42eb846306ccf7dffb11436d228"
Sep 10 19:46:29.735731 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:46:29.735702    2575 scope.go:117] "RemoveContainer" containerID="43ef681f23bb902808eba910d465eefdbc7e9ed88a090e4d06f330823ffb7050"
Sep 10 19:46:45.627511 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:46:45.627466    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:46:46.458703 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:46:46.458667    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qr7rl"]
Sep 10 19:46:47.687802 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:46:47.687762    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:46:49.287604 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:46:49.287567    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:47:15.228546 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:47:15.228493    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:47:15.652021 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:47:15.651936    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qr7rl"]
Sep 10 19:47:16.870571 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:47:16.870508    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:47:23.645271 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:47:23.645235    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qr7rl"]
Sep 10 19:47:31.947314 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:47:31.947282    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qr7rl"]
Sep 10 19:47:49.240484 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:47:49.240446    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qr7rl"]
Sep 10 19:47:55.952413 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:47:55.952377    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qr7rl"]
Sep 10 19:48:35.994086 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:48:35.993991    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:49:23.651162 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:49:23.651126    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qr7rl"]
Sep 10 19:49:29.767092 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:49:29.767058    2575 scope.go:117] "RemoveContainer" containerID="ca0b7caa49e1f96ad05a57d891b6efce93f6049ff4600252e43f6e837fa9a8cc"
Sep 10 19:49:32.057112 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:49:32.057077    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qr7rl"]
Sep 10 19:49:37.522523 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:49:37.522485    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:49:40.649639 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:49:40.649599    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qr7rl"]
Sep 10 19:49:46.005429 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:49:46.005394    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:49:50.251668 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:49:50.251625    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qr7rl"]
Sep 10 19:49:58.768835 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:49:58.768736    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:50:00.381358 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:50:00.381311    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:50:00.954234 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:50:00.954195    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qr7rl"]
Sep 10 19:50:08.958807 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:50:08.958757    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:50:09.962375 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:50:09.962326    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qr7rl"]
Sep 10 19:50:19.059389 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:50:19.059356    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qr7rl"]
Sep 10 19:50:19.452327 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:50:19.452274    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-7d4dc54f4c-f6hlw"]
Sep 10 19:50:19.452867 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:50:19.452847    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="61a281ba-e630-4357-bb77-64fe9085aadb" containerName="istio-proxy"
Sep 10 19:50:19.452922 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:50:19.452871    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="61a281ba-e630-4357-bb77-64fe9085aadb" containerName="istio-proxy"
Sep 10 19:50:19.452922 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:50:19.452898    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="24ce5208-91be-4c34-869b-75b0102f6380" containerName="payload-pre-processing"
Sep 10 19:50:19.452922 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:50:19.452907    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="24ce5208-91be-4c34-869b-75b0102f6380" containerName="payload-pre-processing"
Sep 10 19:50:19.453032 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:50:19.452933    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b0062bc-991f-4271-8b57-27ea0938f924" containerName="revoke-keys"
Sep 10 19:50:19.453032 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:50:19.452943    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b0062bc-991f-4271-8b57-27ea0938f924" containerName="revoke-keys"
Sep 10 19:50:19.453032 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:50:19.453022    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="4b0062bc-991f-4271-8b57-27ea0938f924" containerName="revoke-keys"
Sep 10 19:50:19.453139 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:50:19.453035    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="24ce5208-91be-4c34-869b-75b0102f6380" containerName="payload-pre-processing"
Sep 10 19:50:19.453139 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:50:19.453047    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="61a281ba-e630-4357-bb77-64fe9085aadb" containerName="istio-proxy"
Sep 10 19:50:19.453139 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:50:19.453057    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="61a281ba-e630-4357-bb77-64fe9085aadb" containerName="istio-proxy"
Sep 10 19:50:19.456079 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:50:19.456060    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7d4dc54f4c-f6hlw"
Sep 10 19:50:19.462274 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:50:19.462249    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 19:50:19.462430 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:50:19.462249    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-m8g2m\""
Sep 10 19:50:19.462430 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:50:19.462253    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 10 19:50:19.462430 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:50:19.462254    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 10 19:50:19.466235 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:50:19.466200    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 10 19:50:19.480009 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:50:19.479971    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7d4dc54f4c-f6hlw"]
Sep 10 19:50:19.593687 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:50:19.593653    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/96ccc701-5361-43c9-8c2c-3eeca37893ee-maas-api-tls\") pod \"maas-api-7d4dc54f4c-f6hlw\" (UID: \"96ccc701-5361-43c9-8c2c-3eeca37893ee\") " pod="odh-ai-gateway-infra/maas-api-7d4dc54f4c-f6hlw"
Sep 10 19:50:19.593838 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:50:19.593723    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/96ccc701-5361-43c9-8c2c-3eeca37893ee-openshift-service-ca\") pod \"maas-api-7d4dc54f4c-f6hlw\" (UID: \"96ccc701-5361-43c9-8c2c-3eeca37893ee\") " pod="odh-ai-gateway-infra/maas-api-7d4dc54f4c-f6hlw"
Sep 10 19:50:19.593838 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:50:19.593761    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/96ccc701-5361-43c9-8c2c-3eeca37893ee-metrics-certs\") pod \"maas-api-7d4dc54f4c-f6hlw\" (UID: \"96ccc701-5361-43c9-8c2c-3eeca37893ee\") " pod="odh-ai-gateway-infra/maas-api-7d4dc54f4c-f6hlw"
Sep 10 19:50:19.593838 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:50:19.593782    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nx54q\" (UniqueName: \"kubernetes.io/projected/96ccc701-5361-43c9-8c2c-3eeca37893ee-kube-api-access-nx54q\") pod \"maas-api-7d4dc54f4c-f6hlw\" (UID: \"96ccc701-5361-43c9-8c2c-3eeca37893ee\") " pod="odh-ai-gateway-infra/maas-api-7d4dc54f4c-f6hlw"
Sep 10 19:50:19.695028 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:50:19.694992    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/96ccc701-5361-43c9-8c2c-3eeca37893ee-openshift-service-ca\") pod \"maas-api-7d4dc54f4c-f6hlw\" (UID: \"96ccc701-5361-43c9-8c2c-3eeca37893ee\") " pod="odh-ai-gateway-infra/maas-api-7d4dc54f4c-f6hlw"
Sep 10 19:50:19.695176 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:50:19.695038    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/96ccc701-5361-43c9-8c2c-3eeca37893ee-metrics-certs\") pod \"maas-api-7d4dc54f4c-f6hlw\" (UID: \"96ccc701-5361-43c9-8c2c-3eeca37893ee\") " pod="odh-ai-gateway-infra/maas-api-7d4dc54f4c-f6hlw"
Sep 10 19:50:19.695176 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:50:19.695077    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nx54q\" (UniqueName: \"kubernetes.io/projected/96ccc701-5361-43c9-8c2c-3eeca37893ee-kube-api-access-nx54q\") pod \"maas-api-7d4dc54f4c-f6hlw\" (UID: \"96ccc701-5361-43c9-8c2c-3eeca37893ee\") " pod="odh-ai-gateway-infra/maas-api-7d4dc54f4c-f6hlw"
Sep 10 19:50:19.695176 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:50:19.695112    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/96ccc701-5361-43c9-8c2c-3eeca37893ee-maas-api-tls\") pod \"maas-api-7d4dc54f4c-f6hlw\" (UID: \"96ccc701-5361-43c9-8c2c-3eeca37893ee\") " pod="odh-ai-gateway-infra/maas-api-7d4dc54f4c-f6hlw"
Sep 10 19:50:19.695695 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:50:19.695667    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/96ccc701-5361-43c9-8c2c-3eeca37893ee-openshift-service-ca\") pod \"maas-api-7d4dc54f4c-f6hlw\" (UID: \"96ccc701-5361-43c9-8c2c-3eeca37893ee\") " pod="odh-ai-gateway-infra/maas-api-7d4dc54f4c-f6hlw"
Sep 10 19:50:19.697648 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:50:19.697624    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/96ccc701-5361-43c9-8c2c-3eeca37893ee-maas-api-tls\") pod \"maas-api-7d4dc54f4c-f6hlw\" (UID: \"96ccc701-5361-43c9-8c2c-3eeca37893ee\") " pod="odh-ai-gateway-infra/maas-api-7d4dc54f4c-f6hlw"
Sep 10 19:50:19.697772 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:50:19.697750    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/96ccc701-5361-43c9-8c2c-3eeca37893ee-metrics-certs\") pod \"maas-api-7d4dc54f4c-f6hlw\" (UID: \"96ccc701-5361-43c9-8c2c-3eeca37893ee\") " pod="odh-ai-gateway-infra/maas-api-7d4dc54f4c-f6hlw"
Sep 10 19:50:19.702987 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:50:19.702936    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nx54q\" (UniqueName: \"kubernetes.io/projected/96ccc701-5361-43c9-8c2c-3eeca37893ee-kube-api-access-nx54q\") pod \"maas-api-7d4dc54f4c-f6hlw\" (UID: \"96ccc701-5361-43c9-8c2c-3eeca37893ee\") " pod="odh-ai-gateway-infra/maas-api-7d4dc54f4c-f6hlw"
Sep 10 19:50:19.767301 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:50:19.767271    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7d4dc54f4c-f6hlw"
Sep 10 19:50:20.049363 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:50:20.049260    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7d4dc54f4c-f6hlw"]
Sep 10 19:50:20.052218 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:50:20.052192    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96ccc701_5361_43c9_8c2c_3eeca37893ee.slice/crio-405f6478c95855bc4d43edd49168bafe4062dbd309dee041cdfd7b76daefa2f5 WatchSource:0}: Error finding container 405f6478c95855bc4d43edd49168bafe4062dbd309dee041cdfd7b76daefa2f5: Status 404 returned error can't find the container with id 405f6478c95855bc4d43edd49168bafe4062dbd309dee041cdfd7b76daefa2f5
Sep 10 19:50:20.053471 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:50:20.053455    2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 19:50:21.005425 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:50:21.005325    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7d4dc54f4c-f6hlw" event={"ID":"96ccc701-5361-43c9-8c2c-3eeca37893ee","Type":"ContainerStarted","Data":"405f6478c95855bc4d43edd49168bafe4062dbd309dee041cdfd7b76daefa2f5"}
Sep 10 19:50:23.018468 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:50:23.018433    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7d4dc54f4c-f6hlw" event={"ID":"96ccc701-5361-43c9-8c2c-3eeca37893ee","Type":"ContainerStarted","Data":"dffb498857624c13c9d2a5fdfbeaae5acd1a948fa19eea888962a2df2c64c5e4"}
Sep 10 19:50:23.018923 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:50:23.018567    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-7d4dc54f4c-f6hlw"
Sep 10 19:50:23.041987 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:50:23.041943    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-7d4dc54f4c-f6hlw" podStartSLOduration=1.55225406 podStartE2EDuration="4.041929411s" totalImagesPullingTime="2.489675351s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:50:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:50:23.039850833 +0000 UTC m=+2094.099678502" watchObservedRunningTime="2026-09-10 19:50:23.041929411 +0000 UTC m=+2094.101757079"
Sep 10 19:50:26.850332 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:50:26.850299    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-7d4dc54f4c-f6hlw"]
Sep 10 19:50:26.850784 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:50:26.850522    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-7d4dc54f4c-f6hlw" podUID="96ccc701-5361-43c9-8c2c-3eeca37893ee" containerName="maas-api" containerID="cri-o://dffb498857624c13c9d2a5fdfbeaae5acd1a948fa19eea888962a2df2c64c5e4" gracePeriod=30
Sep 10 19:50:26.854272 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:50:26.854211    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qr7rl"]
Sep 10 19:50:26.886096 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:50:26.886069    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-8cb5f7c95-z5c6s"]
Sep 10 19:50:26.886433 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:50:26.886421    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="61a281ba-e630-4357-bb77-64fe9085aadb" containerName="istio-proxy"
Sep 10 19:50:26.886481 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:50:26.886435    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="61a281ba-e630-4357-bb77-64fe9085aadb" containerName="istio-proxy"
Sep 10 19:50:26.889455 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:50:26.889439    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-8cb5f7c95-z5c6s"
Sep 10 19:50:26.897568 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:50:26.897543    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-8cb5f7c95-z5c6s"]
Sep 10 19:50:27.040171 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:50:27.040139    2575 generic.go:356] "Generic (PLEG): container finished" podID="96ccc701-5361-43c9-8c2c-3eeca37893ee" containerID="dffb498857624c13c9d2a5fdfbeaae5acd1a948fa19eea888962a2df2c64c5e4" exitCode=0
Sep 10 19:50:27.040321 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:50:27.040217    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7d4dc54f4c-f6hlw" event={"ID":"96ccc701-5361-43c9-8c2c-3eeca37893ee","Type":"ContainerDied","Data":"dffb498857624c13c9d2a5fdfbeaae5acd1a948fa19eea888962a2df2c64c5e4"}
Sep 10 19:50:27.054550 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:50:27.054524    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ecfcdb11-b486-4cea-a1a1-c8545512f1ec-openshift-service-ca\") pod \"maas-api-8cb5f7c95-z5c6s\" (UID: \"ecfcdb11-b486-4cea-a1a1-c8545512f1ec\") " pod="odh-ai-gateway-infra/maas-api-8cb5f7c95-z5c6s"
Sep 10 19:50:27.054664 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:50:27.054553    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ecfcdb11-b486-4cea-a1a1-c8545512f1ec-metrics-certs\") pod \"maas-api-8cb5f7c95-z5c6s\" (UID: \"ecfcdb11-b486-4cea-a1a1-c8545512f1ec\") " pod="odh-ai-gateway-infra/maas-api-8cb5f7c95-z5c6s"
Sep 10 19:50:27.054664 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:50:27.054572    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ecfcdb11-b486-4cea-a1a1-c8545512f1ec-maas-api-tls\") pod \"maas-api-8cb5f7c95-z5c6s\" (UID: \"ecfcdb11-b486-4cea-a1a1-c8545512f1ec\") " pod="odh-ai-gateway-infra/maas-api-8cb5f7c95-z5c6s"
Sep 10 19:50:27.054664 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:50:27.054588    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l9jfn\" (UniqueName: \"kubernetes.io/projected/ecfcdb11-b486-4cea-a1a1-c8545512f1ec-kube-api-access-l9jfn\") pod \"maas-api-8cb5f7c95-z5c6s\" (UID: \"ecfcdb11-b486-4cea-a1a1-c8545512f1ec\") " pod="odh-ai-gateway-infra/maas-api-8cb5f7c95-z5c6s"
Sep 10 19:50:27.096671 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:50:27.096652    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7d4dc54f4c-f6hlw"
Sep 10 19:50:27.155768 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:50:27.155708    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ecfcdb11-b486-4cea-a1a1-c8545512f1ec-openshift-service-ca\") pod \"maas-api-8cb5f7c95-z5c6s\" (UID: \"ecfcdb11-b486-4cea-a1a1-c8545512f1ec\") " pod="odh-ai-gateway-infra/maas-api-8cb5f7c95-z5c6s"
Sep 10 19:50:27.155768 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:50:27.155741    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ecfcdb11-b486-4cea-a1a1-c8545512f1ec-metrics-certs\") pod \"maas-api-8cb5f7c95-z5c6s\" (UID: \"ecfcdb11-b486-4cea-a1a1-c8545512f1ec\") " pod="odh-ai-gateway-infra/maas-api-8cb5f7c95-z5c6s"
Sep 10 19:50:27.155895 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:50:27.155767    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ecfcdb11-b486-4cea-a1a1-c8545512f1ec-maas-api-tls\") pod \"maas-api-8cb5f7c95-z5c6s\" (UID: \"ecfcdb11-b486-4cea-a1a1-c8545512f1ec\") " pod="odh-ai-gateway-infra/maas-api-8cb5f7c95-z5c6s"
Sep 10 19:50:27.155948 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:50:27.155930    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l9jfn\" (UniqueName: \"kubernetes.io/projected/ecfcdb11-b486-4cea-a1a1-c8545512f1ec-kube-api-access-l9jfn\") pod \"maas-api-8cb5f7c95-z5c6s\" (UID: \"ecfcdb11-b486-4cea-a1a1-c8545512f1ec\") " pod="odh-ai-gateway-infra/maas-api-8cb5f7c95-z5c6s"
Sep 10 19:50:27.156478 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:50:27.156449    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ecfcdb11-b486-4cea-a1a1-c8545512f1ec-openshift-service-ca\") pod \"maas-api-8cb5f7c95-z5c6s\" (UID: \"ecfcdb11-b486-4cea-a1a1-c8545512f1ec\") " pod="odh-ai-gateway-infra/maas-api-8cb5f7c95-z5c6s"
Sep 10 19:50:27.158123 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:50:27.158104    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ecfcdb11-b486-4cea-a1a1-c8545512f1ec-metrics-certs\") pod \"maas-api-8cb5f7c95-z5c6s\" (UID: \"ecfcdb11-b486-4cea-a1a1-c8545512f1ec\") " pod="odh-ai-gateway-infra/maas-api-8cb5f7c95-z5c6s"
Sep 10 19:50:27.158206 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:50:27.158152    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ecfcdb11-b486-4cea-a1a1-c8545512f1ec-maas-api-tls\") pod \"maas-api-8cb5f7c95-z5c6s\" (UID: \"ecfcdb11-b486-4cea-a1a1-c8545512f1ec\") " pod="odh-ai-gateway-infra/maas-api-8cb5f7c95-z5c6s"
Sep 10 19:50:27.163658 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:50:27.163637    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l9jfn\" (UniqueName: \"kubernetes.io/projected/ecfcdb11-b486-4cea-a1a1-c8545512f1ec-kube-api-access-l9jfn\") pod \"maas-api-8cb5f7c95-z5c6s\" (UID: \"ecfcdb11-b486-4cea-a1a1-c8545512f1ec\") " pod="odh-ai-gateway-infra/maas-api-8cb5f7c95-z5c6s"
Sep 10 19:50:27.225467 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:50:27.225446    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-8cb5f7c95-z5c6s"
Sep 10 19:50:27.256399 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:50:27.256376    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/96ccc701-5361-43c9-8c2c-3eeca37893ee-metrics-certs\") pod \"96ccc701-5361-43c9-8c2c-3eeca37893ee\" (UID: \"96ccc701-5361-43c9-8c2c-3eeca37893ee\") "
Sep 10 19:50:27.256516 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:50:27.256452    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/96ccc701-5361-43c9-8c2c-3eeca37893ee-maas-api-tls\") pod \"96ccc701-5361-43c9-8c2c-3eeca37893ee\" (UID: \"96ccc701-5361-43c9-8c2c-3eeca37893ee\") "
Sep 10 19:50:27.256516 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:50:27.256500    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/96ccc701-5361-43c9-8c2c-3eeca37893ee-openshift-service-ca\") pod \"96ccc701-5361-43c9-8c2c-3eeca37893ee\" (UID: \"96ccc701-5361-43c9-8c2c-3eeca37893ee\") "
Sep 10 19:50:27.256641 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:50:27.256529    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nx54q\" (UniqueName: \"kubernetes.io/projected/96ccc701-5361-43c9-8c2c-3eeca37893ee-kube-api-access-nx54q\") pod \"96ccc701-5361-43c9-8c2c-3eeca37893ee\" (UID: \"96ccc701-5361-43c9-8c2c-3eeca37893ee\") "
Sep 10 19:50:27.256872 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:50:27.256842    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/96ccc701-5361-43c9-8c2c-3eeca37893ee-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "96ccc701-5361-43c9-8c2c-3eeca37893ee" (UID: "96ccc701-5361-43c9-8c2c-3eeca37893ee"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:50:27.258466 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:50:27.258444    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/96ccc701-5361-43c9-8c2c-3eeca37893ee-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "96ccc701-5361-43c9-8c2c-3eeca37893ee" (UID: "96ccc701-5361-43c9-8c2c-3eeca37893ee"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:50:27.258828 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:50:27.258812    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/96ccc701-5361-43c9-8c2c-3eeca37893ee-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "96ccc701-5361-43c9-8c2c-3eeca37893ee" (UID: "96ccc701-5361-43c9-8c2c-3eeca37893ee"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:50:27.258911 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:50:27.258885    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/96ccc701-5361-43c9-8c2c-3eeca37893ee-kube-api-access-nx54q" (OuterVolumeSpecName: "kube-api-access-nx54q") pod "96ccc701-5361-43c9-8c2c-3eeca37893ee" (UID: "96ccc701-5361-43c9-8c2c-3eeca37893ee"). InnerVolumeSpecName "kube-api-access-nx54q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:50:27.357707 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:50:27.357680    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/96ccc701-5361-43c9-8c2c-3eeca37893ee-maas-api-tls\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:50:27.357707 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:50:27.357707    2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/96ccc701-5361-43c9-8c2c-3eeca37893ee-openshift-service-ca\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:50:27.357866 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:50:27.357722    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nx54q\" (UniqueName: \"kubernetes.io/projected/96ccc701-5361-43c9-8c2c-3eeca37893ee-kube-api-access-nx54q\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:50:27.357866 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:50:27.357734    2575 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/96ccc701-5361-43c9-8c2c-3eeca37893ee-metrics-certs\") on node \"ip-10-0-129-114.ec2.internal\" DevicePath \"\""
Sep 10 19:50:27.496017 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:50:27.495993    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-8cb5f7c95-z5c6s"]
Sep 10 19:50:27.498018 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:50:27.497991    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podecfcdb11_b486_4cea_a1a1_c8545512f1ec.slice/crio-a755f9219b238f321716036a99667754d22e5112d7b41be8ae05366a48511b52 WatchSource:0}: Error finding container a755f9219b238f321716036a99667754d22e5112d7b41be8ae05366a48511b52: Status 404 returned error can't find the container with id a755f9219b238f321716036a99667754d22e5112d7b41be8ae05366a48511b52
Sep 10 19:50:28.046801 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:50:28.046775    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7d4dc54f4c-f6hlw"
Sep 10 19:50:28.047205 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:50:28.046811    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7d4dc54f4c-f6hlw" event={"ID":"96ccc701-5361-43c9-8c2c-3eeca37893ee","Type":"ContainerDied","Data":"405f6478c95855bc4d43edd49168bafe4062dbd309dee041cdfd7b76daefa2f5"}
Sep 10 19:50:28.047205 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:50:28.046876    2575 scope.go:117] "RemoveContainer" containerID="dffb498857624c13c9d2a5fdfbeaae5acd1a948fa19eea888962a2df2c64c5e4"
Sep 10 19:50:28.048686 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:50:28.048660    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-8cb5f7c95-z5c6s" event={"ID":"ecfcdb11-b486-4cea-a1a1-c8545512f1ec","Type":"ContainerStarted","Data":"8f19eb0b65bd9c57b78a520e947e6524ab00673b35fa93cde00816bdd8d938fb"}
Sep 10 19:50:28.048818 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:50:28.048703    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-8cb5f7c95-z5c6s" event={"ID":"ecfcdb11-b486-4cea-a1a1-c8545512f1ec","Type":"ContainerStarted","Data":"a755f9219b238f321716036a99667754d22e5112d7b41be8ae05366a48511b52"}
Sep 10 19:50:28.048818 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:50:28.048759    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-8cb5f7c95-z5c6s"
Sep 10 19:50:28.065784 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:50:28.065740    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-8cb5f7c95-z5c6s" podStartSLOduration=1.628967658 podStartE2EDuration="2.065723377s" totalImagesPullingTime="436.755719ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:50:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:50:28.064368705 +0000 UTC m=+2099.124196370" watchObservedRunningTime="2026-09-10 19:50:28.065723377 +0000 UTC m=+2099.125551039"
Sep 10 19:50:28.079831 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:50:28.079805    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-7d4dc54f4c-f6hlw"]
Sep 10 19:50:28.083520 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:50:28.083500    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-7d4dc54f4c-f6hlw"]
Sep 10 19:50:29.385021 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:50:29.384985    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="96ccc701-5361-43c9-8c2c-3eeca37893ee" path="/var/lib/kubelet/pods/96ccc701-5361-43c9-8c2c-3eeca37893ee/volumes"
Sep 10 19:50:29.503744 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:50:29.503720    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rrxfj_c7511dfe-c588-4be1-bd16-1dce34740809/ovn-acl-logging/0.log"
Sep 10 19:50:29.508410 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:50:29.508389    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rrxfj_c7511dfe-c588-4be1-bd16-1dce34740809/ovn-acl-logging/0.log"
Sep 10 19:50:29.792684 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:50:29.792655    2575 scope.go:117] "RemoveContainer" containerID="84fda2327c20a2a97205b16f0c769e7e1e4072598e78cddaa7abdb8486974111"
Sep 10 19:50:33.159163 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:50:33.159127    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:50:33.443672 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:50:33.443597    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qr7rl"]
Sep 10 19:50:34.060095 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:50:34.060063    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-8cb5f7c95-z5c6s"
Sep 10 19:50:39.560548 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:50:39.560514    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qr7rl"]
Sep 10 19:50:46.546798 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:50:46.546763    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qr7rl"]
Sep 10 19:50:49.068711 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:50:49.068678    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-865f7bbd6c-psgbr_1c780b1f-1a09-4477-a61f-5dc520974c3e/authorino/0.log"
Sep 10 19:50:54.473105 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:50:54.473066    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-gdpds_880bf6ca-872f-4005-b5cc-cd60bdf6caec/manager/0.log"
Sep 10 19:50:54.957636 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:50:54.957607    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5d477566c-5h625_d5588d80-e182-43c2-aecb-44038005f945/copy-manifests/0.log"
Sep 10 19:50:54.964537 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:50:54.964514    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5d477566c-5h625_d5588d80-e182-43c2-aecb-44038005f945/manager/0.log"
Sep 10 19:50:55.675865 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:50:55.675839    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-8cb5f7c95-z5c6s_ecfcdb11-b486-4cea-a1a1-c8545512f1ec/maas-api/0.log"
Sep 10 19:50:55.894876 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:50:55.894848    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29817825-j52nd_fdf4ac3e-3fe7-41c1-aff6-c9040a2a7843/cleanup/0.log"
Sep 10 19:50:56.772852 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:50:56.772826    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-865f7bbd6c-psgbr_1c780b1f-1a09-4477-a61f-5dc520974c3e/authorino/0.log"
Sep 10 19:50:57.326302 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:50:57.326281    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-84994f7ffc-qr7rl_d522d914-4f36-4f67-ae2c-43fb14a69285/limitador/0.log"
Sep 10 19:50:58.115688 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:50:58.115658    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5dd7bdc797-q87x9_6d6559e0-b731-4c66-aa5b-7dcbb9de04ad/kube-auth-proxy/0.log"
Sep 10 19:50:58.358556 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:50:58.358529    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-flfb7_8924df89-f4a8-4d61-a636-066edf4dc4f0/payload-pre-processing/0.log"
Sep 10 19:50:59.181090 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:50:59.181063    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-embedding-simulated-kserve-bcc9ff8d8-97zkh_3fb3804c-78bd-4040-b2d3-a2bff8640f5e/main/0.log"
Sep 10 19:50:59.640797 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:50:59.640768    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-gfvhc_a3a1aee4-d4ce-498c-be27-a845c4ef18ac/main/0.log"
Sep 10 19:51:05.094838 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:51:05.094796    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:51:05.184881 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:51:05.184838    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:51:08.661575 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:51:08.661542    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-24lb9_ac571b4d-dbf7-4e62-8d4d-c8c2a527deb3/global-pull-secret-syncer/0.log"
Sep 10 19:51:08.873437 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:51:08.873398    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log"
Sep 10 19:51:08.961562 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:51:08.961532    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-129-114.ec2.internal_3eeea337b91a216332a11d902b8fa0ef/haproxy/0.log"
Sep 10 19:51:13.834196 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:51:13.834165    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-865f7bbd6c-psgbr_1c780b1f-1a09-4477-a61f-5dc520974c3e/authorino/0.log"
Sep 10 19:51:13.999195 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:51:13.999163    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-84994f7ffc-qr7rl_d522d914-4f36-4f67-ae2c-43fb14a69285/limitador/0.log"
Sep 10 19:51:15.578212 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:51:15.578172    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_93a950f6-8e9b-481f-9c1b-0eeb7a373c8d/alertmanager/0.log"
Sep 10 19:51:15.618295 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:51:15.618272    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_93a950f6-8e9b-481f-9c1b-0eeb7a373c8d/config-reloader/0.log"
Sep 10 19:51:15.652001 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:51:15.651980    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_93a950f6-8e9b-481f-9c1b-0eeb7a373c8d/kube-rbac-proxy-web/0.log"
Sep 10 19:51:15.686316 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:51:15.686295    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_93a950f6-8e9b-481f-9c1b-0eeb7a373c8d/kube-rbac-proxy/0.log"
Sep 10 19:51:15.714759 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:51:15.714735    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_93a950f6-8e9b-481f-9c1b-0eeb7a373c8d/kube-rbac-proxy-metric/0.log"
Sep 10 19:51:15.744915 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:51:15.744893    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_93a950f6-8e9b-481f-9c1b-0eeb7a373c8d/prom-label-proxy/0.log"
Sep 10 19:51:15.776465 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:51:15.776446    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_93a950f6-8e9b-481f-9c1b-0eeb7a373c8d/init-config-reloader/0.log"
Sep 10 19:51:16.032245 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:51:16.032222    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6tcld_d970bc06-018c-41e4-86c6-f41f489a21c6/node-exporter/0.log"
Sep 10 19:51:16.054105 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:51:16.054085    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6tcld_d970bc06-018c-41e4-86c6-f41f489a21c6/kube-rbac-proxy/0.log"
Sep 10 19:51:16.078913 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:51:16.078895    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6tcld_d970bc06-018c-41e4-86c6-f41f489a21c6/init-textfile/0.log"
Sep 10 19:51:16.263422 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:51:16.263360    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-lxjbh_152c0c8b-18bd-4977-9e5d-05d4a494cff9/kube-rbac-proxy-main/0.log"
Sep 10 19:51:16.287921 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:51:16.287900    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-lxjbh_152c0c8b-18bd-4977-9e5d-05d4a494cff9/kube-rbac-proxy-self/0.log"
Sep 10 19:51:16.313316 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:51:16.313294    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-lxjbh_152c0c8b-18bd-4977-9e5d-05d4a494cff9/openshift-state-metrics/0.log"
Sep 10 19:51:16.355461 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:51:16.355439    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_64faa9bc-8055-4b10-9a86-046d6507ae4a/prometheus/0.log"
Sep 10 19:51:16.377684 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:51:16.377658    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_64faa9bc-8055-4b10-9a86-046d6507ae4a/config-reloader/0.log"
Sep 10 19:51:16.404777 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:51:16.404757    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_64faa9bc-8055-4b10-9a86-046d6507ae4a/thanos-sidecar/0.log"
Sep 10 19:51:16.430916 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:51:16.430887    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_64faa9bc-8055-4b10-9a86-046d6507ae4a/kube-rbac-proxy-web/0.log"
Sep 10 19:51:16.455616 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:51:16.455597    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_64faa9bc-8055-4b10-9a86-046d6507ae4a/kube-rbac-proxy/0.log"
Sep 10 19:51:16.476659 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:51:16.476639    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_64faa9bc-8055-4b10-9a86-046d6507ae4a/kube-rbac-proxy-thanos/0.log"
Sep 10 19:51:16.498571 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:51:16.498553    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_64faa9bc-8055-4b10-9a86-046d6507ae4a/init-config-reloader/0.log"
Sep 10 19:51:16.530050 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:51:16.529987    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-6f4946db56-ztmr6_4fc69fb7-6f7c-4239-b7ef-7b8379f01e30/prometheus-operator/0.log"
Sep 10 19:51:16.548512 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:51:16.548491    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-6f4946db56-ztmr6_4fc69fb7-6f7c-4239-b7ef-7b8379f01e30/kube-rbac-proxy/0.log"
Sep 10 19:51:16.572917 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:51:16.572897    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-78f599db79-298zr_e2b13e6d-fc2c-47a5-a584-1ed70c44b8e7/prometheus-operator-admission-webhook/0.log"
Sep 10 19:51:18.560026 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:51:18.559990    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-5hbck/perf-node-gather-daemonset-l8tcm"]
Sep 10 19:51:18.560407 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:51:18.560363    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="96ccc701-5361-43c9-8c2c-3eeca37893ee" containerName="maas-api"
Sep 10 19:51:18.560407 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:51:18.560382    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="96ccc701-5361-43c9-8c2c-3eeca37893ee" containerName="maas-api"
Sep 10 19:51:18.560488 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:51:18.560445    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="96ccc701-5361-43c9-8c2c-3eeca37893ee" containerName="maas-api"
Sep 10 19:51:18.563889 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:51:18.563869    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5hbck/perf-node-gather-daemonset-l8tcm"
Sep 10 19:51:18.566229 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:51:18.566213    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-5hbck\"/\"kube-root-ca.crt\""
Sep 10 19:51:18.566618 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:51:18.566601    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-5hbck\"/\"default-dockercfg-r6dbn\""
Sep 10 19:51:18.566667 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:51:18.566601    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-5hbck\"/\"openshift-service-ca.crt\""
Sep 10 19:51:18.573581 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:51:18.573561    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5hbck/perf-node-gather-daemonset-l8tcm"]
Sep 10 19:51:18.653948 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:51:18.653920    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bkw25\" (UniqueName: \"kubernetes.io/projected/75a80dee-0c9e-4e5e-bc5f-7642454e162e-kube-api-access-bkw25\") pod \"perf-node-gather-daemonset-l8tcm\" (UID: \"75a80dee-0c9e-4e5e-bc5f-7642454e162e\") " pod="openshift-must-gather-5hbck/perf-node-gather-daemonset-l8tcm"
Sep 10 19:51:18.654064 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:51:18.653955    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/75a80dee-0c9e-4e5e-bc5f-7642454e162e-lib-modules\") pod \"perf-node-gather-daemonset-l8tcm\" (UID: \"75a80dee-0c9e-4e5e-bc5f-7642454e162e\") " pod="openshift-must-gather-5hbck/perf-node-gather-daemonset-l8tcm"
Sep 10 19:51:18.654064 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:51:18.653987    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/75a80dee-0c9e-4e5e-bc5f-7642454e162e-proc\") pod \"perf-node-gather-daemonset-l8tcm\" (UID: \"75a80dee-0c9e-4e5e-bc5f-7642454e162e\") " pod="openshift-must-gather-5hbck/perf-node-gather-daemonset-l8tcm"
Sep 10 19:51:18.654138 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:51:18.654062    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/75a80dee-0c9e-4e5e-bc5f-7642454e162e-podres\") pod \"perf-node-gather-daemonset-l8tcm\" (UID: \"75a80dee-0c9e-4e5e-bc5f-7642454e162e\") " pod="openshift-must-gather-5hbck/perf-node-gather-daemonset-l8tcm"
Sep 10 19:51:18.654138 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:51:18.654087    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/75a80dee-0c9e-4e5e-bc5f-7642454e162e-sys\") pod \"perf-node-gather-daemonset-l8tcm\" (UID: \"75a80dee-0c9e-4e5e-bc5f-7642454e162e\") " pod="openshift-must-gather-5hbck/perf-node-gather-daemonset-l8tcm"
Sep 10 19:51:18.755211 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:51:18.755180    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bkw25\" (UniqueName: \"kubernetes.io/projected/75a80dee-0c9e-4e5e-bc5f-7642454e162e-kube-api-access-bkw25\") pod \"perf-node-gather-daemonset-l8tcm\" (UID: \"75a80dee-0c9e-4e5e-bc5f-7642454e162e\") " pod="openshift-must-gather-5hbck/perf-node-gather-daemonset-l8tcm"
Sep 10 19:51:18.755368 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:51:18.755215    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/75a80dee-0c9e-4e5e-bc5f-7642454e162e-lib-modules\") pod \"perf-node-gather-daemonset-l8tcm\" (UID: \"75a80dee-0c9e-4e5e-bc5f-7642454e162e\") " pod="openshift-must-gather-5hbck/perf-node-gather-daemonset-l8tcm"
Sep 10 19:51:18.755368 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:51:18.755239    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/75a80dee-0c9e-4e5e-bc5f-7642454e162e-proc\") pod \"perf-node-gather-daemonset-l8tcm\" (UID: \"75a80dee-0c9e-4e5e-bc5f-7642454e162e\") " pod="openshift-must-gather-5hbck/perf-node-gather-daemonset-l8tcm"
Sep 10 19:51:18.755368 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:51:18.755267    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/75a80dee-0c9e-4e5e-bc5f-7642454e162e-podres\") pod \"perf-node-gather-daemonset-l8tcm\" (UID: \"75a80dee-0c9e-4e5e-bc5f-7642454e162e\") " pod="openshift-must-gather-5hbck/perf-node-gather-daemonset-l8tcm"
Sep 10 19:51:18.755368 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:51:18.755281    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/75a80dee-0c9e-4e5e-bc5f-7642454e162e-sys\") pod \"perf-node-gather-daemonset-l8tcm\" (UID: \"75a80dee-0c9e-4e5e-bc5f-7642454e162e\") " pod="openshift-must-gather-5hbck/perf-node-gather-daemonset-l8tcm"
Sep 10 19:51:18.755368 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:51:18.755358    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/75a80dee-0c9e-4e5e-bc5f-7642454e162e-proc\") pod \"perf-node-gather-daemonset-l8tcm\" (UID: \"75a80dee-0c9e-4e5e-bc5f-7642454e162e\") " pod="openshift-must-gather-5hbck/perf-node-gather-daemonset-l8tcm"
Sep 10 19:51:18.755577 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:51:18.755377    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/75a80dee-0c9e-4e5e-bc5f-7642454e162e-sys\") pod \"perf-node-gather-daemonset-l8tcm\" (UID: \"75a80dee-0c9e-4e5e-bc5f-7642454e162e\") " pod="openshift-must-gather-5hbck/perf-node-gather-daemonset-l8tcm"
Sep 10 19:51:18.755577 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:51:18.755396    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/75a80dee-0c9e-4e5e-bc5f-7642454e162e-podres\") pod \"perf-node-gather-daemonset-l8tcm\" (UID: \"75a80dee-0c9e-4e5e-bc5f-7642454e162e\") " pod="openshift-must-gather-5hbck/perf-node-gather-daemonset-l8tcm"
Sep 10 19:51:18.755577 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:51:18.755396    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/75a80dee-0c9e-4e5e-bc5f-7642454e162e-lib-modules\") pod \"perf-node-gather-daemonset-l8tcm\" (UID: \"75a80dee-0c9e-4e5e-bc5f-7642454e162e\") " pod="openshift-must-gather-5hbck/perf-node-gather-daemonset-l8tcm"
Sep 10 19:51:18.770128 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:51:18.770096    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bkw25\" (UniqueName: \"kubernetes.io/projected/75a80dee-0c9e-4e5e-bc5f-7642454e162e-kube-api-access-bkw25\") pod \"perf-node-gather-daemonset-l8tcm\" (UID: \"75a80dee-0c9e-4e5e-bc5f-7642454e162e\") " pod="openshift-must-gather-5hbck/perf-node-gather-daemonset-l8tcm"
Sep 10 19:51:18.873958 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:51:18.873896    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5hbck/perf-node-gather-daemonset-l8tcm"
Sep 10 19:51:19.146772 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:51:19.146688    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5hbck/perf-node-gather-daemonset-l8tcm"]
Sep 10 19:51:19.149862 ip-10-0-129-114 kubenswrapper[2575]: W0910 19:51:19.149831    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod75a80dee_0c9e_4e5e_bc5f_7642454e162e.slice/crio-fafed1378d770f3b69d479beb35f51f94aead6de035ebf11d9c659eb23ca1254 WatchSource:0}: Error finding container fafed1378d770f3b69d479beb35f51f94aead6de035ebf11d9c659eb23ca1254: Status 404 returned error can't find the container with id fafed1378d770f3b69d479beb35f51f94aead6de035ebf11d9c659eb23ca1254
Sep 10 19:51:19.297977 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:51:19.297936    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5hbck/perf-node-gather-daemonset-l8tcm" event={"ID":"75a80dee-0c9e-4e5e-bc5f-7642454e162e","Type":"ContainerStarted","Data":"ff7b4bb144879efe781a66a604baaa48490ed2ad186d3a599f2b8ab96d1e7d50"}
Sep 10 19:51:19.298161 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:51:19.298011    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5hbck/perf-node-gather-daemonset-l8tcm" event={"ID":"75a80dee-0c9e-4e5e-bc5f-7642454e162e","Type":"ContainerStarted","Data":"fafed1378d770f3b69d479beb35f51f94aead6de035ebf11d9c659eb23ca1254"}
Sep 10 19:51:19.298161 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:51:19.298084    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-5hbck/perf-node-gather-daemonset-l8tcm"
Sep 10 19:51:19.319232 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:51:19.319181    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-5hbck/perf-node-gather-daemonset-l8tcm" podStartSLOduration=1.319166927 podStartE2EDuration="1.319166927s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 19:51:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:51:19.318266752 +0000 UTC m=+2150.378094419" watchObservedRunningTime="2026-09-10 19:51:19.319166927 +0000 UTC m=+2150.378994647"
Sep 10 19:51:19.452856 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:51:19.452807    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-85d98c998f-dlmbt_8b439918-46b0-436f-923e-f765d793a7fd/download-server/0.log"
Sep 10 19:51:20.136403 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:51:20.136372    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-695c895bd6-xzxvx_977b75ac-232a-46e1-92d4-5496efa09faa/volume-data-source-validator/0.log"
Sep 10 19:51:21.345673 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:51:21.345640    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-rjdmw_7bcc0525-ded3-4f3b-896e-73c566a68e34/dns/0.log"
Sep 10 19:51:21.365975 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:51:21.365947    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-rjdmw_7bcc0525-ded3-4f3b-896e-73c566a68e34/kube-rbac-proxy/0.log"
Sep 10 19:51:21.490474 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:51:21.490448    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-vmvhn_d093d956-9c62-4201-a4ae-60187edb6fbf/dns-node-resolver/0.log"
Sep 10 19:51:22.110589 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:51:22.110514    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-nh8r2_446e8624-4aaf-471d-9653-2451db8c6889/node-ca/0.log"
Sep 10 19:51:23.359239 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:51:23.359206    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5dd7bdc797-q87x9_6d6559e0-b731-4c66-aa5b-7dcbb9de04ad/kube-auth-proxy/0.log"
Sep 10 19:51:23.444723 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:51:23.444689    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-flfb7_8924df89-f4a8-4d61-a636-066edf4dc4f0/payload-pre-processing/0.log"
Sep 10 19:51:24.400097 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:51:24.400059    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-gbtv9_4cb26dfc-5d83-4cec-b4f2-52f3ff4c47d6/serve-healthcheck-canary/0.log"
Sep 10 19:51:25.021464 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:51:25.021436    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-ktbdm_e06bba47-fb00-41eb-a0d4-34c98888d69a/kube-rbac-proxy/0.log"
Sep 10 19:51:25.042255 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:51:25.042231    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-ktbdm_e06bba47-fb00-41eb-a0d4-34c98888d69a/exporter/0.log"
Sep 10 19:51:25.063585 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:51:25.063564    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-ktbdm_e06bba47-fb00-41eb-a0d4-34c98888d69a/extractor/0.log"
Sep 10 19:51:25.312957 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:51:25.312903    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-5hbck/perf-node-gather-daemonset-l8tcm"
Sep 10 19:51:26.472074 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:51:26.472040    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbmp5_a76b5c99-a97f-461d-bce3-1fc84ef52bae/konnectivity-agent/0.log" limit=1024
Sep 10 19:51:27.762753 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:51:27.762725    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-gdpds_880bf6ca-872f-4005-b5cc-cd60bdf6caec/manager/0.log"
Sep 10 19:51:28.006121 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:51:28.006087    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5d477566c-5h625_d5588d80-e182-43c2-aecb-44038005f945/manager/0.log"
Sep 10 19:51:28.035312 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:51:28.035288    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5d477566c-5h625_d5588d80-e182-43c2-aecb-44038005f945/copy-manifests/0.log"
Sep 10 19:51:29.477695 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:51:29.477670    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5dd7bdc797-q87x9_6d6559e0-b731-4c66-aa5b-7dcbb9de04ad/kube-auth-proxy/0.log"
Sep 10 19:51:29.545994 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:51:29.545949    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-b5db5bfb7-hxskb_6d168b87-3c20-4850-b276-d9fdeb0b53a8/manager/0.log"
Sep 10 19:51:29.548454 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:51:29.548410    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-flfb7_8924df89-f4a8-4d61-a636-066edf4dc4f0/payload-pre-processing/0.log"
Sep 10 19:51:29.572622 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:51:29.572602    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-hg59c_91367cd9-4fe3-4775-90cd-3a5982b38057/openshift-lws-operator/0.log"
Sep 10 19:51:29.814331 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:51:29.814255    2575 scope.go:117] "RemoveContainer" containerID="c2605048c7cf4e60599bd57993645faab5e337690cfc76b9cafc060acd93a7e6"
Sep 10 19:51:29.822092 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:51:29.822062    2575 scope.go:117] "RemoveContainer" containerID="98312b961cbf0779e60d0853a0f58403205a2ad5cd0f9c9167c2abbe10e45af3"
Sep 10 19:51:31.490187 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:51:31.490129    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5dd7bdc797-q87x9_6d6559e0-b731-4c66-aa5b-7dcbb9de04ad/kube-auth-proxy/0.log"
Sep 10 19:51:31.587408 ip-10-0-129-114 kubenswrapper[2575]: I0910 19:51:31.587374    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-flfb7_8924df89-f4a8-4d61-a636-066edf4dc4f0/payload-pre-processing/0.log"