Sep 16 02:12:58.377234 ip-10-0-136-118 systemd[1]: Starting Kubernetes Kubelet...
Sep 16 02:12:58.833728 ip-10-0-136-118 kubenswrapper[2579]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 16 02:12:58.833728 ip-10-0-136-118 kubenswrapper[2579]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 16 02:12:58.833728 ip-10-0-136-118 kubenswrapper[2579]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 16 02:12:58.833728 ip-10-0-136-118 kubenswrapper[2579]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 16 02:12:58.833728 ip-10-0-136-118 kubenswrapper[2579]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 16 02:12:58.834633 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.834550    2579 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 16 02:12:58.841395 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.841378    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 16 02:12:58.841395 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.841393    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 16 02:12:58.841395 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.841397    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 16 02:12:58.841487 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.841400    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 16 02:12:58.841487 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.841403    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 16 02:12:58.841487 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.841407    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 16 02:12:58.841487 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.841410    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 16 02:12:58.841487 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.841413    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 16 02:12:58.841487 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.841415    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 16 02:12:58.841487 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.841418    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 16 02:12:58.841487 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.841421    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 16 02:12:58.841487 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.841424    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 16 02:12:58.841487 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.841427    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 16 02:12:58.841487 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.841430    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 16 02:12:58.841487 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.841432    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 16 02:12:58.841487 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.841435    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 16 02:12:58.841487 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.841437    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 16 02:12:58.841487 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.841440    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 16 02:12:58.841487 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.841443    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 16 02:12:58.841487 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.841445    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 16 02:12:58.841487 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.841448    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 16 02:12:58.841487 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.841450    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 16 02:12:58.841487 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.841452    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 16 02:12:58.842020 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.841455    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 16 02:12:58.842020 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.841457    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 16 02:12:58.842020 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.841460    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 16 02:12:58.842020 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.841462    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 16 02:12:58.842020 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.841465    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 16 02:12:58.842020 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.841468    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 16 02:12:58.842020 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.841470    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 16 02:12:58.842020 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.841472    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 16 02:12:58.842020 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.841475    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 16 02:12:58.842020 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.841477    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 16 02:12:58.842020 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.841479    2579 feature_gate.go:328] unrecognized feature gate: Example2
Sep 16 02:12:58.842020 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.841482    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 16 02:12:58.842020 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.841484    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 16 02:12:58.842020 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.841488    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 16 02:12:58.842020 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.841491    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 16 02:12:58.842020 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.841493    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 16 02:12:58.842020 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.841496    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 16 02:12:58.842020 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.841499    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 16 02:12:58.842020 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.841501    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 16 02:12:58.842020 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.841504    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 16 02:12:58.842511 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.841506    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 16 02:12:58.842511 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.841509    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 16 02:12:58.842511 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.841511    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 16 02:12:58.842511 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.841514    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 16 02:12:58.842511 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.841516    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 16 02:12:58.842511 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.841518    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 16 02:12:58.842511 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.841522    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 16 02:12:58.842511 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.841524    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 16 02:12:58.842511 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.841527    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 16 02:12:58.842511 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.841529    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 16 02:12:58.842511 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.841532    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 16 02:12:58.842511 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.841534    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 16 02:12:58.842511 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.841538    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 16 02:12:58.842511 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.841542    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 16 02:12:58.842511 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.841545    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 16 02:12:58.842511 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.841548    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 16 02:12:58.842511 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.841551    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 16 02:12:58.842511 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.841554    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 16 02:12:58.842511 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.841558    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 16 02:12:58.842991 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.841562    2579 feature_gate.go:328] unrecognized feature gate: Example
Sep 16 02:12:58.842991 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.841564    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 16 02:12:58.842991 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.841567    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 16 02:12:58.842991 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.841570    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 16 02:12:58.842991 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.841573    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 16 02:12:58.842991 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.841577    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 16 02:12:58.842991 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.841580    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 16 02:12:58.842991 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.841583    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 16 02:12:58.842991 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.841585    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 16 02:12:58.842991 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.841588    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 16 02:12:58.842991 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.841590    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 16 02:12:58.842991 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.841592    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 16 02:12:58.842991 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.841595    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 16 02:12:58.842991 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.841597    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 16 02:12:58.842991 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.841601    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 16 02:12:58.842991 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.841604    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 16 02:12:58.842991 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.841607    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 16 02:12:58.842991 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.841610    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 16 02:12:58.842991 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.841613    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 16 02:12:58.842991 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.841616    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 16 02:12:58.843471 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.841619    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 16 02:12:58.843471 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.841621    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 16 02:12:58.843471 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.841624    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 16 02:12:58.843471 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.841627    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 16 02:12:58.843471 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.841630    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 16 02:12:58.843471 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.841633    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 16 02:12:58.843471 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.841635    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 16 02:12:58.843471 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.841638    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 16 02:12:58.843471 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.841640    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 16 02:12:58.843471 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.841643    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 16 02:12:58.843471 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.841645    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 16 02:12:58.843471 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.841648    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 16 02:12:58.843471 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.841650    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 16 02:12:58.843471 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.841653    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 16 02:12:58.843471 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.841655    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 16 02:12:58.843471 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.841658    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 16 02:12:58.843471 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.841660    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 16 02:12:58.843471 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.842361    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 16 02:12:58.843471 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.842368    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 16 02:12:58.843471 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.842371    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 16 02:12:58.843964 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.842373    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 16 02:12:58.843964 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.842376    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 16 02:12:58.843964 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.842379    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 16 02:12:58.843964 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.842381    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 16 02:12:58.843964 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.842384    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 16 02:12:58.843964 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.842387    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 16 02:12:58.843964 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.842389    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 16 02:12:58.843964 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.842392    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 16 02:12:58.843964 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.842395    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 16 02:12:58.843964 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.842397    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 16 02:12:58.843964 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.842400    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 16 02:12:58.843964 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.842403    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 16 02:12:58.843964 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.842405    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 16 02:12:58.843964 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.842408    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 16 02:12:58.843964 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.842410    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 16 02:12:58.843964 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.842412    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 16 02:12:58.843964 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.842415    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 16 02:12:58.843964 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.842418    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 16 02:12:58.843964 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.842421    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 16 02:12:58.843964 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.842423    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 16 02:12:58.844461 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.842426    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 16 02:12:58.844461 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.842430    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 16 02:12:58.844461 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.842434    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 16 02:12:58.844461 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.842438    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 16 02:12:58.844461 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.842441    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 16 02:12:58.844461 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.842444    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 16 02:12:58.844461 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.842446    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 16 02:12:58.844461 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.842449    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 16 02:12:58.844461 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.842451    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 16 02:12:58.844461 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.842455    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 16 02:12:58.844461 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.842457    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 16 02:12:58.844461 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.842460    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 16 02:12:58.844461 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.842462    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 16 02:12:58.844461 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.842465    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 16 02:12:58.844461 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.842467    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 16 02:12:58.844461 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.842470    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 16 02:12:58.844461 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.842472    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 16 02:12:58.844461 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.842474    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 16 02:12:58.844461 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.842477    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 16 02:12:58.844934 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.842479    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 16 02:12:58.844934 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.842482    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 16 02:12:58.844934 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.842485    2579 feature_gate.go:328] unrecognized feature gate: Example2
Sep 16 02:12:58.844934 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.842487    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 16 02:12:58.844934 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.842490    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 16 02:12:58.844934 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.842493    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 16 02:12:58.844934 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.842495    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 16 02:12:58.844934 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.842497    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 16 02:12:58.844934 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.842501    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 16 02:12:58.844934 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.842505    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 16 02:12:58.844934 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.842509    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 16 02:12:58.844934 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.842512    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 16 02:12:58.844934 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.842515    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 16 02:12:58.844934 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.842519    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 16 02:12:58.844934 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.842521    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 16 02:12:58.844934 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.842524    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 16 02:12:58.844934 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.842527    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 16 02:12:58.844934 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.842529    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 16 02:12:58.844934 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.842532    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 16 02:12:58.845409 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.842534    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 16 02:12:58.845409 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.842536    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 16 02:12:58.845409 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.842539    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 16 02:12:58.845409 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.842541    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 16 02:12:58.845409 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.842544    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 16 02:12:58.845409 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.842546    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 16 02:12:58.845409 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.842549    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 16 02:12:58.845409 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.842551    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 16 02:12:58.845409 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.842554    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 16 02:12:58.845409 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.842556    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 16 02:12:58.845409 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.842558    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 16 02:12:58.845409 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.842561    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 16 02:12:58.845409 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.842563    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 16 02:12:58.845409 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.842566    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 16 02:12:58.845409 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.842568    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 16 02:12:58.845409 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.842571    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 16 02:12:58.845409 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.842573    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 16 02:12:58.845409 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.842576    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 16 02:12:58.845409 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.842578    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 16 02:12:58.845409 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.842581    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 16 02:12:58.845935 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.842583    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 16 02:12:58.845935 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.842586    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 16 02:12:58.845935 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.842588    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 16 02:12:58.845935 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.842591    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 16 02:12:58.845935 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.842593    2579 feature_gate.go:328] unrecognized feature gate: Example
Sep 16 02:12:58.845935 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.842596    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 16 02:12:58.845935 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.842598    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 16 02:12:58.845935 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.842600    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 16 02:12:58.845935 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.842603    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 16 02:12:58.845935 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.842605    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 16 02:12:58.845935 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.842608    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 16 02:12:58.845935 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.842610    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 16 02:12:58.845935 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.842612    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 16 02:12:58.845935 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.842615    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 16 02:12:58.845935 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.842617    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 16 02:12:58.845935 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.842620    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 16 02:12:58.845935 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.842622    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 16 02:12:58.845935 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.842625    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 16 02:12:58.845935 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.843008    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 16 02:12:58.845935 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.843013    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 16 02:12:58.846417 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.843016    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 16 02:12:58.846417 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.843019    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 16 02:12:58.846417 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.843021    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 16 02:12:58.846417 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.843024    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 16 02:12:58.846417 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.843027    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 16 02:12:58.846417 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.843029    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 16 02:12:58.846417 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.843032    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 16 02:12:58.846417 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.843035    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 16 02:12:58.846417 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.843037    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 16 02:12:58.846417 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.843040    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 16 02:12:58.846417 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.843043    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 16 02:12:58.846417 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.843046    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 16 02:12:58.846417 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.843048    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 16 02:12:58.846417 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.843051    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 16 02:12:58.846417 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.843054    2579 feature_gate.go:328] unrecognized feature gate: Example2
Sep 16 02:12:58.846417 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.843056    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 16 02:12:58.846417 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.843058    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 16 02:12:58.846417 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.843061    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 16 02:12:58.846417 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.843064    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 16 02:12:58.846417 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.843066    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 16 02:12:58.846915 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.843069    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 16 02:12:58.846915 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.843071    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 16 02:12:58.846915 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.843074    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 16 02:12:58.846915 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.843076    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 16 02:12:58.846915 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.843079    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 16 02:12:58.846915 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.843081    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 16 02:12:58.846915 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.843084    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 16 02:12:58.846915 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.843086    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 16 02:12:58.846915 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.843089    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 16 02:12:58.846915 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.843092    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 16 02:12:58.846915 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.843094    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 16 02:12:58.846915 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.843097    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 16 02:12:58.846915 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.843100    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 16 02:12:58.846915 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.843103    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 16 02:12:58.846915 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.843106    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 16 02:12:58.846915 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.843108    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 16 02:12:58.846915 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.843110    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 16 02:12:58.846915 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.843113    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 16 02:12:58.846915 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.843115    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 16 02:12:58.847411 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.843119    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 16 02:12:58.847411 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.843122    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 16 02:12:58.847411 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.843125    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 16 02:12:58.847411 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.843128    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 16 02:12:58.847411 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.843131    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 16 02:12:58.847411 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.843134    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 16 02:12:58.847411 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.843136    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 16 02:12:58.847411 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.843139    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 16 02:12:58.847411 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.843142    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 16 02:12:58.847411 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.843145    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 16 02:12:58.847411 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.843147    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 16 02:12:58.847411 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.843150    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 16 02:12:58.847411 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.843152    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 16 02:12:58.847411 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.843155    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 16 02:12:58.847411 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.843157    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 16 02:12:58.847411 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.843159    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 16 02:12:58.847411 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.843163    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 16 02:12:58.847411 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.843165    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 16 02:12:58.847411 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.843168    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 16 02:12:58.847411 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.843170    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 16 02:12:58.847903 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.843173    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 16 02:12:58.847903 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.843175    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 16 02:12:58.847903 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.843178    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 16 02:12:58.847903 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.843180    2579 feature_gate.go:328] unrecognized feature gate: Example
Sep 16 02:12:58.847903 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.843183    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 16 02:12:58.847903 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.843185    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 16 02:12:58.847903 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.843187    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 16 02:12:58.847903 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.843190    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 16 02:12:58.847903 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.843192    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 16 02:12:58.847903 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.843195    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 16 02:12:58.847903 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.843197    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 16 02:12:58.847903 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.843199    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 16 02:12:58.847903 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.843202    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 16 02:12:58.847903 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.843204    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 16 02:12:58.847903 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.843207    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 16 02:12:58.847903 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.843209    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 16 02:12:58.847903 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.843211    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 16 02:12:58.847903 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.843216    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 16 02:12:58.847903 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.843219    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 16 02:12:58.848393 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.843222    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 16 02:12:58.848393 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.843225    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 16 02:12:58.848393 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.843228    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 16 02:12:58.848393 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.843230    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 16 02:12:58.848393 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.843233    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 16 02:12:58.848393 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.843235    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 16 02:12:58.848393 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.843237    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 16 02:12:58.848393 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.843240    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 16 02:12:58.848393 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.843242    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 16 02:12:58.848393 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.843245    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 16 02:12:58.848393 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.843247    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 16 02:12:58.848393 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.843249    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 16 02:12:58.848393 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.843251    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 16 02:12:58.848393 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.843254    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 16 02:12:58.848393 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.843257    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 16 02:12:58.848393 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.843259    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 16 02:12:58.848393 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.843262    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 16 02:12:58.848393 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.843264    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 16 02:12:58.848393 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.843267    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 16 02:12:58.848393 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843356    2579 flags.go:64] FLAG: --address="0.0.0.0"
Sep 16 02:12:58.848889 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843409    2579 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 16 02:12:58.848889 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843417    2579 flags.go:64] FLAG: --anonymous-auth="true"
Sep 16 02:12:58.848889 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843421    2579 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 16 02:12:58.848889 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843425    2579 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 16 02:12:58.848889 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843428    2579 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 16 02:12:58.848889 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843432    2579 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 16 02:12:58.848889 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843436    2579 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 16 02:12:58.848889 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843439    2579 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 16 02:12:58.848889 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843442    2579 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 16 02:12:58.848889 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843445    2579 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 16 02:12:58.848889 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843449    2579 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 16 02:12:58.848889 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843453    2579 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 16 02:12:58.848889 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843458    2579 flags.go:64] FLAG: --cgroup-root=""
Sep 16 02:12:58.848889 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843461    2579 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 16 02:12:58.848889 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843463    2579 flags.go:64] FLAG: --client-ca-file=""
Sep 16 02:12:58.848889 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843466    2579 flags.go:64] FLAG: --cloud-provider="external"
Sep 16 02:12:58.848889 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843469    2579 flags.go:64] FLAG: --cluster-dns="[]"
Sep 16 02:12:58.848889 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843473    2579 flags.go:64] FLAG: --cluster-domain=""
Sep 16 02:12:58.848889 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843476    2579 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 16 02:12:58.848889 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843479    2579 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 16 02:12:58.848889 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843482    2579 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 16 02:12:58.848889 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843486    2579 flags.go:64] FLAG: --container-log-max-files="5"
Sep 16 02:12:58.848889 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843489    2579 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 16 02:12:58.848889 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843494    2579 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 16 02:12:58.849472 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843497    2579 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 16 02:12:58.849472 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843500    2579 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 16 02:12:58.849472 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843502    2579 flags.go:64] FLAG: --contention-profiling="false"
Sep 16 02:12:58.849472 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843505    2579 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 16 02:12:58.849472 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843508    2579 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 16 02:12:58.849472 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843511    2579 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 16 02:12:58.849472 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843514    2579 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 16 02:12:58.849472 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843518    2579 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 16 02:12:58.849472 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843520    2579 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 16 02:12:58.849472 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843523    2579 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 16 02:12:58.849472 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843526    2579 flags.go:64] FLAG: --enable-load-reader="false"
Sep 16 02:12:58.849472 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843528    2579 flags.go:64] FLAG: --enable-server="true"
Sep 16 02:12:58.849472 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843531    2579 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 16 02:12:58.849472 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843536    2579 flags.go:64] FLAG: --event-burst="100"
Sep 16 02:12:58.849472 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843539    2579 flags.go:64] FLAG: --event-qps="50"
Sep 16 02:12:58.849472 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843542    2579 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 16 02:12:58.849472 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843544    2579 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 16 02:12:58.849472 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843547    2579 flags.go:64] FLAG: --eviction-hard=""
Sep 16 02:12:58.849472 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843551    2579 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 16 02:12:58.849472 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843554    2579 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 16 02:12:58.849472 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843558    2579 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 16 02:12:58.849472 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843561    2579 flags.go:64] FLAG: --eviction-soft=""
Sep 16 02:12:58.849472 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843564    2579 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 16 02:12:58.849472 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843566    2579 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 16 02:12:58.849472 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843569    2579 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 16 02:12:58.850105 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843572    2579 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 16 02:12:58.850105 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843575    2579 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 16 02:12:58.850105 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843577    2579 flags.go:64] FLAG: --fail-swap-on="true"
Sep 16 02:12:58.850105 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843580    2579 flags.go:64] FLAG: --feature-gates=""
Sep 16 02:12:58.850105 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843584    2579 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 16 02:12:58.850105 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843586    2579 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 16 02:12:58.850105 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843591    2579 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 16 02:12:58.850105 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843594    2579 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 16 02:12:58.850105 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843597    2579 flags.go:64] FLAG: --healthz-port="10248"
Sep 16 02:12:58.850105 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843600    2579 flags.go:64] FLAG: --help="false"
Sep 16 02:12:58.850105 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843603    2579 flags.go:64] FLAG: --hostname-override="ip-10-0-136-118.ec2.internal"
Sep 16 02:12:58.850105 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843606    2579 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 16 02:12:58.850105 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843608    2579 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 16 02:12:58.850105 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843611    2579 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 16 02:12:58.850105 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843615    2579 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 16 02:12:58.850105 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843618    2579 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 16 02:12:58.850105 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843620    2579 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 16 02:12:58.850105 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843623    2579 flags.go:64] FLAG: --image-service-endpoint=""
Sep 16 02:12:58.850105 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843626    2579 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 16 02:12:58.850105 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843629    2579 flags.go:64] FLAG: --kube-api-burst="100"
Sep 16 02:12:58.850105 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843631    2579 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 16 02:12:58.850105 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843634    2579 flags.go:64] FLAG: --kube-api-qps="50"
Sep 16 02:12:58.850105 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843637    2579 flags.go:64] FLAG: --kube-reserved=""
Sep 16 02:12:58.850105 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843640    2579 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 16 02:12:58.850679 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843642    2579 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 16 02:12:58.850679 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843645    2579 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 16 02:12:58.850679 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843648    2579 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 16 02:12:58.850679 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843653    2579 flags.go:64] FLAG: --lock-file=""
Sep 16 02:12:58.850679 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843656    2579 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 16 02:12:58.850679 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843659    2579 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 16 02:12:58.850679 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843661    2579 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 16 02:12:58.850679 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843667    2579 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 16 02:12:58.850679 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843670    2579 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 16 02:12:58.850679 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843672    2579 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 16 02:12:58.850679 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843676    2579 flags.go:64] FLAG: --logging-format="text"
Sep 16 02:12:58.850679 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843679    2579 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 16 02:12:58.850679 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843682    2579 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 16 02:12:58.850679 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843684    2579 flags.go:64] FLAG: --manifest-url=""
Sep 16 02:12:58.850679 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843691    2579 flags.go:64] FLAG: --manifest-url-header=""
Sep 16 02:12:58.850679 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843695    2579 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 16 02:12:58.850679 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843698    2579 flags.go:64] FLAG: --max-open-files="1000000"
Sep 16 02:12:58.850679 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843702    2579 flags.go:64] FLAG: --max-pods="110"
Sep 16 02:12:58.850679 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843705    2579 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 16 02:12:58.850679 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843708    2579 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 16 02:12:58.850679 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843711    2579 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 16 02:12:58.850679 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843714    2579 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 16 02:12:58.850679 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843716    2579 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 16 02:12:58.850679 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843719    2579 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 16 02:12:58.850679 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843722    2579 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 16 02:12:58.851310 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843729    2579 flags.go:64] FLAG: --node-status-max-images="50"
Sep 16 02:12:58.851310 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843732    2579 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 16 02:12:58.851310 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843735    2579 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 16 02:12:58.851310 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843737    2579 flags.go:64] FLAG: --pod-cidr=""
Sep 16 02:12:58.851310 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843740    2579 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:eee392dbe3ac2ed4a38bca29ae3bd486f5b7780406c62c600154cd1906871b10"
Sep 16 02:12:58.851310 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843745    2579 flags.go:64] FLAG: --pod-manifest-path=""
Sep 16 02:12:58.851310 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843748    2579 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 16 02:12:58.851310 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843751    2579 flags.go:64] FLAG: --pods-per-core="0"
Sep 16 02:12:58.851310 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843754    2579 flags.go:64] FLAG: --port="10250"
Sep 16 02:12:58.851310 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843756    2579 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 16 02:12:58.851310 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843759    2579 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-01a9147f1cc1b7293"
Sep 16 02:12:58.851310 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843763    2579 flags.go:64] FLAG: --qos-reserved=""
Sep 16 02:12:58.851310 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843767    2579 flags.go:64] FLAG: --read-only-port="10255"
Sep 16 02:12:58.851310 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843769    2579 flags.go:64] FLAG: --register-node="true"
Sep 16 02:12:58.851310 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843772    2579 flags.go:64] FLAG: --register-with-taints=""
Sep 16 02:12:58.851310 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843776    2579 flags.go:64] FLAG: --registry-burst="10"
Sep 16 02:12:58.851310 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843778    2579 flags.go:64] FLAG: --registry-qps="5"
Sep 16 02:12:58.851310 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843781    2579 flags.go:64] FLAG: --reserved-cpus=""
Sep 16 02:12:58.851310 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843784    2579 flags.go:64] FLAG: --reserved-memory=""
Sep 16 02:12:58.851310 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843787    2579 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 16 02:12:58.851310 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843790    2579 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 16 02:12:58.851310 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843795    2579 flags.go:64] FLAG: --rotate-certificates="false"
Sep 16 02:12:58.851310 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843798    2579 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 16 02:12:58.851310 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843800    2579 flags.go:64] FLAG: --runonce="false"
Sep 16 02:12:58.851310 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843803    2579 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 16 02:12:58.852310 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843806    2579 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 16 02:12:58.852310 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843809    2579 flags.go:64] FLAG: --seccomp-default="false"
Sep 16 02:12:58.852310 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843812    2579 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 16 02:12:58.852310 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843815    2579 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 16 02:12:58.852310 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843818    2579 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 16 02:12:58.852310 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843821    2579 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 16 02:12:58.852310 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843824    2579 flags.go:64] FLAG: --storage-driver-password="root"
Sep 16 02:12:58.852310 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843826    2579 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 16 02:12:58.852310 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843829    2579 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 16 02:12:58.852310 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843832    2579 flags.go:64] FLAG: --storage-driver-user="root"
Sep 16 02:12:58.852310 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843834    2579 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 16 02:12:58.852310 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843838    2579 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 16 02:12:58.852310 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843840    2579 flags.go:64] FLAG: --system-cgroups=""
Sep 16 02:12:58.852310 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843843    2579 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 16 02:12:58.852310 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843863    2579 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 16 02:12:58.852310 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843866    2579 flags.go:64] FLAG: --tls-cert-file=""
Sep 16 02:12:58.852310 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843869    2579 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 16 02:12:58.852310 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843873    2579 flags.go:64] FLAG: --tls-min-version=""
Sep 16 02:12:58.852310 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843877    2579 flags.go:64] FLAG: --tls-private-key-file=""
Sep 16 02:12:58.852310 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843880    2579 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 16 02:12:58.852310 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843883    2579 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 16 02:12:58.852310 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843886    2579 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 16 02:12:58.852310 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843889    2579 flags.go:64] FLAG: --v="2"
Sep 16 02:12:58.852310 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843893    2579 flags.go:64] FLAG: --version="false"
Sep 16 02:12:58.852310 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843896    2579 flags.go:64] FLAG: --vmodule=""
Sep 16 02:12:58.853415 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843900    2579 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 16 02:12:58.853415 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.843903    2579 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 16 02:12:58.853415 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.844014    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 16 02:12:58.853415 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.844019    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 16 02:12:58.853415 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.844023    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 16 02:12:58.853415 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.844026    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 16 02:12:58.853415 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.844028    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 16 02:12:58.853415 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.844031    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 16 02:12:58.853415 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.844034    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 16 02:12:58.853415 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.844036    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 16 02:12:58.853415 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.844038    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 16 02:12:58.853415 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.844043    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 16 02:12:58.853415 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.844047    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 16 02:12:58.853415 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.844050    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 16 02:12:58.853415 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.844053    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 16 02:12:58.853415 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.844056    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 16 02:12:58.853415 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.844059    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 16 02:12:58.853415 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.844062    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 16 02:12:58.853415 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.844064    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 16 02:12:58.854241 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.844067    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 16 02:12:58.854241 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.844070    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 16 02:12:58.854241 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.844072    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 16 02:12:58.854241 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.844075    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 16 02:12:58.854241 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.844077    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 16 02:12:58.854241 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.844080    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 16 02:12:58.854241 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.844083    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 16 02:12:58.854241 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.844085    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 16 02:12:58.854241 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.844088    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 16 02:12:58.854241 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.844091    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 16 02:12:58.854241 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.844093    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 16 02:12:58.854241 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.844096    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 16 02:12:58.854241 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.844098    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 16 02:12:58.854241 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.844101    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 16 02:12:58.854241 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.844103    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 16 02:12:58.854241 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.844106    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 16 02:12:58.854241 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.844109    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 16 02:12:58.854241 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.844111    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 16 02:12:58.854241 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.844114    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 16 02:12:58.854241 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.844116    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 16 02:12:58.855127 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.844119    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 16 02:12:58.855127 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.844121    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 16 02:12:58.855127 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.844124    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 16 02:12:58.855127 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.844126    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 16 02:12:58.855127 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.844128    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 16 02:12:58.855127 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.844131    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 16 02:12:58.855127 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.844134    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 16 02:12:58.855127 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.844136    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 16 02:12:58.855127 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.844138    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 16 02:12:58.855127 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.844141    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 16 02:12:58.855127 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.844143    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 16 02:12:58.855127 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.844146    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 16 02:12:58.855127 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.844148    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 16 02:12:58.855127 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.844150    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 16 02:12:58.855127 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.844153    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 16 02:12:58.855127 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.844155    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 16 02:12:58.855127 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.844158    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 16 02:12:58.855127 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.844160    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 16 02:12:58.855127 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.844163    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 16 02:12:58.855127 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.844165    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 16 02:12:58.855989 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.844168    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 16 02:12:58.855989 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.844170    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 16 02:12:58.855989 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.844173    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 16 02:12:58.855989 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.844175    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 16 02:12:58.855989 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.844178    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 16 02:12:58.855989 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.844180    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 16 02:12:58.855989 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.844182    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 16 02:12:58.855989 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.844185    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 16 02:12:58.855989 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.844188    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 16 02:12:58.855989 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.844191    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 16 02:12:58.855989 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.844193    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 16 02:12:58.855989 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.844195    2579 feature_gate.go:328] unrecognized feature gate: Example
Sep 16 02:12:58.855989 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.844198    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 16 02:12:58.855989 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.844201    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 16 02:12:58.855989 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.844203    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 16 02:12:58.855989 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.844205    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 16 02:12:58.855989 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.844208    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 16 02:12:58.855989 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.844210    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 16 02:12:58.855989 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.844213    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 16 02:12:58.855989 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.844215    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 16 02:12:58.856514 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.844217    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 16 02:12:58.856514 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.844220    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 16 02:12:58.856514 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.844222    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 16 02:12:58.856514 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.844224    2579 feature_gate.go:328] unrecognized feature gate: Example2
Sep 16 02:12:58.856514 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.844228    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 16 02:12:58.856514 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.844231    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 16 02:12:58.856514 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.844233    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 16 02:12:58.856514 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.844235    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 16 02:12:58.856514 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.844238    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 16 02:12:58.856514 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.844240    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 16 02:12:58.856514 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.844243    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 16 02:12:58.856514 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.844246    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 16 02:12:58.856514 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.844250    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 16 02:12:58.856514 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.844252    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 16 02:12:58.856514 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.844255    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 16 02:12:58.856514 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.844265    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 16 02:12:58.856514 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.844269    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 16 02:12:58.856514 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.844272    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 16 02:12:58.856514 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.844276    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 16 02:12:58.857342 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.844281    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 16 02:12:58.857342 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.844287    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 16 02:12:58.857342 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.844290    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 16 02:12:58.857342 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.845034    2579 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 16 02:12:58.857342 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.854243    2579 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 16 02:12:58.857342 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.854260    2579 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 16 02:12:58.857342 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.854289    2579 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 16 02:12:58.857342 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.854298    2579 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 16 02:12:58.857342 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.854406    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 16 02:12:58.857342 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.854414    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 16 02:12:58.857342 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.854420    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 16 02:12:58.857342 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.854425    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 16 02:12:58.857342 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.854429    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 16 02:12:58.857342 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.854435    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 16 02:12:58.857342 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.854439    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 16 02:12:58.858014 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.854444    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 16 02:12:58.858014 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.854448    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 16 02:12:58.858014 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.854453    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 16 02:12:58.858014 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.854459    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 16 02:12:58.858014 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.854489    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 16 02:12:58.858014 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.854495    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 16 02:12:58.858014 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.854500    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 16 02:12:58.858014 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.854505    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 16 02:12:58.858014 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.854509    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 16 02:12:58.858014 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.854514    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 16 02:12:58.858014 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.854518    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 16 02:12:58.858014 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.854522    2579 feature_gate.go:328] unrecognized feature gate: Example2
Sep 16 02:12:58.858014 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.854527    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 16 02:12:58.858014 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.854531    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 16 02:12:58.858014 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.854535    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 16 02:12:58.858014 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.854542    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 16 02:12:58.858014 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.854547    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 16 02:12:58.858014 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.854552    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 16 02:12:58.858014 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.854557    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 16 02:12:58.858481 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.854561    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 16 02:12:58.858481 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.854566    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 16 02:12:58.858481 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.854570    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 16 02:12:58.858481 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.854576    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 16 02:12:58.858481 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.854580    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 16 02:12:58.858481 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.854584    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 16 02:12:58.858481 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.854589    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 16 02:12:58.858481 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.854593    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 16 02:12:58.858481 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.854597    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 16 02:12:58.858481 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.854601    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 16 02:12:58.858481 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.854605    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 16 02:12:58.858481 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.854609    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 16 02:12:58.858481 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.854613    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 16 02:12:58.858481 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.854617    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 16 02:12:58.858481 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.854621    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 16 02:12:58.858481 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.854625    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 16 02:12:58.858481 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.854629    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 16 02:12:58.858481 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.854636    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 16 02:12:58.858481 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.854641    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 16 02:12:58.858481 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.854645    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 16 02:12:58.859119 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.854649    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 16 02:12:58.859119 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.854653    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 16 02:12:58.859119 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.854657    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 16 02:12:58.859119 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.854661    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 16 02:12:58.859119 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.854665    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 16 02:12:58.859119 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.854670    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 16 02:12:58.859119 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.854674    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 16 02:12:58.859119 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.854678    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 16 02:12:58.859119 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.854682    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 16 02:12:58.859119 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.854686    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 16 02:12:58.859119 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.854690    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 16 02:12:58.859119 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.854694    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 16 02:12:58.859119 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.854698    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 16 02:12:58.859119 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.854703    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 16 02:12:58.859119 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.854708    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 16 02:12:58.859119 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.854712    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 16 02:12:58.859119 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.854718    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 16 02:12:58.859119 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.854722    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 16 02:12:58.859119 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.854726    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 16 02:12:58.859119 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.854731    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 16 02:12:58.859608 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.854736    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 16 02:12:58.859608 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.854740    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 16 02:12:58.859608 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.854744    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 16 02:12:58.859608 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.854748    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 16 02:12:58.859608 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.854753    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 16 02:12:58.859608 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.854756    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 16 02:12:58.859608 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.854761    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 16 02:12:58.859608 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.854765    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 16 02:12:58.859608 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.854769    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 16 02:12:58.859608 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.854773    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 16 02:12:58.859608 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.854778    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 16 02:12:58.859608 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.854782    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 16 02:12:58.859608 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.854786    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 16 02:12:58.859608 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.854790    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 16 02:12:58.859608 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.854794    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 16 02:12:58.859608 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.854798    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 16 02:12:58.859608 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.854802    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 16 02:12:58.859608 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.854806    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 16 02:12:58.859608 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.854810    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 16 02:12:58.859608 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.854814    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 16 02:12:58.860267 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.854818    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 16 02:12:58.860267 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.854822    2579 feature_gate.go:328] unrecognized feature gate: Example
Sep 16 02:12:58.860267 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.854826    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 16 02:12:58.860267 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.854830    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 16 02:12:58.860267 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.854834    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 16 02:12:58.860267 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.854838    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 16 02:12:58.860267 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.854842    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 16 02:12:58.860267 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.854847    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 16 02:12:58.860267 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.854869    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 16 02:12:58.860267 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.854874    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 16 02:12:58.860267 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.854878    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 16 02:12:58.860267 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.854883    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 16 02:12:58.860267 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.854886    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 16 02:12:58.860267 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.854894    2579 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 16 02:12:58.860627 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.855097    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 16 02:12:58.860627 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.855108    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 16 02:12:58.860627 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.855113    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 16 02:12:58.860627 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.855117    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 16 02:12:58.860627 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.855121    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 16 02:12:58.860627 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.855125    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 16 02:12:58.860627 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.855129    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 16 02:12:58.860627 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.855132    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 16 02:12:58.860627 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.855136    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 16 02:12:58.860627 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.855140    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 16 02:12:58.860627 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.855144    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 16 02:12:58.860627 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.855147    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 16 02:12:58.860627 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.855152    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 16 02:12:58.860627 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.855156    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 16 02:12:58.860627 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.855160    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 16 02:12:58.860627 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.855166    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 16 02:12:58.860627 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.855172    2579 feature_gate.go:328] unrecognized feature gate: Example2
Sep 16 02:12:58.860627 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.855177    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 16 02:12:58.860627 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.855182    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 16 02:12:58.861109 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.855186    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 16 02:12:58.861109 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.855191    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 16 02:12:58.861109 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.855195    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 16 02:12:58.861109 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.855199    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 16 02:12:58.861109 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.855203    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 16 02:12:58.861109 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.855208    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 16 02:12:58.861109 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.855212    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 16 02:12:58.861109 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.855216    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 16 02:12:58.861109 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.855221    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 16 02:12:58.861109 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.855231    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 16 02:12:58.861109 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.855236    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 16 02:12:58.861109 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.855240    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 16 02:12:58.861109 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.855244    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 16 02:12:58.861109 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.855248    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 16 02:12:58.861109 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.855252    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 16 02:12:58.861109 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.855256    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 16 02:12:58.861109 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.855260    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 16 02:12:58.861109 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.855263    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 16 02:12:58.861109 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.855268    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 16 02:12:58.861109 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.855272    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 16 02:12:58.861607 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.855276    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 16 02:12:58.861607 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.855280    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 16 02:12:58.861607 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.855284    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 16 02:12:58.861607 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.855288    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 16 02:12:58.861607 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.855292    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 16 02:12:58.861607 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.855297    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 16 02:12:58.861607 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.855301    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 16 02:12:58.861607 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.855305    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 16 02:12:58.861607 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.855309    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 16 02:12:58.861607 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.855313    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 16 02:12:58.861607 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.855317    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 16 02:12:58.861607 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.855322    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 16 02:12:58.861607 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.855326    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 16 02:12:58.861607 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.855330    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 16 02:12:58.861607 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.855334    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 16 02:12:58.861607 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.855338    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 16 02:12:58.861607 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.855342    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 16 02:12:58.861607 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.855346    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 16 02:12:58.861607 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.855350    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 16 02:12:58.861607 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.855354    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 16 02:12:58.862115 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.855358    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 16 02:12:58.862115 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.855362    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 16 02:12:58.862115 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.855367    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 16 02:12:58.862115 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.855371    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 16 02:12:58.862115 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.855375    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 16 02:12:58.862115 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.855379    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 16 02:12:58.862115 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.855383    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 16 02:12:58.862115 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.855387    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 16 02:12:58.862115 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.855391    2579 feature_gate.go:328] unrecognized feature gate: Example
Sep 16 02:12:58.862115 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.855395    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 16 02:12:58.862115 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.855399    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 16 02:12:58.862115 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.855403    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 16 02:12:58.862115 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.855407    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 16 02:12:58.862115 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.855411    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 16 02:12:58.862115 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.855415    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 16 02:12:58.862115 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.855419    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 16 02:12:58.862115 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.855423    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 16 02:12:58.862115 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.855427    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 16 02:12:58.862115 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.855431    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 16 02:12:58.862115 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.855435    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 16 02:12:58.862597 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.855439    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 16 02:12:58.862597 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.855443    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 16 02:12:58.862597 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.855447    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 16 02:12:58.862597 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.855450    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 16 02:12:58.862597 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.855454    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 16 02:12:58.862597 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.855460    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 16 02:12:58.862597 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.855466    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 16 02:12:58.862597 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.855470    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 16 02:12:58.862597 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.855474    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 16 02:12:58.862597 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.855479    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 16 02:12:58.862597 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.855482    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 16 02:12:58.862597 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.855487    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 16 02:12:58.862597 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.855491    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 16 02:12:58.862597 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.855495    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 16 02:12:58.862597 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.855499    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 16 02:12:58.862597 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.855504    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 16 02:12:58.862597 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.855508    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 16 02:12:58.862597 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.855512    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 16 02:12:58.862597 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.855516    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 16 02:12:58.862597 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:12:58.855520    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 16 02:12:58.863093 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.855528    2579 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 16 02:12:58.863093 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.855675    2579 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 16 02:12:58.863093 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.858718    2579 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 16 02:12:58.863093 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.859889    2579 server.go:1019] "Starting client certificate rotation"
Sep 16 02:12:58.863093 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.859998    2579 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 16 02:12:58.863093 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.860044    2579 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 16 02:12:58.863093 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.862794    2579 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 16 02:12:58.863894 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.863877    2579 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 16 02:12:58.873528 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.873510    2579 log.go:25] "Validated CRI v1 runtime API"
Sep 16 02:12:58.876661 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.876648    2579 log.go:25] "Validated CRI v1 image API"
Sep 16 02:12:58.879993 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.879970    2579 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 16 02:12:58.883545 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.883527    2579 fs.go:135] Filesystem UUIDs: map[3b2547b1-825f-4cdf-826a-58fd8ac72364:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 d0f955e4-d467-434b-8874-1ff3103aba3d:/dev/nvme0n1p3]
Sep 16 02:12:58.883610 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.883545    2579 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 16 02:12:58.888071 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.888054    2579 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 16 02:12:58.889306 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.889202    2579 manager.go:216] Machine: {Timestamp:2026-09-16 02:12:58.88765441 +0000 UTC m=+0.371785294 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3164632 MemoryCapacity:32812158976 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2eca72558df56b4a56462ef88fb660 SystemUUID:ec2eca72-558d-f56b-4a56-462ef88fb660 BootID:059e6fef-1b7b-440c-b820-8dd89b24d26c Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406081536 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406077440 Type:vfs Inodes:4005390 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:ff:40:b8:23 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:ff:40:b8:23 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:46:53:a8:1d:6d:ee Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812158976 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 16 02:12:58.889306 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.889304    2579 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 16 02:12:58.889422 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.889410    2579 manager.go:232] Version: {KernelVersion:5.14.0-570.140.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260908-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 16 02:12:58.892785 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.892762    2579 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 16 02:12:58.892948 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.892788    2579 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-136-118.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 16 02:12:58.893657 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.893647    2579 topology_manager.go:138] "Creating topology manager with none policy"
Sep 16 02:12:58.893688 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.893660    2579 container_manager_linux.go:309] "Creating device plugin manager"
Sep 16 02:12:58.893688 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.893669    2579 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 16 02:12:58.893688 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.893682    2579 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 16 02:12:58.894971 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.894961    2579 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 16 02:12:58.900415 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.900403    2579 state_mem.go:36] "Initialized new in-memory state store"
Sep 16 02:12:58.900537 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.900527    2579 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 16 02:12:58.900576 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.900557    2579 kubelet.go:486] "Attempting to sync node with API server"
Sep 16 02:12:58.900576 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.900567    2579 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 16 02:12:58.900638 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.900581    2579 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 16 02:12:58.900638 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.900591    2579 kubelet.go:398] "Adding apiserver pod source"
Sep 16 02:12:58.900638 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.900599    2579 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 16 02:12:58.901016 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.901005    2579 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 16 02:12:58.901062 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.901023    2579 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 16 02:12:58.903288 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.903268    2579 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 16 02:12:58.905396 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.905383    2579 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 16 02:12:58.905454 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.905413    2579 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 16 02:12:58.907448 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.907438    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 16 02:12:58.907496 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.907454    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 16 02:12:58.907496 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.907469    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 16 02:12:58.907496 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.907474    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 16 02:12:58.907496 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.907484    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 16 02:12:58.907496 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.907492    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 16 02:12:58.907627 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.907508    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 16 02:12:58.907627 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.907514    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 16 02:12:58.907627 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.907520    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 16 02:12:58.907627 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.907526    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 16 02:12:58.907627 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.907538    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 16 02:12:58.907627 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.907543    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 16 02:12:58.908461 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.908449    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 16 02:12:58.908514 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.908464    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 16 02:12:58.911232 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.911220    2579 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-136-118.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 16 02:12:58.912043 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:12:58.911999    2579 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-136-118.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 16 02:12:58.912084 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:12:58.912000    2579 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 16 02:12:58.912607 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.912595    2579 server.go:1268] "Started kubelet"
Sep 16 02:12:58.912791 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.912765    2579 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 16 02:12:58.912878 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.912796    2579 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 16 02:12:58.912878 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.912874    2579 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 16 02:12:58.913373 ip-10-0-136-118 systemd[1]: Started Kubernetes Kubelet.
Sep 16 02:12:58.916964 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.916939    2579 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 16 02:12:58.918767 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.918746    2579 server.go:310] "Adding debug handlers to kubelet server"
Sep 16 02:12:58.919280 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:12:58.919223    2579 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 16 02:12:58.923089 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:12:58.921973    2579 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-118.ec2.internal.18d5ab4e1bd7cfcb  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-118.ec2.internal,UID:ip-10-0-136-118.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-136-118.ec2.internal,},FirstTimestamp:2026-09-16 02:12:58.912567243 +0000 UTC m=+0.396698128,LastTimestamp:2026-09-16 02:12:58.912567243 +0000 UTC m=+0.396698128,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-118.ec2.internal,}"
Sep 16 02:12:58.923606 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.923586    2579 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 16 02:12:58.923606 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.923599    2579 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 16 02:12:58.923750 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:12:58.923731    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-118.ec2.internal\" not found"
Sep 16 02:12:58.925933 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.924475    2579 factory.go:55] Registering systemd factory
Sep 16 02:12:58.925933 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.924489    2579 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 16 02:12:58.925933 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.924506    2579 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 16 02:12:58.925933 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.924509    2579 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 16 02:12:58.925933 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.924494    2579 factory.go:223] Registration of the systemd container factory successfully
Sep 16 02:12:58.925933 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.924624    2579 reconstruct.go:97] "Volume reconstruction finished"
Sep 16 02:12:58.925933 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.924632    2579 reconciler.go:29] "Reconciler: start to sync state"
Sep 16 02:12:58.925933 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.924725    2579 factory.go:153] Registering CRI-O factory
Sep 16 02:12:58.925933 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.924738    2579 factory.go:223] Registration of the crio container factory successfully
Sep 16 02:12:58.925933 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.924797    2579 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 16 02:12:58.925933 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.924815    2579 factory.go:103] Registering Raw factory
Sep 16 02:12:58.925933 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.924830    2579 manager.go:1196] Started watching for new ooms in manager
Sep 16 02:12:58.925933 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.925345    2579 manager.go:349] Starting recovery of all containers
Sep 16 02:12:58.932491 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:12:58.932452    2579 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-136-118.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 16 02:12:58.932592 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:12:58.932543    2579 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 16 02:12:58.933371 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.933348    2579 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-ngsth"
Sep 16 02:12:58.935113 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.935033    2579 manager.go:354] Recovery completed
Sep 16 02:12:58.939896 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.939882    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 02:12:58.940417 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.940402    2579 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-ngsth"
Sep 16 02:12:58.941499 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.941479    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-118.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 02:12:58.941572 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.941513    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-118.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 02:12:58.941572 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.941524    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-118.ec2.internal" event="NodeHasSufficientPID"
Sep 16 02:12:58.942120 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.942109    2579 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 16 02:12:58.942120 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.942119    2579 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 16 02:12:58.942198 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.942132    2579 state_mem.go:36] "Initialized new in-memory state store"
Sep 16 02:12:58.942198 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.942136    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 02:12:58.942787 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.942771    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-118.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 02:12:58.942839 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.942807    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-118.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 02:12:58.942839 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.942817    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-118.ec2.internal" event="NodeHasSufficientPID"
Sep 16 02:12:58.943295 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:12:58.943239    2579 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-118.ec2.internal.18d5ab4e1d91465c  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-118.ec2.internal,UID:ip-10-0-136-118.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-136-118.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-136-118.ec2.internal,},FirstTimestamp:2026-09-16 02:12:58.941498972 +0000 UTC m=+0.425629857,LastTimestamp:2026-09-16 02:12:58.941498972 +0000 UTC m=+0.425629857,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-118.ec2.internal,}"
Sep 16 02:12:58.944288 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.944273    2579 policy_none.go:49] "None policy: Start"
Sep 16 02:12:58.944363 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.944293    2579 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 16 02:12:58.944363 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.944306    2579 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 16 02:12:58.946560 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.946547    2579 policy_none.go:47] "Start"
Sep 16 02:12:58.979324 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.979305    2579 manager.go:337] "Starting Device Plugin manager"
Sep 16 02:12:58.979436 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:12:58.979352    2579 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 16 02:12:58.979436 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.979366    2579 server.go:87] "Starting device plugin registration server"
Sep 16 02:12:58.979661 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.979646    2579 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 16 02:12:58.979725 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.979663    2579 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 16 02:12:58.980216 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.979820    2579 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 16 02:12:58.980216 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.979934    2579 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 16 02:12:58.980216 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.979944    2579 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 16 02:12:58.980383 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:12:58.980322    2579 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 16 02:12:58.980383 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:12:58.980372    2579 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-136-118.ec2.internal\" not found"
Sep 16 02:12:58.993845 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.993823    2579 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 16 02:12:58.995095 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.995074    2579 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 16 02:12:58.995180 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.995116    2579 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 16 02:12:58.995180 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.995135    2579 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 16 02:12:58.995180 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:12:58.995166    2579 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 16 02:12:58.997316 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:58.997299    2579 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 16 02:12:59.080551 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:59.080526    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 02:12:59.081493 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:59.081477    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-118.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 02:12:59.081609 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:59.081514    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-118.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 02:12:59.081609 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:59.081530    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-118.ec2.internal" event="NodeHasSufficientPID"
Sep 16 02:12:59.081609 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:59.081561    2579 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-136-118.ec2.internal"
Sep 16 02:12:59.089511 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:59.089425    2579 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-136-118.ec2.internal"
Sep 16 02:12:59.089511 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:12:59.089449    2579 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-136-118.ec2.internal\": node \"ip-10-0-136-118.ec2.internal\" not found"
Sep 16 02:12:59.095452 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:59.095434    2579 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-118.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-136-118.ec2.internal"]
Sep 16 02:12:59.095512 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:59.095494    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 02:12:59.097448 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:59.097433    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-118.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 02:12:59.097509 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:59.097464    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-118.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 02:12:59.097509 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:59.097475    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-118.ec2.internal" event="NodeHasSufficientPID"
Sep 16 02:12:59.099820 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:59.099808    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 02:12:59.099977 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:59.099961    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-118.ec2.internal"
Sep 16 02:12:59.100017 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:59.099993    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 02:12:59.100582 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:59.100557    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-118.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 02:12:59.100582 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:59.100577    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-118.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 02:12:59.100697 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:59.100585    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-118.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 02:12:59.100697 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:59.100608    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-118.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 02:12:59.100697 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:59.100622    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-118.ec2.internal" event="NodeHasSufficientPID"
Sep 16 02:12:59.100697 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:59.100588    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-118.ec2.internal" event="NodeHasSufficientPID"
Sep 16 02:12:59.102809 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:59.102788    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-118.ec2.internal"
Sep 16 02:12:59.102926 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:59.102814    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 02:12:59.103721 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:59.103704    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-118.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 02:12:59.103721 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:59.103725    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-118.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 02:12:59.103876 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:59.103734    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-118.ec2.internal" event="NodeHasSufficientPID"
Sep 16 02:12:59.113479 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:12:59.113458    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-118.ec2.internal\" not found"
Sep 16 02:12:59.120369 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:12:59.120355    2579 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-118.ec2.internal\" not found" node="ip-10-0-136-118.ec2.internal"
Sep 16 02:12:59.124284 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:12:59.124268    2579 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-118.ec2.internal\" not found" node="ip-10-0-136-118.ec2.internal"
Sep 16 02:12:59.126216 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:59.126201    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/196354e2e4c46cf552a666282e669d68-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-118.ec2.internal\" (UID: \"196354e2e4c46cf552a666282e669d68\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-118.ec2.internal"
Sep 16 02:12:59.126277 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:59.126226    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/126583346e4bbf13f4d978c46ad24fd3-config\") pod \"kube-apiserver-proxy-ip-10-0-136-118.ec2.internal\" (UID: \"126583346e4bbf13f4d978c46ad24fd3\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-118.ec2.internal"
Sep 16 02:12:59.126277 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:59.126243    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/196354e2e4c46cf552a666282e669d68-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-118.ec2.internal\" (UID: \"196354e2e4c46cf552a666282e669d68\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-118.ec2.internal"
Sep 16 02:12:59.214054 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:12:59.214033    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-118.ec2.internal\" not found"
Sep 16 02:12:59.226382 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:59.226359    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/196354e2e4c46cf552a666282e669d68-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-118.ec2.internal\" (UID: \"196354e2e4c46cf552a666282e669d68\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-118.ec2.internal"
Sep 16 02:12:59.226482 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:59.226432    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/196354e2e4c46cf552a666282e669d68-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-118.ec2.internal\" (UID: \"196354e2e4c46cf552a666282e669d68\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-118.ec2.internal"
Sep 16 02:12:59.226482 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:59.226445    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/196354e2e4c46cf552a666282e669d68-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-118.ec2.internal\" (UID: \"196354e2e4c46cf552a666282e669d68\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-118.ec2.internal"
Sep 16 02:12:59.226482 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:59.226467    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/196354e2e4c46cf552a666282e669d68-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-118.ec2.internal\" (UID: \"196354e2e4c46cf552a666282e669d68\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-118.ec2.internal"
Sep 16 02:12:59.226600 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:59.226494    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/126583346e4bbf13f4d978c46ad24fd3-config\") pod \"kube-apiserver-proxy-ip-10-0-136-118.ec2.internal\" (UID: \"126583346e4bbf13f4d978c46ad24fd3\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-118.ec2.internal"
Sep 16 02:12:59.226600 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:59.226526    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/126583346e4bbf13f4d978c46ad24fd3-config\") pod \"kube-apiserver-proxy-ip-10-0-136-118.ec2.internal\" (UID: \"126583346e4bbf13f4d978c46ad24fd3\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-118.ec2.internal"
Sep 16 02:12:59.314555 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:12:59.314530    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-118.ec2.internal\" not found"
Sep 16 02:12:59.415300 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:12:59.415240    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-118.ec2.internal\" not found"
Sep 16 02:12:59.423400 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:59.423384    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-118.ec2.internal"
Sep 16 02:12:59.428888 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:59.428573    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-118.ec2.internal"
Sep 16 02:12:59.515838 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:12:59.515807    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-118.ec2.internal\" not found"
Sep 16 02:12:59.616382 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:12:59.616347    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-118.ec2.internal\" not found"
Sep 16 02:12:59.716965 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:12:59.716898    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-118.ec2.internal\" not found"
Sep 16 02:12:59.817506 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:12:59.817477    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-118.ec2.internal\" not found"
Sep 16 02:12:59.859922 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:59.859899    2579 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 16 02:12:59.860304 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:59.860037    2579 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 16 02:12:59.917950 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:12:59.917929    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-118.ec2.internal\" not found"
Sep 16 02:12:59.934236 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:59.934217    2579 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 16 02:12:59.946111 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:59.946071    2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-15 02:08:00 +0000 UTC" deadline="2028-05-26 18:27:01.04316962 +0000 UTC"
Sep 16 02:12:59.946207 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:59.946112    2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14848h14m1.09706225s"
Sep 16 02:12:59.946990 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:12:59.946974    2579 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 16 02:13:00.024158 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:00.023966    2579 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-118.ec2.internal"
Sep 16 02:13:00.036697 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:00.036678    2579 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 16 02:13:00.038821 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:00.038803    2579 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-118.ec2.internal"
Sep 16 02:13:00.041679 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:13:00.041648    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod126583346e4bbf13f4d978c46ad24fd3.slice/crio-36924ce6de4d4be65401c4f10efd0713af171de2e52e2863f3416c241aafb815 WatchSource:0}: Error finding container 36924ce6de4d4be65401c4f10efd0713af171de2e52e2863f3416c241aafb815: Status 404 returned error can't find the container with id 36924ce6de4d4be65401c4f10efd0713af171de2e52e2863f3416c241aafb815
Sep 16 02:13:00.046716 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:00.046661    2579 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 16 02:13:00.048064 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:00.047956    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 16 02:13:00.048943 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:13:00.048923    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod196354e2e4c46cf552a666282e669d68.slice/crio-6cf55745db1666d172524ba5f541bde85ba3376a4f240fd8753c57edd1a4f430 WatchSource:0}: Error finding container 6cf55745db1666d172524ba5f541bde85ba3376a4f240fd8753c57edd1a4f430: Status 404 returned error can't find the container with id 6cf55745db1666d172524ba5f541bde85ba3376a4f240fd8753c57edd1a4f430
Sep 16 02:13:00.426400 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:00.426376    2579 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 16 02:13:02.002792 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.002732    2579 apiserver.go:52] "Watching apiserver"
Sep 16 02:13:02.011380 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.011353    2579 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 16 02:13:02.011812 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.011761    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/node-resolver-9522z","openshift-image-registry/node-ca-kwdns","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-118.ec2.internal","openshift-multus/multus-additional-cni-plugins-9g4cg","openshift-multus/network-metrics-daemon-6lwr9","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-85zxn","openshift-cluster-node-tuning-operator/tuned-2kqqm","openshift-multus/multus-ncht9","openshift-network-diagnostics/network-check-target-7b42r","openshift-network-operator/iptables-alerter-tbhqp","openshift-ovn-kubernetes/ovnkube-node-88xqz","kube-system/global-pull-secret-syncer-cdwh7","kube-system/konnectivity-agent-bhd9l","kube-system/kube-apiserver-proxy-ip-10-0-136-118.ec2.internal"]
Sep 16 02:13:02.014342 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.014316    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-2kqqm"
Sep 16 02:13:02.016480 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.016431    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-zffb4\""
Sep 16 02:13:02.016587 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.016557    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 16 02:13:02.016719 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.016698    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 16 02:13:02.016821 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.016736    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-bhd9l"
Sep 16 02:13:02.018746 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.018460    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 16 02:13:02.018746 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.018532    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-9tckg\""
Sep 16 02:13:02.018746 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.018566    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 16 02:13:02.018882 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.018786    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-ncht9"
Sep 16 02:13:02.020640 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.020613    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 16 02:13:02.020640 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.020635    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 16 02:13:02.020984 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.020962    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-2fl9k\""
Sep 16 02:13:02.021090 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.021075    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 16 02:13:02.021152 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.021140    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 16 02:13:02.021607 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.021328    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-9g4cg"
Sep 16 02:13:02.023539 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.023518    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6lwr9"
Sep 16 02:13:02.023629 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:02.023594    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6lwr9" podUID="e3472392-891d-49a1-83a7-57f1f99f4b79"
Sep 16 02:13:02.023984 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.023967    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 16 02:13:02.024168 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.024149    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 16 02:13:02.024259 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.024240    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-tjwdl\""
Sep 16 02:13:02.024403 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.024355    2579 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 16 02:13:02.025816 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.025798    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-85zxn"
Sep 16 02:13:02.027653 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.027634    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 16 02:13:02.027653 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.027655    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 16 02:13:02.028037 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.027994    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 16 02:13:02.028037 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.028011    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-9nb5k\""
Sep 16 02:13:02.028167 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.028015    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7b42r"
Sep 16 02:13:02.028167 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:02.028157    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7b42r" podUID="28c55959-36b5-4d35-9e1a-f046d6d36f74"
Sep 16 02:13:02.030604 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.030299    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-9522z"
Sep 16 02:13:02.032335 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.032316    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 16 02:13:02.032620 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.032584    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-xv64h\""
Sep 16 02:13:02.032680 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.032658    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 16 02:13:02.032975 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.032959    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-kwdns"
Sep 16 02:13:02.034954 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.034842    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ae9f48ae-4a32-4f07-8ea3-ce26faa19349-etc-selinux\") pod \"aws-ebs-csi-driver-node-85zxn\" (UID: \"ae9f48ae-4a32-4f07-8ea3-ce26faa19349\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-85zxn"
Sep 16 02:13:02.035427 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.034885    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/4b370e61-b8a4-4423-8673-4b604907e2f7-run\") pod \"tuned-2kqqm\" (UID: \"4b370e61-b8a4-4423-8673-4b604907e2f7\") " pod="openshift-cluster-node-tuning-operator/tuned-2kqqm"
Sep 16 02:13:02.035545 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.035458    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-tbhqp"
Sep 16 02:13:02.035545 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.035454    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/98489f25-0768-47e2-b8f0-e2a7c7af2063-agent-certs\") pod \"konnectivity-agent-bhd9l\" (UID: \"98489f25-0768-47e2-b8f0-e2a7c7af2063\") " pod="kube-system/konnectivity-agent-bhd9l"
Sep 16 02:13:02.035646 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.035594    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/bc90ef77-c833-4991-9fe8-656fdf2f7457-hostroot\") pod \"multus-ncht9\" (UID: \"bc90ef77-c833-4991-9fe8-656fdf2f7457\") " pod="openshift-multus/multus-ncht9"
Sep 16 02:13:02.035646 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.035625    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bc90ef77-c833-4991-9fe8-656fdf2f7457-etc-kubernetes\") pod \"multus-ncht9\" (UID: \"bc90ef77-c833-4991-9fe8-656fdf2f7457\") " pod="openshift-multus/multus-ncht9"
Sep 16 02:13:02.035708 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.035670    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/34a0d09d-a96b-4242-9d17-61fc3b8d88bd-system-cni-dir\") pod \"multus-additional-cni-plugins-9g4cg\" (UID: \"34a0d09d-a96b-4242-9d17-61fc3b8d88bd\") " pod="openshift-multus/multus-additional-cni-plugins-9g4cg"
Sep 16 02:13:02.035769 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.035708    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4b370e61-b8a4-4423-8673-4b604907e2f7-sys\") pod \"tuned-2kqqm\" (UID: \"4b370e61-b8a4-4423-8673-4b604907e2f7\") " pod="openshift-cluster-node-tuning-operator/tuned-2kqqm"
Sep 16 02:13:02.035885 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.035839    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bc90ef77-c833-4991-9fe8-656fdf2f7457-cnibin\") pod \"multus-ncht9\" (UID: \"bc90ef77-c833-4991-9fe8-656fdf2f7457\") " pod="openshift-multus/multus-ncht9"
Sep 16 02:13:02.035955 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.035932    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/bc90ef77-c833-4991-9fe8-656fdf2f7457-multus-daemon-config\") pod \"multus-ncht9\" (UID: \"bc90ef77-c833-4991-9fe8-656fdf2f7457\") " pod="openshift-multus/multus-ncht9"
Sep 16 02:13:02.036030 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.036007    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/bc90ef77-c833-4991-9fe8-656fdf2f7457-host-run-multus-certs\") pod \"multus-ncht9\" (UID: \"bc90ef77-c833-4991-9fe8-656fdf2f7457\") " pod="openshift-multus/multus-ncht9"
Sep 16 02:13:02.036089 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.036043    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ae9f48ae-4a32-4f07-8ea3-ce26faa19349-kubelet-dir\") pod \"aws-ebs-csi-driver-node-85zxn\" (UID: \"ae9f48ae-4a32-4f07-8ea3-ce26faa19349\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-85zxn"
Sep 16 02:13:02.036089 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.036070    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l5fl9\" (UniqueName: \"kubernetes.io/projected/ae9f48ae-4a32-4f07-8ea3-ce26faa19349-kube-api-access-l5fl9\") pod \"aws-ebs-csi-driver-node-85zxn\" (UID: \"ae9f48ae-4a32-4f07-8ea3-ce26faa19349\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-85zxn"
Sep 16 02:13:02.036089 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.036093    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4b370e61-b8a4-4423-8673-4b604907e2f7-lib-modules\") pod \"tuned-2kqqm\" (UID: \"4b370e61-b8a4-4423-8673-4b604907e2f7\") " pod="openshift-cluster-node-tuning-operator/tuned-2kqqm"
Sep 16 02:13:02.036249 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.036177    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/4b370e61-b8a4-4423-8673-4b604907e2f7-etc-tuned\") pod \"tuned-2kqqm\" (UID: \"4b370e61-b8a4-4423-8673-4b604907e2f7\") " pod="openshift-cluster-node-tuning-operator/tuned-2kqqm"
Sep 16 02:13:02.036301 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.036249    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bc90ef77-c833-4991-9fe8-656fdf2f7457-system-cni-dir\") pod \"multus-ncht9\" (UID: \"bc90ef77-c833-4991-9fe8-656fdf2f7457\") " pod="openshift-multus/multus-ncht9"
Sep 16 02:13:02.036353 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.036298    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bc90ef77-c833-4991-9fe8-656fdf2f7457-multus-cni-dir\") pod \"multus-ncht9\" (UID: \"bc90ef77-c833-4991-9fe8-656fdf2f7457\") " pod="openshift-multus/multus-ncht9"
Sep 16 02:13:02.036353 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.036315    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 16 02:13:02.036353 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.036329    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bc90ef77-c833-4991-9fe8-656fdf2f7457-host-var-lib-cni-bin\") pod \"multus-ncht9\" (UID: \"bc90ef77-c833-4991-9fe8-656fdf2f7457\") " pod="openshift-multus/multus-ncht9"
Sep 16 02:13:02.036520 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.036358    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/bc90ef77-c833-4991-9fe8-656fdf2f7457-host-var-lib-cni-multus\") pod \"multus-ncht9\" (UID: \"bc90ef77-c833-4991-9fe8-656fdf2f7457\") " pod="openshift-multus/multus-ncht9"
Sep 16 02:13:02.036520 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.036380    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bc90ef77-c833-4991-9fe8-656fdf2f7457-host-var-lib-kubelet\") pod \"multus-ncht9\" (UID: \"bc90ef77-c833-4991-9fe8-656fdf2f7457\") " pod="openshift-multus/multus-ncht9"
Sep 16 02:13:02.036520 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.036401    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bc90ef77-c833-4991-9fe8-656fdf2f7457-multus-conf-dir\") pod \"multus-ncht9\" (UID: \"bc90ef77-c833-4991-9fe8-656fdf2f7457\") " pod="openshift-multus/multus-ncht9"
Sep 16 02:13:02.036520 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.036422    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/34a0d09d-a96b-4242-9d17-61fc3b8d88bd-cni-binary-copy\") pod \"multus-additional-cni-plugins-9g4cg\" (UID: \"34a0d09d-a96b-4242-9d17-61fc3b8d88bd\") " pod="openshift-multus/multus-additional-cni-plugins-9g4cg"
Sep 16 02:13:02.036520 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.036483    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/34a0d09d-a96b-4242-9d17-61fc3b8d88bd-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-9g4cg\" (UID: \"34a0d09d-a96b-4242-9d17-61fc3b8d88bd\") " pod="openshift-multus/multus-additional-cni-plugins-9g4cg"
Sep 16 02:13:02.036520 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.036488    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 16 02:13:02.036815 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.036554    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lxlhf\" (UniqueName: \"kubernetes.io/projected/bc90ef77-c833-4991-9fe8-656fdf2f7457-kube-api-access-lxlhf\") pod \"multus-ncht9\" (UID: \"bc90ef77-c833-4991-9fe8-656fdf2f7457\") " pod="openshift-multus/multus-ncht9"
Sep 16 02:13:02.036815 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.036621    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/34a0d09d-a96b-4242-9d17-61fc3b8d88bd-cnibin\") pod \"multus-additional-cni-plugins-9g4cg\" (UID: \"34a0d09d-a96b-4242-9d17-61fc3b8d88bd\") " pod="openshift-multus/multus-additional-cni-plugins-9g4cg"
Sep 16 02:13:02.036815 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.036564    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 16 02:13:02.036815 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.036666    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ae9f48ae-4a32-4f07-8ea3-ce26faa19349-device-dir\") pod \"aws-ebs-csi-driver-node-85zxn\" (UID: \"ae9f48ae-4a32-4f07-8ea3-ce26faa19349\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-85zxn"
Sep 16 02:13:02.036815 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.036702    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ae9f48ae-4a32-4f07-8ea3-ce26faa19349-sys-fs\") pod \"aws-ebs-csi-driver-node-85zxn\" (UID: \"ae9f48ae-4a32-4f07-8ea3-ce26faa19349\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-85zxn"
Sep 16 02:13:02.036815 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.036707    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-djzp6\""
Sep 16 02:13:02.036815 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.036728    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/4b370e61-b8a4-4423-8673-4b604907e2f7-etc-sysconfig\") pod \"tuned-2kqqm\" (UID: \"4b370e61-b8a4-4423-8673-4b604907e2f7\") " pod="openshift-cluster-node-tuning-operator/tuned-2kqqm"
Sep 16 02:13:02.036815 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.036788    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4b370e61-b8a4-4423-8673-4b604907e2f7-etc-kubernetes\") pod \"tuned-2kqqm\" (UID: \"4b370e61-b8a4-4423-8673-4b604907e2f7\") " pod="openshift-cluster-node-tuning-operator/tuned-2kqqm"
Sep 16 02:13:02.036815 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.036814    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/4b370e61-b8a4-4423-8673-4b604907e2f7-etc-sysctl-conf\") pod \"tuned-2kqqm\" (UID: \"4b370e61-b8a4-4423-8673-4b604907e2f7\") " pod="openshift-cluster-node-tuning-operator/tuned-2kqqm"
Sep 16 02:13:02.037195 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.036838    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/bc90ef77-c833-4991-9fe8-656fdf2f7457-multus-socket-dir-parent\") pod \"multus-ncht9\" (UID: \"bc90ef77-c833-4991-9fe8-656fdf2f7457\") " pod="openshift-multus/multus-ncht9"
Sep 16 02:13:02.037195 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.036862    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qpqhz\" (UniqueName: \"kubernetes.io/projected/34a0d09d-a96b-4242-9d17-61fc3b8d88bd-kube-api-access-qpqhz\") pod \"multus-additional-cni-plugins-9g4cg\" (UID: \"34a0d09d-a96b-4242-9d17-61fc3b8d88bd\") " pod="openshift-multus/multus-additional-cni-plugins-9g4cg"
Sep 16 02:13:02.037195 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.036920    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ae9f48ae-4a32-4f07-8ea3-ce26faa19349-socket-dir\") pod \"aws-ebs-csi-driver-node-85zxn\" (UID: \"ae9f48ae-4a32-4f07-8ea3-ce26faa19349\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-85zxn"
Sep 16 02:13:02.037195 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.036946    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wlf8b\" (UniqueName: \"kubernetes.io/projected/28c55959-36b5-4d35-9e1a-f046d6d36f74-kube-api-access-wlf8b\") pod \"network-check-target-7b42r\" (UID: \"28c55959-36b5-4d35-9e1a-f046d6d36f74\") " pod="openshift-network-diagnostics/network-check-target-7b42r"
Sep 16 02:13:02.037195 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.036998    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/4b370e61-b8a4-4423-8673-4b604907e2f7-etc-systemd\") pod \"tuned-2kqqm\" (UID: \"4b370e61-b8a4-4423-8673-4b604907e2f7\") " pod="openshift-cluster-node-tuning-operator/tuned-2kqqm"
Sep 16 02:13:02.037195 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.037054    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4b370e61-b8a4-4423-8673-4b604907e2f7-var-lib-kubelet\") pod \"tuned-2kqqm\" (UID: \"4b370e61-b8a4-4423-8673-4b604907e2f7\") " pod="openshift-cluster-node-tuning-operator/tuned-2kqqm"
Sep 16 02:13:02.037195 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.037096    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6wb6w\" (UniqueName: \"kubernetes.io/projected/4b370e61-b8a4-4423-8673-4b604907e2f7-kube-api-access-6wb6w\") pod \"tuned-2kqqm\" (UID: \"4b370e61-b8a4-4423-8673-4b604907e2f7\") " pod="openshift-cluster-node-tuning-operator/tuned-2kqqm"
Sep 16 02:13:02.037195 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.037123    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bc90ef77-c833-4991-9fe8-656fdf2f7457-host-run-netns\") pod \"multus-ncht9\" (UID: \"bc90ef77-c833-4991-9fe8-656fdf2f7457\") " pod="openshift-multus/multus-ncht9"
Sep 16 02:13:02.037195 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.037138    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/34a0d09d-a96b-4242-9d17-61fc3b8d88bd-tuning-conf-dir\") pod \"multus-additional-cni-plugins-9g4cg\" (UID: \"34a0d09d-a96b-4242-9d17-61fc3b8d88bd\") " pod="openshift-multus/multus-additional-cni-plugins-9g4cg"
Sep 16 02:13:02.037195 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.037165    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/34a0d09d-a96b-4242-9d17-61fc3b8d88bd-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-9g4cg\" (UID: \"34a0d09d-a96b-4242-9d17-61fc3b8d88bd\") " pod="openshift-multus/multus-additional-cni-plugins-9g4cg"
Sep 16 02:13:02.037195 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.037195    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/4b370e61-b8a4-4423-8673-4b604907e2f7-etc-sysctl-d\") pod \"tuned-2kqqm\" (UID: \"4b370e61-b8a4-4423-8673-4b604907e2f7\") " pod="openshift-cluster-node-tuning-operator/tuned-2kqqm"
Sep 16 02:13:02.037658 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.037230    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4b370e61-b8a4-4423-8673-4b604907e2f7-host\") pod \"tuned-2kqqm\" (UID: \"4b370e61-b8a4-4423-8673-4b604907e2f7\") " pod="openshift-cluster-node-tuning-operator/tuned-2kqqm"
Sep 16 02:13:02.037658 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.037258    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4b370e61-b8a4-4423-8673-4b604907e2f7-tmp\") pod \"tuned-2kqqm\" (UID: \"4b370e61-b8a4-4423-8673-4b604907e2f7\") " pod="openshift-cluster-node-tuning-operator/tuned-2kqqm"
Sep 16 02:13:02.037658 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.037277    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/98489f25-0768-47e2-b8f0-e2a7c7af2063-konnectivity-ca\") pod \"konnectivity-agent-bhd9l\" (UID: \"98489f25-0768-47e2-b8f0-e2a7c7af2063\") " pod="kube-system/konnectivity-agent-bhd9l"
Sep 16 02:13:02.037658 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.037307    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/bc90ef77-c833-4991-9fe8-656fdf2f7457-host-run-k8s-cni-cncf-io\") pod \"multus-ncht9\" (UID: \"bc90ef77-c833-4991-9fe8-656fdf2f7457\") " pod="openshift-multus/multus-ncht9"
Sep 16 02:13:02.037658 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.037365    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8fhvr\" (UniqueName: \"kubernetes.io/projected/e3472392-891d-49a1-83a7-57f1f99f4b79-kube-api-access-8fhvr\") pod \"network-metrics-daemon-6lwr9\" (UID: \"e3472392-891d-49a1-83a7-57f1f99f4b79\") " pod="openshift-multus/network-metrics-daemon-6lwr9"
Sep 16 02:13:02.037658 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.037399    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/4b370e61-b8a4-4423-8673-4b604907e2f7-etc-modprobe-d\") pod \"tuned-2kqqm\" (UID: \"4b370e61-b8a4-4423-8673-4b604907e2f7\") " pod="openshift-cluster-node-tuning-operator/tuned-2kqqm"
Sep 16 02:13:02.037658 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.037448    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bc90ef77-c833-4991-9fe8-656fdf2f7457-cni-binary-copy\") pod \"multus-ncht9\" (UID: \"bc90ef77-c833-4991-9fe8-656fdf2f7457\") " pod="openshift-multus/multus-ncht9"
Sep 16 02:13:02.037658 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.037487    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3472392-891d-49a1-83a7-57f1f99f4b79-metrics-certs\") pod \"network-metrics-daemon-6lwr9\" (UID: \"e3472392-891d-49a1-83a7-57f1f99f4b79\") " pod="openshift-multus/network-metrics-daemon-6lwr9"
Sep 16 02:13:02.037658 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.037535    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ae9f48ae-4a32-4f07-8ea3-ce26faa19349-registration-dir\") pod \"aws-ebs-csi-driver-node-85zxn\" (UID: \"ae9f48ae-4a32-4f07-8ea3-ce26faa19349\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-85zxn"
Sep 16 02:13:02.038142 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.037755    2579 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 16 02:13:02.039929 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.039911    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-6hzhd\""
Sep 16 02:13:02.040298 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.040282    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 16 02:13:02.040487 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.040453    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 16 02:13:02.042792 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.040655    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 16 02:13:02.044776 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.044755    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-88xqz"
Sep 16 02:13:02.047711 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.047152    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 16 02:13:02.047711 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.047165    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 16 02:13:02.047711 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.047177    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 16 02:13:02.047711 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.047291    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 16 02:13:02.047711 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.047355    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-2l64p\""
Sep 16 02:13:02.047711 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.047454    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 16 02:13:02.047711 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.047455    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 16 02:13:02.048097 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.047931    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cdwh7"
Sep 16 02:13:02.048097 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:02.047986    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cdwh7" podUID="2b6aaa85-d6dd-420d-a818-f56d06ce6467"
Sep 16 02:13:02.064697 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.064682    2579 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 16 02:13:02.101131 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.101086    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-118.ec2.internal" event={"ID":"196354e2e4c46cf552a666282e669d68","Type":"ContainerStarted","Data":"6cf55745db1666d172524ba5f541bde85ba3376a4f240fd8753c57edd1a4f430"}
Sep 16 02:13:02.102713 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.102596    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-118.ec2.internal" event={"ID":"126583346e4bbf13f4d978c46ad24fd3","Type":"ContainerStarted","Data":"36924ce6de4d4be65401c4f10efd0713af171de2e52e2863f3416c241aafb815"}
Sep 16 02:13:02.126714 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.126692    2579 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 16 02:13:02.137762 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.137733    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ae9f48ae-4a32-4f07-8ea3-ce26faa19349-device-dir\") pod \"aws-ebs-csi-driver-node-85zxn\" (UID: \"ae9f48ae-4a32-4f07-8ea3-ce26faa19349\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-85zxn"
Sep 16 02:13:02.137868 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.137791    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ae9f48ae-4a32-4f07-8ea3-ce26faa19349-sys-fs\") pod \"aws-ebs-csi-driver-node-85zxn\" (UID: \"ae9f48ae-4a32-4f07-8ea3-ce26faa19349\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-85zxn"
Sep 16 02:13:02.137868 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.137809    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/4b370e61-b8a4-4423-8673-4b604907e2f7-etc-sysconfig\") pod \"tuned-2kqqm\" (UID: \"4b370e61-b8a4-4423-8673-4b604907e2f7\") " pod="openshift-cluster-node-tuning-operator/tuned-2kqqm"
Sep 16 02:13:02.137868 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.137830    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4b370e61-b8a4-4423-8673-4b604907e2f7-etc-kubernetes\") pod \"tuned-2kqqm\" (UID: \"4b370e61-b8a4-4423-8673-4b604907e2f7\") " pod="openshift-cluster-node-tuning-operator/tuned-2kqqm"
Sep 16 02:13:02.137868 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.137847    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/4b370e61-b8a4-4423-8673-4b604907e2f7-etc-sysctl-conf\") pod \"tuned-2kqqm\" (UID: \"4b370e61-b8a4-4423-8673-4b604907e2f7\") " pod="openshift-cluster-node-tuning-operator/tuned-2kqqm"
Sep 16 02:13:02.137868 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.137856    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ae9f48ae-4a32-4f07-8ea3-ce26faa19349-device-dir\") pod \"aws-ebs-csi-driver-node-85zxn\" (UID: \"ae9f48ae-4a32-4f07-8ea3-ce26faa19349\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-85zxn"
Sep 16 02:13:02.137868 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.137869    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/de5aef44-71c6-4440-afe1-7973e763c35d-systemd-units\") pod \"ovnkube-node-88xqz\" (UID: \"de5aef44-71c6-4440-afe1-7973e763c35d\") " pod="openshift-ovn-kubernetes/ovnkube-node-88xqz"
Sep 16 02:13:02.138046 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.137885    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lxlhf\" (UniqueName: \"kubernetes.io/projected/bc90ef77-c833-4991-9fe8-656fdf2f7457-kube-api-access-lxlhf\") pod \"multus-ncht9\" (UID: \"bc90ef77-c833-4991-9fe8-656fdf2f7457\") " pod="openshift-multus/multus-ncht9"
Sep 16 02:13:02.138046 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.137900    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/de5aef44-71c6-4440-afe1-7973e763c35d-etc-openvswitch\") pod \"ovnkube-node-88xqz\" (UID: \"de5aef44-71c6-4440-afe1-7973e763c35d\") " pod="openshift-ovn-kubernetes/ovnkube-node-88xqz"
Sep 16 02:13:02.138046 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.137913    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/de5aef44-71c6-4440-afe1-7973e763c35d-host-cni-netd\") pod \"ovnkube-node-88xqz\" (UID: \"de5aef44-71c6-4440-afe1-7973e763c35d\") " pod="openshift-ovn-kubernetes/ovnkube-node-88xqz"
Sep 16 02:13:02.138046 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.137947    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/835b007c-0c9a-4c8f-94dd-68f8a3b3a137-tmp-dir\") pod \"node-resolver-9522z\" (UID: \"835b007c-0c9a-4c8f-94dd-68f8a3b3a137\") " pod="openshift-dns/node-resolver-9522z"
Sep 16 02:13:02.138046 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.137965    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wlf8b\" (UniqueName: \"kubernetes.io/projected/28c55959-36b5-4d35-9e1a-f046d6d36f74-kube-api-access-wlf8b\") pod \"network-check-target-7b42r\" (UID: \"28c55959-36b5-4d35-9e1a-f046d6d36f74\") " pod="openshift-network-diagnostics/network-check-target-7b42r"
Sep 16 02:13:02.138046 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.137981    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/98489f25-0768-47e2-b8f0-e2a7c7af2063-konnectivity-ca\") pod \"konnectivity-agent-bhd9l\" (UID: \"98489f25-0768-47e2-b8f0-e2a7c7af2063\") " pod="kube-system/konnectivity-agent-bhd9l"
Sep 16 02:13:02.138046 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.137984    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4b370e61-b8a4-4423-8673-4b604907e2f7-etc-kubernetes\") pod \"tuned-2kqqm\" (UID: \"4b370e61-b8a4-4423-8673-4b604907e2f7\") " pod="openshift-cluster-node-tuning-operator/tuned-2kqqm"
Sep 16 02:13:02.138046 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.137994    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/496c15b3-4d67-473f-b894-2e66b47df562-serviceca\") pod \"node-ca-kwdns\" (UID: \"496c15b3-4d67-473f-b894-2e66b47df562\") " pod="openshift-image-registry/node-ca-kwdns"
Sep 16 02:13:02.138046 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.138036    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ae9f48ae-4a32-4f07-8ea3-ce26faa19349-sys-fs\") pod \"aws-ebs-csi-driver-node-85zxn\" (UID: \"ae9f48ae-4a32-4f07-8ea3-ce26faa19349\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-85zxn"
Sep 16 02:13:02.138394 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.138066    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2b6aaa85-d6dd-420d-a818-f56d06ce6467-original-pull-secret\") pod \"global-pull-secret-syncer-cdwh7\" (UID: \"2b6aaa85-d6dd-420d-a818-f56d06ce6467\") " pod="kube-system/global-pull-secret-syncer-cdwh7"
Sep 16 02:13:02.138394 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.138095    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bc90ef77-c833-4991-9fe8-656fdf2f7457-host-run-netns\") pod \"multus-ncht9\" (UID: \"bc90ef77-c833-4991-9fe8-656fdf2f7457\") " pod="openshift-multus/multus-ncht9"
Sep 16 02:13:02.138394 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.138118    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/34a0d09d-a96b-4242-9d17-61fc3b8d88bd-tuning-conf-dir\") pod \"multus-additional-cni-plugins-9g4cg\" (UID: \"34a0d09d-a96b-4242-9d17-61fc3b8d88bd\") " pod="openshift-multus/multus-additional-cni-plugins-9g4cg"
Sep 16 02:13:02.138394 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.138124    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/4b370e61-b8a4-4423-8673-4b604907e2f7-etc-sysctl-conf\") pod \"tuned-2kqqm\" (UID: \"4b370e61-b8a4-4423-8673-4b604907e2f7\") " pod="openshift-cluster-node-tuning-operator/tuned-2kqqm"
Sep 16 02:13:02.138394 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.138144    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/34a0d09d-a96b-4242-9d17-61fc3b8d88bd-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-9g4cg\" (UID: \"34a0d09d-a96b-4242-9d17-61fc3b8d88bd\") " pod="openshift-multus/multus-additional-cni-plugins-9g4cg"
Sep 16 02:13:02.138394 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.138170    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/4b370e61-b8a4-4423-8673-4b604907e2f7-etc-sysctl-d\") pod \"tuned-2kqqm\" (UID: \"4b370e61-b8a4-4423-8673-4b604907e2f7\") " pod="openshift-cluster-node-tuning-operator/tuned-2kqqm"
Sep 16 02:13:02.138394 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.138196    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4b370e61-b8a4-4423-8673-4b604907e2f7-host\") pod \"tuned-2kqqm\" (UID: \"4b370e61-b8a4-4423-8673-4b604907e2f7\") " pod="openshift-cluster-node-tuning-operator/tuned-2kqqm"
Sep 16 02:13:02.138394 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.138223    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/de5aef44-71c6-4440-afe1-7973e763c35d-var-lib-openvswitch\") pod \"ovnkube-node-88xqz\" (UID: \"de5aef44-71c6-4440-afe1-7973e763c35d\") " pod="openshift-ovn-kubernetes/ovnkube-node-88xqz"
Sep 16 02:13:02.138394 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.138246    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/de5aef44-71c6-4440-afe1-7973e763c35d-run-ovn\") pod \"ovnkube-node-88xqz\" (UID: \"de5aef44-71c6-4440-afe1-7973e763c35d\") " pod="openshift-ovn-kubernetes/ovnkube-node-88xqz"
Sep 16 02:13:02.138394 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.138269    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/de5aef44-71c6-4440-afe1-7973e763c35d-host-cni-bin\") pod \"ovnkube-node-88xqz\" (UID: \"de5aef44-71c6-4440-afe1-7973e763c35d\") " pod="openshift-ovn-kubernetes/ovnkube-node-88xqz"
Sep 16 02:13:02.138394 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.138305    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/bc90ef77-c833-4991-9fe8-656fdf2f7457-host-run-k8s-cni-cncf-io\") pod \"multus-ncht9\" (UID: \"bc90ef77-c833-4991-9fe8-656fdf2f7457\") " pod="openshift-multus/multus-ncht9"
Sep 16 02:13:02.138394 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.138332    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8fhvr\" (UniqueName: \"kubernetes.io/projected/e3472392-891d-49a1-83a7-57f1f99f4b79-kube-api-access-8fhvr\") pod \"network-metrics-daemon-6lwr9\" (UID: \"e3472392-891d-49a1-83a7-57f1f99f4b79\") " pod="openshift-multus/network-metrics-daemon-6lwr9"
Sep 16 02:13:02.138394 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.138366    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/4b370e61-b8a4-4423-8673-4b604907e2f7-etc-modprobe-d\") pod \"tuned-2kqqm\" (UID: \"4b370e61-b8a4-4423-8673-4b604907e2f7\") " pod="openshift-cluster-node-tuning-operator/tuned-2kqqm"
Sep 16 02:13:02.138834 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.138440    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/bc90ef77-c833-4991-9fe8-656fdf2f7457-host-run-k8s-cni-cncf-io\") pod \"multus-ncht9\" (UID: \"bc90ef77-c833-4991-9fe8-656fdf2f7457\") " pod="openshift-multus/multus-ncht9"
Sep 16 02:13:02.138935 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.138913    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/98489f25-0768-47e2-b8f0-e2a7c7af2063-konnectivity-ca\") pod \"konnectivity-agent-bhd9l\" (UID: \"98489f25-0768-47e2-b8f0-e2a7c7af2063\") " pod="kube-system/konnectivity-agent-bhd9l"
Sep 16 02:13:02.139029 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.138965    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c27d76ec-6b7e-46e9-b66b-d61c0f3ae3f6-iptables-alerter-script\") pod \"iptables-alerter-tbhqp\" (UID: \"c27d76ec-6b7e-46e9-b66b-d61c0f3ae3f6\") " pod="openshift-network-operator/iptables-alerter-tbhqp"
Sep 16 02:13:02.139029 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.138994    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2b6aaa85-d6dd-420d-a818-f56d06ce6467-dbus\") pod \"global-pull-secret-syncer-cdwh7\" (UID: \"2b6aaa85-d6dd-420d-a818-f56d06ce6467\") " pod="kube-system/global-pull-secret-syncer-cdwh7"
Sep 16 02:13:02.139029 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.138998    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/34a0d09d-a96b-4242-9d17-61fc3b8d88bd-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-9g4cg\" (UID: \"34a0d09d-a96b-4242-9d17-61fc3b8d88bd\") " pod="openshift-multus/multus-additional-cni-plugins-9g4cg"
Sep 16 02:13:02.139029 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.139023    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bc90ef77-c833-4991-9fe8-656fdf2f7457-cni-binary-copy\") pod \"multus-ncht9\" (UID: \"bc90ef77-c833-4991-9fe8-656fdf2f7457\") " pod="openshift-multus/multus-ncht9"
Sep 16 02:13:02.139239 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.139038    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3472392-891d-49a1-83a7-57f1f99f4b79-metrics-certs\") pod \"network-metrics-daemon-6lwr9\" (UID: \"e3472392-891d-49a1-83a7-57f1f99f4b79\") " pod="openshift-multus/network-metrics-daemon-6lwr9"
Sep 16 02:13:02.139239 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.139056    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ae9f48ae-4a32-4f07-8ea3-ce26faa19349-etc-selinux\") pod \"aws-ebs-csi-driver-node-85zxn\" (UID: \"ae9f48ae-4a32-4f07-8ea3-ce26faa19349\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-85zxn"
Sep 16 02:13:02.139239 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.139107    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c27d76ec-6b7e-46e9-b66b-d61c0f3ae3f6-host-slash\") pod \"iptables-alerter-tbhqp\" (UID: \"c27d76ec-6b7e-46e9-b66b-d61c0f3ae3f6\") " pod="openshift-network-operator/iptables-alerter-tbhqp"
Sep 16 02:13:02.139239 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.139131    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/bc90ef77-c833-4991-9fe8-656fdf2f7457-hostroot\") pod \"multus-ncht9\" (UID: \"bc90ef77-c833-4991-9fe8-656fdf2f7457\") " pod="openshift-multus/multus-ncht9"
Sep 16 02:13:02.139239 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.139152    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/34a0d09d-a96b-4242-9d17-61fc3b8d88bd-system-cni-dir\") pod \"multus-additional-cni-plugins-9g4cg\" (UID: \"34a0d09d-a96b-4242-9d17-61fc3b8d88bd\") " pod="openshift-multus/multus-additional-cni-plugins-9g4cg"
Sep 16 02:13:02.139239 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.139205    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hjzwc\" (UniqueName: \"kubernetes.io/projected/c27d76ec-6b7e-46e9-b66b-d61c0f3ae3f6-kube-api-access-hjzwc\") pod \"iptables-alerter-tbhqp\" (UID: \"c27d76ec-6b7e-46e9-b66b-d61c0f3ae3f6\") " pod="openshift-network-operator/iptables-alerter-tbhqp"
Sep 16 02:13:02.139548 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.139227    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/de5aef44-71c6-4440-afe1-7973e763c35d-host-kubelet\") pod \"ovnkube-node-88xqz\" (UID: \"de5aef44-71c6-4440-afe1-7973e763c35d\") " pod="openshift-ovn-kubernetes/ovnkube-node-88xqz"
Sep 16 02:13:02.139548 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.139276    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ae9f48ae-4a32-4f07-8ea3-ce26faa19349-kubelet-dir\") pod \"aws-ebs-csi-driver-node-85zxn\" (UID: \"ae9f48ae-4a32-4f07-8ea3-ce26faa19349\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-85zxn"
Sep 16 02:13:02.139548 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.139296    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/de5aef44-71c6-4440-afe1-7973e763c35d-env-overrides\") pod \"ovnkube-node-88xqz\" (UID: \"de5aef44-71c6-4440-afe1-7973e763c35d\") " pod="openshift-ovn-kubernetes/ovnkube-node-88xqz"
Sep 16 02:13:02.139548 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.139301    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/4b370e61-b8a4-4423-8673-4b604907e2f7-etc-sysconfig\") pod \"tuned-2kqqm\" (UID: \"4b370e61-b8a4-4423-8673-4b604907e2f7\") " pod="openshift-cluster-node-tuning-operator/tuned-2kqqm"
Sep 16 02:13:02.139548 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.139315    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/835b007c-0c9a-4c8f-94dd-68f8a3b3a137-hosts-file\") pod \"node-resolver-9522z\" (UID: \"835b007c-0c9a-4c8f-94dd-68f8a3b3a137\") " pod="openshift-dns/node-resolver-9522z"
Sep 16 02:13:02.139548 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.139330    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bc90ef77-c833-4991-9fe8-656fdf2f7457-multus-cni-dir\") pod \"multus-ncht9\" (UID: \"bc90ef77-c833-4991-9fe8-656fdf2f7457\") " pod="openshift-multus/multus-ncht9"
Sep 16 02:13:02.139548 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.139354    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bc90ef77-c833-4991-9fe8-656fdf2f7457-host-var-lib-cni-bin\") pod \"multus-ncht9\" (UID: \"bc90ef77-c833-4991-9fe8-656fdf2f7457\") " pod="openshift-multus/multus-ncht9"
Sep 16 02:13:02.139548 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.139358    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bc90ef77-c833-4991-9fe8-656fdf2f7457-host-run-netns\") pod \"multus-ncht9\" (UID: \"bc90ef77-c833-4991-9fe8-656fdf2f7457\") " pod="openshift-multus/multus-ncht9"
Sep 16 02:13:02.139548 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.139393    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/bc90ef77-c833-4991-9fe8-656fdf2f7457-host-var-lib-cni-multus\") pod \"multus-ncht9\" (UID: \"bc90ef77-c833-4991-9fe8-656fdf2f7457\") " pod="openshift-multus/multus-ncht9"
Sep 16 02:13:02.139548 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.139399    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bc90ef77-c833-4991-9fe8-656fdf2f7457-host-var-lib-cni-bin\") pod \"multus-ncht9\" (UID: \"bc90ef77-c833-4991-9fe8-656fdf2f7457\") " pod="openshift-multus/multus-ncht9"
Sep 16 02:13:02.139548 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.139423    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bc90ef77-c833-4991-9fe8-656fdf2f7457-host-var-lib-kubelet\") pod \"multus-ncht9\" (UID: \"bc90ef77-c833-4991-9fe8-656fdf2f7457\") " pod="openshift-multus/multus-ncht9"
Sep 16 02:13:02.139548 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.139429    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/34a0d09d-a96b-4242-9d17-61fc3b8d88bd-system-cni-dir\") pod \"multus-additional-cni-plugins-9g4cg\" (UID: \"34a0d09d-a96b-4242-9d17-61fc3b8d88bd\") " pod="openshift-multus/multus-additional-cni-plugins-9g4cg"
Sep 16 02:13:02.139548 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.139449    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bc90ef77-c833-4991-9fe8-656fdf2f7457-multus-conf-dir\") pod \"multus-ncht9\" (UID: \"bc90ef77-c833-4991-9fe8-656fdf2f7457\") " pod="openshift-multus/multus-ncht9"
Sep 16 02:13:02.139548 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.139507    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/34a0d09d-a96b-4242-9d17-61fc3b8d88bd-cni-binary-copy\") pod \"multus-additional-cni-plugins-9g4cg\" (UID: \"34a0d09d-a96b-4242-9d17-61fc3b8d88bd\") " pod="openshift-multus/multus-additional-cni-plugins-9g4cg"
Sep 16 02:13:02.139548 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.139517    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/4b370e61-b8a4-4423-8673-4b604907e2f7-etc-modprobe-d\") pod \"tuned-2kqqm\" (UID: \"4b370e61-b8a4-4423-8673-4b604907e2f7\") " pod="openshift-cluster-node-tuning-operator/tuned-2kqqm"
Sep 16 02:13:02.139548 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.139532    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4b370e61-b8a4-4423-8673-4b604907e2f7-tmp\") pod \"tuned-2kqqm\" (UID: \"4b370e61-b8a4-4423-8673-4b604907e2f7\") " pod="openshift-cluster-node-tuning-operator/tuned-2kqqm"
Sep 16 02:13:02.140325 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.139658    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/4b370e61-b8a4-4423-8673-4b604907e2f7-etc-sysctl-d\") pod \"tuned-2kqqm\" (UID: \"4b370e61-b8a4-4423-8673-4b604907e2f7\") " pod="openshift-cluster-node-tuning-operator/tuned-2kqqm"
Sep 16 02:13:02.140325 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.139882    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4b370e61-b8a4-4423-8673-4b604907e2f7-host\") pod \"tuned-2kqqm\" (UID: \"4b370e61-b8a4-4423-8673-4b604907e2f7\") " pod="openshift-cluster-node-tuning-operator/tuned-2kqqm"
Sep 16 02:13:02.140325 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.139999    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/34a0d09d-a96b-4242-9d17-61fc3b8d88bd-cni-binary-copy\") pod \"multus-additional-cni-plugins-9g4cg\" (UID: \"34a0d09d-a96b-4242-9d17-61fc3b8d88bd\") " pod="openshift-multus/multus-additional-cni-plugins-9g4cg"
Sep 16 02:13:02.140325 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:02.140097    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 02:13:02.140325 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:02.140219    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3472392-891d-49a1-83a7-57f1f99f4b79-metrics-certs podName:e3472392-891d-49a1-83a7-57f1f99f4b79 nodeName:}" failed. No retries permitted until 2026-09-16 02:13:02.640182922 +0000 UTC m=+3.023698170 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e3472392-891d-49a1-83a7-57f1f99f4b79-metrics-certs") pod "network-metrics-daemon-6lwr9" (UID: "e3472392-891d-49a1-83a7-57f1f99f4b79") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 02:13:02.140596 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.140380    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bc90ef77-c833-4991-9fe8-656fdf2f7457-cni-binary-copy\") pod \"multus-ncht9\" (UID: \"bc90ef77-c833-4991-9fe8-656fdf2f7457\") " pod="openshift-multus/multus-ncht9"
Sep 16 02:13:02.140596 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.140409    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ae9f48ae-4a32-4f07-8ea3-ce26faa19349-etc-selinux\") pod \"aws-ebs-csi-driver-node-85zxn\" (UID: \"ae9f48ae-4a32-4f07-8ea3-ce26faa19349\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-85zxn"
Sep 16 02:13:02.140596 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.140430    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/de5aef44-71c6-4440-afe1-7973e763c35d-run-systemd\") pod \"ovnkube-node-88xqz\" (UID: \"de5aef44-71c6-4440-afe1-7973e763c35d\") " pod="openshift-ovn-kubernetes/ovnkube-node-88xqz"
Sep 16 02:13:02.140596 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.140455    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/bc90ef77-c833-4991-9fe8-656fdf2f7457-hostroot\") pod \"multus-ncht9\" (UID: \"bc90ef77-c833-4991-9fe8-656fdf2f7457\") " pod="openshift-multus/multus-ncht9"
Sep 16 02:13:02.140596 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.140461    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/34a0d09d-a96b-4242-9d17-61fc3b8d88bd-cnibin\") pod \"multus-additional-cni-plugins-9g4cg\" (UID: \"34a0d09d-a96b-4242-9d17-61fc3b8d88bd\") " pod="openshift-multus/multus-additional-cni-plugins-9g4cg"
Sep 16 02:13:02.140596 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.140511    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/4b370e61-b8a4-4423-8673-4b604907e2f7-etc-systemd\") pod \"tuned-2kqqm\" (UID: \"4b370e61-b8a4-4423-8673-4b604907e2f7\") " pod="openshift-cluster-node-tuning-operator/tuned-2kqqm"
Sep 16 02:13:02.140596 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.140544    2579 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-9kbd7"
Sep 16 02:13:02.140596 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.140558    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/de5aef44-71c6-4440-afe1-7973e763c35d-host-slash\") pod \"ovnkube-node-88xqz\" (UID: \"de5aef44-71c6-4440-afe1-7973e763c35d\") " pod="openshift-ovn-kubernetes/ovnkube-node-88xqz"
Sep 16 02:13:02.140596 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.140576    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/bc90ef77-c833-4991-9fe8-656fdf2f7457-host-var-lib-cni-multus\") pod \"multus-ncht9\" (UID: \"bc90ef77-c833-4991-9fe8-656fdf2f7457\") " pod="openshift-multus/multus-ncht9"
Sep 16 02:13:02.140596 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.140515    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ae9f48ae-4a32-4f07-8ea3-ce26faa19349-kubelet-dir\") pod \"aws-ebs-csi-driver-node-85zxn\" (UID: \"ae9f48ae-4a32-4f07-8ea3-ce26faa19349\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-85zxn"
Sep 16 02:13:02.141058 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.140616    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/de5aef44-71c6-4440-afe1-7973e763c35d-log-socket\") pod \"ovnkube-node-88xqz\" (UID: \"de5aef44-71c6-4440-afe1-7973e763c35d\") " pod="openshift-ovn-kubernetes/ovnkube-node-88xqz"
Sep 16 02:13:02.141058 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.140634    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/de5aef44-71c6-4440-afe1-7973e763c35d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-88xqz\" (UID: \"de5aef44-71c6-4440-afe1-7973e763c35d\") " pod="openshift-ovn-kubernetes/ovnkube-node-88xqz"
Sep 16 02:13:02.141058 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.140658    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t6psc\" (UniqueName: \"kubernetes.io/projected/496c15b3-4d67-473f-b894-2e66b47df562-kube-api-access-t6psc\") pod \"node-ca-kwdns\" (UID: \"496c15b3-4d67-473f-b894-2e66b47df562\") " pod="openshift-image-registry/node-ca-kwdns"
Sep 16 02:13:02.141058 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.140677    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/bc90ef77-c833-4991-9fe8-656fdf2f7457-multus-socket-dir-parent\") pod \"multus-ncht9\" (UID: \"bc90ef77-c833-4991-9fe8-656fdf2f7457\") " pod="openshift-multus/multus-ncht9"
Sep 16 02:13:02.141058 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.140705    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qpqhz\" (UniqueName: \"kubernetes.io/projected/34a0d09d-a96b-4242-9d17-61fc3b8d88bd-kube-api-access-qpqhz\") pod \"multus-additional-cni-plugins-9g4cg\" (UID: \"34a0d09d-a96b-4242-9d17-61fc3b8d88bd\") " pod="openshift-multus/multus-additional-cni-plugins-9g4cg"
Sep 16 02:13:02.141058 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.140760    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bc90ef77-c833-4991-9fe8-656fdf2f7457-multus-cni-dir\") pod \"multus-ncht9\" (UID: \"bc90ef77-c833-4991-9fe8-656fdf2f7457\") " pod="openshift-multus/multus-ncht9"
Sep 16 02:13:02.141058 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.140762    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ae9f48ae-4a32-4f07-8ea3-ce26faa19349-socket-dir\") pod \"aws-ebs-csi-driver-node-85zxn\" (UID: \"ae9f48ae-4a32-4f07-8ea3-ce26faa19349\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-85zxn"
Sep 16 02:13:02.141058 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.140800    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4b370e61-b8a4-4423-8673-4b604907e2f7-var-lib-kubelet\") pod \"tuned-2kqqm\" (UID: \"4b370e61-b8a4-4423-8673-4b604907e2f7\") " pod="openshift-cluster-node-tuning-operator/tuned-2kqqm"
Sep 16 02:13:02.141058 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.140824    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/de5aef44-71c6-4440-afe1-7973e763c35d-run-openvswitch\") pod \"ovnkube-node-88xqz\" (UID: \"de5aef44-71c6-4440-afe1-7973e763c35d\") " pod="openshift-ovn-kubernetes/ovnkube-node-88xqz"
Sep 16 02:13:02.141058 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.140846    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ae9f48ae-4a32-4f07-8ea3-ce26faa19349-socket-dir\") pod \"aws-ebs-csi-driver-node-85zxn\" (UID: \"ae9f48ae-4a32-4f07-8ea3-ce26faa19349\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-85zxn"
Sep 16 02:13:02.141058 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.140845    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/de5aef44-71c6-4440-afe1-7973e763c35d-node-log\") pod \"ovnkube-node-88xqz\" (UID: \"de5aef44-71c6-4440-afe1-7973e763c35d\") " pod="openshift-ovn-kubernetes/ovnkube-node-88xqz"
Sep 16 02:13:02.141058 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.140895    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4b370e61-b8a4-4423-8673-4b604907e2f7-var-lib-kubelet\") pod \"tuned-2kqqm\" (UID: \"4b370e61-b8a4-4423-8673-4b604907e2f7\") " pod="openshift-cluster-node-tuning-operator/tuned-2kqqm"
Sep 16 02:13:02.141058 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.140904    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/496c15b3-4d67-473f-b894-2e66b47df562-host\") pod \"node-ca-kwdns\" (UID: \"496c15b3-4d67-473f-b894-2e66b47df562\") " pod="openshift-image-registry/node-ca-kwdns"
Sep 16 02:13:02.141058 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.140931    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/de5aef44-71c6-4440-afe1-7973e763c35d-host-run-ovn-kubernetes\") pod \"ovnkube-node-88xqz\" (UID: \"de5aef44-71c6-4440-afe1-7973e763c35d\") " pod="openshift-ovn-kubernetes/ovnkube-node-88xqz"
Sep 16 02:13:02.141058 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.140958    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/de5aef44-71c6-4440-afe1-7973e763c35d-ovn-node-metrics-cert\") pod \"ovnkube-node-88xqz\" (UID: \"de5aef44-71c6-4440-afe1-7973e763c35d\") " pod="openshift-ovn-kubernetes/ovnkube-node-88xqz"
Sep 16 02:13:02.141058 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.140993    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2b6aaa85-d6dd-420d-a818-f56d06ce6467-kubelet-config\") pod \"global-pull-secret-syncer-cdwh7\" (UID: \"2b6aaa85-d6dd-420d-a818-f56d06ce6467\") " pod="kube-system/global-pull-secret-syncer-cdwh7"
Sep 16 02:13:02.141058 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.141035    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/de5aef44-71c6-4440-afe1-7973e763c35d-ovnkube-config\") pod \"ovnkube-node-88xqz\" (UID: \"de5aef44-71c6-4440-afe1-7973e763c35d\") " pod="openshift-ovn-kubernetes/ovnkube-node-88xqz"
Sep 16 02:13:02.141920 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.141069    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vdvtq\" (UniqueName: \"kubernetes.io/projected/835b007c-0c9a-4c8f-94dd-68f8a3b3a137-kube-api-access-vdvtq\") pod \"node-resolver-9522z\" (UID: \"835b007c-0c9a-4c8f-94dd-68f8a3b3a137\") " pod="openshift-dns/node-resolver-9522z"
Sep 16 02:13:02.141920 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.141103    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ae9f48ae-4a32-4f07-8ea3-ce26faa19349-registration-dir\") pod \"aws-ebs-csi-driver-node-85zxn\" (UID: \"ae9f48ae-4a32-4f07-8ea3-ce26faa19349\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-85zxn"
Sep 16 02:13:02.141920 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.141123    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/4b370e61-b8a4-4423-8673-4b604907e2f7-run\") pod \"tuned-2kqqm\" (UID: \"4b370e61-b8a4-4423-8673-4b604907e2f7\") " pod="openshift-cluster-node-tuning-operator/tuned-2kqqm"
Sep 16 02:13:02.141920 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.141148    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/98489f25-0768-47e2-b8f0-e2a7c7af2063-agent-certs\") pod \"konnectivity-agent-bhd9l\" (UID: \"98489f25-0768-47e2-b8f0-e2a7c7af2063\") " pod="kube-system/konnectivity-agent-bhd9l"
Sep 16 02:13:02.141920 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.141162    2579 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 16 02:13:02.141920 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.141181    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bc90ef77-c833-4991-9fe8-656fdf2f7457-host-var-lib-kubelet\") pod \"multus-ncht9\" (UID: \"bc90ef77-c833-4991-9fe8-656fdf2f7457\") " pod="openshift-multus/multus-ncht9"
Sep 16 02:13:02.141920 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.141241    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/4b370e61-b8a4-4423-8673-4b604907e2f7-run\") pod \"tuned-2kqqm\" (UID: \"4b370e61-b8a4-4423-8673-4b604907e2f7\") " pod="openshift-cluster-node-tuning-operator/tuned-2kqqm"
Sep 16 02:13:02.141920 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.141258    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bc90ef77-c833-4991-9fe8-656fdf2f7457-multus-conf-dir\") pod \"multus-ncht9\" (UID: \"bc90ef77-c833-4991-9fe8-656fdf2f7457\") " pod="openshift-multus/multus-ncht9"
Sep 16 02:13:02.141920 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.141288    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bc90ef77-c833-4991-9fe8-656fdf2f7457-etc-kubernetes\") pod \"multus-ncht9\" (UID: \"bc90ef77-c833-4991-9fe8-656fdf2f7457\") " pod="openshift-multus/multus-ncht9"
Sep 16 02:13:02.141920 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.141312    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4b370e61-b8a4-4423-8673-4b604907e2f7-sys\") pod \"tuned-2kqqm\" (UID: \"4b370e61-b8a4-4423-8673-4b604907e2f7\") " pod="openshift-cluster-node-tuning-operator/tuned-2kqqm"
Sep 16 02:13:02.141920 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.141359    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/de5aef44-71c6-4440-afe1-7973e763c35d-host-run-netns\") pod \"ovnkube-node-88xqz\" (UID: \"de5aef44-71c6-4440-afe1-7973e763c35d\") " pod="openshift-ovn-kubernetes/ovnkube-node-88xqz"
Sep 16 02:13:02.141920 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.141365    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/34a0d09d-a96b-4242-9d17-61fc3b8d88bd-tuning-conf-dir\") pod \"multus-additional-cni-plugins-9g4cg\" (UID: \"34a0d09d-a96b-4242-9d17-61fc3b8d88bd\") " pod="openshift-multus/multus-additional-cni-plugins-9g4cg"
Sep 16 02:13:02.141920 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.141383    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/de5aef44-71c6-4440-afe1-7973e763c35d-ovnkube-script-lib\") pod \"ovnkube-node-88xqz\" (UID: \"de5aef44-71c6-4440-afe1-7973e763c35d\") " pod="openshift-ovn-kubernetes/ovnkube-node-88xqz"
Sep 16 02:13:02.141920 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.141408    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wt9fv\" (UniqueName: \"kubernetes.io/projected/de5aef44-71c6-4440-afe1-7973e763c35d-kube-api-access-wt9fv\") pod \"ovnkube-node-88xqz\" (UID: \"de5aef44-71c6-4440-afe1-7973e763c35d\") " pod="openshift-ovn-kubernetes/ovnkube-node-88xqz"
Sep 16 02:13:02.141920 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.141411    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/34a0d09d-a96b-4242-9d17-61fc3b8d88bd-cnibin\") pod \"multus-additional-cni-plugins-9g4cg\" (UID: \"34a0d09d-a96b-4242-9d17-61fc3b8d88bd\") " pod="openshift-multus/multus-additional-cni-plugins-9g4cg"
Sep 16 02:13:02.141920 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.141436    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6wb6w\" (UniqueName: \"kubernetes.io/projected/4b370e61-b8a4-4423-8673-4b604907e2f7-kube-api-access-6wb6w\") pod \"tuned-2kqqm\" (UID: \"4b370e61-b8a4-4423-8673-4b604907e2f7\") " pod="openshift-cluster-node-tuning-operator/tuned-2kqqm"
Sep 16 02:13:02.141920 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.141461    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/bc90ef77-c833-4991-9fe8-656fdf2f7457-multus-socket-dir-parent\") pod \"multus-ncht9\" (UID: \"bc90ef77-c833-4991-9fe8-656fdf2f7457\") " pod="openshift-multus/multus-ncht9"
Sep 16 02:13:02.141920 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.141540    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4b370e61-b8a4-4423-8673-4b604907e2f7-sys\") pod \"tuned-2kqqm\" (UID: \"4b370e61-b8a4-4423-8673-4b604907e2f7\") " pod="openshift-cluster-node-tuning-operator/tuned-2kqqm"
Sep 16 02:13:02.142808 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.141599    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ae9f48ae-4a32-4f07-8ea3-ce26faa19349-registration-dir\") pod \"aws-ebs-csi-driver-node-85zxn\" (UID: \"ae9f48ae-4a32-4f07-8ea3-ce26faa19349\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-85zxn"
Sep 16 02:13:02.142808 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.141641    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bc90ef77-c833-4991-9fe8-656fdf2f7457-etc-kubernetes\") pod \"multus-ncht9\" (UID: \"bc90ef77-c833-4991-9fe8-656fdf2f7457\") " pod="openshift-multus/multus-ncht9"
Sep 16 02:13:02.142808 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.141795    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/4b370e61-b8a4-4423-8673-4b604907e2f7-etc-systemd\") pod \"tuned-2kqqm\" (UID: \"4b370e61-b8a4-4423-8673-4b604907e2f7\") " pod="openshift-cluster-node-tuning-operator/tuned-2kqqm"
Sep 16 02:13:02.142808 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.141884    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bc90ef77-c833-4991-9fe8-656fdf2f7457-cnibin\") pod \"multus-ncht9\" (UID: \"bc90ef77-c833-4991-9fe8-656fdf2f7457\") " pod="openshift-multus/multus-ncht9"
Sep 16 02:13:02.142808 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.141945    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bc90ef77-c833-4991-9fe8-656fdf2f7457-cnibin\") pod \"multus-ncht9\" (UID: \"bc90ef77-c833-4991-9fe8-656fdf2f7457\") " pod="openshift-multus/multus-ncht9"
Sep 16 02:13:02.142808 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.141944    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/bc90ef77-c833-4991-9fe8-656fdf2f7457-multus-daemon-config\") pod \"multus-ncht9\" (UID: \"bc90ef77-c833-4991-9fe8-656fdf2f7457\") " pod="openshift-multus/multus-ncht9"
Sep 16 02:13:02.142808 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.141998    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/bc90ef77-c833-4991-9fe8-656fdf2f7457-host-run-multus-certs\") pod \"multus-ncht9\" (UID: \"bc90ef77-c833-4991-9fe8-656fdf2f7457\") " pod="openshift-multus/multus-ncht9"
Sep 16 02:13:02.142808 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.142025    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l5fl9\" (UniqueName: \"kubernetes.io/projected/ae9f48ae-4a32-4f07-8ea3-ce26faa19349-kube-api-access-l5fl9\") pod \"aws-ebs-csi-driver-node-85zxn\" (UID: \"ae9f48ae-4a32-4f07-8ea3-ce26faa19349\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-85zxn"
Sep 16 02:13:02.142808 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.142054    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4b370e61-b8a4-4423-8673-4b604907e2f7-lib-modules\") pod \"tuned-2kqqm\" (UID: \"4b370e61-b8a4-4423-8673-4b604907e2f7\") " pod="openshift-cluster-node-tuning-operator/tuned-2kqqm"
Sep 16 02:13:02.142808 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.142075    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/4b370e61-b8a4-4423-8673-4b604907e2f7-etc-tuned\") pod \"tuned-2kqqm\" (UID: \"4b370e61-b8a4-4423-8673-4b604907e2f7\") " pod="openshift-cluster-node-tuning-operator/tuned-2kqqm"
Sep 16 02:13:02.142808 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.142098    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bc90ef77-c833-4991-9fe8-656fdf2f7457-system-cni-dir\") pod \"multus-ncht9\" (UID: \"bc90ef77-c833-4991-9fe8-656fdf2f7457\") " pod="openshift-multus/multus-ncht9"
Sep 16 02:13:02.142808 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.142153    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/34a0d09d-a96b-4242-9d17-61fc3b8d88bd-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-9g4cg\" (UID: \"34a0d09d-a96b-4242-9d17-61fc3b8d88bd\") " pod="openshift-multus/multus-additional-cni-plugins-9g4cg"
Sep 16 02:13:02.142808 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.142394    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/bc90ef77-c833-4991-9fe8-656fdf2f7457-multus-daemon-config\") pod \"multus-ncht9\" (UID: \"bc90ef77-c833-4991-9fe8-656fdf2f7457\") " pod="openshift-multus/multus-ncht9"
Sep 16 02:13:02.142808 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.142435    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/bc90ef77-c833-4991-9fe8-656fdf2f7457-host-run-multus-certs\") pod \"multus-ncht9\" (UID: \"bc90ef77-c833-4991-9fe8-656fdf2f7457\") " pod="openshift-multus/multus-ncht9"
Sep 16 02:13:02.142808 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.142533    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4b370e61-b8a4-4423-8673-4b604907e2f7-lib-modules\") pod \"tuned-2kqqm\" (UID: \"4b370e61-b8a4-4423-8673-4b604907e2f7\") " pod="openshift-cluster-node-tuning-operator/tuned-2kqqm"
Sep 16 02:13:02.142808 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.142571    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bc90ef77-c833-4991-9fe8-656fdf2f7457-system-cni-dir\") pod \"multus-ncht9\" (UID: \"bc90ef77-c833-4991-9fe8-656fdf2f7457\") " pod="openshift-multus/multus-ncht9"
Sep 16 02:13:02.142808 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.142624    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/34a0d09d-a96b-4242-9d17-61fc3b8d88bd-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-9g4cg\" (UID: \"34a0d09d-a96b-4242-9d17-61fc3b8d88bd\") " pod="openshift-multus/multus-additional-cni-plugins-9g4cg"
Sep 16 02:13:02.146179 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.146155    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4b370e61-b8a4-4423-8673-4b604907e2f7-tmp\") pod \"tuned-2kqqm\" (UID: \"4b370e61-b8a4-4423-8673-4b604907e2f7\") " pod="openshift-cluster-node-tuning-operator/tuned-2kqqm"
Sep 16 02:13:02.146585 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.146563    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/4b370e61-b8a4-4423-8673-4b604907e2f7-etc-tuned\") pod \"tuned-2kqqm\" (UID: \"4b370e61-b8a4-4423-8673-4b604907e2f7\") " pod="openshift-cluster-node-tuning-operator/tuned-2kqqm"
Sep 16 02:13:02.146671 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.146661    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/98489f25-0768-47e2-b8f0-e2a7c7af2063-agent-certs\") pod \"konnectivity-agent-bhd9l\" (UID: \"98489f25-0768-47e2-b8f0-e2a7c7af2063\") " pod="kube-system/konnectivity-agent-bhd9l"
Sep 16 02:13:02.147386 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.147365    2579 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-9kbd7"
Sep 16 02:13:02.149258 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:02.149240    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 02:13:02.149358 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:02.149266    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 02:13:02.149358 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:02.149287    2579 projected.go:196] Error preparing data for projected volume kube-api-access-wlf8b for pod openshift-network-diagnostics/network-check-target-7b42r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 02:13:02.149358 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:02.149342    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/28c55959-36b5-4d35-9e1a-f046d6d36f74-kube-api-access-wlf8b podName:28c55959-36b5-4d35-9e1a-f046d6d36f74 nodeName:}" failed. No retries permitted until 2026-09-16 02:13:02.649325204 +0000 UTC m=+3.032840445 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-wlf8b" (UniqueName: "kubernetes.io/projected/28c55959-36b5-4d35-9e1a-f046d6d36f74-kube-api-access-wlf8b") pod "network-check-target-7b42r" (UID: "28c55959-36b5-4d35-9e1a-f046d6d36f74") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 02:13:02.152260 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.152237    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8fhvr\" (UniqueName: \"kubernetes.io/projected/e3472392-891d-49a1-83a7-57f1f99f4b79-kube-api-access-8fhvr\") pod \"network-metrics-daemon-6lwr9\" (UID: \"e3472392-891d-49a1-83a7-57f1f99f4b79\") " pod="openshift-multus/network-metrics-daemon-6lwr9"
Sep 16 02:13:02.152550 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.152532    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lxlhf\" (UniqueName: \"kubernetes.io/projected/bc90ef77-c833-4991-9fe8-656fdf2f7457-kube-api-access-lxlhf\") pod \"multus-ncht9\" (UID: \"bc90ef77-c833-4991-9fe8-656fdf2f7457\") " pod="openshift-multus/multus-ncht9"
Sep 16 02:13:02.156812 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.156788    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6wb6w\" (UniqueName: \"kubernetes.io/projected/4b370e61-b8a4-4423-8673-4b604907e2f7-kube-api-access-6wb6w\") pod \"tuned-2kqqm\" (UID: \"4b370e61-b8a4-4423-8673-4b604907e2f7\") " pod="openshift-cluster-node-tuning-operator/tuned-2kqqm"
Sep 16 02:13:02.157055 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.157030    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l5fl9\" (UniqueName: \"kubernetes.io/projected/ae9f48ae-4a32-4f07-8ea3-ce26faa19349-kube-api-access-l5fl9\") pod \"aws-ebs-csi-driver-node-85zxn\" (UID: \"ae9f48ae-4a32-4f07-8ea3-ce26faa19349\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-85zxn"
Sep 16 02:13:02.157918 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.157899    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qpqhz\" (UniqueName: \"kubernetes.io/projected/34a0d09d-a96b-4242-9d17-61fc3b8d88bd-kube-api-access-qpqhz\") pod \"multus-additional-cni-plugins-9g4cg\" (UID: \"34a0d09d-a96b-4242-9d17-61fc3b8d88bd\") " pod="openshift-multus/multus-additional-cni-plugins-9g4cg"
Sep 16 02:13:02.243021 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.242991    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/de5aef44-71c6-4440-afe1-7973e763c35d-etc-openvswitch\") pod \"ovnkube-node-88xqz\" (UID: \"de5aef44-71c6-4440-afe1-7973e763c35d\") " pod="openshift-ovn-kubernetes/ovnkube-node-88xqz"
Sep 16 02:13:02.243175 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.243030    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/de5aef44-71c6-4440-afe1-7973e763c35d-host-cni-netd\") pod \"ovnkube-node-88xqz\" (UID: \"de5aef44-71c6-4440-afe1-7973e763c35d\") " pod="openshift-ovn-kubernetes/ovnkube-node-88xqz"
Sep 16 02:13:02.243175 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.243081    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/835b007c-0c9a-4c8f-94dd-68f8a3b3a137-tmp-dir\") pod \"node-resolver-9522z\" (UID: \"835b007c-0c9a-4c8f-94dd-68f8a3b3a137\") " pod="openshift-dns/node-resolver-9522z"
Sep 16 02:13:02.243175 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.243097    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/de5aef44-71c6-4440-afe1-7973e763c35d-etc-openvswitch\") pod \"ovnkube-node-88xqz\" (UID: \"de5aef44-71c6-4440-afe1-7973e763c35d\") " pod="openshift-ovn-kubernetes/ovnkube-node-88xqz"
Sep 16 02:13:02.243175 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.243138    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/496c15b3-4d67-473f-b894-2e66b47df562-serviceca\") pod \"node-ca-kwdns\" (UID: \"496c15b3-4d67-473f-b894-2e66b47df562\") " pod="openshift-image-registry/node-ca-kwdns"
Sep 16 02:13:02.243175 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.243162    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2b6aaa85-d6dd-420d-a818-f56d06ce6467-original-pull-secret\") pod \"global-pull-secret-syncer-cdwh7\" (UID: \"2b6aaa85-d6dd-420d-a818-f56d06ce6467\") " pod="kube-system/global-pull-secret-syncer-cdwh7"
Sep 16 02:13:02.243508 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.243181    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/de5aef44-71c6-4440-afe1-7973e763c35d-var-lib-openvswitch\") pod \"ovnkube-node-88xqz\" (UID: \"de5aef44-71c6-4440-afe1-7973e763c35d\") " pod="openshift-ovn-kubernetes/ovnkube-node-88xqz"
Sep 16 02:13:02.243508 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.243197    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/de5aef44-71c6-4440-afe1-7973e763c35d-run-ovn\") pod \"ovnkube-node-88xqz\" (UID: \"de5aef44-71c6-4440-afe1-7973e763c35d\") " pod="openshift-ovn-kubernetes/ovnkube-node-88xqz"
Sep 16 02:13:02.243508 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.243210    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/de5aef44-71c6-4440-afe1-7973e763c35d-host-cni-bin\") pod \"ovnkube-node-88xqz\" (UID: \"de5aef44-71c6-4440-afe1-7973e763c35d\") " pod="openshift-ovn-kubernetes/ovnkube-node-88xqz"
Sep 16 02:13:02.243508 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.243232    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c27d76ec-6b7e-46e9-b66b-d61c0f3ae3f6-iptables-alerter-script\") pod \"iptables-alerter-tbhqp\" (UID: \"c27d76ec-6b7e-46e9-b66b-d61c0f3ae3f6\") " pod="openshift-network-operator/iptables-alerter-tbhqp"
Sep 16 02:13:02.243508 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.243246    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2b6aaa85-d6dd-420d-a818-f56d06ce6467-dbus\") pod \"global-pull-secret-syncer-cdwh7\" (UID: \"2b6aaa85-d6dd-420d-a818-f56d06ce6467\") " pod="kube-system/global-pull-secret-syncer-cdwh7"
Sep 16 02:13:02.243508 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.243272    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c27d76ec-6b7e-46e9-b66b-d61c0f3ae3f6-host-slash\") pod \"iptables-alerter-tbhqp\" (UID: \"c27d76ec-6b7e-46e9-b66b-d61c0f3ae3f6\") " pod="openshift-network-operator/iptables-alerter-tbhqp"
Sep 16 02:13:02.243508 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.243288    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hjzwc\" (UniqueName: \"kubernetes.io/projected/c27d76ec-6b7e-46e9-b66b-d61c0f3ae3f6-kube-api-access-hjzwc\") pod \"iptables-alerter-tbhqp\" (UID: \"c27d76ec-6b7e-46e9-b66b-d61c0f3ae3f6\") " pod="openshift-network-operator/iptables-alerter-tbhqp"
Sep 16 02:13:02.243508 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.243309    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/de5aef44-71c6-4440-afe1-7973e763c35d-host-kubelet\") pod \"ovnkube-node-88xqz\" (UID: \"de5aef44-71c6-4440-afe1-7973e763c35d\") " pod="openshift-ovn-kubernetes/ovnkube-node-88xqz"
Sep 16 02:13:02.243508 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.243331    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/de5aef44-71c6-4440-afe1-7973e763c35d-env-overrides\") pod \"ovnkube-node-88xqz\" (UID: \"de5aef44-71c6-4440-afe1-7973e763c35d\") " pod="openshift-ovn-kubernetes/ovnkube-node-88xqz"
Sep 16 02:13:02.243508 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.243345    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/835b007c-0c9a-4c8f-94dd-68f8a3b3a137-hosts-file\") pod \"node-resolver-9522z\" (UID: \"835b007c-0c9a-4c8f-94dd-68f8a3b3a137\") " pod="openshift-dns/node-resolver-9522z"
Sep 16 02:13:02.243508 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.243365    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/de5aef44-71c6-4440-afe1-7973e763c35d-run-systemd\") pod \"ovnkube-node-88xqz\" (UID: \"de5aef44-71c6-4440-afe1-7973e763c35d\") " pod="openshift-ovn-kubernetes/ovnkube-node-88xqz"
Sep 16 02:13:02.243508 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.243386    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/de5aef44-71c6-4440-afe1-7973e763c35d-host-slash\") pod \"ovnkube-node-88xqz\" (UID: \"de5aef44-71c6-4440-afe1-7973e763c35d\") " pod="openshift-ovn-kubernetes/ovnkube-node-88xqz"
Sep 16 02:13:02.243508 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.243409    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/de5aef44-71c6-4440-afe1-7973e763c35d-log-socket\") pod \"ovnkube-node-88xqz\" (UID: \"de5aef44-71c6-4440-afe1-7973e763c35d\") " pod="openshift-ovn-kubernetes/ovnkube-node-88xqz"
Sep 16 02:13:02.243508 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.243414    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/835b007c-0c9a-4c8f-94dd-68f8a3b3a137-tmp-dir\") pod \"node-resolver-9522z\" (UID: \"835b007c-0c9a-4c8f-94dd-68f8a3b3a137\") " pod="openshift-dns/node-resolver-9522z"
Sep 16 02:13:02.243508 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.243424    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/de5aef44-71c6-4440-afe1-7973e763c35d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-88xqz\" (UID: \"de5aef44-71c6-4440-afe1-7973e763c35d\") " pod="openshift-ovn-kubernetes/ovnkube-node-88xqz"
Sep 16 02:13:02.243508 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.243440    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t6psc\" (UniqueName: \"kubernetes.io/projected/496c15b3-4d67-473f-b894-2e66b47df562-kube-api-access-t6psc\") pod \"node-ca-kwdns\" (UID: \"496c15b3-4d67-473f-b894-2e66b47df562\") " pod="openshift-image-registry/node-ca-kwdns"
Sep 16 02:13:02.243508 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.243491    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/de5aef44-71c6-4440-afe1-7973e763c35d-run-openvswitch\") pod \"ovnkube-node-88xqz\" (UID: \"de5aef44-71c6-4440-afe1-7973e763c35d\") " pod="openshift-ovn-kubernetes/ovnkube-node-88xqz"
Sep 16 02:13:02.244180 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.243493    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/de5aef44-71c6-4440-afe1-7973e763c35d-host-cni-netd\") pod \"ovnkube-node-88xqz\" (UID: \"de5aef44-71c6-4440-afe1-7973e763c35d\") " pod="openshift-ovn-kubernetes/ovnkube-node-88xqz"
Sep 16 02:13:02.244180 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.243505    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/de5aef44-71c6-4440-afe1-7973e763c35d-node-log\") pod \"ovnkube-node-88xqz\" (UID: \"de5aef44-71c6-4440-afe1-7973e763c35d\") " pod="openshift-ovn-kubernetes/ovnkube-node-88xqz"
Sep 16 02:13:02.244180 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.243519    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/496c15b3-4d67-473f-b894-2e66b47df562-host\") pod \"node-ca-kwdns\" (UID: \"496c15b3-4d67-473f-b894-2e66b47df562\") " pod="openshift-image-registry/node-ca-kwdns"
Sep 16 02:13:02.244180 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.243537    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/de5aef44-71c6-4440-afe1-7973e763c35d-host-run-ovn-kubernetes\") pod \"ovnkube-node-88xqz\" (UID: \"de5aef44-71c6-4440-afe1-7973e763c35d\") " pod="openshift-ovn-kubernetes/ovnkube-node-88xqz"
Sep 16 02:13:02.244180 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.243555    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/de5aef44-71c6-4440-afe1-7973e763c35d-ovn-node-metrics-cert\") pod \"ovnkube-node-88xqz\" (UID: \"de5aef44-71c6-4440-afe1-7973e763c35d\") " pod="openshift-ovn-kubernetes/ovnkube-node-88xqz"
Sep 16 02:13:02.244180 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.243570    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2b6aaa85-d6dd-420d-a818-f56d06ce6467-kubelet-config\") pod \"global-pull-secret-syncer-cdwh7\" (UID: \"2b6aaa85-d6dd-420d-a818-f56d06ce6467\") " pod="kube-system/global-pull-secret-syncer-cdwh7"
Sep 16 02:13:02.244180 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.243584    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/de5aef44-71c6-4440-afe1-7973e763c35d-ovnkube-config\") pod \"ovnkube-node-88xqz\" (UID: \"de5aef44-71c6-4440-afe1-7973e763c35d\") " pod="openshift-ovn-kubernetes/ovnkube-node-88xqz"
Sep 16 02:13:02.244180 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.243598    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vdvtq\" (UniqueName: \"kubernetes.io/projected/835b007c-0c9a-4c8f-94dd-68f8a3b3a137-kube-api-access-vdvtq\") pod \"node-resolver-9522z\" (UID: \"835b007c-0c9a-4c8f-94dd-68f8a3b3a137\") " pod="openshift-dns/node-resolver-9522z"
Sep 16 02:13:02.244180 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.243621    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/de5aef44-71c6-4440-afe1-7973e763c35d-host-run-netns\") pod \"ovnkube-node-88xqz\" (UID: \"de5aef44-71c6-4440-afe1-7973e763c35d\") " pod="openshift-ovn-kubernetes/ovnkube-node-88xqz"
Sep 16 02:13:02.244180 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.243639    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/de5aef44-71c6-4440-afe1-7973e763c35d-ovnkube-script-lib\") pod \"ovnkube-node-88xqz\" (UID: \"de5aef44-71c6-4440-afe1-7973e763c35d\") " pod="openshift-ovn-kubernetes/ovnkube-node-88xqz"
Sep 16 02:13:02.244180 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.243658    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wt9fv\" (UniqueName: \"kubernetes.io/projected/de5aef44-71c6-4440-afe1-7973e763c35d-kube-api-access-wt9fv\") pod \"ovnkube-node-88xqz\" (UID: \"de5aef44-71c6-4440-afe1-7973e763c35d\") " pod="openshift-ovn-kubernetes/ovnkube-node-88xqz"
Sep 16 02:13:02.244180 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.243685    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/de5aef44-71c6-4440-afe1-7973e763c35d-systemd-units\") pod \"ovnkube-node-88xqz\" (UID: \"de5aef44-71c6-4440-afe1-7973e763c35d\") " pod="openshift-ovn-kubernetes/ovnkube-node-88xqz"
Sep 16 02:13:02.244180 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.243741    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/de5aef44-71c6-4440-afe1-7973e763c35d-systemd-units\") pod \"ovnkube-node-88xqz\" (UID: \"de5aef44-71c6-4440-afe1-7973e763c35d\") " pod="openshift-ovn-kubernetes/ovnkube-node-88xqz"
Sep 16 02:13:02.244180 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.243773    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/de5aef44-71c6-4440-afe1-7973e763c35d-host-kubelet\") pod \"ovnkube-node-88xqz\" (UID: \"de5aef44-71c6-4440-afe1-7973e763c35d\") " pod="openshift-ovn-kubernetes/ovnkube-node-88xqz"
Sep 16 02:13:02.244180 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.244033    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/496c15b3-4d67-473f-b894-2e66b47df562-serviceca\") pod \"node-ca-kwdns\" (UID: \"496c15b3-4d67-473f-b894-2e66b47df562\") " pod="openshift-image-registry/node-ca-kwdns"
Sep 16 02:13:02.244180 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:02.244096    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 02:13:02.244180 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:02.244141    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b6aaa85-d6dd-420d-a818-f56d06ce6467-original-pull-secret podName:2b6aaa85-d6dd-420d-a818-f56d06ce6467 nodeName:}" failed. No retries permitted until 2026-09-16 02:13:02.744125499 +0000 UTC m=+3.127640731 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2b6aaa85-d6dd-420d-a818-f56d06ce6467-original-pull-secret") pod "global-pull-secret-syncer-cdwh7" (UID: "2b6aaa85-d6dd-420d-a818-f56d06ce6467") : object "kube-system"/"original-pull-secret" not registered
Sep 16 02:13:02.245037 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.244198    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/de5aef44-71c6-4440-afe1-7973e763c35d-env-overrides\") pod \"ovnkube-node-88xqz\" (UID: \"de5aef44-71c6-4440-afe1-7973e763c35d\") " pod="openshift-ovn-kubernetes/ovnkube-node-88xqz"
Sep 16 02:13:02.245037 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.244260    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/835b007c-0c9a-4c8f-94dd-68f8a3b3a137-hosts-file\") pod \"node-resolver-9522z\" (UID: \"835b007c-0c9a-4c8f-94dd-68f8a3b3a137\") " pod="openshift-dns/node-resolver-9522z"
Sep 16 02:13:02.245037 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.244301    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/de5aef44-71c6-4440-afe1-7973e763c35d-run-systemd\") pod \"ovnkube-node-88xqz\" (UID: \"de5aef44-71c6-4440-afe1-7973e763c35d\") " pod="openshift-ovn-kubernetes/ovnkube-node-88xqz"
Sep 16 02:13:02.245037 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.244304    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/de5aef44-71c6-4440-afe1-7973e763c35d-var-lib-openvswitch\") pod \"ovnkube-node-88xqz\" (UID: \"de5aef44-71c6-4440-afe1-7973e763c35d\") " pod="openshift-ovn-kubernetes/ovnkube-node-88xqz"
Sep 16 02:13:02.245037 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.244329    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/de5aef44-71c6-4440-afe1-7973e763c35d-run-ovn\") pod \"ovnkube-node-88xqz\" (UID: \"de5aef44-71c6-4440-afe1-7973e763c35d\") " pod="openshift-ovn-kubernetes/ovnkube-node-88xqz"
Sep 16 02:13:02.245037 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.244343    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/de5aef44-71c6-4440-afe1-7973e763c35d-host-slash\") pod \"ovnkube-node-88xqz\" (UID: \"de5aef44-71c6-4440-afe1-7973e763c35d\") " pod="openshift-ovn-kubernetes/ovnkube-node-88xqz"
Sep 16 02:13:02.245037 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.244392    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/de5aef44-71c6-4440-afe1-7973e763c35d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-88xqz\" (UID: \"de5aef44-71c6-4440-afe1-7973e763c35d\") " pod="openshift-ovn-kubernetes/ovnkube-node-88xqz"
Sep 16 02:13:02.245037 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.244526    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/de5aef44-71c6-4440-afe1-7973e763c35d-host-run-netns\") pod \"ovnkube-node-88xqz\" (UID: \"de5aef44-71c6-4440-afe1-7973e763c35d\") " pod="openshift-ovn-kubernetes/ovnkube-node-88xqz"
Sep 16 02:13:02.245037 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.244584    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/de5aef44-71c6-4440-afe1-7973e763c35d-run-openvswitch\") pod \"ovnkube-node-88xqz\" (UID: \"de5aef44-71c6-4440-afe1-7973e763c35d\") " pod="openshift-ovn-kubernetes/ovnkube-node-88xqz"
Sep 16 02:13:02.245037 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.244592    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/de5aef44-71c6-4440-afe1-7973e763c35d-node-log\") pod \"ovnkube-node-88xqz\" (UID: \"de5aef44-71c6-4440-afe1-7973e763c35d\") " pod="openshift-ovn-kubernetes/ovnkube-node-88xqz"
Sep 16 02:13:02.245037 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.244630    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2b6aaa85-d6dd-420d-a818-f56d06ce6467-kubelet-config\") pod \"global-pull-secret-syncer-cdwh7\" (UID: \"2b6aaa85-d6dd-420d-a818-f56d06ce6467\") " pod="kube-system/global-pull-secret-syncer-cdwh7"
Sep 16 02:13:02.245037 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.244632    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c27d76ec-6b7e-46e9-b66b-d61c0f3ae3f6-host-slash\") pod \"iptables-alerter-tbhqp\" (UID: \"c27d76ec-6b7e-46e9-b66b-d61c0f3ae3f6\") " pod="openshift-network-operator/iptables-alerter-tbhqp"
Sep 16 02:13:02.245037 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.244667    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/496c15b3-4d67-473f-b894-2e66b47df562-host\") pod \"node-ca-kwdns\" (UID: \"496c15b3-4d67-473f-b894-2e66b47df562\") " pod="openshift-image-registry/node-ca-kwdns"
Sep 16 02:13:02.245037 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.244678    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/de5aef44-71c6-4440-afe1-7973e763c35d-log-socket\") pod \"ovnkube-node-88xqz\" (UID: \"de5aef44-71c6-4440-afe1-7973e763c35d\") " pod="openshift-ovn-kubernetes/ovnkube-node-88xqz"
Sep 16 02:13:02.245037 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.244702    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/de5aef44-71c6-4440-afe1-7973e763c35d-host-cni-bin\") pod \"ovnkube-node-88xqz\" (UID: \"de5aef44-71c6-4440-afe1-7973e763c35d\") " pod="openshift-ovn-kubernetes/ovnkube-node-88xqz"
Sep 16 02:13:02.245037 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.244738    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/de5aef44-71c6-4440-afe1-7973e763c35d-host-run-ovn-kubernetes\") pod \"ovnkube-node-88xqz\" (UID: \"de5aef44-71c6-4440-afe1-7973e763c35d\") " pod="openshift-ovn-kubernetes/ovnkube-node-88xqz"
Sep 16 02:13:02.245037 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.244790    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2b6aaa85-d6dd-420d-a818-f56d06ce6467-dbus\") pod \"global-pull-secret-syncer-cdwh7\" (UID: \"2b6aaa85-d6dd-420d-a818-f56d06ce6467\") " pod="kube-system/global-pull-secret-syncer-cdwh7"
Sep 16 02:13:02.245806 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.245114    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/de5aef44-71c6-4440-afe1-7973e763c35d-ovnkube-script-lib\") pod \"ovnkube-node-88xqz\" (UID: \"de5aef44-71c6-4440-afe1-7973e763c35d\") " pod="openshift-ovn-kubernetes/ovnkube-node-88xqz"
Sep 16 02:13:02.245806 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.245116    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c27d76ec-6b7e-46e9-b66b-d61c0f3ae3f6-iptables-alerter-script\") pod \"iptables-alerter-tbhqp\" (UID: \"c27d76ec-6b7e-46e9-b66b-d61c0f3ae3f6\") " pod="openshift-network-operator/iptables-alerter-tbhqp"
Sep 16 02:13:02.245806 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.245150    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/de5aef44-71c6-4440-afe1-7973e763c35d-ovnkube-config\") pod \"ovnkube-node-88xqz\" (UID: \"de5aef44-71c6-4440-afe1-7973e763c35d\") " pod="openshift-ovn-kubernetes/ovnkube-node-88xqz"
Sep 16 02:13:02.247230 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.247212    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/de5aef44-71c6-4440-afe1-7973e763c35d-ovn-node-metrics-cert\") pod \"ovnkube-node-88xqz\" (UID: \"de5aef44-71c6-4440-afe1-7973e763c35d\") " pod="openshift-ovn-kubernetes/ovnkube-node-88xqz"
Sep 16 02:13:02.251724 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.251702    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hjzwc\" (UniqueName: \"kubernetes.io/projected/c27d76ec-6b7e-46e9-b66b-d61c0f3ae3f6-kube-api-access-hjzwc\") pod \"iptables-alerter-tbhqp\" (UID: \"c27d76ec-6b7e-46e9-b66b-d61c0f3ae3f6\") " pod="openshift-network-operator/iptables-alerter-tbhqp"
Sep 16 02:13:02.252029 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.252010    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t6psc\" (UniqueName: \"kubernetes.io/projected/496c15b3-4d67-473f-b894-2e66b47df562-kube-api-access-t6psc\") pod \"node-ca-kwdns\" (UID: \"496c15b3-4d67-473f-b894-2e66b47df562\") " pod="openshift-image-registry/node-ca-kwdns"
Sep 16 02:13:02.252520 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.252497    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wt9fv\" (UniqueName: \"kubernetes.io/projected/de5aef44-71c6-4440-afe1-7973e763c35d-kube-api-access-wt9fv\") pod \"ovnkube-node-88xqz\" (UID: \"de5aef44-71c6-4440-afe1-7973e763c35d\") " pod="openshift-ovn-kubernetes/ovnkube-node-88xqz"
Sep 16 02:13:02.252623 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.252526    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vdvtq\" (UniqueName: \"kubernetes.io/projected/835b007c-0c9a-4c8f-94dd-68f8a3b3a137-kube-api-access-vdvtq\") pod \"node-resolver-9522z\" (UID: \"835b007c-0c9a-4c8f-94dd-68f8a3b3a137\") " pod="openshift-dns/node-resolver-9522z"
Sep 16 02:13:02.326541 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.326455    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-2kqqm"
Sep 16 02:13:02.335369 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.335343    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-bhd9l"
Sep 16 02:13:02.347381 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.347347    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-ncht9"
Sep 16 02:13:02.354138 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.354116    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-9g4cg"
Sep 16 02:13:02.363807 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.363789    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-85zxn"
Sep 16 02:13:02.369269 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.369248    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-9522z"
Sep 16 02:13:02.376885 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.376866    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-kwdns"
Sep 16 02:13:02.385496 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.385463    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-tbhqp"
Sep 16 02:13:02.391066 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.391048    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-88xqz"
Sep 16 02:13:02.646239 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.646157    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3472392-891d-49a1-83a7-57f1f99f4b79-metrics-certs\") pod \"network-metrics-daemon-6lwr9\" (UID: \"e3472392-891d-49a1-83a7-57f1f99f4b79\") " pod="openshift-multus/network-metrics-daemon-6lwr9"
Sep 16 02:13:02.646402 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:02.646252    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 02:13:02.646402 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:02.646298    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3472392-891d-49a1-83a7-57f1f99f4b79-metrics-certs podName:e3472392-891d-49a1-83a7-57f1f99f4b79 nodeName:}" failed. No retries permitted until 2026-09-16 02:13:03.646285459 +0000 UTC m=+4.029800681 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e3472392-891d-49a1-83a7-57f1f99f4b79-metrics-certs") pod "network-metrics-daemon-6lwr9" (UID: "e3472392-891d-49a1-83a7-57f1f99f4b79") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 02:13:02.746509 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.746464    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wlf8b\" (UniqueName: \"kubernetes.io/projected/28c55959-36b5-4d35-9e1a-f046d6d36f74-kube-api-access-wlf8b\") pod \"network-check-target-7b42r\" (UID: \"28c55959-36b5-4d35-9e1a-f046d6d36f74\") " pod="openshift-network-diagnostics/network-check-target-7b42r"
Sep 16 02:13:02.746690 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:02.746520    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2b6aaa85-d6dd-420d-a818-f56d06ce6467-original-pull-secret\") pod \"global-pull-secret-syncer-cdwh7\" (UID: \"2b6aaa85-d6dd-420d-a818-f56d06ce6467\") " pod="kube-system/global-pull-secret-syncer-cdwh7"
Sep 16 02:13:02.746690 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:02.746628    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 02:13:02.746690 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:02.746677    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b6aaa85-d6dd-420d-a818-f56d06ce6467-original-pull-secret podName:2b6aaa85-d6dd-420d-a818-f56d06ce6467 nodeName:}" failed. No retries permitted until 2026-09-16 02:13:03.746664011 +0000 UTC m=+4.130179248 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2b6aaa85-d6dd-420d-a818-f56d06ce6467-original-pull-secret") pod "global-pull-secret-syncer-cdwh7" (UID: "2b6aaa85-d6dd-420d-a818-f56d06ce6467") : object "kube-system"/"original-pull-secret" not registered
Sep 16 02:13:02.746852 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:02.746630    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 02:13:02.746852 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:02.746728    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 02:13:02.746852 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:02.746745    2579 projected.go:196] Error preparing data for projected volume kube-api-access-wlf8b for pod openshift-network-diagnostics/network-check-target-7b42r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 02:13:02.746852 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:02.746818    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/28c55959-36b5-4d35-9e1a-f046d6d36f74-kube-api-access-wlf8b podName:28c55959-36b5-4d35-9e1a-f046d6d36f74 nodeName:}" failed. No retries permitted until 2026-09-16 02:13:03.74679086 +0000 UTC m=+4.130306088 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-wlf8b" (UniqueName: "kubernetes.io/projected/28c55959-36b5-4d35-9e1a-f046d6d36f74-kube-api-access-wlf8b") pod "network-check-target-7b42r" (UID: "28c55959-36b5-4d35-9e1a-f046d6d36f74") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 02:13:03.151899 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:03.151859    2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-15 02:08:02 +0000 UTC" deadline="2028-05-08 11:35:50.771047592 +0000 UTC"
Sep 16 02:13:03.151899 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:03.151897    2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14409h22m47.619154593s"
Sep 16 02:13:03.375661 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:13:03.375630    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34a0d09d_a96b_4242_9d17_61fc3b8d88bd.slice/crio-2782f875bbcc570c504fd3c3c9a4c6a7a287dab22c5e6b18d671cb7381c847b8 WatchSource:0}: Error finding container 2782f875bbcc570c504fd3c3c9a4c6a7a287dab22c5e6b18d671cb7381c847b8: Status 404 returned error can't find the container with id 2782f875bbcc570c504fd3c3c9a4c6a7a287dab22c5e6b18d671cb7381c847b8
Sep 16 02:13:03.401200 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:13:03.401173    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc90ef77_c833_4991_9fe8_656fdf2f7457.slice/crio-379225b0b1fb35111207924eabedacbbba29175aeab95779f56cfeed42182c53 WatchSource:0}: Error finding container 379225b0b1fb35111207924eabedacbbba29175aeab95779f56cfeed42182c53: Status 404 returned error can't find the container with id 379225b0b1fb35111207924eabedacbbba29175aeab95779f56cfeed42182c53
Sep 16 02:13:03.401766 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:13:03.401745    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod496c15b3_4d67_473f_b894_2e66b47df562.slice/crio-00ac7861125c10cd2772297b0e75d9c8110d151700a9fcf17fafeb3f35aeee55 WatchSource:0}: Error finding container 00ac7861125c10cd2772297b0e75d9c8110d151700a9fcf17fafeb3f35aeee55: Status 404 returned error can't find the container with id 00ac7861125c10cd2772297b0e75d9c8110d151700a9fcf17fafeb3f35aeee55
Sep 16 02:13:03.403233 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:13:03.403173    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc27d76ec_6b7e_46e9_b66b_d61c0f3ae3f6.slice/crio-0b20be59aa1f6b7143e881c21a4bddb4da5adf5e6e7a1dd519686c25ded67e92 WatchSource:0}: Error finding container 0b20be59aa1f6b7143e881c21a4bddb4da5adf5e6e7a1dd519686c25ded67e92: Status 404 returned error can't find the container with id 0b20be59aa1f6b7143e881c21a4bddb4da5adf5e6e7a1dd519686c25ded67e92
Sep 16 02:13:03.403991 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:13:03.403970    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae9f48ae_4a32_4f07_8ea3_ce26faa19349.slice/crio-1a26890439de8723cdd739ec37e3d3d91d4f6e2f4afbc0e9f9d27748d4df67d4 WatchSource:0}: Error finding container 1a26890439de8723cdd739ec37e3d3d91d4f6e2f4afbc0e9f9d27748d4df67d4: Status 404 returned error can't find the container with id 1a26890439de8723cdd739ec37e3d3d91d4f6e2f4afbc0e9f9d27748d4df67d4
Sep 16 02:13:03.404774 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:13:03.404736    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde5aef44_71c6_4440_afe1_7973e763c35d.slice/crio-86df6423af5c88fee96830b4d73f996537a9f8280c2d8c80ee0fee45a3112da1 WatchSource:0}: Error finding container 86df6423af5c88fee96830b4d73f996537a9f8280c2d8c80ee0fee45a3112da1: Status 404 returned error can't find the container with id 86df6423af5c88fee96830b4d73f996537a9f8280c2d8c80ee0fee45a3112da1
Sep 16 02:13:03.405637 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:13:03.405618    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b370e61_b8a4_4423_8673_4b604907e2f7.slice/crio-b7901142ee0261bc57ccd38f2c4a16c13d7cdf450ca886998c05f081c2658e89 WatchSource:0}: Error finding container b7901142ee0261bc57ccd38f2c4a16c13d7cdf450ca886998c05f081c2658e89: Status 404 returned error can't find the container with id b7901142ee0261bc57ccd38f2c4a16c13d7cdf450ca886998c05f081c2658e89
Sep 16 02:13:03.406373 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:13:03.406298    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98489f25_0768_47e2_b8f0_e2a7c7af2063.slice/crio-b8a792f9f2884e4111fa3280aff0dff28992ffcf89adba863cc91e0b324aa07f WatchSource:0}: Error finding container b8a792f9f2884e4111fa3280aff0dff28992ffcf89adba863cc91e0b324aa07f: Status 404 returned error can't find the container with id b8a792f9f2884e4111fa3280aff0dff28992ffcf89adba863cc91e0b324aa07f
Sep 16 02:13:03.410780 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:13:03.410712    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod835b007c_0c9a_4c8f_94dd_68f8a3b3a137.slice/crio-3fac4ab460dbed8bbf21559cc683e291189f8e65482b4033457a20d9322d159e WatchSource:0}: Error finding container 3fac4ab460dbed8bbf21559cc683e291189f8e65482b4033457a20d9322d159e: Status 404 returned error can't find the container with id 3fac4ab460dbed8bbf21559cc683e291189f8e65482b4033457a20d9322d159e
Sep 16 02:13:03.653286 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:03.653239    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3472392-891d-49a1-83a7-57f1f99f4b79-metrics-certs\") pod \"network-metrics-daemon-6lwr9\" (UID: \"e3472392-891d-49a1-83a7-57f1f99f4b79\") " pod="openshift-multus/network-metrics-daemon-6lwr9"
Sep 16 02:13:03.653418 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:03.653399    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 02:13:03.653505 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:03.653493    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3472392-891d-49a1-83a7-57f1f99f4b79-metrics-certs podName:e3472392-891d-49a1-83a7-57f1f99f4b79 nodeName:}" failed. No retries permitted until 2026-09-16 02:13:05.653456804 +0000 UTC m=+6.036972045 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e3472392-891d-49a1-83a7-57f1f99f4b79-metrics-certs") pod "network-metrics-daemon-6lwr9" (UID: "e3472392-891d-49a1-83a7-57f1f99f4b79") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 02:13:03.753696 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:03.753666    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wlf8b\" (UniqueName: \"kubernetes.io/projected/28c55959-36b5-4d35-9e1a-f046d6d36f74-kube-api-access-wlf8b\") pod \"network-check-target-7b42r\" (UID: \"28c55959-36b5-4d35-9e1a-f046d6d36f74\") " pod="openshift-network-diagnostics/network-check-target-7b42r"
Sep 16 02:13:03.753820 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:03.753702    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2b6aaa85-d6dd-420d-a818-f56d06ce6467-original-pull-secret\") pod \"global-pull-secret-syncer-cdwh7\" (UID: \"2b6aaa85-d6dd-420d-a818-f56d06ce6467\") " pod="kube-system/global-pull-secret-syncer-cdwh7"
Sep 16 02:13:03.753820 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:03.753816    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 02:13:03.753984 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:03.753843    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 02:13:03.753984 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:03.753864    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b6aaa85-d6dd-420d-a818-f56d06ce6467-original-pull-secret podName:2b6aaa85-d6dd-420d-a818-f56d06ce6467 nodeName:}" failed. No retries permitted until 2026-09-16 02:13:05.753852246 +0000 UTC m=+6.137367469 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2b6aaa85-d6dd-420d-a818-f56d06ce6467-original-pull-secret") pod "global-pull-secret-syncer-cdwh7" (UID: "2b6aaa85-d6dd-420d-a818-f56d06ce6467") : object "kube-system"/"original-pull-secret" not registered
Sep 16 02:13:03.753984 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:03.753866    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 02:13:03.753984 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:03.753881    2579 projected.go:196] Error preparing data for projected volume kube-api-access-wlf8b for pod openshift-network-diagnostics/network-check-target-7b42r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 02:13:03.753984 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:03.753940    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/28c55959-36b5-4d35-9e1a-f046d6d36f74-kube-api-access-wlf8b podName:28c55959-36b5-4d35-9e1a-f046d6d36f74 nodeName:}" failed. No retries permitted until 2026-09-16 02:13:05.753923812 +0000 UTC m=+6.137439052 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-wlf8b" (UniqueName: "kubernetes.io/projected/28c55959-36b5-4d35-9e1a-f046d6d36f74-kube-api-access-wlf8b") pod "network-check-target-7b42r" (UID: "28c55959-36b5-4d35-9e1a-f046d6d36f74") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 02:13:04.099110 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:04.099054    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7b42r"
Sep 16 02:13:04.099312 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:04.099173    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7b42r" podUID="28c55959-36b5-4d35-9e1a-f046d6d36f74"
Sep 16 02:13:04.099613 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:04.099594    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cdwh7"
Sep 16 02:13:04.099715 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:04.099696    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cdwh7" podUID="2b6aaa85-d6dd-420d-a818-f56d06ce6467"
Sep 16 02:13:04.099796 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:04.099782    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6lwr9"
Sep 16 02:13:04.099889 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:04.099872    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6lwr9" podUID="e3472392-891d-49a1-83a7-57f1f99f4b79"
Sep 16 02:13:04.127317 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:04.126038    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-118.ec2.internal" event={"ID":"126583346e4bbf13f4d978c46ad24fd3","Type":"ContainerStarted","Data":"070ba35fd34caf8e3048ab61db66f0921a050e5f6af32dc7041a264f4153df2d"}
Sep 16 02:13:04.141494 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:04.131067    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-bhd9l" event={"ID":"98489f25-0768-47e2-b8f0-e2a7c7af2063","Type":"ContainerStarted","Data":"b8a792f9f2884e4111fa3280aff0dff28992ffcf89adba863cc91e0b324aa07f"}
Sep 16 02:13:04.141494 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:04.139381    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-88xqz" event={"ID":"de5aef44-71c6-4440-afe1-7973e763c35d","Type":"ContainerStarted","Data":"86df6423af5c88fee96830b4d73f996537a9f8280c2d8c80ee0fee45a3112da1"}
Sep 16 02:13:04.143685 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:04.143432    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-tbhqp" event={"ID":"c27d76ec-6b7e-46e9-b66b-d61c0f3ae3f6","Type":"ContainerStarted","Data":"0b20be59aa1f6b7143e881c21a4bddb4da5adf5e6e7a1dd519686c25ded67e92"}
Sep 16 02:13:04.145812 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:04.145759    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-ncht9" event={"ID":"bc90ef77-c833-4991-9fe8-656fdf2f7457","Type":"ContainerStarted","Data":"379225b0b1fb35111207924eabedacbbba29175aeab95779f56cfeed42182c53"}
Sep 16 02:13:04.149246 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:04.149218    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9g4cg" event={"ID":"34a0d09d-a96b-4242-9d17-61fc3b8d88bd","Type":"ContainerStarted","Data":"2782f875bbcc570c504fd3c3c9a4c6a7a287dab22c5e6b18d671cb7381c847b8"}
Sep 16 02:13:04.152566 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:04.152195    2579 generic.go:356] "Generic (PLEG): container finished" podID="196354e2e4c46cf552a666282e669d68" containerID="fc18fccfa9dd3b5506eeee19e5301b72128baf2e00cc13dcfec624e8c77f10e0" exitCode=0
Sep 16 02:13:04.152566 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:04.152275    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-118.ec2.internal" event={"ID":"196354e2e4c46cf552a666282e669d68","Type":"ContainerDied","Data":"fc18fccfa9dd3b5506eeee19e5301b72128baf2e00cc13dcfec624e8c77f10e0"}
Sep 16 02:13:04.152566 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:04.152349    2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-15 02:08:02 +0000 UTC" deadline="2028-06-06 09:41:55.176294276 +0000 UTC"
Sep 16 02:13:04.152566 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:04.152386    2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15103h28m51.023912195s"
Sep 16 02:13:04.158054 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:04.157814    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-9522z" event={"ID":"835b007c-0c9a-4c8f-94dd-68f8a3b3a137","Type":"ContainerStarted","Data":"3fac4ab460dbed8bbf21559cc683e291189f8e65482b4033457a20d9322d159e"}
Sep 16 02:13:04.162853 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:04.162825    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-2kqqm" event={"ID":"4b370e61-b8a4-4423-8673-4b604907e2f7","Type":"ContainerStarted","Data":"b7901142ee0261bc57ccd38f2c4a16c13d7cdf450ca886998c05f081c2658e89"}
Sep 16 02:13:04.166990 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:04.166944    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-118.ec2.internal" podStartSLOduration=3.166927646 podStartE2EDuration="3.166927646s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 02:13:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 02:13:04.141069191 +0000 UTC m=+4.524584436" watchObservedRunningTime="2026-09-16 02:13:04.166927646 +0000 UTC m=+4.550442893"
Sep 16 02:13:04.171203 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:04.171144    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-85zxn" event={"ID":"ae9f48ae-4a32-4f07-8ea3-ce26faa19349","Type":"ContainerStarted","Data":"1a26890439de8723cdd739ec37e3d3d91d4f6e2f4afbc0e9f9d27748d4df67d4"}
Sep 16 02:13:04.177232 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:04.177208    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-kwdns" event={"ID":"496c15b3-4d67-473f-b894-2e66b47df562","Type":"ContainerStarted","Data":"00ac7861125c10cd2772297b0e75d9c8110d151700a9fcf17fafeb3f35aeee55"}
Sep 16 02:13:05.191578 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:05.191530    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-118.ec2.internal" event={"ID":"196354e2e4c46cf552a666282e669d68","Type":"ContainerStarted","Data":"c32a96d603829a5c59ae3d89fd2b7246f7cd3c92b5214eafc0e440f4979eef8f"}
Sep 16 02:13:05.671700 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:05.671082    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3472392-891d-49a1-83a7-57f1f99f4b79-metrics-certs\") pod \"network-metrics-daemon-6lwr9\" (UID: \"e3472392-891d-49a1-83a7-57f1f99f4b79\") " pod="openshift-multus/network-metrics-daemon-6lwr9"
Sep 16 02:13:05.671700 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:05.671222    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 02:13:05.671700 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:05.671294    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3472392-891d-49a1-83a7-57f1f99f4b79-metrics-certs podName:e3472392-891d-49a1-83a7-57f1f99f4b79 nodeName:}" failed. No retries permitted until 2026-09-16 02:13:09.67127698 +0000 UTC m=+10.054792209 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e3472392-891d-49a1-83a7-57f1f99f4b79-metrics-certs") pod "network-metrics-daemon-6lwr9" (UID: "e3472392-891d-49a1-83a7-57f1f99f4b79") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 02:13:05.773132 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:05.772195    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wlf8b\" (UniqueName: \"kubernetes.io/projected/28c55959-36b5-4d35-9e1a-f046d6d36f74-kube-api-access-wlf8b\") pod \"network-check-target-7b42r\" (UID: \"28c55959-36b5-4d35-9e1a-f046d6d36f74\") " pod="openshift-network-diagnostics/network-check-target-7b42r"
Sep 16 02:13:05.773132 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:05.772246    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2b6aaa85-d6dd-420d-a818-f56d06ce6467-original-pull-secret\") pod \"global-pull-secret-syncer-cdwh7\" (UID: \"2b6aaa85-d6dd-420d-a818-f56d06ce6467\") " pod="kube-system/global-pull-secret-syncer-cdwh7"
Sep 16 02:13:05.773132 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:05.772530    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 02:13:05.773132 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:05.772552    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 02:13:05.773132 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:05.772565    2579 projected.go:196] Error preparing data for projected volume kube-api-access-wlf8b for pod openshift-network-diagnostics/network-check-target-7b42r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 02:13:05.773132 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:05.772623    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/28c55959-36b5-4d35-9e1a-f046d6d36f74-kube-api-access-wlf8b podName:28c55959-36b5-4d35-9e1a-f046d6d36f74 nodeName:}" failed. No retries permitted until 2026-09-16 02:13:09.772604499 +0000 UTC m=+10.156119725 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-wlf8b" (UniqueName: "kubernetes.io/projected/28c55959-36b5-4d35-9e1a-f046d6d36f74-kube-api-access-wlf8b") pod "network-check-target-7b42r" (UID: "28c55959-36b5-4d35-9e1a-f046d6d36f74") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 02:13:05.773132 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:05.773003    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 02:13:05.773132 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:05.773049    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b6aaa85-d6dd-420d-a818-f56d06ce6467-original-pull-secret podName:2b6aaa85-d6dd-420d-a818-f56d06ce6467 nodeName:}" failed. No retries permitted until 2026-09-16 02:13:09.773035438 +0000 UTC m=+10.156550668 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2b6aaa85-d6dd-420d-a818-f56d06ce6467-original-pull-secret") pod "global-pull-secret-syncer-cdwh7" (UID: "2b6aaa85-d6dd-420d-a818-f56d06ce6467") : object "kube-system"/"original-pull-secret" not registered
Sep 16 02:13:06.099459 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:06.098639    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cdwh7"
Sep 16 02:13:06.099459 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:06.098760    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cdwh7" podUID="2b6aaa85-d6dd-420d-a818-f56d06ce6467"
Sep 16 02:13:06.099459 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:06.099146    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6lwr9"
Sep 16 02:13:06.099459 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:06.099257    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6lwr9" podUID="e3472392-891d-49a1-83a7-57f1f99f4b79"
Sep 16 02:13:06.099459 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:06.099323    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7b42r"
Sep 16 02:13:06.099459 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:06.099389    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7b42r" podUID="28c55959-36b5-4d35-9e1a-f046d6d36f74"
Sep 16 02:13:08.097079 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:08.096564    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cdwh7"
Sep 16 02:13:08.097079 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:08.096613    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7b42r"
Sep 16 02:13:08.097079 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:08.096743    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cdwh7" podUID="2b6aaa85-d6dd-420d-a818-f56d06ce6467"
Sep 16 02:13:08.097079 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:08.096801    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7b42r" podUID="28c55959-36b5-4d35-9e1a-f046d6d36f74"
Sep 16 02:13:08.097079 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:08.096856    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6lwr9"
Sep 16 02:13:08.097079 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:08.096931    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6lwr9" podUID="e3472392-891d-49a1-83a7-57f1f99f4b79"
Sep 16 02:13:09.704415 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:09.704374    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3472392-891d-49a1-83a7-57f1f99f4b79-metrics-certs\") pod \"network-metrics-daemon-6lwr9\" (UID: \"e3472392-891d-49a1-83a7-57f1f99f4b79\") " pod="openshift-multus/network-metrics-daemon-6lwr9"
Sep 16 02:13:09.704878 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:09.704571    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 02:13:09.704878 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:09.704641    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3472392-891d-49a1-83a7-57f1f99f4b79-metrics-certs podName:e3472392-891d-49a1-83a7-57f1f99f4b79 nodeName:}" failed. No retries permitted until 2026-09-16 02:13:17.704621024 +0000 UTC m=+18.088136263 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e3472392-891d-49a1-83a7-57f1f99f4b79-metrics-certs") pod "network-metrics-daemon-6lwr9" (UID: "e3472392-891d-49a1-83a7-57f1f99f4b79") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 02:13:09.805141 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:09.805103    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wlf8b\" (UniqueName: \"kubernetes.io/projected/28c55959-36b5-4d35-9e1a-f046d6d36f74-kube-api-access-wlf8b\") pod \"network-check-target-7b42r\" (UID: \"28c55959-36b5-4d35-9e1a-f046d6d36f74\") " pod="openshift-network-diagnostics/network-check-target-7b42r"
Sep 16 02:13:09.805295 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:09.805153    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2b6aaa85-d6dd-420d-a818-f56d06ce6467-original-pull-secret\") pod \"global-pull-secret-syncer-cdwh7\" (UID: \"2b6aaa85-d6dd-420d-a818-f56d06ce6467\") " pod="kube-system/global-pull-secret-syncer-cdwh7"
Sep 16 02:13:09.805295 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:09.805274    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 02:13:09.805382 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:09.805310    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 02:13:09.805382 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:09.805326    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b6aaa85-d6dd-420d-a818-f56d06ce6467-original-pull-secret podName:2b6aaa85-d6dd-420d-a818-f56d06ce6467 nodeName:}" failed. No retries permitted until 2026-09-16 02:13:17.805313446 +0000 UTC m=+18.188828668 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2b6aaa85-d6dd-420d-a818-f56d06ce6467-original-pull-secret") pod "global-pull-secret-syncer-cdwh7" (UID: "2b6aaa85-d6dd-420d-a818-f56d06ce6467") : object "kube-system"/"original-pull-secret" not registered
Sep 16 02:13:09.805382 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:09.805340    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 02:13:09.805382 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:09.805353    2579 projected.go:196] Error preparing data for projected volume kube-api-access-wlf8b for pod openshift-network-diagnostics/network-check-target-7b42r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 02:13:09.805618 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:09.805403    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/28c55959-36b5-4d35-9e1a-f046d6d36f74-kube-api-access-wlf8b podName:28c55959-36b5-4d35-9e1a-f046d6d36f74 nodeName:}" failed. No retries permitted until 2026-09-16 02:13:17.8053871 +0000 UTC m=+18.188902338 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-wlf8b" (UniqueName: "kubernetes.io/projected/28c55959-36b5-4d35-9e1a-f046d6d36f74-kube-api-access-wlf8b") pod "network-check-target-7b42r" (UID: "28c55959-36b5-4d35-9e1a-f046d6d36f74") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 02:13:10.097238 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:10.097166    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cdwh7"
Sep 16 02:13:10.097380 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:10.097259    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cdwh7" podUID="2b6aaa85-d6dd-420d-a818-f56d06ce6467"
Sep 16 02:13:10.097380 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:10.097361    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6lwr9"
Sep 16 02:13:10.097559 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:10.097493    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6lwr9" podUID="e3472392-891d-49a1-83a7-57f1f99f4b79"
Sep 16 02:13:10.097559 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:10.097538    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7b42r"
Sep 16 02:13:10.097652 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:10.097596    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7b42r" podUID="28c55959-36b5-4d35-9e1a-f046d6d36f74"
Sep 16 02:13:12.098240 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:12.098203    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cdwh7"
Sep 16 02:13:12.098713 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:12.098333    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cdwh7" podUID="2b6aaa85-d6dd-420d-a818-f56d06ce6467"
Sep 16 02:13:12.098713 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:12.098692    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6lwr9"
Sep 16 02:13:12.098819 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:12.098766    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6lwr9" podUID="e3472392-891d-49a1-83a7-57f1f99f4b79"
Sep 16 02:13:12.098819 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:12.098809    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7b42r"
Sep 16 02:13:12.098899 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:12.098876    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7b42r" podUID="28c55959-36b5-4d35-9e1a-f046d6d36f74"
Sep 16 02:13:14.096878 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:14.096841    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7b42r"
Sep 16 02:13:14.097413 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:14.096842    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6lwr9"
Sep 16 02:13:14.097413 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:14.096962    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7b42r" podUID="28c55959-36b5-4d35-9e1a-f046d6d36f74"
Sep 16 02:13:14.097413 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:14.096842    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cdwh7"
Sep 16 02:13:14.097413 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:14.097034    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6lwr9" podUID="e3472392-891d-49a1-83a7-57f1f99f4b79"
Sep 16 02:13:14.097413 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:14.097095    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cdwh7" podUID="2b6aaa85-d6dd-420d-a818-f56d06ce6467"
Sep 16 02:13:16.096082 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:16.096048    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7b42r"
Sep 16 02:13:16.096082 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:16.096082    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6lwr9"
Sep 16 02:13:16.096607 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:16.096168    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7b42r" podUID="28c55959-36b5-4d35-9e1a-f046d6d36f74"
Sep 16 02:13:16.096607 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:16.096050    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cdwh7"
Sep 16 02:13:16.096607 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:16.096290    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6lwr9" podUID="e3472392-891d-49a1-83a7-57f1f99f4b79"
Sep 16 02:13:16.096607 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:16.096383    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cdwh7" podUID="2b6aaa85-d6dd-420d-a818-f56d06ce6467"
Sep 16 02:13:17.766772 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:17.766736    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3472392-891d-49a1-83a7-57f1f99f4b79-metrics-certs\") pod \"network-metrics-daemon-6lwr9\" (UID: \"e3472392-891d-49a1-83a7-57f1f99f4b79\") " pod="openshift-multus/network-metrics-daemon-6lwr9"
Sep 16 02:13:17.767268 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:17.766869    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 02:13:17.767268 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:17.766936    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3472392-891d-49a1-83a7-57f1f99f4b79-metrics-certs podName:e3472392-891d-49a1-83a7-57f1f99f4b79 nodeName:}" failed. No retries permitted until 2026-09-16 02:13:33.76691526 +0000 UTC m=+34.150430500 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e3472392-891d-49a1-83a7-57f1f99f4b79-metrics-certs") pod "network-metrics-daemon-6lwr9" (UID: "e3472392-891d-49a1-83a7-57f1f99f4b79") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 02:13:17.867362 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:17.867322    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wlf8b\" (UniqueName: \"kubernetes.io/projected/28c55959-36b5-4d35-9e1a-f046d6d36f74-kube-api-access-wlf8b\") pod \"network-check-target-7b42r\" (UID: \"28c55959-36b5-4d35-9e1a-f046d6d36f74\") " pod="openshift-network-diagnostics/network-check-target-7b42r"
Sep 16 02:13:17.867564 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:17.867374    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2b6aaa85-d6dd-420d-a818-f56d06ce6467-original-pull-secret\") pod \"global-pull-secret-syncer-cdwh7\" (UID: \"2b6aaa85-d6dd-420d-a818-f56d06ce6467\") " pod="kube-system/global-pull-secret-syncer-cdwh7"
Sep 16 02:13:17.867564 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:17.867525    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 02:13:17.867564 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:17.867548    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 02:13:17.867564 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:17.867551    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 02:13:17.867564 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:17.867561    2579 projected.go:196] Error preparing data for projected volume kube-api-access-wlf8b for pod openshift-network-diagnostics/network-check-target-7b42r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 02:13:17.867783 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:17.867613    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b6aaa85-d6dd-420d-a818-f56d06ce6467-original-pull-secret podName:2b6aaa85-d6dd-420d-a818-f56d06ce6467 nodeName:}" failed. No retries permitted until 2026-09-16 02:13:33.867595663 +0000 UTC m=+34.251110903 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2b6aaa85-d6dd-420d-a818-f56d06ce6467-original-pull-secret") pod "global-pull-secret-syncer-cdwh7" (UID: "2b6aaa85-d6dd-420d-a818-f56d06ce6467") : object "kube-system"/"original-pull-secret" not registered
Sep 16 02:13:17.867783 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:17.867633    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/28c55959-36b5-4d35-9e1a-f046d6d36f74-kube-api-access-wlf8b podName:28c55959-36b5-4d35-9e1a-f046d6d36f74 nodeName:}" failed. No retries permitted until 2026-09-16 02:13:33.86762245 +0000 UTC m=+34.251137676 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-wlf8b" (UniqueName: "kubernetes.io/projected/28c55959-36b5-4d35-9e1a-f046d6d36f74-kube-api-access-wlf8b") pod "network-check-target-7b42r" (UID: "28c55959-36b5-4d35-9e1a-f046d6d36f74") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 02:13:18.096743 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:18.096710    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6lwr9"
Sep 16 02:13:18.096743 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:18.096740    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7b42r"
Sep 16 02:13:18.096944 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:18.096853    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6lwr9" podUID="e3472392-891d-49a1-83a7-57f1f99f4b79"
Sep 16 02:13:18.097217 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:18.097174    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7b42r" podUID="28c55959-36b5-4d35-9e1a-f046d6d36f74"
Sep 16 02:13:18.097217 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:18.097198    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cdwh7"
Sep 16 02:13:18.097344 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:18.097286    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cdwh7" podUID="2b6aaa85-d6dd-420d-a818-f56d06ce6467"
Sep 16 02:13:20.097124 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:20.097089    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cdwh7"
Sep 16 02:13:20.097630 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:20.097191    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6lwr9"
Sep 16 02:13:20.097630 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:20.097210    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cdwh7" podUID="2b6aaa85-d6dd-420d-a818-f56d06ce6467"
Sep 16 02:13:20.097630 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:20.097313    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6lwr9" podUID="e3472392-891d-49a1-83a7-57f1f99f4b79"
Sep 16 02:13:20.097630 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:20.097345    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7b42r"
Sep 16 02:13:20.097630 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:20.097415    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7b42r" podUID="28c55959-36b5-4d35-9e1a-f046d6d36f74"
Sep 16 02:13:22.096887 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:22.096684    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6lwr9"
Sep 16 02:13:22.097730 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:22.096996    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6lwr9" podUID="e3472392-891d-49a1-83a7-57f1f99f4b79"
Sep 16 02:13:22.097730 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:22.096723    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cdwh7"
Sep 16 02:13:22.097730 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:22.097079    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cdwh7" podUID="2b6aaa85-d6dd-420d-a818-f56d06ce6467"
Sep 16 02:13:22.097730 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:22.096697    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7b42r"
Sep 16 02:13:22.097730 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:22.097138    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7b42r" podUID="28c55959-36b5-4d35-9e1a-f046d6d36f74"
Sep 16 02:13:22.239085 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:22.239040    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-bhd9l" event={"ID":"98489f25-0768-47e2-b8f0-e2a7c7af2063","Type":"ContainerStarted","Data":"a379a002b68c415b3fdfc0031a6913571021e76c3f2849a4aa0a6f69960dc75b"}
Sep 16 02:13:22.241934 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:22.241906    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-88xqz" event={"ID":"de5aef44-71c6-4440-afe1-7973e763c35d","Type":"ContainerStarted","Data":"3547fd136582c6646740bdc9a9ce94b3381e6c3f4fabfbcfec9e62cc47c9153e"}
Sep 16 02:13:22.242049 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:22.241970    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-88xqz" event={"ID":"de5aef44-71c6-4440-afe1-7973e763c35d","Type":"ContainerStarted","Data":"bf64c6bc7eaf2a10e0a4746ed983a384e54f47d742549d4b77ff465b95ea3dcf"}
Sep 16 02:13:22.242049 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:22.242004    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-88xqz" event={"ID":"de5aef44-71c6-4440-afe1-7973e763c35d","Type":"ContainerStarted","Data":"65cc254017094a9cdeab259151188c054f94b69e6d6f1d782726e383b03f3d4c"}
Sep 16 02:13:22.242126 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:22.242048    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-88xqz" event={"ID":"de5aef44-71c6-4440-afe1-7973e763c35d","Type":"ContainerStarted","Data":"0cad80bb6ef911ba387bea0d80464646559873f27796b109f084d2c2d2ae1494"}
Sep 16 02:13:22.242126 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:22.242094    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-88xqz" event={"ID":"de5aef44-71c6-4440-afe1-7973e763c35d","Type":"ContainerStarted","Data":"0cd97e251b3a75ddfc59bb9c997b9a4223b2391622267535ec7652d2973b7f9d"}
Sep 16 02:13:22.242126 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:22.242124    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-88xqz" event={"ID":"de5aef44-71c6-4440-afe1-7973e763c35d","Type":"ContainerStarted","Data":"e0f79fe3b07402b039b12b49b3675e2bea356979fec932631aeb47e533f500f9"}
Sep 16 02:13:22.243401 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:22.243379    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-ncht9" event={"ID":"bc90ef77-c833-4991-9fe8-656fdf2f7457","Type":"ContainerStarted","Data":"4112ff258c17d1e61892dac567397b40ba0ec29102be4f9baa5d4200d9f15ae9"}
Sep 16 02:13:22.244779 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:22.244758    2579 generic.go:356] "Generic (PLEG): container finished" podID="34a0d09d-a96b-4242-9d17-61fc3b8d88bd" containerID="f940db3943f867d9b9cd67c7b1db4ad79bc1ee4f2d44608b2e09523b25c2d3dc" exitCode=0
Sep 16 02:13:22.244889 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:22.244820    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9g4cg" event={"ID":"34a0d09d-a96b-4242-9d17-61fc3b8d88bd","Type":"ContainerDied","Data":"f940db3943f867d9b9cd67c7b1db4ad79bc1ee4f2d44608b2e09523b25c2d3dc"}
Sep 16 02:13:22.246254 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:22.246229    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-9522z" event={"ID":"835b007c-0c9a-4c8f-94dd-68f8a3b3a137","Type":"ContainerStarted","Data":"e274d4d765f5488506126376851ac352bfd48f1e2a96dceee01e980cd5023505"}
Sep 16 02:13:22.247823 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:22.247765    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-2kqqm" event={"ID":"4b370e61-b8a4-4423-8673-4b604907e2f7","Type":"ContainerStarted","Data":"879ac5071df8fdd12f1ead2446730268de8c4574cf6c285478231ac026ee8a21"}
Sep 16 02:13:22.249353 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:22.249336    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-85zxn" event={"ID":"ae9f48ae-4a32-4f07-8ea3-ce26faa19349","Type":"ContainerStarted","Data":"d6f874b428d35230b2956b36bb6bcd9651cc55bcc04ae861c553193e67716ea0"}
Sep 16 02:13:22.250707 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:22.250686    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-kwdns" event={"ID":"496c15b3-4d67-473f-b894-2e66b47df562","Type":"ContainerStarted","Data":"6d583a94c068fe261ed8179113a35a78c94a561ff9ad399f6e7d0a83deda444c"}
Sep 16 02:13:22.252301 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:22.252264    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-118.ec2.internal" podStartSLOduration=21.25224955 podStartE2EDuration="21.25224955s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 02:13:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 02:13:05.206150459 +0000 UTC m=+5.589665706" watchObservedRunningTime="2026-09-16 02:13:22.25224955 +0000 UTC m=+22.635764797"
Sep 16 02:13:22.252843 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:22.252820    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-bhd9l" podStartSLOduration=4.280486667 podStartE2EDuration="22.252814309s" totalImagesPullingTime="17.972327642s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 02:13:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 02:13:22.252139142 +0000 UTC m=+22.635654397" watchObservedRunningTime="2026-09-16 02:13:22.252814309 +0000 UTC m=+22.636329553"
Sep 16 02:13:22.266612 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:22.266570    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-ncht9" podStartSLOduration=4.228966731 podStartE2EDuration="22.266558957s" totalImagesPullingTime="18.037592226s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 02:13:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 02:13:22.266406865 +0000 UTC m=+22.649922112" watchObservedRunningTime="2026-09-16 02:13:22.266558957 +0000 UTC m=+22.650074202"
Sep 16 02:13:22.297888 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:22.297848    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-9522z" podStartSLOduration=4.328323611 podStartE2EDuration="22.297837631s" totalImagesPullingTime="17.96951402s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 02:13:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 02:13:22.297281986 +0000 UTC m=+22.680797230" watchObservedRunningTime="2026-09-16 02:13:22.297837631 +0000 UTC m=+22.681352873"
Sep 16 02:13:22.311237 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:22.311202    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-2kqqm" podStartSLOduration=4.337430015 podStartE2EDuration="22.311191678s" totalImagesPullingTime="17.973761663s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 02:13:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 02:13:22.31096734 +0000 UTC m=+22.694482586" watchObservedRunningTime="2026-09-16 02:13:22.311191678 +0000 UTC m=+22.694706937"
Sep 16 02:13:22.322385 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:22.322352    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-kwdns" podStartSLOduration=4.343632183 podStartE2EDuration="22.322346404s" totalImagesPullingTime="17.978714221s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 02:13:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 02:13:22.322163591 +0000 UTC m=+22.705678835" watchObservedRunningTime="2026-09-16 02:13:22.322346404 +0000 UTC m=+22.705861661"
Sep 16 02:13:22.900196 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:22.900169    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-bhd9l"
Sep 16 02:13:22.900884 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:22.900865    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-bhd9l"
Sep 16 02:13:22.905551 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:22.905534    2579 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 16 02:13:23.097706 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:23.097615    2579 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-16T02:13:22.905547816Z","UUID":"459c6e45-2940-4b98-b6c1-76958203a03d","Handler":null,"Name":"","Endpoint":""}
Sep 16 02:13:23.099303 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:23.099279    2579 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 16 02:13:23.099424 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:23.099314    2579 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 16 02:13:23.254943 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:23.254912    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-85zxn" event={"ID":"ae9f48ae-4a32-4f07-8ea3-ce26faa19349","Type":"ContainerStarted","Data":"7fccd058bfddf740b6bcaaf1fe55841090c4583a09daa11c697de24c4250d8f4"}
Sep 16 02:13:23.256374 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:23.256340    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-tbhqp" event={"ID":"c27d76ec-6b7e-46e9-b66b-d61c0f3ae3f6","Type":"ContainerStarted","Data":"a6c77af3b5fd22b403a276753e3affc4829776a6fe4d38d78f7c68ef550379fd"}
Sep 16 02:13:23.268676 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:23.268638    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-tbhqp" podStartSLOduration=5.290217387 podStartE2EDuration="23.268624352s" totalImagesPullingTime="17.978406965s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 02:13:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 02:13:23.268146411 +0000 UTC m=+23.651661656" watchObservedRunningTime="2026-09-16 02:13:23.268624352 +0000 UTC m=+23.652139597"
Sep 16 02:13:24.098081 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:24.096409    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6lwr9"
Sep 16 02:13:24.098081 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:24.096447    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7b42r"
Sep 16 02:13:24.098081 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:24.096578    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6lwr9" podUID="e3472392-891d-49a1-83a7-57f1f99f4b79"
Sep 16 02:13:24.098081 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:24.096654    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7b42r" podUID="28c55959-36b5-4d35-9e1a-f046d6d36f74"
Sep 16 02:13:24.098081 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:24.096702    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cdwh7"
Sep 16 02:13:24.098081 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:24.096816    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cdwh7" podUID="2b6aaa85-d6dd-420d-a818-f56d06ce6467"
Sep 16 02:13:24.262797 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:24.262760    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-88xqz" event={"ID":"de5aef44-71c6-4440-afe1-7973e763c35d","Type":"ContainerStarted","Data":"a1f24aa455c92ab0688fca652faf797f65d513879b50f77c38e9fa754a2ac411"}
Sep 16 02:13:24.265336 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:24.265269    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-85zxn" event={"ID":"ae9f48ae-4a32-4f07-8ea3-ce26faa19349","Type":"ContainerStarted","Data":"ac8d49b136f84308f3836e05d8e454ac0a048325d6b4f8104e0f2be203edb9ca"}
Sep 16 02:13:24.265336 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:24.265316    2579 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 16 02:13:24.282503 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:24.282443    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-85zxn" podStartSLOduration=4.176621631 podStartE2EDuration="24.282432121s" totalImagesPullingTime="20.10581049s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 02:13:00 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 02:13:24.281991785 +0000 UTC m=+24.665507031" watchObservedRunningTime="2026-09-16 02:13:24.282432121 +0000 UTC m=+24.665947395"
Sep 16 02:13:26.096768 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:26.096736    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6lwr9"
Sep 16 02:13:26.096768 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:26.096756    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cdwh7"
Sep 16 02:13:26.097277 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:26.096736    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7b42r"
Sep 16 02:13:26.097277 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:26.096869    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6lwr9" podUID="e3472392-891d-49a1-83a7-57f1f99f4b79"
Sep 16 02:13:26.097277 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:26.096925    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7b42r" podUID="28c55959-36b5-4d35-9e1a-f046d6d36f74"
Sep 16 02:13:26.097277 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:26.096981    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cdwh7" podUID="2b6aaa85-d6dd-420d-a818-f56d06ce6467"
Sep 16 02:13:28.097208 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:28.096986    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cdwh7"
Sep 16 02:13:28.097947 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:28.097019    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6lwr9"
Sep 16 02:13:28.097947 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:28.097289    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cdwh7" podUID="2b6aaa85-d6dd-420d-a818-f56d06ce6467"
Sep 16 02:13:28.097947 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:28.097348    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6lwr9" podUID="e3472392-891d-49a1-83a7-57f1f99f4b79"
Sep 16 02:13:28.097947 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:28.097031    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7b42r"
Sep 16 02:13:28.097947 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:28.097454    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7b42r" podUID="28c55959-36b5-4d35-9e1a-f046d6d36f74"
Sep 16 02:13:28.277337 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:28.277294    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-88xqz" event={"ID":"de5aef44-71c6-4440-afe1-7973e763c35d","Type":"ContainerStarted","Data":"37fe827b0585f6c46072da955ab81c96896a2d91c7117b45586b315dc76ae68c"}
Sep 16 02:13:28.277617 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:28.277562    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-88xqz"
Sep 16 02:13:28.279250 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:28.279225    2579 generic.go:356] "Generic (PLEG): container finished" podID="34a0d09d-a96b-4242-9d17-61fc3b8d88bd" containerID="97b71de893292a966d476c7e0b2d3c0cf469ecb3a0d00781933d8a76517674ba" exitCode=0
Sep 16 02:13:28.279435 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:28.279259    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9g4cg" event={"ID":"34a0d09d-a96b-4242-9d17-61fc3b8d88bd","Type":"ContainerDied","Data":"97b71de893292a966d476c7e0b2d3c0cf469ecb3a0d00781933d8a76517674ba"}
Sep 16 02:13:28.293412 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:28.293391    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-88xqz"
Sep 16 02:13:28.301579 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:28.301533    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-88xqz" podStartSLOduration=10.262342285999999 podStartE2EDuration="28.301520337s" totalImagesPullingTime="18.039178051s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 02:13:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 02:13:28.300718565 +0000 UTC m=+28.684233811" watchObservedRunningTime="2026-09-16 02:13:28.301520337 +0000 UTC m=+28.685035583"
Sep 16 02:13:28.522922 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:28.522901    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-bhd9l"
Sep 16 02:13:28.523034 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:28.522995    2579 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 16 02:13:28.523650 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:28.523628    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-bhd9l"
Sep 16 02:13:29.286686 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:29.286659    2579 generic.go:356] "Generic (PLEG): container finished" podID="34a0d09d-a96b-4242-9d17-61fc3b8d88bd" containerID="190de901390466c1ffdc81357913aa35e5a313882a153208acaf6eda016ee1b0" exitCode=0
Sep 16 02:13:29.287161 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:29.286740    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9g4cg" event={"ID":"34a0d09d-a96b-4242-9d17-61fc3b8d88bd","Type":"ContainerDied","Data":"190de901390466c1ffdc81357913aa35e5a313882a153208acaf6eda016ee1b0"}
Sep 16 02:13:29.287548 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:29.287430    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-88xqz"
Sep 16 02:13:29.287548 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:29.287509    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-88xqz"
Sep 16 02:13:29.302133 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:29.301973    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-88xqz"
Sep 16 02:13:29.380045 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:29.379972    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-7b42r"]
Sep 16 02:13:29.380145 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:29.380085    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7b42r"
Sep 16 02:13:29.380202 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:29.380185    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7b42r" podUID="28c55959-36b5-4d35-9e1a-f046d6d36f74"
Sep 16 02:13:29.382484 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:29.382448    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-cdwh7"]
Sep 16 02:13:29.382579 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:29.382544    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cdwh7"
Sep 16 02:13:29.382627 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:29.382612    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cdwh7" podUID="2b6aaa85-d6dd-420d-a818-f56d06ce6467"
Sep 16 02:13:29.393403 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:29.393382    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-6lwr9"]
Sep 16 02:13:29.393507 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:29.393496    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6lwr9"
Sep 16 02:13:29.393618 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:29.393601    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6lwr9" podUID="e3472392-891d-49a1-83a7-57f1f99f4b79"
Sep 16 02:13:30.291762 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:30.291736    2579 generic.go:356] "Generic (PLEG): container finished" podID="34a0d09d-a96b-4242-9d17-61fc3b8d88bd" containerID="4337bfa32e2936a1ef4e211ee73c98e849520b4c153118b7f0a709b30b3fc5e2" exitCode=0
Sep 16 02:13:30.292135 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:30.291825    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9g4cg" event={"ID":"34a0d09d-a96b-4242-9d17-61fc3b8d88bd","Type":"ContainerDied","Data":"4337bfa32e2936a1ef4e211ee73c98e849520b4c153118b7f0a709b30b3fc5e2"}
Sep 16 02:13:31.096059 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:31.096015    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6lwr9"
Sep 16 02:13:31.096059 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:31.096057    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7b42r"
Sep 16 02:13:31.096297 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:31.096057    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cdwh7"
Sep 16 02:13:31.096297 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:31.096142    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6lwr9" podUID="e3472392-891d-49a1-83a7-57f1f99f4b79"
Sep 16 02:13:31.096297 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:31.096243    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7b42r" podUID="28c55959-36b5-4d35-9e1a-f046d6d36f74"
Sep 16 02:13:31.096442 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:31.096339    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cdwh7" podUID="2b6aaa85-d6dd-420d-a818-f56d06ce6467"
Sep 16 02:13:33.096078 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:33.096043    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cdwh7"
Sep 16 02:13:33.096078 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:33.096077    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6lwr9"
Sep 16 02:13:33.096628 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:33.096164    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cdwh7" podUID="2b6aaa85-d6dd-420d-a818-f56d06ce6467"
Sep 16 02:13:33.096628 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:33.096220    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7b42r"
Sep 16 02:13:33.096628 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:33.096305    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6lwr9" podUID="e3472392-891d-49a1-83a7-57f1f99f4b79"
Sep 16 02:13:33.096628 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:33.096381    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7b42r" podUID="28c55959-36b5-4d35-9e1a-f046d6d36f74"
Sep 16 02:13:33.784622 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:33.784576    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3472392-891d-49a1-83a7-57f1f99f4b79-metrics-certs\") pod \"network-metrics-daemon-6lwr9\" (UID: \"e3472392-891d-49a1-83a7-57f1f99f4b79\") " pod="openshift-multus/network-metrics-daemon-6lwr9"
Sep 16 02:13:33.784875 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:33.784738    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 02:13:33.784875 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:33.784803    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3472392-891d-49a1-83a7-57f1f99f4b79-metrics-certs podName:e3472392-891d-49a1-83a7-57f1f99f4b79 nodeName:}" failed. No retries permitted until 2026-09-16 02:14:05.784787956 +0000 UTC m=+66.168303182 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e3472392-891d-49a1-83a7-57f1f99f4b79-metrics-certs") pod "network-metrics-daemon-6lwr9" (UID: "e3472392-891d-49a1-83a7-57f1f99f4b79") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 02:13:33.885359 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:33.885322    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wlf8b\" (UniqueName: \"kubernetes.io/projected/28c55959-36b5-4d35-9e1a-f046d6d36f74-kube-api-access-wlf8b\") pod \"network-check-target-7b42r\" (UID: \"28c55959-36b5-4d35-9e1a-f046d6d36f74\") " pod="openshift-network-diagnostics/network-check-target-7b42r"
Sep 16 02:13:33.885359 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:33.885363    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2b6aaa85-d6dd-420d-a818-f56d06ce6467-original-pull-secret\") pod \"global-pull-secret-syncer-cdwh7\" (UID: \"2b6aaa85-d6dd-420d-a818-f56d06ce6467\") " pod="kube-system/global-pull-secret-syncer-cdwh7"
Sep 16 02:13:33.885631 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:33.885518    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 02:13:33.885631 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:33.885538    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 02:13:33.885631 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:33.885537    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 02:13:33.885631 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:33.885552    2579 projected.go:196] Error preparing data for projected volume kube-api-access-wlf8b for pod openshift-network-diagnostics/network-check-target-7b42r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 02:13:33.885631 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:33.885616    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/28c55959-36b5-4d35-9e1a-f046d6d36f74-kube-api-access-wlf8b podName:28c55959-36b5-4d35-9e1a-f046d6d36f74 nodeName:}" failed. No retries permitted until 2026-09-16 02:14:05.885599102 +0000 UTC m=+66.269114337 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-wlf8b" (UniqueName: "kubernetes.io/projected/28c55959-36b5-4d35-9e1a-f046d6d36f74-kube-api-access-wlf8b") pod "network-check-target-7b42r" (UID: "28c55959-36b5-4d35-9e1a-f046d6d36f74") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 02:13:33.885883 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:33.885638    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b6aaa85-d6dd-420d-a818-f56d06ce6467-original-pull-secret podName:2b6aaa85-d6dd-420d-a818-f56d06ce6467 nodeName:}" failed. No retries permitted until 2026-09-16 02:14:05.885628918 +0000 UTC m=+66.269144140 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2b6aaa85-d6dd-420d-a818-f56d06ce6467-original-pull-secret") pod "global-pull-secret-syncer-cdwh7" (UID: "2b6aaa85-d6dd-420d-a818-f56d06ce6467") : object "kube-system"/"original-pull-secret" not registered
Sep 16 02:13:34.402828 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:34.402795    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-118.ec2.internal" event="NodeReady"
Sep 16 02:13:34.403215 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:34.402909    2579 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 16 02:13:34.434850 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:34.433670    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-5dcbc7df6b-jtz8l"]
Sep 16 02:13:34.447962 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:34.447923    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-649f6698d9-cmzbc"]
Sep 16 02:13:34.448095 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:34.448052    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-jtz8l"
Sep 16 02:13:34.450572 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:34.450545    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Sep 16 02:13:34.450572 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:34.450567    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Sep 16 02:13:34.450747 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:34.450568    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-cfd5b\""
Sep 16 02:13:34.464129 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:34.464109    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-5dcbc7df6b-jtz8l"]
Sep 16 02:13:34.464230 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:34.464144    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-vbpq2"]
Sep 16 02:13:34.464230 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:34.464211    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-649f6698d9-cmzbc"
Sep 16 02:13:34.466677 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:34.466655    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-9j6zg\""
Sep 16 02:13:34.466795 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:34.466660    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 16 02:13:34.466795 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:34.466718    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 16 02:13:34.466795 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:34.466662    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 16 02:13:34.471164 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:34.471144    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 16 02:13:34.484320 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:34.484299    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-649f6698d9-cmzbc"]
Sep 16 02:13:34.484414 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:34.484336    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-q6tmq"]
Sep 16 02:13:34.484455 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:34.484418    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-vbpq2"
Sep 16 02:13:34.486450 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:34.486432    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 16 02:13:34.486568 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:34.486491    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 16 02:13:34.486568 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:34.486541    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-lfqpp\""
Sep 16 02:13:34.486666 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:34.486594    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 16 02:13:34.506529 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:34.506505    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-vbpq2"]
Sep 16 02:13:34.506619 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:34.506549    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-q6tmq"]
Sep 16 02:13:34.506619 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:34.506561    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-q6tmq"
Sep 16 02:13:34.508728 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:34.508699    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 16 02:13:34.508728 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:34.508704    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 16 02:13:34.508859 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:34.508843    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-h6q9g\""
Sep 16 02:13:34.587660 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:34.587626    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:13:34.590725 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:34.590704    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/35bd8e04-3e55-44b6-b78e-83566dcf43cc-bound-sa-token\") pod \"image-registry-649f6698d9-cmzbc\" (UID: \"35bd8e04-3e55-44b6-b78e-83566dcf43cc\") " pod="openshift-image-registry/image-registry-649f6698d9-cmzbc"
Sep 16 02:13:34.590790 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:34.590752    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8e7a4943-f231-4531-9e5d-4bcf2a844cf9-tmp-dir\") pod \"dns-default-q6tmq\" (UID: \"8e7a4943-f231-4531-9e5d-4bcf2a844cf9\") " pod="openshift-dns/dns-default-q6tmq"
Sep 16 02:13:34.590790 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:34.590769    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bwkc4\" (UniqueName: \"kubernetes.io/projected/8e7a4943-f231-4531-9e5d-4bcf2a844cf9-kube-api-access-bwkc4\") pod \"dns-default-q6tmq\" (UID: \"8e7a4943-f231-4531-9e5d-4bcf2a844cf9\") " pod="openshift-dns/dns-default-q6tmq"
Sep 16 02:13:34.590866 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:34.590800    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5caaf390-c372-40a3-8303-e99ab52b762b-cert\") pod \"ingress-canary-vbpq2\" (UID: \"5caaf390-c372-40a3-8303-e99ab52b762b\") " pod="openshift-ingress-canary/ingress-canary-vbpq2"
Sep 16 02:13:34.590866 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:34.590816    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vcl5w\" (UniqueName: \"kubernetes.io/projected/5caaf390-c372-40a3-8303-e99ab52b762b-kube-api-access-vcl5w\") pod \"ingress-canary-vbpq2\" (UID: \"5caaf390-c372-40a3-8303-e99ab52b762b\") " pod="openshift-ingress-canary/ingress-canary-vbpq2"
Sep 16 02:13:34.590866 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:34.590832    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8ff5e4be-2aa6-4203-83ac-e715f2f0f565-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-jtz8l\" (UID: \"8ff5e4be-2aa6-4203-83ac-e715f2f0f565\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-jtz8l"
Sep 16 02:13:34.590866 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:34.590854    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/35bd8e04-3e55-44b6-b78e-83566dcf43cc-image-registry-private-configuration\") pod \"image-registry-649f6698d9-cmzbc\" (UID: \"35bd8e04-3e55-44b6-b78e-83566dcf43cc\") " pod="openshift-image-registry/image-registry-649f6698d9-cmzbc"
Sep 16 02:13:34.591050 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:34.590871    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/8ff5e4be-2aa6-4203-83ac-e715f2f0f565-nginx-conf\") pod \"networking-console-plugin-5dcbc7df6b-jtz8l\" (UID: \"8ff5e4be-2aa6-4203-83ac-e715f2f0f565\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-jtz8l"
Sep 16 02:13:34.591050 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:34.590937    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/35bd8e04-3e55-44b6-b78e-83566dcf43cc-registry-tls\") pod \"image-registry-649f6698d9-cmzbc\" (UID: \"35bd8e04-3e55-44b6-b78e-83566dcf43cc\") " pod="openshift-image-registry/image-registry-649f6698d9-cmzbc"
Sep 16 02:13:34.591050 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:34.590979    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/35bd8e04-3e55-44b6-b78e-83566dcf43cc-installation-pull-secrets\") pod \"image-registry-649f6698d9-cmzbc\" (UID: \"35bd8e04-3e55-44b6-b78e-83566dcf43cc\") " pod="openshift-image-registry/image-registry-649f6698d9-cmzbc"
Sep 16 02:13:34.591050 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:34.591015    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/8e7a4943-f231-4531-9e5d-4bcf2a844cf9-config-volume\") pod \"dns-default-q6tmq\" (UID: \"8e7a4943-f231-4531-9e5d-4bcf2a844cf9\") " pod="openshift-dns/dns-default-q6tmq"
Sep 16 02:13:34.591050 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:34.591040    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/35bd8e04-3e55-44b6-b78e-83566dcf43cc-ca-trust-extracted\") pod \"image-registry-649f6698d9-cmzbc\" (UID: \"35bd8e04-3e55-44b6-b78e-83566dcf43cc\") " pod="openshift-image-registry/image-registry-649f6698d9-cmzbc"
Sep 16 02:13:34.591266 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:34.591065    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/35bd8e04-3e55-44b6-b78e-83566dcf43cc-registry-certificates\") pod \"image-registry-649f6698d9-cmzbc\" (UID: \"35bd8e04-3e55-44b6-b78e-83566dcf43cc\") " pod="openshift-image-registry/image-registry-649f6698d9-cmzbc"
Sep 16 02:13:34.591266 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:34.591089    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/35bd8e04-3e55-44b6-b78e-83566dcf43cc-trusted-ca\") pod \"image-registry-649f6698d9-cmzbc\" (UID: \"35bd8e04-3e55-44b6-b78e-83566dcf43cc\") " pod="openshift-image-registry/image-registry-649f6698d9-cmzbc"
Sep 16 02:13:34.591266 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:34.591119    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-grvhj\" (UniqueName: \"kubernetes.io/projected/35bd8e04-3e55-44b6-b78e-83566dcf43cc-kube-api-access-grvhj\") pod \"image-registry-649f6698d9-cmzbc\" (UID: \"35bd8e04-3e55-44b6-b78e-83566dcf43cc\") " pod="openshift-image-registry/image-registry-649f6698d9-cmzbc"
Sep 16 02:13:34.591266 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:34.591148    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8e7a4943-f231-4531-9e5d-4bcf2a844cf9-metrics-tls\") pod \"dns-default-q6tmq\" (UID: \"8e7a4943-f231-4531-9e5d-4bcf2a844cf9\") " pod="openshift-dns/dns-default-q6tmq"
Sep 16 02:13:34.668095 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:34.668067    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:13:34.692346 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:34.692326    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/8e7a4943-f231-4531-9e5d-4bcf2a844cf9-config-volume\") pod \"dns-default-q6tmq\" (UID: \"8e7a4943-f231-4531-9e5d-4bcf2a844cf9\") " pod="openshift-dns/dns-default-q6tmq"
Sep 16 02:13:34.692448 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:34.692361    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/35bd8e04-3e55-44b6-b78e-83566dcf43cc-ca-trust-extracted\") pod \"image-registry-649f6698d9-cmzbc\" (UID: \"35bd8e04-3e55-44b6-b78e-83566dcf43cc\") " pod="openshift-image-registry/image-registry-649f6698d9-cmzbc"
Sep 16 02:13:34.692448 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:34.692388    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/35bd8e04-3e55-44b6-b78e-83566dcf43cc-registry-certificates\") pod \"image-registry-649f6698d9-cmzbc\" (UID: \"35bd8e04-3e55-44b6-b78e-83566dcf43cc\") " pod="openshift-image-registry/image-registry-649f6698d9-cmzbc"
Sep 16 02:13:34.692583 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:34.692511    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/35bd8e04-3e55-44b6-b78e-83566dcf43cc-trusted-ca\") pod \"image-registry-649f6698d9-cmzbc\" (UID: \"35bd8e04-3e55-44b6-b78e-83566dcf43cc\") " pod="openshift-image-registry/image-registry-649f6698d9-cmzbc"
Sep 16 02:13:34.692583 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:34.692548    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-grvhj\" (UniqueName: \"kubernetes.io/projected/35bd8e04-3e55-44b6-b78e-83566dcf43cc-kube-api-access-grvhj\") pod \"image-registry-649f6698d9-cmzbc\" (UID: \"35bd8e04-3e55-44b6-b78e-83566dcf43cc\") " pod="openshift-image-registry/image-registry-649f6698d9-cmzbc"
Sep 16 02:13:34.692583 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:34.692577    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8e7a4943-f231-4531-9e5d-4bcf2a844cf9-metrics-tls\") pod \"dns-default-q6tmq\" (UID: \"8e7a4943-f231-4531-9e5d-4bcf2a844cf9\") " pod="openshift-dns/dns-default-q6tmq"
Sep 16 02:13:34.692732 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:34.692632    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/35bd8e04-3e55-44b6-b78e-83566dcf43cc-bound-sa-token\") pod \"image-registry-649f6698d9-cmzbc\" (UID: \"35bd8e04-3e55-44b6-b78e-83566dcf43cc\") " pod="openshift-image-registry/image-registry-649f6698d9-cmzbc"
Sep 16 02:13:34.692732 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:34.692678    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8e7a4943-f231-4531-9e5d-4bcf2a844cf9-tmp-dir\") pod \"dns-default-q6tmq\" (UID: \"8e7a4943-f231-4531-9e5d-4bcf2a844cf9\") " pod="openshift-dns/dns-default-q6tmq"
Sep 16 02:13:34.692732 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:34.692704    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bwkc4\" (UniqueName: \"kubernetes.io/projected/8e7a4943-f231-4531-9e5d-4bcf2a844cf9-kube-api-access-bwkc4\") pod \"dns-default-q6tmq\" (UID: \"8e7a4943-f231-4531-9e5d-4bcf2a844cf9\") " pod="openshift-dns/dns-default-q6tmq"
Sep 16 02:13:34.692732 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:34.692711    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 02:13:34.692732 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:34.692725    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/35bd8e04-3e55-44b6-b78e-83566dcf43cc-ca-trust-extracted\") pod \"image-registry-649f6698d9-cmzbc\" (UID: \"35bd8e04-3e55-44b6-b78e-83566dcf43cc\") " pod="openshift-image-registry/image-registry-649f6698d9-cmzbc"
Sep 16 02:13:34.692968 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:34.692740    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5caaf390-c372-40a3-8303-e99ab52b762b-cert\") pod \"ingress-canary-vbpq2\" (UID: \"5caaf390-c372-40a3-8303-e99ab52b762b\") " pod="openshift-ingress-canary/ingress-canary-vbpq2"
Sep 16 02:13:34.692968 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:34.692786    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8e7a4943-f231-4531-9e5d-4bcf2a844cf9-metrics-tls podName:8e7a4943-f231-4531-9e5d-4bcf2a844cf9 nodeName:}" failed. No retries permitted until 2026-09-16 02:13:35.19276598 +0000 UTC m=+35.576281230 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8e7a4943-f231-4531-9e5d-4bcf2a844cf9-metrics-tls") pod "dns-default-q6tmq" (UID: "8e7a4943-f231-4531-9e5d-4bcf2a844cf9") : secret "dns-default-metrics-tls" not found
Sep 16 02:13:34.692968 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:34.692809    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vcl5w\" (UniqueName: \"kubernetes.io/projected/5caaf390-c372-40a3-8303-e99ab52b762b-kube-api-access-vcl5w\") pod \"ingress-canary-vbpq2\" (UID: \"5caaf390-c372-40a3-8303-e99ab52b762b\") " pod="openshift-ingress-canary/ingress-canary-vbpq2"
Sep 16 02:13:34.692968 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:34.692841    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8ff5e4be-2aa6-4203-83ac-e715f2f0f565-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-jtz8l\" (UID: \"8ff5e4be-2aa6-4203-83ac-e715f2f0f565\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-jtz8l"
Sep 16 02:13:34.692968 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:34.692882    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/35bd8e04-3e55-44b6-b78e-83566dcf43cc-image-registry-private-configuration\") pod \"image-registry-649f6698d9-cmzbc\" (UID: \"35bd8e04-3e55-44b6-b78e-83566dcf43cc\") " pod="openshift-image-registry/image-registry-649f6698d9-cmzbc"
Sep 16 02:13:34.692968 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:34.692908    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/8ff5e4be-2aa6-4203-83ac-e715f2f0f565-nginx-conf\") pod \"networking-console-plugin-5dcbc7df6b-jtz8l\" (UID: \"8ff5e4be-2aa6-4203-83ac-e715f2f0f565\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-jtz8l"
Sep 16 02:13:34.692968 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:34.692934    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/35bd8e04-3e55-44b6-b78e-83566dcf43cc-registry-tls\") pod \"image-registry-649f6698d9-cmzbc\" (UID: \"35bd8e04-3e55-44b6-b78e-83566dcf43cc\") " pod="openshift-image-registry/image-registry-649f6698d9-cmzbc"
Sep 16 02:13:34.693382 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:34.692973    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/35bd8e04-3e55-44b6-b78e-83566dcf43cc-installation-pull-secrets\") pod \"image-registry-649f6698d9-cmzbc\" (UID: \"35bd8e04-3e55-44b6-b78e-83566dcf43cc\") " pod="openshift-image-registry/image-registry-649f6698d9-cmzbc"
Sep 16 02:13:34.693382 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:34.693017    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/8e7a4943-f231-4531-9e5d-4bcf2a844cf9-config-volume\") pod \"dns-default-q6tmq\" (UID: \"8e7a4943-f231-4531-9e5d-4bcf2a844cf9\") " pod="openshift-dns/dns-default-q6tmq"
Sep 16 02:13:34.693382 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:34.693016    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8e7a4943-f231-4531-9e5d-4bcf2a844cf9-tmp-dir\") pod \"dns-default-q6tmq\" (UID: \"8e7a4943-f231-4531-9e5d-4bcf2a844cf9\") " pod="openshift-dns/dns-default-q6tmq"
Sep 16 02:13:34.693382 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:34.693033    2579 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 16 02:13:34.693382 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:34.693096    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8ff5e4be-2aa6-4203-83ac-e715f2f0f565-networking-console-plugin-cert podName:8ff5e4be-2aa6-4203-83ac-e715f2f0f565 nodeName:}" failed. No retries permitted until 2026-09-16 02:13:35.193080319 +0000 UTC m=+35.576595557 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/8ff5e4be-2aa6-4203-83ac-e715f2f0f565-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-jtz8l" (UID: "8ff5e4be-2aa6-4203-83ac-e715f2f0f565") : secret "networking-console-plugin-cert" not found
Sep 16 02:13:34.693382 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:34.693100    2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 16 02:13:34.693382 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:34.693019    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/35bd8e04-3e55-44b6-b78e-83566dcf43cc-registry-certificates\") pod \"image-registry-649f6698d9-cmzbc\" (UID: \"35bd8e04-3e55-44b6-b78e-83566dcf43cc\") " pod="openshift-image-registry/image-registry-649f6698d9-cmzbc"
Sep 16 02:13:34.693382 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:34.693113    2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-649f6698d9-cmzbc: secret "image-registry-tls" not found
Sep 16 02:13:34.693382 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:34.693151    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/35bd8e04-3e55-44b6-b78e-83566dcf43cc-registry-tls podName:35bd8e04-3e55-44b6-b78e-83566dcf43cc nodeName:}" failed. No retries permitted until 2026-09-16 02:13:35.193138188 +0000 UTC m=+35.576653428 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/35bd8e04-3e55-44b6-b78e-83566dcf43cc-registry-tls") pod "image-registry-649f6698d9-cmzbc" (UID: "35bd8e04-3e55-44b6-b78e-83566dcf43cc") : secret "image-registry-tls" not found
Sep 16 02:13:34.693770 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:34.693416    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/35bd8e04-3e55-44b6-b78e-83566dcf43cc-trusted-ca\") pod \"image-registry-649f6698d9-cmzbc\" (UID: \"35bd8e04-3e55-44b6-b78e-83566dcf43cc\") " pod="openshift-image-registry/image-registry-649f6698d9-cmzbc"
Sep 16 02:13:34.693770 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:34.693691    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/8ff5e4be-2aa6-4203-83ac-e715f2f0f565-nginx-conf\") pod \"networking-console-plugin-5dcbc7df6b-jtz8l\" (UID: \"8ff5e4be-2aa6-4203-83ac-e715f2f0f565\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-jtz8l"
Sep 16 02:13:34.696886 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:34.696859    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/35bd8e04-3e55-44b6-b78e-83566dcf43cc-image-registry-private-configuration\") pod \"image-registry-649f6698d9-cmzbc\" (UID: \"35bd8e04-3e55-44b6-b78e-83566dcf43cc\") " pod="openshift-image-registry/image-registry-649f6698d9-cmzbc"
Sep 16 02:13:34.696969 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:34.696861    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/35bd8e04-3e55-44b6-b78e-83566dcf43cc-installation-pull-secrets\") pod \"image-registry-649f6698d9-cmzbc\" (UID: \"35bd8e04-3e55-44b6-b78e-83566dcf43cc\") " pod="openshift-image-registry/image-registry-649f6698d9-cmzbc"
Sep 16 02:13:34.696969 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:34.696868    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5caaf390-c372-40a3-8303-e99ab52b762b-cert\") pod \"ingress-canary-vbpq2\" (UID: \"5caaf390-c372-40a3-8303-e99ab52b762b\") " pod="openshift-ingress-canary/ingress-canary-vbpq2"
Sep 16 02:13:34.701124 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:34.701103    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bwkc4\" (UniqueName: \"kubernetes.io/projected/8e7a4943-f231-4531-9e5d-4bcf2a844cf9-kube-api-access-bwkc4\") pod \"dns-default-q6tmq\" (UID: \"8e7a4943-f231-4531-9e5d-4bcf2a844cf9\") " pod="openshift-dns/dns-default-q6tmq"
Sep 16 02:13:34.701560 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:34.701528    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vcl5w\" (UniqueName: \"kubernetes.io/projected/5caaf390-c372-40a3-8303-e99ab52b762b-kube-api-access-vcl5w\") pod \"ingress-canary-vbpq2\" (UID: \"5caaf390-c372-40a3-8303-e99ab52b762b\") " pod="openshift-ingress-canary/ingress-canary-vbpq2"
Sep 16 02:13:34.702028 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:34.701999    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/35bd8e04-3e55-44b6-b78e-83566dcf43cc-bound-sa-token\") pod \"image-registry-649f6698d9-cmzbc\" (UID: \"35bd8e04-3e55-44b6-b78e-83566dcf43cc\") " pod="openshift-image-registry/image-registry-649f6698d9-cmzbc"
Sep 16 02:13:34.702445 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:34.702420    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-grvhj\" (UniqueName: \"kubernetes.io/projected/35bd8e04-3e55-44b6-b78e-83566dcf43cc-kube-api-access-grvhj\") pod \"image-registry-649f6698d9-cmzbc\" (UID: \"35bd8e04-3e55-44b6-b78e-83566dcf43cc\") " pod="openshift-image-registry/image-registry-649f6698d9-cmzbc"
Sep 16 02:13:34.791870 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:34.791843    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-vbpq2"
Sep 16 02:13:35.097167 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:35.096956    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cdwh7"
Sep 16 02:13:35.097307 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:35.096956    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6lwr9"
Sep 16 02:13:35.097307 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:35.096978    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7b42r"
Sep 16 02:13:35.099571 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:35.099550    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 16 02:13:35.099571 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:35.099568    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 16 02:13:35.099817 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:35.099615    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 16 02:13:35.099817 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:35.099674    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 16 02:13:35.099817 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:35.099687    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-2ht6m\""
Sep 16 02:13:35.099920 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:35.099820    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-ksfsh\""
Sep 16 02:13:35.109952 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:35.109571    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-vbpq2"]
Sep 16 02:13:35.112354 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:13:35.112331    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5caaf390_c372_40a3_8303_e99ab52b762b.slice/crio-b92111ab18174a67b086d059aef68a3cb637403734398c80feddeacfa27d6fe2 WatchSource:0}: Error finding container b92111ab18174a67b086d059aef68a3cb637403734398c80feddeacfa27d6fe2: Status 404 returned error can't find the container with id b92111ab18174a67b086d059aef68a3cb637403734398c80feddeacfa27d6fe2
Sep 16 02:13:35.197731 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:35.197705    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8ff5e4be-2aa6-4203-83ac-e715f2f0f565-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-jtz8l\" (UID: \"8ff5e4be-2aa6-4203-83ac-e715f2f0f565\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-jtz8l"
Sep 16 02:13:35.197851 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:35.197757    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/35bd8e04-3e55-44b6-b78e-83566dcf43cc-registry-tls\") pod \"image-registry-649f6698d9-cmzbc\" (UID: \"35bd8e04-3e55-44b6-b78e-83566dcf43cc\") " pod="openshift-image-registry/image-registry-649f6698d9-cmzbc"
Sep 16 02:13:35.197851 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:35.197791    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8e7a4943-f231-4531-9e5d-4bcf2a844cf9-metrics-tls\") pod \"dns-default-q6tmq\" (UID: \"8e7a4943-f231-4531-9e5d-4bcf2a844cf9\") " pod="openshift-dns/dns-default-q6tmq"
Sep 16 02:13:35.197954 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:35.197895    2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 16 02:13:35.197954 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:35.197910    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 02:13:35.197954 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:35.197913    2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-649f6698d9-cmzbc: secret "image-registry-tls" not found
Sep 16 02:13:35.197954 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:35.197909    2579 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 16 02:13:35.198088 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:35.197956    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8e7a4943-f231-4531-9e5d-4bcf2a844cf9-metrics-tls podName:8e7a4943-f231-4531-9e5d-4bcf2a844cf9 nodeName:}" failed. No retries permitted until 2026-09-16 02:13:36.197943712 +0000 UTC m=+36.581458935 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8e7a4943-f231-4531-9e5d-4bcf2a844cf9-metrics-tls") pod "dns-default-q6tmq" (UID: "8e7a4943-f231-4531-9e5d-4bcf2a844cf9") : secret "dns-default-metrics-tls" not found
Sep 16 02:13:35.198088 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:35.197969    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/35bd8e04-3e55-44b6-b78e-83566dcf43cc-registry-tls podName:35bd8e04-3e55-44b6-b78e-83566dcf43cc nodeName:}" failed. No retries permitted until 2026-09-16 02:13:36.197962958 +0000 UTC m=+36.581478181 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/35bd8e04-3e55-44b6-b78e-83566dcf43cc-registry-tls") pod "image-registry-649f6698d9-cmzbc" (UID: "35bd8e04-3e55-44b6-b78e-83566dcf43cc") : secret "image-registry-tls" not found
Sep 16 02:13:35.198088 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:35.197986    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8ff5e4be-2aa6-4203-83ac-e715f2f0f565-networking-console-plugin-cert podName:8ff5e4be-2aa6-4203-83ac-e715f2f0f565 nodeName:}" failed. No retries permitted until 2026-09-16 02:13:36.197973168 +0000 UTC m=+36.581488391 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/8ff5e4be-2aa6-4203-83ac-e715f2f0f565-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-jtz8l" (UID: "8ff5e4be-2aa6-4203-83ac-e715f2f0f565") : secret "networking-console-plugin-cert" not found
Sep 16 02:13:35.306949 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:35.306920    2579 generic.go:356] "Generic (PLEG): container finished" podID="34a0d09d-a96b-4242-9d17-61fc3b8d88bd" containerID="c053b805f3d704c41e40376f3b6335976196ab4cf2c104dd7a3a555af39bcc5c" exitCode=0
Sep 16 02:13:35.307089 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:35.307005    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9g4cg" event={"ID":"34a0d09d-a96b-4242-9d17-61fc3b8d88bd","Type":"ContainerDied","Data":"c053b805f3d704c41e40376f3b6335976196ab4cf2c104dd7a3a555af39bcc5c"}
Sep 16 02:13:35.308231 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:35.308205    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-vbpq2" event={"ID":"5caaf390-c372-40a3-8303-e99ab52b762b","Type":"ContainerStarted","Data":"b92111ab18174a67b086d059aef68a3cb637403734398c80feddeacfa27d6fe2"}
Sep 16 02:13:36.206024 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:36.205983    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8e7a4943-f231-4531-9e5d-4bcf2a844cf9-metrics-tls\") pod \"dns-default-q6tmq\" (UID: \"8e7a4943-f231-4531-9e5d-4bcf2a844cf9\") " pod="openshift-dns/dns-default-q6tmq"
Sep 16 02:13:36.206683 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:36.206076    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8ff5e4be-2aa6-4203-83ac-e715f2f0f565-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-jtz8l\" (UID: \"8ff5e4be-2aa6-4203-83ac-e715f2f0f565\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-jtz8l"
Sep 16 02:13:36.206683 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:36.206116    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/35bd8e04-3e55-44b6-b78e-83566dcf43cc-registry-tls\") pod \"image-registry-649f6698d9-cmzbc\" (UID: \"35bd8e04-3e55-44b6-b78e-83566dcf43cc\") " pod="openshift-image-registry/image-registry-649f6698d9-cmzbc"
Sep 16 02:13:36.206683 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:36.206137    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 02:13:36.206683 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:36.206197    2579 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 16 02:13:36.206683 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:36.206210    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8e7a4943-f231-4531-9e5d-4bcf2a844cf9-metrics-tls podName:8e7a4943-f231-4531-9e5d-4bcf2a844cf9 nodeName:}" failed. No retries permitted until 2026-09-16 02:13:38.206188968 +0000 UTC m=+38.589704206 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8e7a4943-f231-4531-9e5d-4bcf2a844cf9-metrics-tls") pod "dns-default-q6tmq" (UID: "8e7a4943-f231-4531-9e5d-4bcf2a844cf9") : secret "dns-default-metrics-tls" not found
Sep 16 02:13:36.206683 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:36.206210    2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 16 02:13:36.206683 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:36.206226    2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-649f6698d9-cmzbc: secret "image-registry-tls" not found
Sep 16 02:13:36.206683 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:36.206245    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8ff5e4be-2aa6-4203-83ac-e715f2f0f565-networking-console-plugin-cert podName:8ff5e4be-2aa6-4203-83ac-e715f2f0f565 nodeName:}" failed. No retries permitted until 2026-09-16 02:13:38.206233339 +0000 UTC m=+38.589748563 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/8ff5e4be-2aa6-4203-83ac-e715f2f0f565-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-jtz8l" (UID: "8ff5e4be-2aa6-4203-83ac-e715f2f0f565") : secret "networking-console-plugin-cert" not found
Sep 16 02:13:36.206683 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:36.206266    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/35bd8e04-3e55-44b6-b78e-83566dcf43cc-registry-tls podName:35bd8e04-3e55-44b6-b78e-83566dcf43cc nodeName:}" failed. No retries permitted until 2026-09-16 02:13:38.206252206 +0000 UTC m=+38.589767444 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/35bd8e04-3e55-44b6-b78e-83566dcf43cc-registry-tls") pod "image-registry-649f6698d9-cmzbc" (UID: "35bd8e04-3e55-44b6-b78e-83566dcf43cc") : secret "image-registry-tls" not found
Sep 16 02:13:36.314913 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:36.314839    2579 generic.go:356] "Generic (PLEG): container finished" podID="34a0d09d-a96b-4242-9d17-61fc3b8d88bd" containerID="237d251184de7f212e3c168bafac52663a9077fedfb3e82e520e3b9e9bf09c68" exitCode=0
Sep 16 02:13:36.315072 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:36.314928    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9g4cg" event={"ID":"34a0d09d-a96b-4242-9d17-61fc3b8d88bd","Type":"ContainerDied","Data":"237d251184de7f212e3c168bafac52663a9077fedfb3e82e520e3b9e9bf09c68"}
Sep 16 02:13:36.639130 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:36.638844    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:13:36.704247 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:36.704214    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:13:36.809259 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:36.809226    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:13:36.855851 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:36.855820    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:13:36.985710 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:36.985664    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:13:37.088994 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:37.088960    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:13:37.322378 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:37.322276    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9g4cg" event={"ID":"34a0d09d-a96b-4242-9d17-61fc3b8d88bd","Type":"ContainerStarted","Data":"7eee1c272f1a505c74b6309b567accf863c1ebf0ed1a3b6e8d172743f80ac282"}
Sep 16 02:13:37.344564 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:37.344499    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-9g4cg" podStartSLOduration=8.147008771 podStartE2EDuration="37.344459681s" totalImagesPullingTime="29.19745091s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 02:13:00 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 02:13:37.34154242 +0000 UTC m=+37.725057666" watchObservedRunningTime="2026-09-16 02:13:37.344459681 +0000 UTC m=+37.727974931"
Sep 16 02:13:37.458806 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:37.458773    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:13:37.515892 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:37.515848    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:13:37.750993 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:37.750967    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:13:38.121704 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:38.121619    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:13:38.174455 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:38.174417    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:13:38.226110 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:38.226079    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8ff5e4be-2aa6-4203-83ac-e715f2f0f565-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-jtz8l\" (UID: \"8ff5e4be-2aa6-4203-83ac-e715f2f0f565\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-jtz8l"
Sep 16 02:13:38.226256 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:38.226135    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/35bd8e04-3e55-44b6-b78e-83566dcf43cc-registry-tls\") pod \"image-registry-649f6698d9-cmzbc\" (UID: \"35bd8e04-3e55-44b6-b78e-83566dcf43cc\") " pod="openshift-image-registry/image-registry-649f6698d9-cmzbc"
Sep 16 02:13:38.226256 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:38.226182    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8e7a4943-f231-4531-9e5d-4bcf2a844cf9-metrics-tls\") pod \"dns-default-q6tmq\" (UID: \"8e7a4943-f231-4531-9e5d-4bcf2a844cf9\") " pod="openshift-dns/dns-default-q6tmq"
Sep 16 02:13:38.226256 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:38.226227    2579 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 16 02:13:38.226419 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:38.226293    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8ff5e4be-2aa6-4203-83ac-e715f2f0f565-networking-console-plugin-cert podName:8ff5e4be-2aa6-4203-83ac-e715f2f0f565 nodeName:}" failed. No retries permitted until 2026-09-16 02:13:42.226276505 +0000 UTC m=+42.609791744 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/8ff5e4be-2aa6-4203-83ac-e715f2f0f565-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-jtz8l" (UID: "8ff5e4be-2aa6-4203-83ac-e715f2f0f565") : secret "networking-console-plugin-cert" not found
Sep 16 02:13:38.226419 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:38.226322    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 02:13:38.226419 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:38.226326    2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 16 02:13:38.226419 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:38.226343    2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-649f6698d9-cmzbc: secret "image-registry-tls" not found
Sep 16 02:13:38.226419 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:38.226382    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8e7a4943-f231-4531-9e5d-4bcf2a844cf9-metrics-tls podName:8e7a4943-f231-4531-9e5d-4bcf2a844cf9 nodeName:}" failed. No retries permitted until 2026-09-16 02:13:42.226366209 +0000 UTC m=+42.609881448 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8e7a4943-f231-4531-9e5d-4bcf2a844cf9-metrics-tls") pod "dns-default-q6tmq" (UID: "8e7a4943-f231-4531-9e5d-4bcf2a844cf9") : secret "dns-default-metrics-tls" not found
Sep 16 02:13:38.226419 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:38.226399    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/35bd8e04-3e55-44b6-b78e-83566dcf43cc-registry-tls podName:35bd8e04-3e55-44b6-b78e-83566dcf43cc nodeName:}" failed. No retries permitted until 2026-09-16 02:13:42.226390119 +0000 UTC m=+42.609905344 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/35bd8e04-3e55-44b6-b78e-83566dcf43cc-registry-tls") pod "image-registry-649f6698d9-cmzbc" (UID: "35bd8e04-3e55-44b6-b78e-83566dcf43cc") : secret "image-registry-tls" not found
Sep 16 02:13:38.325889 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:38.325856    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-vbpq2" event={"ID":"5caaf390-c372-40a3-8303-e99ab52b762b","Type":"ContainerStarted","Data":"e8e14371a83e7d45f0349ba33d3e4bda2239cbfc994021495cd5307ee61e57d4"}
Sep 16 02:13:38.339645 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:38.339590    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-vbpq2" podStartSLOduration=1.579332207 podStartE2EDuration="4.339568259s" totalImagesPullingTime="2.760236052s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 02:13:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 02:13:38.33905053 +0000 UTC m=+38.722565775" watchObservedRunningTime="2026-09-16 02:13:38.339568259 +0000 UTC m=+38.723083505"
Sep 16 02:13:39.798117 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:39.798085    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:13:39.858667 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:39.858629    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:13:41.501914 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:41.501880    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:13:41.602224 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:41.602195    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:13:41.691735 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:41.691700    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:13:42.256279 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:42.256245    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8e7a4943-f231-4531-9e5d-4bcf2a844cf9-metrics-tls\") pod \"dns-default-q6tmq\" (UID: \"8e7a4943-f231-4531-9e5d-4bcf2a844cf9\") " pod="openshift-dns/dns-default-q6tmq"
Sep 16 02:13:42.256426 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:42.256304    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8ff5e4be-2aa6-4203-83ac-e715f2f0f565-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-jtz8l\" (UID: \"8ff5e4be-2aa6-4203-83ac-e715f2f0f565\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-jtz8l"
Sep 16 02:13:42.256426 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:42.256335    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/35bd8e04-3e55-44b6-b78e-83566dcf43cc-registry-tls\") pod \"image-registry-649f6698d9-cmzbc\" (UID: \"35bd8e04-3e55-44b6-b78e-83566dcf43cc\") " pod="openshift-image-registry/image-registry-649f6698d9-cmzbc"
Sep 16 02:13:42.256426 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:42.256406    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 02:13:42.256426 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:42.256422    2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 16 02:13:42.256600 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:42.256433    2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-649f6698d9-cmzbc: secret "image-registry-tls" not found
Sep 16 02:13:42.256600 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:42.256444    2579 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 16 02:13:42.256600 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:42.256508    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8e7a4943-f231-4531-9e5d-4bcf2a844cf9-metrics-tls podName:8e7a4943-f231-4531-9e5d-4bcf2a844cf9 nodeName:}" failed. No retries permitted until 2026-09-16 02:13:50.256484964 +0000 UTC m=+50.640000201 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8e7a4943-f231-4531-9e5d-4bcf2a844cf9-metrics-tls") pod "dns-default-q6tmq" (UID: "8e7a4943-f231-4531-9e5d-4bcf2a844cf9") : secret "dns-default-metrics-tls" not found
Sep 16 02:13:42.256600 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:42.256522    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8ff5e4be-2aa6-4203-83ac-e715f2f0f565-networking-console-plugin-cert podName:8ff5e4be-2aa6-4203-83ac-e715f2f0f565 nodeName:}" failed. No retries permitted until 2026-09-16 02:13:50.256516374 +0000 UTC m=+50.640031596 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/8ff5e4be-2aa6-4203-83ac-e715f2f0f565-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-jtz8l" (UID: "8ff5e4be-2aa6-4203-83ac-e715f2f0f565") : secret "networking-console-plugin-cert" not found
Sep 16 02:13:42.256600 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:42.256535    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/35bd8e04-3e55-44b6-b78e-83566dcf43cc-registry-tls podName:35bd8e04-3e55-44b6-b78e-83566dcf43cc nodeName:}" failed. No retries permitted until 2026-09-16 02:13:50.256529773 +0000 UTC m=+50.640044996 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/35bd8e04-3e55-44b6-b78e-83566dcf43cc-registry-tls") pod "image-registry-649f6698d9-cmzbc" (UID: "35bd8e04-3e55-44b6-b78e-83566dcf43cc") : secret "image-registry-tls" not found
Sep 16 02:13:45.791691 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:45.791655    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:13:45.838969 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:45.838941    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:13:46.064456 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:46.064377    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:13:46.164701 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:46.164668    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:13:47.825150 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:47.825080    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:13:47.880319 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:47.880281    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:13:50.314949 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:50.314910    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8ff5e4be-2aa6-4203-83ac-e715f2f0f565-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-jtz8l\" (UID: \"8ff5e4be-2aa6-4203-83ac-e715f2f0f565\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-jtz8l"
Sep 16 02:13:50.315456 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:50.314962    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/35bd8e04-3e55-44b6-b78e-83566dcf43cc-registry-tls\") pod \"image-registry-649f6698d9-cmzbc\" (UID: \"35bd8e04-3e55-44b6-b78e-83566dcf43cc\") " pod="openshift-image-registry/image-registry-649f6698d9-cmzbc"
Sep 16 02:13:50.315456 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:50.315004    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8e7a4943-f231-4531-9e5d-4bcf2a844cf9-metrics-tls\") pod \"dns-default-q6tmq\" (UID: \"8e7a4943-f231-4531-9e5d-4bcf2a844cf9\") " pod="openshift-dns/dns-default-q6tmq"
Sep 16 02:13:50.315456 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:50.315060    2579 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 16 02:13:50.315456 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:50.315108    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 02:13:50.315456 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:50.315110    2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 16 02:13:50.315456 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:50.315128    2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-649f6698d9-cmzbc: secret "image-registry-tls" not found
Sep 16 02:13:50.315456 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:50.315151    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8ff5e4be-2aa6-4203-83ac-e715f2f0f565-networking-console-plugin-cert podName:8ff5e4be-2aa6-4203-83ac-e715f2f0f565 nodeName:}" failed. No retries permitted until 2026-09-16 02:14:06.315129428 +0000 UTC m=+66.698644653 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/8ff5e4be-2aa6-4203-83ac-e715f2f0f565-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-jtz8l" (UID: "8ff5e4be-2aa6-4203-83ac-e715f2f0f565") : secret "networking-console-plugin-cert" not found
Sep 16 02:13:50.315456 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:50.315170    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8e7a4943-f231-4531-9e5d-4bcf2a844cf9-metrics-tls podName:8e7a4943-f231-4531-9e5d-4bcf2a844cf9 nodeName:}" failed. No retries permitted until 2026-09-16 02:14:06.315160967 +0000 UTC m=+66.698676195 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8e7a4943-f231-4531-9e5d-4bcf2a844cf9-metrics-tls") pod "dns-default-q6tmq" (UID: "8e7a4943-f231-4531-9e5d-4bcf2a844cf9") : secret "dns-default-metrics-tls" not found
Sep 16 02:13:50.315456 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:13:50.315202    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/35bd8e04-3e55-44b6-b78e-83566dcf43cc-registry-tls podName:35bd8e04-3e55-44b6-b78e-83566dcf43cc nodeName:}" failed. No retries permitted until 2026-09-16 02:14:06.315191058 +0000 UTC m=+66.698706309 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/35bd8e04-3e55-44b6-b78e-83566dcf43cc-registry-tls") pod "image-registry-649f6698d9-cmzbc" (UID: "35bd8e04-3e55-44b6-b78e-83566dcf43cc") : secret "image-registry-tls" not found
Sep 16 02:13:53.815769 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:53.815739    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:13:53.866759 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:13:53.866736    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:14:01.305206 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:14:01.305179    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-88xqz"
Sep 16 02:14:01.448872 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:14:01.448838    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:14:01.507403 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:14:01.507377    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:14:05.824365 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:14:05.824330    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3472392-891d-49a1-83a7-57f1f99f4b79-metrics-certs\") pod \"network-metrics-daemon-6lwr9\" (UID: \"e3472392-891d-49a1-83a7-57f1f99f4b79\") " pod="openshift-multus/network-metrics-daemon-6lwr9"
Sep 16 02:14:05.826600 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:14:05.826582    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 16 02:14:05.834842 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:14:05.834825    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 16 02:14:05.834885 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:14:05.834880    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3472392-891d-49a1-83a7-57f1f99f4b79-metrics-certs podName:e3472392-891d-49a1-83a7-57f1f99f4b79 nodeName:}" failed. No retries permitted until 2026-09-16 02:15:09.834863529 +0000 UTC m=+130.218378751 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e3472392-891d-49a1-83a7-57f1f99f4b79-metrics-certs") pod "network-metrics-daemon-6lwr9" (UID: "e3472392-891d-49a1-83a7-57f1f99f4b79") : secret "metrics-daemon-secret" not found
Sep 16 02:14:05.925194 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:14:05.925164    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wlf8b\" (UniqueName: \"kubernetes.io/projected/28c55959-36b5-4d35-9e1a-f046d6d36f74-kube-api-access-wlf8b\") pod \"network-check-target-7b42r\" (UID: \"28c55959-36b5-4d35-9e1a-f046d6d36f74\") " pod="openshift-network-diagnostics/network-check-target-7b42r"
Sep 16 02:14:05.925274 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:14:05.925199    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2b6aaa85-d6dd-420d-a818-f56d06ce6467-original-pull-secret\") pod \"global-pull-secret-syncer-cdwh7\" (UID: \"2b6aaa85-d6dd-420d-a818-f56d06ce6467\") " pod="kube-system/global-pull-secret-syncer-cdwh7"
Sep 16 02:14:05.927283 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:14:05.927268    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 16 02:14:05.927403 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:14:05.927390    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 16 02:14:05.937745 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:14:05.937730    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 16 02:14:05.939765 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:14:05.939747    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2b6aaa85-d6dd-420d-a818-f56d06ce6467-original-pull-secret\") pod \"global-pull-secret-syncer-cdwh7\" (UID: \"2b6aaa85-d6dd-420d-a818-f56d06ce6467\") " pod="kube-system/global-pull-secret-syncer-cdwh7"
Sep 16 02:14:05.948918 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:14:05.948887    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wlf8b\" (UniqueName: \"kubernetes.io/projected/28c55959-36b5-4d35-9e1a-f046d6d36f74-kube-api-access-wlf8b\") pod \"network-check-target-7b42r\" (UID: \"28c55959-36b5-4d35-9e1a-f046d6d36f74\") " pod="openshift-network-diagnostics/network-check-target-7b42r"
Sep 16 02:14:06.006811 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:14:06.006782    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cdwh7"
Sep 16 02:14:06.023219 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:14:06.023199    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-ksfsh\""
Sep 16 02:14:06.031251 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:14:06.031229    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7b42r"
Sep 16 02:14:06.307375 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:14:06.307346    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-cdwh7"]
Sep 16 02:14:06.310524 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:14:06.310501    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b6aaa85_d6dd_420d_a818_f56d06ce6467.slice/crio-b65a854e72469a1d61ccb4c85748d1f3718a37090c6f18f20269d8f339cc2e8b WatchSource:0}: Error finding container b65a854e72469a1d61ccb4c85748d1f3718a37090c6f18f20269d8f339cc2e8b: Status 404 returned error can't find the container with id b65a854e72469a1d61ccb4c85748d1f3718a37090c6f18f20269d8f339cc2e8b
Sep 16 02:14:06.328343 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:14:06.328319    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8ff5e4be-2aa6-4203-83ac-e715f2f0f565-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-jtz8l\" (UID: \"8ff5e4be-2aa6-4203-83ac-e715f2f0f565\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-jtz8l"
Sep 16 02:14:06.328443 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:14:06.328369    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/35bd8e04-3e55-44b6-b78e-83566dcf43cc-registry-tls\") pod \"image-registry-649f6698d9-cmzbc\" (UID: \"35bd8e04-3e55-44b6-b78e-83566dcf43cc\") " pod="openshift-image-registry/image-registry-649f6698d9-cmzbc"
Sep 16 02:14:06.328443 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:14:06.328422    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8e7a4943-f231-4531-9e5d-4bcf2a844cf9-metrics-tls\") pod \"dns-default-q6tmq\" (UID: \"8e7a4943-f231-4531-9e5d-4bcf2a844cf9\") " pod="openshift-dns/dns-default-q6tmq"
Sep 16 02:14:06.328601 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:14:06.328445    2579 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 16 02:14:06.328601 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:14:06.328520    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8ff5e4be-2aa6-4203-83ac-e715f2f0f565-networking-console-plugin-cert podName:8ff5e4be-2aa6-4203-83ac-e715f2f0f565 nodeName:}" failed. No retries permitted until 2026-09-16 02:14:38.328506423 +0000 UTC m=+98.712021646 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/8ff5e4be-2aa6-4203-83ac-e715f2f0f565-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-jtz8l" (UID: "8ff5e4be-2aa6-4203-83ac-e715f2f0f565") : secret "networking-console-plugin-cert" not found
Sep 16 02:14:06.328601 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:14:06.328546    2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 16 02:14:06.328601 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:14:06.328551    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 02:14:06.328601 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:14:06.328564    2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-649f6698d9-cmzbc: secret "image-registry-tls" not found
Sep 16 02:14:06.328768 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:14:06.328613    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8e7a4943-f231-4531-9e5d-4bcf2a844cf9-metrics-tls podName:8e7a4943-f231-4531-9e5d-4bcf2a844cf9 nodeName:}" failed. No retries permitted until 2026-09-16 02:14:38.328596486 +0000 UTC m=+98.712111724 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8e7a4943-f231-4531-9e5d-4bcf2a844cf9-metrics-tls") pod "dns-default-q6tmq" (UID: "8e7a4943-f231-4531-9e5d-4bcf2a844cf9") : secret "dns-default-metrics-tls" not found
Sep 16 02:14:06.328768 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:14:06.328636    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/35bd8e04-3e55-44b6-b78e-83566dcf43cc-registry-tls podName:35bd8e04-3e55-44b6-b78e-83566dcf43cc nodeName:}" failed. No retries permitted until 2026-09-16 02:14:38.32862112 +0000 UTC m=+98.712136358 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/35bd8e04-3e55-44b6-b78e-83566dcf43cc-registry-tls") pod "image-registry-649f6698d9-cmzbc" (UID: "35bd8e04-3e55-44b6-b78e-83566dcf43cc") : secret "image-registry-tls" not found
Sep 16 02:14:06.337912 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:14:06.337893    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-7b42r"]
Sep 16 02:14:06.340911 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:14:06.340882    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28c55959_36b5_4d35_9e1a_f046d6d36f74.slice/crio-772b7f6dfbb67d89520137e13292339bed351de12aba748ca82d29148ad4fb60 WatchSource:0}: Error finding container 772b7f6dfbb67d89520137e13292339bed351de12aba748ca82d29148ad4fb60: Status 404 returned error can't find the container with id 772b7f6dfbb67d89520137e13292339bed351de12aba748ca82d29148ad4fb60
Sep 16 02:14:06.390966 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:14:06.390939    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-7b42r" event={"ID":"28c55959-36b5-4d35-9e1a-f046d6d36f74","Type":"ContainerStarted","Data":"772b7f6dfbb67d89520137e13292339bed351de12aba748ca82d29148ad4fb60"}
Sep 16 02:14:06.391968 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:14:06.391946    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-cdwh7" event={"ID":"2b6aaa85-d6dd-420d-a818-f56d06ce6467","Type":"ContainerStarted","Data":"b65a854e72469a1d61ccb4c85748d1f3718a37090c6f18f20269d8f339cc2e8b"}
Sep 16 02:14:08.051515 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:14:08.051457    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:14:08.151917 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:14:08.151880    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:14:11.406782 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:14:11.406689    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-cdwh7" event={"ID":"2b6aaa85-d6dd-420d-a818-f56d06ce6467","Type":"ContainerStarted","Data":"d91da07d57452c349dd48c641d821d5797526fa318c927ca0d10363289ca3ec3"}
Sep 16 02:14:11.408151 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:14:11.408125    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-7b42r" event={"ID":"28c55959-36b5-4d35-9e1a-f046d6d36f74","Type":"ContainerStarted","Data":"2ee363b544fed43a4427e2728dfeabde2825b555361f708aab2a8a8327db3c63"}
Sep 16 02:14:11.408265 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:14:11.408244    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-7b42r"
Sep 16 02:14:11.420048 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:14:11.420007    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-cdwh7" podStartSLOduration=66.65629708 podStartE2EDuration="1m11.419990927s" totalImagesPullingTime="4.763693847s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 02:13:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 02:14:11.419590456 +0000 UTC m=+71.803105702" watchObservedRunningTime="2026-09-16 02:14:11.419990927 +0000 UTC m=+71.803506175"
Sep 16 02:14:11.432418 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:14:11.432380    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-7b42r" podStartSLOduration=66.703562868 podStartE2EDuration="1m11.432369792s" totalImagesPullingTime="4.728806924s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 02:13:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 02:14:11.431641755 +0000 UTC m=+71.815157000" watchObservedRunningTime="2026-09-16 02:14:11.432369792 +0000 UTC m=+71.815885036"
Sep 16 02:14:13.742735 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:14:13.742698    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:14:13.798462 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:14:13.798435    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:14:37.701313 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:14:37.701191    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:14:37.798874 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:14:37.798844    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:14:38.351663 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:14:38.351621    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8e7a4943-f231-4531-9e5d-4bcf2a844cf9-metrics-tls\") pod \"dns-default-q6tmq\" (UID: \"8e7a4943-f231-4531-9e5d-4bcf2a844cf9\") " pod="openshift-dns/dns-default-q6tmq"
Sep 16 02:14:38.351846 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:14:38.351679    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8ff5e4be-2aa6-4203-83ac-e715f2f0f565-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-jtz8l\" (UID: \"8ff5e4be-2aa6-4203-83ac-e715f2f0f565\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-jtz8l"
Sep 16 02:14:38.351846 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:14:38.351712    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/35bd8e04-3e55-44b6-b78e-83566dcf43cc-registry-tls\") pod \"image-registry-649f6698d9-cmzbc\" (UID: \"35bd8e04-3e55-44b6-b78e-83566dcf43cc\") " pod="openshift-image-registry/image-registry-649f6698d9-cmzbc"
Sep 16 02:14:38.351846 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:14:38.351769    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 02:14:38.351846 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:14:38.351809    2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 16 02:14:38.351846 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:14:38.351815    2579 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 16 02:14:38.352031 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:14:38.351853    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8e7a4943-f231-4531-9e5d-4bcf2a844cf9-metrics-tls podName:8e7a4943-f231-4531-9e5d-4bcf2a844cf9 nodeName:}" failed. No retries permitted until 2026-09-16 02:15:42.351833648 +0000 UTC m=+162.735348894 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8e7a4943-f231-4531-9e5d-4bcf2a844cf9-metrics-tls") pod "dns-default-q6tmq" (UID: "8e7a4943-f231-4531-9e5d-4bcf2a844cf9") : secret "dns-default-metrics-tls" not found
Sep 16 02:14:38.352031 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:14:38.351873    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8ff5e4be-2aa6-4203-83ac-e715f2f0f565-networking-console-plugin-cert podName:8ff5e4be-2aa6-4203-83ac-e715f2f0f565 nodeName:}" failed. No retries permitted until 2026-09-16 02:15:42.351859659 +0000 UTC m=+162.735374886 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/8ff5e4be-2aa6-4203-83ac-e715f2f0f565-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-jtz8l" (UID: "8ff5e4be-2aa6-4203-83ac-e715f2f0f565") : secret "networking-console-plugin-cert" not found
Sep 16 02:14:38.352031 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:14:38.351819    2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-649f6698d9-cmzbc: secret "image-registry-tls" not found
Sep 16 02:14:38.352031 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:14:38.351900    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/35bd8e04-3e55-44b6-b78e-83566dcf43cc-registry-tls podName:35bd8e04-3e55-44b6-b78e-83566dcf43cc nodeName:}" failed. No retries permitted until 2026-09-16 02:15:42.351893789 +0000 UTC m=+162.735409043 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/35bd8e04-3e55-44b6-b78e-83566dcf43cc-registry-tls") pod "image-registry-649f6698d9-cmzbc" (UID: "35bd8e04-3e55-44b6-b78e-83566dcf43cc") : secret "image-registry-tls" not found
Sep 16 02:14:42.412871 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:14:42.412843    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-7b42r"
Sep 16 02:15:09.864520 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:15:09.864451    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3472392-891d-49a1-83a7-57f1f99f4b79-metrics-certs\") pod \"network-metrics-daemon-6lwr9\" (UID: \"e3472392-891d-49a1-83a7-57f1f99f4b79\") " pod="openshift-multus/network-metrics-daemon-6lwr9"
Sep 16 02:15:09.865018 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:15:09.864602    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 16 02:15:09.865018 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:15:09.864679    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3472392-891d-49a1-83a7-57f1f99f4b79-metrics-certs podName:e3472392-891d-49a1-83a7-57f1f99f4b79 nodeName:}" failed. No retries permitted until 2026-09-16 02:17:11.864662817 +0000 UTC m=+252.248178045 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e3472392-891d-49a1-83a7-57f1f99f4b79-metrics-certs") pod "network-metrics-daemon-6lwr9" (UID: "e3472392-891d-49a1-83a7-57f1f99f4b79") : secret "metrics-daemon-secret" not found
Sep 16 02:15:37.458442 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:15:37.458385    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-jtz8l" podUID="8ff5e4be-2aa6-4203-83ac-e715f2f0f565"
Sep 16 02:15:37.473216 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:15:37.473181    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-649f6698d9-cmzbc" podUID="35bd8e04-3e55-44b6-b78e-83566dcf43cc"
Sep 16 02:15:37.513985 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:15:37.513961    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-q6tmq" podUID="8e7a4943-f231-4531-9e5d-4bcf2a844cf9"
Sep 16 02:15:37.611484 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:15:37.611452    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-q6tmq"
Sep 16 02:15:37.611599 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:15:37.611452    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-jtz8l"
Sep 16 02:15:37.611649 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:15:37.611454    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-649f6698d9-cmzbc"
Sep 16 02:15:38.114521 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:15:38.114492    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-6lwr9" podUID="e3472392-891d-49a1-83a7-57f1f99f4b79"
Sep 16 02:15:42.385593 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:15:42.385543    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8ff5e4be-2aa6-4203-83ac-e715f2f0f565-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-jtz8l\" (UID: \"8ff5e4be-2aa6-4203-83ac-e715f2f0f565\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-jtz8l"
Sep 16 02:15:42.386139 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:15:42.385614    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/35bd8e04-3e55-44b6-b78e-83566dcf43cc-registry-tls\") pod \"image-registry-649f6698d9-cmzbc\" (UID: \"35bd8e04-3e55-44b6-b78e-83566dcf43cc\") " pod="openshift-image-registry/image-registry-649f6698d9-cmzbc"
Sep 16 02:15:42.386139 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:15:42.385657    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8e7a4943-f231-4531-9e5d-4bcf2a844cf9-metrics-tls\") pod \"dns-default-q6tmq\" (UID: \"8e7a4943-f231-4531-9e5d-4bcf2a844cf9\") " pod="openshift-dns/dns-default-q6tmq"
Sep 16 02:15:42.386139 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:15:42.385704    2579 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 16 02:15:42.386139 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:15:42.385747    2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 16 02:15:42.386139 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:15:42.385762    2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-649f6698d9-cmzbc: secret "image-registry-tls" not found
Sep 16 02:15:42.386139 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:15:42.385768    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 02:15:42.386139 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:15:42.385789    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8ff5e4be-2aa6-4203-83ac-e715f2f0f565-networking-console-plugin-cert podName:8ff5e4be-2aa6-4203-83ac-e715f2f0f565 nodeName:}" failed. No retries permitted until 2026-09-16 02:17:44.38576838 +0000 UTC m=+284.769283609 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/8ff5e4be-2aa6-4203-83ac-e715f2f0f565-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-jtz8l" (UID: "8ff5e4be-2aa6-4203-83ac-e715f2f0f565") : secret "networking-console-plugin-cert" not found
Sep 16 02:15:42.386139 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:15:42.385810    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/35bd8e04-3e55-44b6-b78e-83566dcf43cc-registry-tls podName:35bd8e04-3e55-44b6-b78e-83566dcf43cc nodeName:}" failed. No retries permitted until 2026-09-16 02:17:44.385797565 +0000 UTC m=+284.769312805 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/35bd8e04-3e55-44b6-b78e-83566dcf43cc-registry-tls") pod "image-registry-649f6698d9-cmzbc" (UID: "35bd8e04-3e55-44b6-b78e-83566dcf43cc") : secret "image-registry-tls" not found
Sep 16 02:15:42.386139 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:15:42.385826    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8e7a4943-f231-4531-9e5d-4bcf2a844cf9-metrics-tls podName:8e7a4943-f231-4531-9e5d-4bcf2a844cf9 nodeName:}" failed. No retries permitted until 2026-09-16 02:17:44.385818847 +0000 UTC m=+284.769334071 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8e7a4943-f231-4531-9e5d-4bcf2a844cf9-metrics-tls") pod "dns-default-q6tmq" (UID: "8e7a4943-f231-4531-9e5d-4bcf2a844cf9") : secret "dns-default-metrics-tls" not found
Sep 16 02:15:49.096435 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:15:49.096382    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6lwr9"
Sep 16 02:16:45.061214 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:16:45.061180    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:17:10.902675 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:10.902643    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-6f8b768455-8cp89"]
Sep 16 02:17:10.906589 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:10.906527    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f8b768455-8cp89"
Sep 16 02:17:10.908707 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:10.908668    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cd954b897-s5pxd"]
Sep 16 02:17:10.909397 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:10.909376    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-j6bnv\""
Sep 16 02:17:10.911252 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:10.911234    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-696884fb9c-c8q8z"]
Sep 16 02:17:10.911340 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:10.911329    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-s5pxd"
Sep 16 02:17:10.913797 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:10.913777    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-758df9dfb-pnlzd"]
Sep 16 02:17:10.913913 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:10.913896    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-696884fb9c-c8q8z"
Sep 16 02:17:10.914756 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:10.914534    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 16 02:17:10.914756 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:10.913803    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Sep 16 02:17:10.914956 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:10.914925    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-flrfp\""
Sep 16 02:17:10.915160 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:10.915141    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 16 02:17:10.915347 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:10.915323    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Sep 16 02:17:10.916080 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:10.916061    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Sep 16 02:17:10.916181 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:10.916092    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-2jvrs\""
Sep 16 02:17:10.916443 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:10.916423    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Sep 16 02:17:10.916640 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:10.916623    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f8b768455-8cp89"]
Sep 16 02:17:10.916726 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:10.916709    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-758df9dfb-pnlzd"
Sep 16 02:17:10.917081 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:10.917061    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Sep 16 02:17:10.917550 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:10.917530    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Sep 16 02:17:10.918436 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:10.918417    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Sep 16 02:17:10.918661 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:10.918647    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Sep 16 02:17:10.918725 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:10.918668    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 16 02:17:10.919180 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:10.919159    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Sep 16 02:17:10.919280 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:10.919165    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-5597j\""
Sep 16 02:17:10.919385 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:10.919369    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 16 02:17:10.919731 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:10.919716    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Sep 16 02:17:10.922821 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:10.922801    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Sep 16 02:17:10.924639 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:10.924622    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-696884fb9c-c8q8z"]
Sep 16 02:17:10.929543 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:10.929524    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cd954b897-s5pxd"]
Sep 16 02:17:10.932632 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:10.932575    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-758df9dfb-pnlzd"]
Sep 16 02:17:10.990364 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:10.990328    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-47g4b\" (UniqueName: \"kubernetes.io/projected/91fb29dc-53d2-4a7e-adc8-365d8791dfbc-kube-api-access-47g4b\") pod \"router-default-758df9dfb-pnlzd\" (UID: \"91fb29dc-53d2-4a7e-adc8-365d8791dfbc\") " pod="openshift-ingress/router-default-758df9dfb-pnlzd"
Sep 16 02:17:10.990534 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:10.990380    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ebf846fa-af7d-4fdd-9662-b7c533c36ddd-serving-cert\") pod \"console-operator-696884fb9c-c8q8z\" (UID: \"ebf846fa-af7d-4fdd-9662-b7c533c36ddd\") " pod="openshift-console-operator/console-operator-696884fb9c-c8q8z"
Sep 16 02:17:10.990534 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:10.990412    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/91fb29dc-53d2-4a7e-adc8-365d8791dfbc-stats-auth\") pod \"router-default-758df9dfb-pnlzd\" (UID: \"91fb29dc-53d2-4a7e-adc8-365d8791dfbc\") " pod="openshift-ingress/router-default-758df9dfb-pnlzd"
Sep 16 02:17:10.990534 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:10.990448    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/91fb29dc-53d2-4a7e-adc8-365d8791dfbc-service-ca-bundle\") pod \"router-default-758df9dfb-pnlzd\" (UID: \"91fb29dc-53d2-4a7e-adc8-365d8791dfbc\") " pod="openshift-ingress/router-default-758df9dfb-pnlzd"
Sep 16 02:17:10.990534 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:10.990498    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d9vsz\" (UniqueName: \"kubernetes.io/projected/de358882-93c3-45d7-baef-9c6c86f6b6fb-kube-api-access-d9vsz\") pod \"network-check-source-6f8b768455-8cp89\" (UID: \"de358882-93c3-45d7-baef-9c6c86f6b6fb\") " pod="openshift-network-diagnostics/network-check-source-6f8b768455-8cp89"
Sep 16 02:17:10.990534 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:10.990532    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/91fb29dc-53d2-4a7e-adc8-365d8791dfbc-metrics-certs\") pod \"router-default-758df9dfb-pnlzd\" (UID: \"91fb29dc-53d2-4a7e-adc8-365d8791dfbc\") " pod="openshift-ingress/router-default-758df9dfb-pnlzd"
Sep 16 02:17:10.990716 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:10.990556    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kxpc8\" (UniqueName: \"kubernetes.io/projected/e74b93a9-2fe3-43bd-81a4-b928b46db246-kube-api-access-kxpc8\") pod \"cluster-monitoring-operator-6cd954b897-s5pxd\" (UID: \"e74b93a9-2fe3-43bd-81a4-b928b46db246\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-s5pxd"
Sep 16 02:17:10.990716 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:10.990608    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e74b93a9-2fe3-43bd-81a4-b928b46db246-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-s5pxd\" (UID: \"e74b93a9-2fe3-43bd-81a4-b928b46db246\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-s5pxd"
Sep 16 02:17:10.990716 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:10.990662    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ebf846fa-af7d-4fdd-9662-b7c533c36ddd-trusted-ca\") pod \"console-operator-696884fb9c-c8q8z\" (UID: \"ebf846fa-af7d-4fdd-9662-b7c533c36ddd\") " pod="openshift-console-operator/console-operator-696884fb9c-c8q8z"
Sep 16 02:17:10.990805 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:10.990733    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/91fb29dc-53d2-4a7e-adc8-365d8791dfbc-default-certificate\") pod \"router-default-758df9dfb-pnlzd\" (UID: \"91fb29dc-53d2-4a7e-adc8-365d8791dfbc\") " pod="openshift-ingress/router-default-758df9dfb-pnlzd"
Sep 16 02:17:10.990805 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:10.990753    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-drq2n\" (UniqueName: \"kubernetes.io/projected/ebf846fa-af7d-4fdd-9662-b7c533c36ddd-kube-api-access-drq2n\") pod \"console-operator-696884fb9c-c8q8z\" (UID: \"ebf846fa-af7d-4fdd-9662-b7c533c36ddd\") " pod="openshift-console-operator/console-operator-696884fb9c-c8q8z"
Sep 16 02:17:10.990805 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:10.990777    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/e74b93a9-2fe3-43bd-81a4-b928b46db246-telemetry-config\") pod \"cluster-monitoring-operator-6cd954b897-s5pxd\" (UID: \"e74b93a9-2fe3-43bd-81a4-b928b46db246\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-s5pxd"
Sep 16 02:17:10.990904 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:10.990808    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ebf846fa-af7d-4fdd-9662-b7c533c36ddd-config\") pod \"console-operator-696884fb9c-c8q8z\" (UID: \"ebf846fa-af7d-4fdd-9662-b7c533c36ddd\") " pod="openshift-console-operator/console-operator-696884fb9c-c8q8z"
Sep 16 02:17:11.000602 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:11.000578    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-cf7c776b7-bmj9c"]
Sep 16 02:17:11.003777 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:11.003761    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5844b856db-hdtv6"]
Sep 16 02:17:11.003903 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:11.003889    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-bmj9c"
Sep 16 02:17:11.005815 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:11.005793    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Sep 16 02:17:11.005926 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:11.005877    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Sep 16 02:17:11.006142 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:11.006127    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Sep 16 02:17:11.006195 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:11.006132    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-9b54h\""
Sep 16 02:17:11.006195 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:11.006179    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Sep 16 02:17:11.006486 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:11.006454    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-s9rbv"]
Sep 16 02:17:11.006581 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:11.006569    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5844b856db-hdtv6"
Sep 16 02:17:11.008761 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:11.008743    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 16 02:17:11.008875 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:11.008772    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-9g8mb\""
Sep 16 02:17:11.008875 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:11.008748    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Sep 16 02:17:11.008875 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:11.008747    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 16 02:17:11.009024 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:11.008885    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Sep 16 02:17:11.009233 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:11.009216    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-s9rbv"
Sep 16 02:17:11.011284 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:11.011264    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Sep 16 02:17:11.011632 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:11.011616    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Sep 16 02:17:11.011718 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:11.011622    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Sep 16 02:17:11.011905 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:11.011888    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-v7mdd\""
Sep 16 02:17:11.014025 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:11.014006    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Sep 16 02:17:11.016126 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:11.016090    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-cf7c776b7-bmj9c"]
Sep 16 02:17:11.017525 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:11.017505    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-s9rbv"]
Sep 16 02:17:11.018757 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:11.018738    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5844b856db-hdtv6"]
Sep 16 02:17:11.091340 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:11.091312    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d9ffb572-8b90-4503-82c1-84deef377ca7-service-ca-bundle\") pod \"insights-operator-5844b856db-hdtv6\" (UID: \"d9ffb572-8b90-4503-82c1-84deef377ca7\") " pod="openshift-insights/insights-operator-5844b856db-hdtv6"
Sep 16 02:17:11.091340 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:11.091345    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/91fb29dc-53d2-4a7e-adc8-365d8791dfbc-metrics-certs\") pod \"router-default-758df9dfb-pnlzd\" (UID: \"91fb29dc-53d2-4a7e-adc8-365d8791dfbc\") " pod="openshift-ingress/router-default-758df9dfb-pnlzd"
Sep 16 02:17:11.091604 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:11.091362    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kxpc8\" (UniqueName: \"kubernetes.io/projected/e74b93a9-2fe3-43bd-81a4-b928b46db246-kube-api-access-kxpc8\") pod \"cluster-monitoring-operator-6cd954b897-s5pxd\" (UID: \"e74b93a9-2fe3-43bd-81a4-b928b46db246\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-s5pxd"
Sep 16 02:17:11.091604 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:11.091378    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e74b93a9-2fe3-43bd-81a4-b928b46db246-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-s5pxd\" (UID: \"e74b93a9-2fe3-43bd-81a4-b928b46db246\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-s5pxd"
Sep 16 02:17:11.091604 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:11.091396    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d9ffb572-8b90-4503-82c1-84deef377ca7-trusted-ca-bundle\") pod \"insights-operator-5844b856db-hdtv6\" (UID: \"d9ffb572-8b90-4503-82c1-84deef377ca7\") " pod="openshift-insights/insights-operator-5844b856db-hdtv6"
Sep 16 02:17:11.091604 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:11.091414    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ebf846fa-af7d-4fdd-9662-b7c533c36ddd-trusted-ca\") pod \"console-operator-696884fb9c-c8q8z\" (UID: \"ebf846fa-af7d-4fdd-9662-b7c533c36ddd\") " pod="openshift-console-operator/console-operator-696884fb9c-c8q8z"
Sep 16 02:17:11.091604 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:11.091458    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/d9ffb572-8b90-4503-82c1-84deef377ca7-snapshots\") pod \"insights-operator-5844b856db-hdtv6\" (UID: \"d9ffb572-8b90-4503-82c1-84deef377ca7\") " pod="openshift-insights/insights-operator-5844b856db-hdtv6"
Sep 16 02:17:11.091604 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:17:11.091509    2579 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 16 02:17:11.091604 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:11.091520    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jqgcm\" (UniqueName: \"kubernetes.io/projected/5fe7f38e-ec3a-487f-8011-2744a5e25f2f-kube-api-access-jqgcm\") pod \"cluster-samples-operator-86499c89c4-s9rbv\" (UID: \"5fe7f38e-ec3a-487f-8011-2744a5e25f2f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-s9rbv"
Sep 16 02:17:11.091604 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:11.091545    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/111d4c95-703e-4afe-bfca-fbf61d55154f-serving-cert\") pod \"service-ca-operator-cf7c776b7-bmj9c\" (UID: \"111d4c95-703e-4afe-bfca-fbf61d55154f\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-bmj9c"
Sep 16 02:17:11.091604 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:17:11.091550    2579 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 16 02:17:11.091604 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:17:11.091579    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/91fb29dc-53d2-4a7e-adc8-365d8791dfbc-metrics-certs podName:91fb29dc-53d2-4a7e-adc8-365d8791dfbc nodeName:}" failed. No retries permitted until 2026-09-16 02:17:11.591557959 +0000 UTC m=+251.975073197 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/91fb29dc-53d2-4a7e-adc8-365d8791dfbc-metrics-certs") pod "router-default-758df9dfb-pnlzd" (UID: "91fb29dc-53d2-4a7e-adc8-365d8791dfbc") : secret "router-metrics-certs-default" not found
Sep 16 02:17:11.091604 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:17:11.091608    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e74b93a9-2fe3-43bd-81a4-b928b46db246-cluster-monitoring-operator-tls podName:e74b93a9-2fe3-43bd-81a4-b928b46db246 nodeName:}" failed. No retries permitted until 2026-09-16 02:17:11.591594951 +0000 UTC m=+251.975110192 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e74b93a9-2fe3-43bd-81a4-b928b46db246-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-s5pxd" (UID: "e74b93a9-2fe3-43bd-81a4-b928b46db246") : secret "cluster-monitoring-operator-tls" not found
Sep 16 02:17:11.092027 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:11.091756    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5fe7f38e-ec3a-487f-8011-2744a5e25f2f-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-s9rbv\" (UID: \"5fe7f38e-ec3a-487f-8011-2744a5e25f2f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-s9rbv"
Sep 16 02:17:11.092027 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:11.091788    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/91fb29dc-53d2-4a7e-adc8-365d8791dfbc-default-certificate\") pod \"router-default-758df9dfb-pnlzd\" (UID: \"91fb29dc-53d2-4a7e-adc8-365d8791dfbc\") " pod="openshift-ingress/router-default-758df9dfb-pnlzd"
Sep 16 02:17:11.092027 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:11.091813    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/111d4c95-703e-4afe-bfca-fbf61d55154f-config\") pod \"service-ca-operator-cf7c776b7-bmj9c\" (UID: \"111d4c95-703e-4afe-bfca-fbf61d55154f\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-bmj9c"
Sep 16 02:17:11.092027 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:11.091845    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-drq2n\" (UniqueName: \"kubernetes.io/projected/ebf846fa-af7d-4fdd-9662-b7c533c36ddd-kube-api-access-drq2n\") pod \"console-operator-696884fb9c-c8q8z\" (UID: \"ebf846fa-af7d-4fdd-9662-b7c533c36ddd\") " pod="openshift-console-operator/console-operator-696884fb9c-c8q8z"
Sep 16 02:17:11.092027 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:11.091883    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d9ffb572-8b90-4503-82c1-84deef377ca7-tmp\") pod \"insights-operator-5844b856db-hdtv6\" (UID: \"d9ffb572-8b90-4503-82c1-84deef377ca7\") " pod="openshift-insights/insights-operator-5844b856db-hdtv6"
Sep 16 02:17:11.092027 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:11.091911    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/e74b93a9-2fe3-43bd-81a4-b928b46db246-telemetry-config\") pod \"cluster-monitoring-operator-6cd954b897-s5pxd\" (UID: \"e74b93a9-2fe3-43bd-81a4-b928b46db246\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-s5pxd"
Sep 16 02:17:11.092027 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:11.091927    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ebf846fa-af7d-4fdd-9662-b7c533c36ddd-config\") pod \"console-operator-696884fb9c-c8q8z\" (UID: \"ebf846fa-af7d-4fdd-9662-b7c533c36ddd\") " pod="openshift-console-operator/console-operator-696884fb9c-c8q8z"
Sep 16 02:17:11.092027 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:11.091962    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-47g4b\" (UniqueName: \"kubernetes.io/projected/91fb29dc-53d2-4a7e-adc8-365d8791dfbc-kube-api-access-47g4b\") pod \"router-default-758df9dfb-pnlzd\" (UID: \"91fb29dc-53d2-4a7e-adc8-365d8791dfbc\") " pod="openshift-ingress/router-default-758df9dfb-pnlzd"
Sep 16 02:17:11.092027 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:11.092001    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ebf846fa-af7d-4fdd-9662-b7c533c36ddd-serving-cert\") pod \"console-operator-696884fb9c-c8q8z\" (UID: \"ebf846fa-af7d-4fdd-9662-b7c533c36ddd\") " pod="openshift-console-operator/console-operator-696884fb9c-c8q8z"
Sep 16 02:17:11.092027 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:11.092025    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g27bh\" (UniqueName: \"kubernetes.io/projected/d9ffb572-8b90-4503-82c1-84deef377ca7-kube-api-access-g27bh\") pod \"insights-operator-5844b856db-hdtv6\" (UID: \"d9ffb572-8b90-4503-82c1-84deef377ca7\") " pod="openshift-insights/insights-operator-5844b856db-hdtv6"
Sep 16 02:17:11.092602 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:11.092059    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/91fb29dc-53d2-4a7e-adc8-365d8791dfbc-stats-auth\") pod \"router-default-758df9dfb-pnlzd\" (UID: \"91fb29dc-53d2-4a7e-adc8-365d8791dfbc\") " pod="openshift-ingress/router-default-758df9dfb-pnlzd"
Sep 16 02:17:11.092602 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:11.092101    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d9ffb572-8b90-4503-82c1-84deef377ca7-serving-cert\") pod \"insights-operator-5844b856db-hdtv6\" (UID: \"d9ffb572-8b90-4503-82c1-84deef377ca7\") " pod="openshift-insights/insights-operator-5844b856db-hdtv6"
Sep 16 02:17:11.092602 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:11.092129    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/91fb29dc-53d2-4a7e-adc8-365d8791dfbc-service-ca-bundle\") pod \"router-default-758df9dfb-pnlzd\" (UID: \"91fb29dc-53d2-4a7e-adc8-365d8791dfbc\") " pod="openshift-ingress/router-default-758df9dfb-pnlzd"
Sep 16 02:17:11.092602 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:11.092156    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d9vsz\" (UniqueName: \"kubernetes.io/projected/de358882-93c3-45d7-baef-9c6c86f6b6fb-kube-api-access-d9vsz\") pod \"network-check-source-6f8b768455-8cp89\" (UID: \"de358882-93c3-45d7-baef-9c6c86f6b6fb\") " pod="openshift-network-diagnostics/network-check-source-6f8b768455-8cp89"
Sep 16 02:17:11.092602 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:11.092180    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fcqfj\" (UniqueName: \"kubernetes.io/projected/111d4c95-703e-4afe-bfca-fbf61d55154f-kube-api-access-fcqfj\") pod \"service-ca-operator-cf7c776b7-bmj9c\" (UID: \"111d4c95-703e-4afe-bfca-fbf61d55154f\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-bmj9c"
Sep 16 02:17:11.092602 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:17:11.092518    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/91fb29dc-53d2-4a7e-adc8-365d8791dfbc-service-ca-bundle podName:91fb29dc-53d2-4a7e-adc8-365d8791dfbc nodeName:}" failed. No retries permitted until 2026-09-16 02:17:11.592498563 +0000 UTC m=+251.976013786 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/91fb29dc-53d2-4a7e-adc8-365d8791dfbc-service-ca-bundle") pod "router-default-758df9dfb-pnlzd" (UID: "91fb29dc-53d2-4a7e-adc8-365d8791dfbc") : configmap references non-existent config key: service-ca.crt
Sep 16 02:17:11.092865 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:11.092746    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ebf846fa-af7d-4fdd-9662-b7c533c36ddd-config\") pod \"console-operator-696884fb9c-c8q8z\" (UID: \"ebf846fa-af7d-4fdd-9662-b7c533c36ddd\") " pod="openshift-console-operator/console-operator-696884fb9c-c8q8z"
Sep 16 02:17:11.092865 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:11.092808    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ebf846fa-af7d-4fdd-9662-b7c533c36ddd-trusted-ca\") pod \"console-operator-696884fb9c-c8q8z\" (UID: \"ebf846fa-af7d-4fdd-9662-b7c533c36ddd\") " pod="openshift-console-operator/console-operator-696884fb9c-c8q8z"
Sep 16 02:17:11.093141 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:11.093121    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/e74b93a9-2fe3-43bd-81a4-b928b46db246-telemetry-config\") pod \"cluster-monitoring-operator-6cd954b897-s5pxd\" (UID: \"e74b93a9-2fe3-43bd-81a4-b928b46db246\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-s5pxd"
Sep 16 02:17:11.094544 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:11.094524    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/91fb29dc-53d2-4a7e-adc8-365d8791dfbc-default-certificate\") pod \"router-default-758df9dfb-pnlzd\" (UID: \"91fb29dc-53d2-4a7e-adc8-365d8791dfbc\") " pod="openshift-ingress/router-default-758df9dfb-pnlzd"
Sep 16 02:17:11.094634 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:11.094523    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ebf846fa-af7d-4fdd-9662-b7c533c36ddd-serving-cert\") pod \"console-operator-696884fb9c-c8q8z\" (UID: \"ebf846fa-af7d-4fdd-9662-b7c533c36ddd\") " pod="openshift-console-operator/console-operator-696884fb9c-c8q8z"
Sep 16 02:17:11.094738 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:11.094722    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/91fb29dc-53d2-4a7e-adc8-365d8791dfbc-stats-auth\") pod \"router-default-758df9dfb-pnlzd\" (UID: \"91fb29dc-53d2-4a7e-adc8-365d8791dfbc\") " pod="openshift-ingress/router-default-758df9dfb-pnlzd"
Sep 16 02:17:11.099391 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:11.099369    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kxpc8\" (UniqueName: \"kubernetes.io/projected/e74b93a9-2fe3-43bd-81a4-b928b46db246-kube-api-access-kxpc8\") pod \"cluster-monitoring-operator-6cd954b897-s5pxd\" (UID: \"e74b93a9-2fe3-43bd-81a4-b928b46db246\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-s5pxd"
Sep 16 02:17:11.102295 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:11.102278    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-47g4b\" (UniqueName: \"kubernetes.io/projected/91fb29dc-53d2-4a7e-adc8-365d8791dfbc-kube-api-access-47g4b\") pod \"router-default-758df9dfb-pnlzd\" (UID: \"91fb29dc-53d2-4a7e-adc8-365d8791dfbc\") " pod="openshift-ingress/router-default-758df9dfb-pnlzd"
Sep 16 02:17:11.105574 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:11.105553    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d9vsz\" (UniqueName: \"kubernetes.io/projected/de358882-93c3-45d7-baef-9c6c86f6b6fb-kube-api-access-d9vsz\") pod \"network-check-source-6f8b768455-8cp89\" (UID: \"de358882-93c3-45d7-baef-9c6c86f6b6fb\") " pod="openshift-network-diagnostics/network-check-source-6f8b768455-8cp89"
Sep 16 02:17:11.105738 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:11.105718    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-drq2n\" (UniqueName: \"kubernetes.io/projected/ebf846fa-af7d-4fdd-9662-b7c533c36ddd-kube-api-access-drq2n\") pod \"console-operator-696884fb9c-c8q8z\" (UID: \"ebf846fa-af7d-4fdd-9662-b7c533c36ddd\") " pod="openshift-console-operator/console-operator-696884fb9c-c8q8z"
Sep 16 02:17:11.121503 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:11.121463    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:17:11.192777 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:11.192746    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d9ffb572-8b90-4503-82c1-84deef377ca7-trusted-ca-bundle\") pod \"insights-operator-5844b856db-hdtv6\" (UID: \"d9ffb572-8b90-4503-82c1-84deef377ca7\") " pod="openshift-insights/insights-operator-5844b856db-hdtv6"
Sep 16 02:17:11.192877 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:11.192782    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/d9ffb572-8b90-4503-82c1-84deef377ca7-snapshots\") pod \"insights-operator-5844b856db-hdtv6\" (UID: \"d9ffb572-8b90-4503-82c1-84deef377ca7\") " pod="openshift-insights/insights-operator-5844b856db-hdtv6"
Sep 16 02:17:11.192877 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:11.192805    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jqgcm\" (UniqueName: \"kubernetes.io/projected/5fe7f38e-ec3a-487f-8011-2744a5e25f2f-kube-api-access-jqgcm\") pod \"cluster-samples-operator-86499c89c4-s9rbv\" (UID: \"5fe7f38e-ec3a-487f-8011-2744a5e25f2f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-s9rbv"
Sep 16 02:17:11.192877 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:11.192831    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/111d4c95-703e-4afe-bfca-fbf61d55154f-serving-cert\") pod \"service-ca-operator-cf7c776b7-bmj9c\" (UID: \"111d4c95-703e-4afe-bfca-fbf61d55154f\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-bmj9c"
Sep 16 02:17:11.193017 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:11.192873    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5fe7f38e-ec3a-487f-8011-2744a5e25f2f-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-s9rbv\" (UID: \"5fe7f38e-ec3a-487f-8011-2744a5e25f2f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-s9rbv"
Sep 16 02:17:11.193017 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:11.192908    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/111d4c95-703e-4afe-bfca-fbf61d55154f-config\") pod \"service-ca-operator-cf7c776b7-bmj9c\" (UID: \"111d4c95-703e-4afe-bfca-fbf61d55154f\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-bmj9c"
Sep 16 02:17:11.193017 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:11.192940    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d9ffb572-8b90-4503-82c1-84deef377ca7-tmp\") pod \"insights-operator-5844b856db-hdtv6\" (UID: \"d9ffb572-8b90-4503-82c1-84deef377ca7\") " pod="openshift-insights/insights-operator-5844b856db-hdtv6"
Sep 16 02:17:11.193017 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:11.193009    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g27bh\" (UniqueName: \"kubernetes.io/projected/d9ffb572-8b90-4503-82c1-84deef377ca7-kube-api-access-g27bh\") pod \"insights-operator-5844b856db-hdtv6\" (UID: \"d9ffb572-8b90-4503-82c1-84deef377ca7\") " pod="openshift-insights/insights-operator-5844b856db-hdtv6"
Sep 16 02:17:11.193417 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:11.193054    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d9ffb572-8b90-4503-82c1-84deef377ca7-serving-cert\") pod \"insights-operator-5844b856db-hdtv6\" (UID: \"d9ffb572-8b90-4503-82c1-84deef377ca7\") " pod="openshift-insights/insights-operator-5844b856db-hdtv6"
Sep 16 02:17:11.193417 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:11.193094    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fcqfj\" (UniqueName: \"kubernetes.io/projected/111d4c95-703e-4afe-bfca-fbf61d55154f-kube-api-access-fcqfj\") pod \"service-ca-operator-cf7c776b7-bmj9c\" (UID: \"111d4c95-703e-4afe-bfca-fbf61d55154f\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-bmj9c"
Sep 16 02:17:11.193417 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:11.193132    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d9ffb572-8b90-4503-82c1-84deef377ca7-service-ca-bundle\") pod \"insights-operator-5844b856db-hdtv6\" (UID: \"d9ffb572-8b90-4503-82c1-84deef377ca7\") " pod="openshift-insights/insights-operator-5844b856db-hdtv6"
Sep 16 02:17:11.193417 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:11.193403    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d9ffb572-8b90-4503-82c1-84deef377ca7-tmp\") pod \"insights-operator-5844b856db-hdtv6\" (UID: \"d9ffb572-8b90-4503-82c1-84deef377ca7\") " pod="openshift-insights/insights-operator-5844b856db-hdtv6"
Sep 16 02:17:11.193716 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:11.193659    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/111d4c95-703e-4afe-bfca-fbf61d55154f-config\") pod \"service-ca-operator-cf7c776b7-bmj9c\" (UID: \"111d4c95-703e-4afe-bfca-fbf61d55154f\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-bmj9c"
Sep 16 02:17:11.193919 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:17:11.193742    2579 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 16 02:17:11.193919 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:11.193753    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d9ffb572-8b90-4503-82c1-84deef377ca7-service-ca-bundle\") pod \"insights-operator-5844b856db-hdtv6\" (UID: \"d9ffb572-8b90-4503-82c1-84deef377ca7\") " pod="openshift-insights/insights-operator-5844b856db-hdtv6"
Sep 16 02:17:11.193919 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:17:11.193808    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5fe7f38e-ec3a-487f-8011-2744a5e25f2f-samples-operator-tls podName:5fe7f38e-ec3a-487f-8011-2744a5e25f2f nodeName:}" failed. No retries permitted until 2026-09-16 02:17:11.693795539 +0000 UTC m=+252.077310761 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/5fe7f38e-ec3a-487f-8011-2744a5e25f2f-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-s9rbv" (UID: "5fe7f38e-ec3a-487f-8011-2744a5e25f2f") : secret "samples-operator-tls" not found
Sep 16 02:17:11.193919 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:11.193751    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/d9ffb572-8b90-4503-82c1-84deef377ca7-snapshots\") pod \"insights-operator-5844b856db-hdtv6\" (UID: \"d9ffb572-8b90-4503-82c1-84deef377ca7\") " pod="openshift-insights/insights-operator-5844b856db-hdtv6"
Sep 16 02:17:11.193919 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:11.193850    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d9ffb572-8b90-4503-82c1-84deef377ca7-trusted-ca-bundle\") pod \"insights-operator-5844b856db-hdtv6\" (UID: \"d9ffb572-8b90-4503-82c1-84deef377ca7\") " pod="openshift-insights/insights-operator-5844b856db-hdtv6"
Sep 16 02:17:11.195431 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:11.195408    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d9ffb572-8b90-4503-82c1-84deef377ca7-serving-cert\") pod \"insights-operator-5844b856db-hdtv6\" (UID: \"d9ffb572-8b90-4503-82c1-84deef377ca7\") " pod="openshift-insights/insights-operator-5844b856db-hdtv6"
Sep 16 02:17:11.195534 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:11.195431    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/111d4c95-703e-4afe-bfca-fbf61d55154f-serving-cert\") pod \"service-ca-operator-cf7c776b7-bmj9c\" (UID: \"111d4c95-703e-4afe-bfca-fbf61d55154f\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-bmj9c"
Sep 16 02:17:11.200054 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:11.200029    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jqgcm\" (UniqueName: \"kubernetes.io/projected/5fe7f38e-ec3a-487f-8011-2744a5e25f2f-kube-api-access-jqgcm\") pod \"cluster-samples-operator-86499c89c4-s9rbv\" (UID: \"5fe7f38e-ec3a-487f-8011-2744a5e25f2f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-s9rbv"
Sep 16 02:17:11.200174 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:11.200087    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g27bh\" (UniqueName: \"kubernetes.io/projected/d9ffb572-8b90-4503-82c1-84deef377ca7-kube-api-access-g27bh\") pod \"insights-operator-5844b856db-hdtv6\" (UID: \"d9ffb572-8b90-4503-82c1-84deef377ca7\") " pod="openshift-insights/insights-operator-5844b856db-hdtv6"
Sep 16 02:17:11.201030 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:11.201011    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fcqfj\" (UniqueName: \"kubernetes.io/projected/111d4c95-703e-4afe-bfca-fbf61d55154f-kube-api-access-fcqfj\") pod \"service-ca-operator-cf7c776b7-bmj9c\" (UID: \"111d4c95-703e-4afe-bfca-fbf61d55154f\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-bmj9c"
Sep 16 02:17:11.219082 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:11.219062    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f8b768455-8cp89"
Sep 16 02:17:11.222500 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:11.222460    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:17:11.233522 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:11.233503    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-696884fb9c-c8q8z"
Sep 16 02:17:11.315042 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:11.315019    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-bmj9c"
Sep 16 02:17:11.322998 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:11.322974    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5844b856db-hdtv6"
Sep 16 02:17:11.487073 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:11.487003    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f8b768455-8cp89"]
Sep 16 02:17:11.490158 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:17:11.490134    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde358882_93c3_45d7_baef_9c6c86f6b6fb.slice/crio-9d58cc966762203144efe1f67a66b4edf82d3e25c511b013ac0f6958821a80a4 WatchSource:0}: Error finding container 9d58cc966762203144efe1f67a66b4edf82d3e25c511b013ac0f6958821a80a4: Status 404 returned error can't find the container with id 9d58cc966762203144efe1f67a66b4edf82d3e25c511b013ac0f6958821a80a4
Sep 16 02:17:11.509933 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:11.509887    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-696884fb9c-c8q8z"]
Sep 16 02:17:11.516336 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:17:11.516313    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podebf846fa_af7d_4fdd_9662_b7c533c36ddd.slice/crio-3c744d16fdd095d1758cb80080dc1cd257533c4514fd6a979c4f8b5524ad2e5a WatchSource:0}: Error finding container 3c744d16fdd095d1758cb80080dc1cd257533c4514fd6a979c4f8b5524ad2e5a: Status 404 returned error can't find the container with id 3c744d16fdd095d1758cb80080dc1cd257533c4514fd6a979c4f8b5524ad2e5a
Sep 16 02:17:11.583711 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:11.583679    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-cf7c776b7-bmj9c"]
Sep 16 02:17:11.587126 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:17:11.587085    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod111d4c95_703e_4afe_bfca_fbf61d55154f.slice/crio-a5d3eb6a318a59e00afe83251d33c1ce70f15ad64fe130053c4012c67b749b2b WatchSource:0}: Error finding container a5d3eb6a318a59e00afe83251d33c1ce70f15ad64fe130053c4012c67b749b2b: Status 404 returned error can't find the container with id a5d3eb6a318a59e00afe83251d33c1ce70f15ad64fe130053c4012c67b749b2b
Sep 16 02:17:11.597570 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:11.597451    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/91fb29dc-53d2-4a7e-adc8-365d8791dfbc-service-ca-bundle\") pod \"router-default-758df9dfb-pnlzd\" (UID: \"91fb29dc-53d2-4a7e-adc8-365d8791dfbc\") " pod="openshift-ingress/router-default-758df9dfb-pnlzd"
Sep 16 02:17:11.597570 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:11.597522    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/91fb29dc-53d2-4a7e-adc8-365d8791dfbc-metrics-certs\") pod \"router-default-758df9dfb-pnlzd\" (UID: \"91fb29dc-53d2-4a7e-adc8-365d8791dfbc\") " pod="openshift-ingress/router-default-758df9dfb-pnlzd"
Sep 16 02:17:11.597570 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:11.597543    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e74b93a9-2fe3-43bd-81a4-b928b46db246-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-s5pxd\" (UID: \"e74b93a9-2fe3-43bd-81a4-b928b46db246\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-s5pxd"
Sep 16 02:17:11.597728 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:17:11.597621    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/91fb29dc-53d2-4a7e-adc8-365d8791dfbc-service-ca-bundle podName:91fb29dc-53d2-4a7e-adc8-365d8791dfbc nodeName:}" failed. No retries permitted until 2026-09-16 02:17:12.597602834 +0000 UTC m=+252.981118057 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/91fb29dc-53d2-4a7e-adc8-365d8791dfbc-service-ca-bundle") pod "router-default-758df9dfb-pnlzd" (UID: "91fb29dc-53d2-4a7e-adc8-365d8791dfbc") : configmap references non-existent config key: service-ca.crt
Sep 16 02:17:11.597728 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:17:11.597660    2579 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 16 02:17:11.597728 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:17:11.597671    2579 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 16 02:17:11.597728 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:17:11.597703    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/91fb29dc-53d2-4a7e-adc8-365d8791dfbc-metrics-certs podName:91fb29dc-53d2-4a7e-adc8-365d8791dfbc nodeName:}" failed. No retries permitted until 2026-09-16 02:17:12.59769261 +0000 UTC m=+252.981207848 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/91fb29dc-53d2-4a7e-adc8-365d8791dfbc-metrics-certs") pod "router-default-758df9dfb-pnlzd" (UID: "91fb29dc-53d2-4a7e-adc8-365d8791dfbc") : secret "router-metrics-certs-default" not found
Sep 16 02:17:11.597728 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:17:11.597725    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e74b93a9-2fe3-43bd-81a4-b928b46db246-cluster-monitoring-operator-tls podName:e74b93a9-2fe3-43bd-81a4-b928b46db246 nodeName:}" failed. No retries permitted until 2026-09-16 02:17:12.597708756 +0000 UTC m=+252.981223983 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e74b93a9-2fe3-43bd-81a4-b928b46db246-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-s5pxd" (UID: "e74b93a9-2fe3-43bd-81a4-b928b46db246") : secret "cluster-monitoring-operator-tls" not found
Sep 16 02:17:11.603508 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:11.603485    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5844b856db-hdtv6"]
Sep 16 02:17:11.609173 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:17:11.609153    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9ffb572_8b90_4503_82c1_84deef377ca7.slice/crio-13b32233a77cdf6d08bec53b822e69a23f50c3b72642003f7edf9a7bfe7ea8f4 WatchSource:0}: Error finding container 13b32233a77cdf6d08bec53b822e69a23f50c3b72642003f7edf9a7bfe7ea8f4: Status 404 returned error can't find the container with id 13b32233a77cdf6d08bec53b822e69a23f50c3b72642003f7edf9a7bfe7ea8f4
Sep 16 02:17:11.698425 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:11.698399    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5fe7f38e-ec3a-487f-8011-2744a5e25f2f-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-s9rbv\" (UID: \"5fe7f38e-ec3a-487f-8011-2744a5e25f2f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-s9rbv"
Sep 16 02:17:11.698569 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:17:11.698550    2579 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 16 02:17:11.698622 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:17:11.698613    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5fe7f38e-ec3a-487f-8011-2744a5e25f2f-samples-operator-tls podName:5fe7f38e-ec3a-487f-8011-2744a5e25f2f nodeName:}" failed. No retries permitted until 2026-09-16 02:17:12.69859735 +0000 UTC m=+253.082112573 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/5fe7f38e-ec3a-487f-8011-2744a5e25f2f-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-s9rbv" (UID: "5fe7f38e-ec3a-487f-8011-2744a5e25f2f") : secret "samples-operator-tls" not found
Sep 16 02:17:11.831292 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:11.831206    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-hdtv6" event={"ID":"d9ffb572-8b90-4503-82c1-84deef377ca7","Type":"ContainerStarted","Data":"13b32233a77cdf6d08bec53b822e69a23f50c3b72642003f7edf9a7bfe7ea8f4"}
Sep 16 02:17:11.832626 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:11.832597    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f8b768455-8cp89" event={"ID":"de358882-93c3-45d7-baef-9c6c86f6b6fb","Type":"ContainerStarted","Data":"3228dd25afdcdb12587ad7693747101af94d1afb71fb3e3e62a4e8d3e255e4ea"}
Sep 16 02:17:11.832773 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:11.832648    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f8b768455-8cp89" event={"ID":"de358882-93c3-45d7-baef-9c6c86f6b6fb","Type":"ContainerStarted","Data":"9d58cc966762203144efe1f67a66b4edf82d3e25c511b013ac0f6958821a80a4"}
Sep 16 02:17:11.833779 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:11.833756    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-bmj9c" event={"ID":"111d4c95-703e-4afe-bfca-fbf61d55154f","Type":"ContainerStarted","Data":"a5d3eb6a318a59e00afe83251d33c1ce70f15ad64fe130053c4012c67b749b2b"}
Sep 16 02:17:11.834834 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:11.834812    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-c8q8z" event={"ID":"ebf846fa-af7d-4fdd-9662-b7c533c36ddd","Type":"ContainerStarted","Data":"3c744d16fdd095d1758cb80080dc1cd257533c4514fd6a979c4f8b5524ad2e5a"}
Sep 16 02:17:11.846852 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:11.846810    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-6f8b768455-8cp89" podStartSLOduration=1.84679748 podStartE2EDuration="1.84679748s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 02:17:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 02:17:11.846126538 +0000 UTC m=+252.229641784" watchObservedRunningTime="2026-09-16 02:17:11.84679748 +0000 UTC m=+252.230312724"
Sep 16 02:17:11.899882 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:11.899861    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3472392-891d-49a1-83a7-57f1f99f4b79-metrics-certs\") pod \"network-metrics-daemon-6lwr9\" (UID: \"e3472392-891d-49a1-83a7-57f1f99f4b79\") " pod="openshift-multus/network-metrics-daemon-6lwr9"
Sep 16 02:17:11.900032 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:17:11.900013    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 16 02:17:11.900098 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:17:11.900085    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3472392-891d-49a1-83a7-57f1f99f4b79-metrics-certs podName:e3472392-891d-49a1-83a7-57f1f99f4b79 nodeName:}" failed. No retries permitted until 2026-09-16 02:19:13.900066419 +0000 UTC m=+374.283581645 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e3472392-891d-49a1-83a7-57f1f99f4b79-metrics-certs") pod "network-metrics-daemon-6lwr9" (UID: "e3472392-891d-49a1-83a7-57f1f99f4b79") : secret "metrics-daemon-secret" not found
Sep 16 02:17:12.607761 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:12.607723    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/91fb29dc-53d2-4a7e-adc8-365d8791dfbc-metrics-certs\") pod \"router-default-758df9dfb-pnlzd\" (UID: \"91fb29dc-53d2-4a7e-adc8-365d8791dfbc\") " pod="openshift-ingress/router-default-758df9dfb-pnlzd"
Sep 16 02:17:12.608226 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:12.607779    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e74b93a9-2fe3-43bd-81a4-b928b46db246-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-s5pxd\" (UID: \"e74b93a9-2fe3-43bd-81a4-b928b46db246\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-s5pxd"
Sep 16 02:17:12.608226 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:12.607922    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/91fb29dc-53d2-4a7e-adc8-365d8791dfbc-service-ca-bundle\") pod \"router-default-758df9dfb-pnlzd\" (UID: \"91fb29dc-53d2-4a7e-adc8-365d8791dfbc\") " pod="openshift-ingress/router-default-758df9dfb-pnlzd"
Sep 16 02:17:12.608226 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:17:12.608065    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/91fb29dc-53d2-4a7e-adc8-365d8791dfbc-service-ca-bundle podName:91fb29dc-53d2-4a7e-adc8-365d8791dfbc nodeName:}" failed. No retries permitted until 2026-09-16 02:17:14.60804585 +0000 UTC m=+254.991561077 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/91fb29dc-53d2-4a7e-adc8-365d8791dfbc-service-ca-bundle") pod "router-default-758df9dfb-pnlzd" (UID: "91fb29dc-53d2-4a7e-adc8-365d8791dfbc") : configmap references non-existent config key: service-ca.crt
Sep 16 02:17:12.608490 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:17:12.608445    2579 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 16 02:17:12.608573 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:17:12.608482    2579 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 16 02:17:12.608573 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:17:12.608534    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e74b93a9-2fe3-43bd-81a4-b928b46db246-cluster-monitoring-operator-tls podName:e74b93a9-2fe3-43bd-81a4-b928b46db246 nodeName:}" failed. No retries permitted until 2026-09-16 02:17:14.608517195 +0000 UTC m=+254.992032423 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e74b93a9-2fe3-43bd-81a4-b928b46db246-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-s5pxd" (UID: "e74b93a9-2fe3-43bd-81a4-b928b46db246") : secret "cluster-monitoring-operator-tls" not found
Sep 16 02:17:12.608573 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:17:12.608554    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/91fb29dc-53d2-4a7e-adc8-365d8791dfbc-metrics-certs podName:91fb29dc-53d2-4a7e-adc8-365d8791dfbc nodeName:}" failed. No retries permitted until 2026-09-16 02:17:14.608544654 +0000 UTC m=+254.992059878 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/91fb29dc-53d2-4a7e-adc8-365d8791dfbc-metrics-certs") pod "router-default-758df9dfb-pnlzd" (UID: "91fb29dc-53d2-4a7e-adc8-365d8791dfbc") : secret "router-metrics-certs-default" not found
Sep 16 02:17:12.708860 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:12.708820    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5fe7f38e-ec3a-487f-8011-2744a5e25f2f-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-s9rbv\" (UID: \"5fe7f38e-ec3a-487f-8011-2744a5e25f2f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-s9rbv"
Sep 16 02:17:12.710104 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:17:12.709109    2579 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 16 02:17:12.710104 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:17:12.709189    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5fe7f38e-ec3a-487f-8011-2744a5e25f2f-samples-operator-tls podName:5fe7f38e-ec3a-487f-8011-2744a5e25f2f nodeName:}" failed. No retries permitted until 2026-09-16 02:17:14.709165655 +0000 UTC m=+255.092680881 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/5fe7f38e-ec3a-487f-8011-2744a5e25f2f-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-s9rbv" (UID: "5fe7f38e-ec3a-487f-8011-2744a5e25f2f") : secret "samples-operator-tls" not found
Sep 16 02:17:14.254239 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:14.254210    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:17:14.624436 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:14.624339    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/91fb29dc-53d2-4a7e-adc8-365d8791dfbc-service-ca-bundle\") pod \"router-default-758df9dfb-pnlzd\" (UID: \"91fb29dc-53d2-4a7e-adc8-365d8791dfbc\") " pod="openshift-ingress/router-default-758df9dfb-pnlzd"
Sep 16 02:17:14.624436 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:14.624387    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/91fb29dc-53d2-4a7e-adc8-365d8791dfbc-metrics-certs\") pod \"router-default-758df9dfb-pnlzd\" (UID: \"91fb29dc-53d2-4a7e-adc8-365d8791dfbc\") " pod="openshift-ingress/router-default-758df9dfb-pnlzd"
Sep 16 02:17:14.624436 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:14.624410    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e74b93a9-2fe3-43bd-81a4-b928b46db246-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-s5pxd\" (UID: \"e74b93a9-2fe3-43bd-81a4-b928b46db246\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-s5pxd"
Sep 16 02:17:14.624738 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:17:14.624542    2579 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 16 02:17:14.624738 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:17:14.624542    2579 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 16 02:17:14.624738 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:17:14.624543    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/91fb29dc-53d2-4a7e-adc8-365d8791dfbc-service-ca-bundle podName:91fb29dc-53d2-4a7e-adc8-365d8791dfbc nodeName:}" failed. No retries permitted until 2026-09-16 02:17:18.624520562 +0000 UTC m=+259.008035788 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/91fb29dc-53d2-4a7e-adc8-365d8791dfbc-service-ca-bundle") pod "router-default-758df9dfb-pnlzd" (UID: "91fb29dc-53d2-4a7e-adc8-365d8791dfbc") : configmap references non-existent config key: service-ca.crt
Sep 16 02:17:14.624738 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:17:14.624729    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e74b93a9-2fe3-43bd-81a4-b928b46db246-cluster-monitoring-operator-tls podName:e74b93a9-2fe3-43bd-81a4-b928b46db246 nodeName:}" failed. No retries permitted until 2026-09-16 02:17:18.62471431 +0000 UTC m=+259.008229539 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e74b93a9-2fe3-43bd-81a4-b928b46db246-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-s5pxd" (UID: "e74b93a9-2fe3-43bd-81a4-b928b46db246") : secret "cluster-monitoring-operator-tls" not found
Sep 16 02:17:14.624886 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:17:14.624745    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/91fb29dc-53d2-4a7e-adc8-365d8791dfbc-metrics-certs podName:91fb29dc-53d2-4a7e-adc8-365d8791dfbc nodeName:}" failed. No retries permitted until 2026-09-16 02:17:18.624736214 +0000 UTC m=+259.008251437 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/91fb29dc-53d2-4a7e-adc8-365d8791dfbc-metrics-certs") pod "router-default-758df9dfb-pnlzd" (UID: "91fb29dc-53d2-4a7e-adc8-365d8791dfbc") : secret "router-metrics-certs-default" not found
Sep 16 02:17:14.725749 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:14.725708    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5fe7f38e-ec3a-487f-8011-2744a5e25f2f-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-s9rbv\" (UID: \"5fe7f38e-ec3a-487f-8011-2744a5e25f2f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-s9rbv"
Sep 16 02:17:14.725937 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:17:14.725894    2579 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 16 02:17:14.725998 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:17:14.725974    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5fe7f38e-ec3a-487f-8011-2744a5e25f2f-samples-operator-tls podName:5fe7f38e-ec3a-487f-8011-2744a5e25f2f nodeName:}" failed. No retries permitted until 2026-09-16 02:17:18.725955839 +0000 UTC m=+259.109471101 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/5fe7f38e-ec3a-487f-8011-2744a5e25f2f-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-s9rbv" (UID: "5fe7f38e-ec3a-487f-8011-2744a5e25f2f") : secret "samples-operator-tls" not found
Sep 16 02:17:15.090307 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:15.090270    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:17:15.191904 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:15.191873    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:17:15.290213 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:15.290181    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:17:15.850248 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:15.850206    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-bmj9c" event={"ID":"111d4c95-703e-4afe-bfca-fbf61d55154f","Type":"ContainerStarted","Data":"e18c2ebfc4eb7b6ed7011b031f7f35449fcb4c4ec0049896c639fa8972643356"}
Sep 16 02:17:15.852055 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:15.852033    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-c8q8z_ebf846fa-af7d-4fdd-9662-b7c533c36ddd/console-operator/0.log"
Sep 16 02:17:15.852174 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:15.852074    2579 generic.go:356] "Generic (PLEG): container finished" podID="ebf846fa-af7d-4fdd-9662-b7c533c36ddd" containerID="f8d14a0e2e92d06af60cfb941d2d14c8ef84de4fc1ad31564f6c7eeed4a8d33b" exitCode=255
Sep 16 02:17:15.852174 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:15.852153    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-c8q8z" event={"ID":"ebf846fa-af7d-4fdd-9662-b7c533c36ddd","Type":"ContainerDied","Data":"f8d14a0e2e92d06af60cfb941d2d14c8ef84de4fc1ad31564f6c7eeed4a8d33b"}
Sep 16 02:17:15.852329 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:15.852314    2579 scope.go:117] "RemoveContainer" containerID="f8d14a0e2e92d06af60cfb941d2d14c8ef84de4fc1ad31564f6c7eeed4a8d33b"
Sep 16 02:17:15.853897 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:15.853879    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-hdtv6" event={"ID":"d9ffb572-8b90-4503-82c1-84deef377ca7","Type":"ContainerStarted","Data":"ff61760bba35cf7fdbfa297a36f13fe8fadff711912d50acf4faebd39beca134"}
Sep 16 02:17:15.864954 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:15.864920    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-bmj9c" podStartSLOduration=2.629521769 podStartE2EDuration="5.864907942s" totalImagesPullingTime="3.235386173s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 02:17:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 02:17:15.863592191 +0000 UTC m=+256.247107435" watchObservedRunningTime="2026-09-16 02:17:15.864907942 +0000 UTC m=+256.248423188"
Sep 16 02:17:15.895430 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:15.895374    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5844b856db-hdtv6" podStartSLOduration=2.687916773 podStartE2EDuration="5.895353469s" totalImagesPullingTime="3.207436696s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 02:17:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 02:17:15.89387204 +0000 UTC m=+256.277387358" watchObservedRunningTime="2026-09-16 02:17:15.895353469 +0000 UTC m=+256.278868718"
Sep 16 02:17:16.090054 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:16.090018    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:17:16.190071 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:16.190033    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-84f2w"]
Sep 16 02:17:16.193167 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:16.193144    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-84f2w"
Sep 16 02:17:16.196823 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:16.196590    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 16 02:17:16.196823 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:16.196623    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-8h898\""
Sep 16 02:17:16.196823 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:16.196784    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 16 02:17:16.207140 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:16.207117    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:17:16.212013 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:16.211982    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-84f2w"]
Sep 16 02:17:16.279509 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:16.279460    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:17:16.340005 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:16.339978    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/aefd9717-5d49-4029-96d1-e195cfc682ff-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-84f2w\" (UID: \"aefd9717-5d49-4029-96d1-e195cfc682ff\") " pod="openshift-insights/insights-runtime-extractor-84f2w"
Sep 16 02:17:16.340396 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:16.340018    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/aefd9717-5d49-4029-96d1-e195cfc682ff-data-volume\") pod \"insights-runtime-extractor-84f2w\" (UID: \"aefd9717-5d49-4029-96d1-e195cfc682ff\") " pod="openshift-insights/insights-runtime-extractor-84f2w"
Sep 16 02:17:16.340396 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:16.340039    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/aefd9717-5d49-4029-96d1-e195cfc682ff-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-84f2w\" (UID: \"aefd9717-5d49-4029-96d1-e195cfc682ff\") " pod="openshift-insights/insights-runtime-extractor-84f2w"
Sep 16 02:17:16.340396 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:16.340105    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wh82p\" (UniqueName: \"kubernetes.io/projected/aefd9717-5d49-4029-96d1-e195cfc682ff-kube-api-access-wh82p\") pod \"insights-runtime-extractor-84f2w\" (UID: \"aefd9717-5d49-4029-96d1-e195cfc682ff\") " pod="openshift-insights/insights-runtime-extractor-84f2w"
Sep 16 02:17:16.340396 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:16.340144    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/aefd9717-5d49-4029-96d1-e195cfc682ff-crio-socket\") pod \"insights-runtime-extractor-84f2w\" (UID: \"aefd9717-5d49-4029-96d1-e195cfc682ff\") " pod="openshift-insights/insights-runtime-extractor-84f2w"
Sep 16 02:17:16.441398 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:16.441318    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/aefd9717-5d49-4029-96d1-e195cfc682ff-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-84f2w\" (UID: \"aefd9717-5d49-4029-96d1-e195cfc682ff\") " pod="openshift-insights/insights-runtime-extractor-84f2w"
Sep 16 02:17:16.441398 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:16.441376    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/aefd9717-5d49-4029-96d1-e195cfc682ff-data-volume\") pod \"insights-runtime-extractor-84f2w\" (UID: \"aefd9717-5d49-4029-96d1-e195cfc682ff\") " pod="openshift-insights/insights-runtime-extractor-84f2w"
Sep 16 02:17:16.441633 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:16.441399    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/aefd9717-5d49-4029-96d1-e195cfc682ff-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-84f2w\" (UID: \"aefd9717-5d49-4029-96d1-e195cfc682ff\") " pod="openshift-insights/insights-runtime-extractor-84f2w"
Sep 16 02:17:16.441633 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:16.441431    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wh82p\" (UniqueName: \"kubernetes.io/projected/aefd9717-5d49-4029-96d1-e195cfc682ff-kube-api-access-wh82p\") pod \"insights-runtime-extractor-84f2w\" (UID: \"aefd9717-5d49-4029-96d1-e195cfc682ff\") " pod="openshift-insights/insights-runtime-extractor-84f2w"
Sep 16 02:17:16.441633 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:17:16.441447    2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 16 02:17:16.441633 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:16.441539    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/aefd9717-5d49-4029-96d1-e195cfc682ff-crio-socket\") pod \"insights-runtime-extractor-84f2w\" (UID: \"aefd9717-5d49-4029-96d1-e195cfc682ff\") " pod="openshift-insights/insights-runtime-extractor-84f2w"
Sep 16 02:17:16.441633 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:17:16.441550    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/aefd9717-5d49-4029-96d1-e195cfc682ff-kube-rbac-proxy-cm podName:aefd9717-5d49-4029-96d1-e195cfc682ff nodeName:}" failed. No retries permitted until 2026-09-16 02:17:16.941530218 +0000 UTC m=+257.325045446 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/aefd9717-5d49-4029-96d1-e195cfc682ff-kube-rbac-proxy-cm") pod "insights-runtime-extractor-84f2w" (UID: "aefd9717-5d49-4029-96d1-e195cfc682ff") : configmap "kube-rbac-proxy" not found
Sep 16 02:17:16.441633 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:16.441456    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/aefd9717-5d49-4029-96d1-e195cfc682ff-crio-socket\") pod \"insights-runtime-extractor-84f2w\" (UID: \"aefd9717-5d49-4029-96d1-e195cfc682ff\") " pod="openshift-insights/insights-runtime-extractor-84f2w"
Sep 16 02:17:16.441633 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:17:16.441561    2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 16 02:17:16.441633 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:17:16.441623    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aefd9717-5d49-4029-96d1-e195cfc682ff-insights-runtime-extractor-tls podName:aefd9717-5d49-4029-96d1-e195cfc682ff nodeName:}" failed. No retries permitted until 2026-09-16 02:17:16.941606478 +0000 UTC m=+257.325121706 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/aefd9717-5d49-4029-96d1-e195cfc682ff-insights-runtime-extractor-tls") pod "insights-runtime-extractor-84f2w" (UID: "aefd9717-5d49-4029-96d1-e195cfc682ff") : secret "insights-runtime-extractor-tls" not found
Sep 16 02:17:16.441933 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:16.441740    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/aefd9717-5d49-4029-96d1-e195cfc682ff-data-volume\") pod \"insights-runtime-extractor-84f2w\" (UID: \"aefd9717-5d49-4029-96d1-e195cfc682ff\") " pod="openshift-insights/insights-runtime-extractor-84f2w"
Sep 16 02:17:16.451746 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:16.451723    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wh82p\" (UniqueName: \"kubernetes.io/projected/aefd9717-5d49-4029-96d1-e195cfc682ff-kube-api-access-wh82p\") pod \"insights-runtime-extractor-84f2w\" (UID: \"aefd9717-5d49-4029-96d1-e195cfc682ff\") " pod="openshift-insights/insights-runtime-extractor-84f2w"
Sep 16 02:17:16.521649 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:16.521621    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:17:16.574709 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:16.574680    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:17:16.859067 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:16.858998    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-c8q8z_ebf846fa-af7d-4fdd-9662-b7c533c36ddd/console-operator/1.log"
Sep 16 02:17:16.859389 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:16.859375    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-c8q8z_ebf846fa-af7d-4fdd-9662-b7c533c36ddd/console-operator/0.log"
Sep 16 02:17:16.859440 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:16.859409    2579 generic.go:356] "Generic (PLEG): container finished" podID="ebf846fa-af7d-4fdd-9662-b7c533c36ddd" containerID="53573f0eedd3204a2ac5aea4deafae460c834f1a9c1e509e515b8d69c15044fe" exitCode=255
Sep 16 02:17:16.859523 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:16.859499    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-c8q8z" event={"ID":"ebf846fa-af7d-4fdd-9662-b7c533c36ddd","Type":"ContainerDied","Data":"53573f0eedd3204a2ac5aea4deafae460c834f1a9c1e509e515b8d69c15044fe"}
Sep 16 02:17:16.859573 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:16.859555    2579 scope.go:117] "RemoveContainer" containerID="f8d14a0e2e92d06af60cfb941d2d14c8ef84de4fc1ad31564f6c7eeed4a8d33b"
Sep 16 02:17:16.859800 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:16.859782    2579 scope.go:117] "RemoveContainer" containerID="53573f0eedd3204a2ac5aea4deafae460c834f1a9c1e509e515b8d69c15044fe"
Sep 16 02:17:16.859975 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:17:16.859957    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-696884fb9c-c8q8z_openshift-console-operator(ebf846fa-af7d-4fdd-9662-b7c533c36ddd)\"" pod="openshift-console-operator/console-operator-696884fb9c-c8q8z" podUID="ebf846fa-af7d-4fdd-9662-b7c533c36ddd"
Sep 16 02:17:16.866985 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:16.866959    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:17:16.945589 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:16.945562    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/aefd9717-5d49-4029-96d1-e195cfc682ff-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-84f2w\" (UID: \"aefd9717-5d49-4029-96d1-e195cfc682ff\") " pod="openshift-insights/insights-runtime-extractor-84f2w"
Sep 16 02:17:16.945732 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:17:16.945715    2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 16 02:17:16.945797 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:17:16.945784    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aefd9717-5d49-4029-96d1-e195cfc682ff-insights-runtime-extractor-tls podName:aefd9717-5d49-4029-96d1-e195cfc682ff nodeName:}" failed. No retries permitted until 2026-09-16 02:17:17.945763329 +0000 UTC m=+258.329278555 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/aefd9717-5d49-4029-96d1-e195cfc682ff-insights-runtime-extractor-tls") pod "insights-runtime-extractor-84f2w" (UID: "aefd9717-5d49-4029-96d1-e195cfc682ff") : secret "insights-runtime-extractor-tls" not found
Sep 16 02:17:16.945931 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:16.945915    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/aefd9717-5d49-4029-96d1-e195cfc682ff-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-84f2w\" (UID: \"aefd9717-5d49-4029-96d1-e195cfc682ff\") " pod="openshift-insights/insights-runtime-extractor-84f2w"
Sep 16 02:17:16.946047 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:17:16.946031    2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 16 02:17:16.946125 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:17:16.946087    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/aefd9717-5d49-4029-96d1-e195cfc682ff-kube-rbac-proxy-cm podName:aefd9717-5d49-4029-96d1-e195cfc682ff nodeName:}" failed. No retries permitted until 2026-09-16 02:17:17.946069955 +0000 UTC m=+258.329585181 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/aefd9717-5d49-4029-96d1-e195cfc682ff-kube-rbac-proxy-cm") pod "insights-runtime-extractor-84f2w" (UID: "aefd9717-5d49-4029-96d1-e195cfc682ff") : configmap "kube-rbac-proxy" not found
Sep 16 02:17:16.968151 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:16.968125    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:17:17.267835 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:17.267798    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:17:17.369002 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:17.368971    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:17:17.864281 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:17.864254    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-c8q8z_ebf846fa-af7d-4fdd-9662-b7c533c36ddd/console-operator/1.log"
Sep 16 02:17:17.864575 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:17.864560    2579 scope.go:117] "RemoveContainer" containerID="53573f0eedd3204a2ac5aea4deafae460c834f1a9c1e509e515b8d69c15044fe"
Sep 16 02:17:17.864718 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:17:17.864699    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-696884fb9c-c8q8z_openshift-console-operator(ebf846fa-af7d-4fdd-9662-b7c533c36ddd)\"" pod="openshift-console-operator/console-operator-696884fb9c-c8q8z" podUID="ebf846fa-af7d-4fdd-9662-b7c533c36ddd"
Sep 16 02:17:17.955809 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:17.955779    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/aefd9717-5d49-4029-96d1-e195cfc682ff-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-84f2w\" (UID: \"aefd9717-5d49-4029-96d1-e195cfc682ff\") " pod="openshift-insights/insights-runtime-extractor-84f2w"
Sep 16 02:17:17.955965 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:17.955837    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/aefd9717-5d49-4029-96d1-e195cfc682ff-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-84f2w\" (UID: \"aefd9717-5d49-4029-96d1-e195cfc682ff\") " pod="openshift-insights/insights-runtime-extractor-84f2w"
Sep 16 02:17:17.955965 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:17:17.955920    2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 16 02:17:17.956082 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:17:17.955980    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aefd9717-5d49-4029-96d1-e195cfc682ff-insights-runtime-extractor-tls podName:aefd9717-5d49-4029-96d1-e195cfc682ff nodeName:}" failed. No retries permitted until 2026-09-16 02:17:19.955961652 +0000 UTC m=+260.339476878 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/aefd9717-5d49-4029-96d1-e195cfc682ff-insights-runtime-extractor-tls") pod "insights-runtime-extractor-84f2w" (UID: "aefd9717-5d49-4029-96d1-e195cfc682ff") : secret "insights-runtime-extractor-tls" not found
Sep 16 02:17:17.956082 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:17:17.955922    2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 16 02:17:17.956174 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:17:17.956098    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/aefd9717-5d49-4029-96d1-e195cfc682ff-kube-rbac-proxy-cm podName:aefd9717-5d49-4029-96d1-e195cfc682ff nodeName:}" failed. No retries permitted until 2026-09-16 02:17:19.956086857 +0000 UTC m=+260.339602107 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/aefd9717-5d49-4029-96d1-e195cfc682ff-kube-rbac-proxy-cm") pod "insights-runtime-extractor-84f2w" (UID: "aefd9717-5d49-4029-96d1-e195cfc682ff") : configmap "kube-rbac-proxy" not found
Sep 16 02:17:18.448661 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:18.448621    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:17:18.501284 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:18.501256    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:17:18.663008 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:18.662974    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/91fb29dc-53d2-4a7e-adc8-365d8791dfbc-service-ca-bundle\") pod \"router-default-758df9dfb-pnlzd\" (UID: \"91fb29dc-53d2-4a7e-adc8-365d8791dfbc\") " pod="openshift-ingress/router-default-758df9dfb-pnlzd"
Sep 16 02:17:18.663154 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:18.663019    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/91fb29dc-53d2-4a7e-adc8-365d8791dfbc-metrics-certs\") pod \"router-default-758df9dfb-pnlzd\" (UID: \"91fb29dc-53d2-4a7e-adc8-365d8791dfbc\") " pod="openshift-ingress/router-default-758df9dfb-pnlzd"
Sep 16 02:17:18.663154 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:18.663043    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e74b93a9-2fe3-43bd-81a4-b928b46db246-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-s5pxd\" (UID: \"e74b93a9-2fe3-43bd-81a4-b928b46db246\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-s5pxd"
Sep 16 02:17:18.663154 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:17:18.663134    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/91fb29dc-53d2-4a7e-adc8-365d8791dfbc-service-ca-bundle podName:91fb29dc-53d2-4a7e-adc8-365d8791dfbc nodeName:}" failed. No retries permitted until 2026-09-16 02:17:26.663116896 +0000 UTC m=+267.046632124 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/91fb29dc-53d2-4a7e-adc8-365d8791dfbc-service-ca-bundle") pod "router-default-758df9dfb-pnlzd" (UID: "91fb29dc-53d2-4a7e-adc8-365d8791dfbc") : configmap references non-existent config key: service-ca.crt
Sep 16 02:17:18.663154 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:17:18.663148    2579 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 16 02:17:18.663304 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:17:18.663205    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e74b93a9-2fe3-43bd-81a4-b928b46db246-cluster-monitoring-operator-tls podName:e74b93a9-2fe3-43bd-81a4-b928b46db246 nodeName:}" failed. No retries permitted until 2026-09-16 02:17:26.663186979 +0000 UTC m=+267.046702216 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e74b93a9-2fe3-43bd-81a4-b928b46db246-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-s5pxd" (UID: "e74b93a9-2fe3-43bd-81a4-b928b46db246") : secret "cluster-monitoring-operator-tls" not found
Sep 16 02:17:18.663304 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:17:18.663236    2579 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 16 02:17:18.663304 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:17:18.663295    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/91fb29dc-53d2-4a7e-adc8-365d8791dfbc-metrics-certs podName:91fb29dc-53d2-4a7e-adc8-365d8791dfbc nodeName:}" failed. No retries permitted until 2026-09-16 02:17:26.66328338 +0000 UTC m=+267.046798621 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/91fb29dc-53d2-4a7e-adc8-365d8791dfbc-metrics-certs") pod "router-default-758df9dfb-pnlzd" (UID: "91fb29dc-53d2-4a7e-adc8-365d8791dfbc") : secret "router-metrics-certs-default" not found
Sep 16 02:17:18.763558 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:18.763464    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5fe7f38e-ec3a-487f-8011-2744a5e25f2f-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-s9rbv\" (UID: \"5fe7f38e-ec3a-487f-8011-2744a5e25f2f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-s9rbv"
Sep 16 02:17:18.763700 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:17:18.763610    2579 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 16 02:17:18.763700 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:17:18.763672    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5fe7f38e-ec3a-487f-8011-2744a5e25f2f-samples-operator-tls podName:5fe7f38e-ec3a-487f-8011-2744a5e25f2f nodeName:}" failed. No retries permitted until 2026-09-16 02:17:26.763654208 +0000 UTC m=+267.147169434 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/5fe7f38e-ec3a-487f-8011-2744a5e25f2f-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-s9rbv" (UID: "5fe7f38e-ec3a-487f-8011-2744a5e25f2f") : secret "samples-operator-tls" not found
Sep 16 02:17:18.764359 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:18.764336    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-9522z_835b007c-0c9a-4c8f-94dd-68f8a3b3a137/dns-node-resolver/0.log"
Sep 16 02:17:19.238543 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:19.238506    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:17:19.964101 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:19.964076    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-kwdns_496c15b3-4d67-473f-b894-2e66b47df562/node-ca/0.log"
Sep 16 02:17:19.974415 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:19.974387    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/aefd9717-5d49-4029-96d1-e195cfc682ff-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-84f2w\" (UID: \"aefd9717-5d49-4029-96d1-e195cfc682ff\") " pod="openshift-insights/insights-runtime-extractor-84f2w"
Sep 16 02:17:19.974540 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:19.974518    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/aefd9717-5d49-4029-96d1-e195cfc682ff-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-84f2w\" (UID: \"aefd9717-5d49-4029-96d1-e195cfc682ff\") " pod="openshift-insights/insights-runtime-extractor-84f2w"
Sep 16 02:17:19.974593 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:17:19.974551    2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 16 02:17:19.974593 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:17:19.974585    2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 16 02:17:19.974655 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:17:19.974611    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aefd9717-5d49-4029-96d1-e195cfc682ff-insights-runtime-extractor-tls podName:aefd9717-5d49-4029-96d1-e195cfc682ff nodeName:}" failed. No retries permitted until 2026-09-16 02:17:23.974593562 +0000 UTC m=+264.358108812 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/aefd9717-5d49-4029-96d1-e195cfc682ff-insights-runtime-extractor-tls") pod "insights-runtime-extractor-84f2w" (UID: "aefd9717-5d49-4029-96d1-e195cfc682ff") : secret "insights-runtime-extractor-tls" not found
Sep 16 02:17:19.974655 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:17:19.974626    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/aefd9717-5d49-4029-96d1-e195cfc682ff-kube-rbac-proxy-cm podName:aefd9717-5d49-4029-96d1-e195cfc682ff nodeName:}" failed. No retries permitted until 2026-09-16 02:17:23.974620368 +0000 UTC m=+264.358135591 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/aefd9717-5d49-4029-96d1-e195cfc682ff-kube-rbac-proxy-cm") pod "insights-runtime-extractor-84f2w" (UID: "aefd9717-5d49-4029-96d1-e195cfc682ff") : configmap "kube-rbac-proxy" not found
Sep 16 02:17:20.964607 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:20.964578    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-vbpq2_5caaf390-c372-40a3-8303-e99ab52b762b/serve-healthcheck-canary/0.log"
Sep 16 02:17:21.233743 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:21.233719    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-696884fb9c-c8q8z"
Sep 16 02:17:21.233743 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:21.233761    2579 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-696884fb9c-c8q8z"
Sep 16 02:17:21.234061 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:21.234049    2579 scope.go:117] "RemoveContainer" containerID="53573f0eedd3204a2ac5aea4deafae460c834f1a9c1e509e515b8d69c15044fe"
Sep 16 02:17:21.234208 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:17:21.234191    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-696884fb9c-c8q8z_openshift-console-operator(ebf846fa-af7d-4fdd-9662-b7c533c36ddd)\"" pod="openshift-console-operator/console-operator-696884fb9c-c8q8z" podUID="ebf846fa-af7d-4fdd-9662-b7c533c36ddd"
Sep 16 02:17:22.270464 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:22.270430    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:17:22.666536 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:22.666435    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:17:22.820674 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:22.820632    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:17:22.893152 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:22.893109    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:17:22.993636 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:22.993602    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:17:23.055114 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:23.055077    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:17:23.962488 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:23.962435    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:17:24.008814 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:24.008785    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/aefd9717-5d49-4029-96d1-e195cfc682ff-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-84f2w\" (UID: \"aefd9717-5d49-4029-96d1-e195cfc682ff\") " pod="openshift-insights/insights-runtime-extractor-84f2w"
Sep 16 02:17:24.008954 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:24.008826    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/aefd9717-5d49-4029-96d1-e195cfc682ff-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-84f2w\" (UID: \"aefd9717-5d49-4029-96d1-e195cfc682ff\") " pod="openshift-insights/insights-runtime-extractor-84f2w"
Sep 16 02:17:24.008954 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:17:24.008905    2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 16 02:17:24.009023 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:17:24.008966    2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 16 02:17:24.009023 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:17:24.008971    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/aefd9717-5d49-4029-96d1-e195cfc682ff-kube-rbac-proxy-cm podName:aefd9717-5d49-4029-96d1-e195cfc682ff nodeName:}" failed. No retries permitted until 2026-09-16 02:17:32.008954828 +0000 UTC m=+272.392470056 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/aefd9717-5d49-4029-96d1-e195cfc682ff-kube-rbac-proxy-cm") pod "insights-runtime-extractor-84f2w" (UID: "aefd9717-5d49-4029-96d1-e195cfc682ff") : configmap "kube-rbac-proxy" not found
Sep 16 02:17:24.009095 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:17:24.009031    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aefd9717-5d49-4029-96d1-e195cfc682ff-insights-runtime-extractor-tls podName:aefd9717-5d49-4029-96d1-e195cfc682ff nodeName:}" failed. No retries permitted until 2026-09-16 02:17:32.009015044 +0000 UTC m=+272.392530273 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/aefd9717-5d49-4029-96d1-e195cfc682ff-insights-runtime-extractor-tls") pod "insights-runtime-extractor-84f2w" (UID: "aefd9717-5d49-4029-96d1-e195cfc682ff") : secret "insights-runtime-extractor-tls" not found
Sep 16 02:17:24.362653 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:24.362558    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:17:24.565949 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:24.565911    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:17:24.759645 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:24.759599    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:17:24.962568 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:24.962535    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:17:25.613440 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:25.613404    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:17:25.954260 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:25.954225    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:17:26.003335 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:26.003304    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:17:26.365532 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:26.365416    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:17:26.562439 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:26.562409    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:17:26.611801 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:26.611764    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:17:26.732188 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:26.732153    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/91fb29dc-53d2-4a7e-adc8-365d8791dfbc-metrics-certs\") pod \"router-default-758df9dfb-pnlzd\" (UID: \"91fb29dc-53d2-4a7e-adc8-365d8791dfbc\") " pod="openshift-ingress/router-default-758df9dfb-pnlzd"
Sep 16 02:17:26.732365 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:26.732191    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e74b93a9-2fe3-43bd-81a4-b928b46db246-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-s5pxd\" (UID: \"e74b93a9-2fe3-43bd-81a4-b928b46db246\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-s5pxd"
Sep 16 02:17:26.732365 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:17:26.732286    2579 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 16 02:17:26.732365 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:17:26.732287    2579 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 16 02:17:26.732365 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:17:26.732344    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e74b93a9-2fe3-43bd-81a4-b928b46db246-cluster-monitoring-operator-tls podName:e74b93a9-2fe3-43bd-81a4-b928b46db246 nodeName:}" failed. No retries permitted until 2026-09-16 02:17:42.732327595 +0000 UTC m=+283.115842839 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e74b93a9-2fe3-43bd-81a4-b928b46db246-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-s5pxd" (UID: "e74b93a9-2fe3-43bd-81a4-b928b46db246") : secret "cluster-monitoring-operator-tls" not found
Sep 16 02:17:26.732365 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:17:26.732363    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/91fb29dc-53d2-4a7e-adc8-365d8791dfbc-metrics-certs podName:91fb29dc-53d2-4a7e-adc8-365d8791dfbc nodeName:}" failed. No retries permitted until 2026-09-16 02:17:42.732348983 +0000 UTC m=+283.115864206 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/91fb29dc-53d2-4a7e-adc8-365d8791dfbc-metrics-certs") pod "router-default-758df9dfb-pnlzd" (UID: "91fb29dc-53d2-4a7e-adc8-365d8791dfbc") : secret "router-metrics-certs-default" not found
Sep 16 02:17:26.732564 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:26.732427    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/91fb29dc-53d2-4a7e-adc8-365d8791dfbc-service-ca-bundle\") pod \"router-default-758df9dfb-pnlzd\" (UID: \"91fb29dc-53d2-4a7e-adc8-365d8791dfbc\") " pod="openshift-ingress/router-default-758df9dfb-pnlzd"
Sep 16 02:17:26.732564 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:17:26.732530    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/91fb29dc-53d2-4a7e-adc8-365d8791dfbc-service-ca-bundle podName:91fb29dc-53d2-4a7e-adc8-365d8791dfbc nodeName:}" failed. No retries permitted until 2026-09-16 02:17:42.732522243 +0000 UTC m=+283.116037468 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/91fb29dc-53d2-4a7e-adc8-365d8791dfbc-service-ca-bundle") pod "router-default-758df9dfb-pnlzd" (UID: "91fb29dc-53d2-4a7e-adc8-365d8791dfbc") : configmap references non-existent config key: service-ca.crt
Sep 16 02:17:26.761517 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:26.761450    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:17:26.833772 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:26.833742    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5fe7f38e-ec3a-487f-8011-2744a5e25f2f-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-s9rbv\" (UID: \"5fe7f38e-ec3a-487f-8011-2744a5e25f2f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-s9rbv"
Sep 16 02:17:26.836228 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:26.836198    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5fe7f38e-ec3a-487f-8011-2744a5e25f2f-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-s9rbv\" (UID: \"5fe7f38e-ec3a-487f-8011-2744a5e25f2f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-s9rbv"
Sep 16 02:17:26.928219 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:26.928189    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-s9rbv"
Sep 16 02:17:26.961240 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:26.961212    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:17:27.193786 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:27.193759    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-s9rbv"]
Sep 16 02:17:27.896661 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:27.896626    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-s9rbv" event={"ID":"5fe7f38e-ec3a-487f-8011-2744a5e25f2f","Type":"ContainerStarted","Data":"37afd73ca775e91f4cc89738d024c441263f4e847bcc02248b9ead88c3bf4ec0"}
Sep 16 02:17:27.970237 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:27.970201    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:17:28.247732 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:28.247699    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:17:28.363735 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:28.363701    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:17:28.568039 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:28.567956    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:17:28.761464 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:28.761430    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:17:29.166042 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:29.166014    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:17:29.267226 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:29.267203    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:17:29.762874 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:29.762838    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:17:29.906969 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:29.906938    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-s9rbv" event={"ID":"5fe7f38e-ec3a-487f-8011-2744a5e25f2f","Type":"ContainerStarted","Data":"79d5e9212294414344adf9f5c1f3c023658306a43db2384142f031b7fd9a5bcc"}
Sep 16 02:17:29.907142 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:29.907017    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-s9rbv" event={"ID":"5fe7f38e-ec3a-487f-8011-2744a5e25f2f","Type":"ContainerStarted","Data":"4463904139aede4e4ebb272c4cac37f3da4a63bd80a3a59ece22aaf501187d22"}
Sep 16 02:17:29.920770 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:29.920726    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-s9rbv" podStartSLOduration=18.008539436 podStartE2EDuration="19.920713803s" totalImagesPullingTime="1.912174367s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 02:17:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 02:17:29.919638308 +0000 UTC m=+270.303153553" watchObservedRunningTime="2026-09-16 02:17:29.920713803 +0000 UTC m=+270.304229047"
Sep 16 02:17:30.168740 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:30.168643    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:17:30.563881 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:30.563854    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:17:30.965136 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:30.965105    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:17:31.359720 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:31.359642    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:17:31.765762 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:31.765721    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:17:32.075176 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:32.075097    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/aefd9717-5d49-4029-96d1-e195cfc682ff-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-84f2w\" (UID: \"aefd9717-5d49-4029-96d1-e195cfc682ff\") " pod="openshift-insights/insights-runtime-extractor-84f2w"
Sep 16 02:17:32.075176 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:32.075148    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/aefd9717-5d49-4029-96d1-e195cfc682ff-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-84f2w\" (UID: \"aefd9717-5d49-4029-96d1-e195cfc682ff\") " pod="openshift-insights/insights-runtime-extractor-84f2w"
Sep 16 02:17:32.075354 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:17:32.075267    2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 16 02:17:32.075354 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:17:32.075346    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aefd9717-5d49-4029-96d1-e195cfc682ff-insights-runtime-extractor-tls podName:aefd9717-5d49-4029-96d1-e195cfc682ff nodeName:}" failed. No retries permitted until 2026-09-16 02:17:48.075328038 +0000 UTC m=+288.458843263 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/aefd9717-5d49-4029-96d1-e195cfc682ff-insights-runtime-extractor-tls") pod "insights-runtime-extractor-84f2w" (UID: "aefd9717-5d49-4029-96d1-e195cfc682ff") : secret "insights-runtime-extractor-tls" not found
Sep 16 02:17:32.075797 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:32.075777    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/aefd9717-5d49-4029-96d1-e195cfc682ff-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-84f2w\" (UID: \"aefd9717-5d49-4029-96d1-e195cfc682ff\") " pod="openshift-insights/insights-runtime-extractor-84f2w"
Sep 16 02:17:32.165716 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:32.165680    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:17:32.595161 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:32.595132    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:17:32.763866 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:32.763827    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:17:33.164502 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:33.164451    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:17:33.365021 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:33.364987    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:17:35.096683 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:35.096655    2579 scope.go:117] "RemoveContainer" containerID="53573f0eedd3204a2ac5aea4deafae460c834f1a9c1e509e515b8d69c15044fe"
Sep 16 02:17:35.925732 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:35.925707    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-c8q8z_ebf846fa-af7d-4fdd-9662-b7c533c36ddd/console-operator/2.log"
Sep 16 02:17:35.926085 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:35.926070    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-c8q8z_ebf846fa-af7d-4fdd-9662-b7c533c36ddd/console-operator/1.log"
Sep 16 02:17:35.926149 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:35.926105    2579 generic.go:356] "Generic (PLEG): container finished" podID="ebf846fa-af7d-4fdd-9662-b7c533c36ddd" containerID="af8faa2a6f130af678d8d07d5ba3021db5df3934a3b04a67786e3417884e5e0e" exitCode=255
Sep 16 02:17:35.926184 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:35.926154    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-c8q8z" event={"ID":"ebf846fa-af7d-4fdd-9662-b7c533c36ddd","Type":"ContainerDied","Data":"af8faa2a6f130af678d8d07d5ba3021db5df3934a3b04a67786e3417884e5e0e"}
Sep 16 02:17:35.926217 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:35.926203    2579 scope.go:117] "RemoveContainer" containerID="53573f0eedd3204a2ac5aea4deafae460c834f1a9c1e509e515b8d69c15044fe"
Sep 16 02:17:35.926602 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:35.926582    2579 scope.go:117] "RemoveContainer" containerID="af8faa2a6f130af678d8d07d5ba3021db5df3934a3b04a67786e3417884e5e0e"
Sep 16 02:17:35.926797 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:17:35.926774    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-696884fb9c-c8q8z_openshift-console-operator(ebf846fa-af7d-4fdd-9662-b7c533c36ddd)\"" pod="openshift-console-operator/console-operator-696884fb9c-c8q8z" podUID="ebf846fa-af7d-4fdd-9662-b7c533c36ddd"
Sep 16 02:17:36.930272 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:36.930247    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-c8q8z_ebf846fa-af7d-4fdd-9662-b7c533c36ddd/console-operator/2.log"
Sep 16 02:17:36.995327 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:36.995285    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:17:37.043568 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:37.043536    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:17:37.929271 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:37.929240    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:17:38.028640 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:38.028605    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:17:40.612194 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:17:40.612162    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-jtz8l" podUID="8ff5e4be-2aa6-4203-83ac-e715f2f0f565"
Sep 16 02:17:40.612194 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:17:40.612161    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-q6tmq" podUID="8e7a4943-f231-4531-9e5d-4bcf2a844cf9"
Sep 16 02:17:40.612194 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:17:40.612170    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-649f6698d9-cmzbc" podUID="35bd8e04-3e55-44b6-b78e-83566dcf43cc"
Sep 16 02:17:40.941518 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:40.941494    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-jtz8l"
Sep 16 02:17:40.941665 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:40.941494    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-q6tmq"
Sep 16 02:17:40.941704 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:40.941494    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-649f6698d9-cmzbc"
Sep 16 02:17:41.234646 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:41.234554    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-696884fb9c-c8q8z"
Sep 16 02:17:41.234646 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:41.234647    2579 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-696884fb9c-c8q8z"
Sep 16 02:17:41.234930 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:41.234918    2579 scope.go:117] "RemoveContainer" containerID="af8faa2a6f130af678d8d07d5ba3021db5df3934a3b04a67786e3417884e5e0e"
Sep 16 02:17:41.235081 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:17:41.235062    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-696884fb9c-c8q8z_openshift-console-operator(ebf846fa-af7d-4fdd-9662-b7c533c36ddd)\"" pod="openshift-console-operator/console-operator-696884fb9c-c8q8z" podUID="ebf846fa-af7d-4fdd-9662-b7c533c36ddd"
Sep 16 02:17:42.759207 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:42.759167    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/91fb29dc-53d2-4a7e-adc8-365d8791dfbc-service-ca-bundle\") pod \"router-default-758df9dfb-pnlzd\" (UID: \"91fb29dc-53d2-4a7e-adc8-365d8791dfbc\") " pod="openshift-ingress/router-default-758df9dfb-pnlzd"
Sep 16 02:17:42.759585 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:42.759214    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/91fb29dc-53d2-4a7e-adc8-365d8791dfbc-metrics-certs\") pod \"router-default-758df9dfb-pnlzd\" (UID: \"91fb29dc-53d2-4a7e-adc8-365d8791dfbc\") " pod="openshift-ingress/router-default-758df9dfb-pnlzd"
Sep 16 02:17:42.759585 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:42.759242    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e74b93a9-2fe3-43bd-81a4-b928b46db246-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-s5pxd\" (UID: \"e74b93a9-2fe3-43bd-81a4-b928b46db246\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-s5pxd"
Sep 16 02:17:42.759877 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:42.759856    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/91fb29dc-53d2-4a7e-adc8-365d8791dfbc-service-ca-bundle\") pod \"router-default-758df9dfb-pnlzd\" (UID: \"91fb29dc-53d2-4a7e-adc8-365d8791dfbc\") " pod="openshift-ingress/router-default-758df9dfb-pnlzd"
Sep 16 02:17:42.761849 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:42.761808    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/91fb29dc-53d2-4a7e-adc8-365d8791dfbc-metrics-certs\") pod \"router-default-758df9dfb-pnlzd\" (UID: \"91fb29dc-53d2-4a7e-adc8-365d8791dfbc\") " pod="openshift-ingress/router-default-758df9dfb-pnlzd"
Sep 16 02:17:42.761849 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:42.761807    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e74b93a9-2fe3-43bd-81a4-b928b46db246-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-s5pxd\" (UID: \"e74b93a9-2fe3-43bd-81a4-b928b46db246\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-s5pxd"
Sep 16 02:17:43.026709 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:43.026634    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-s5pxd"
Sep 16 02:17:43.039053 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:43.039027    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-758df9dfb-pnlzd"
Sep 16 02:17:43.303321 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:43.303244    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cd954b897-s5pxd"]
Sep 16 02:17:43.306174 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:17:43.306144    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode74b93a9_2fe3_43bd_81a4_b928b46db246.slice/crio-f92c7765431e8396f89c1e41406e3568b3fd449137d2d0568c6ad33de830fe9f WatchSource:0}: Error finding container f92c7765431e8396f89c1e41406e3568b3fd449137d2d0568c6ad33de830fe9f: Status 404 returned error can't find the container with id f92c7765431e8396f89c1e41406e3568b3fd449137d2d0568c6ad33de830fe9f
Sep 16 02:17:43.328648 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:43.328627    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-758df9dfb-pnlzd"]
Sep 16 02:17:43.330517 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:17:43.330495    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91fb29dc_53d2_4a7e_adc8_365d8791dfbc.slice/crio-a3ccd0d1caad921d4b72bcbfd7ed00a11d21f9de89161f3f04aaab7772a83f03 WatchSource:0}: Error finding container a3ccd0d1caad921d4b72bcbfd7ed00a11d21f9de89161f3f04aaab7772a83f03: Status 404 returned error can't find the container with id a3ccd0d1caad921d4b72bcbfd7ed00a11d21f9de89161f3f04aaab7772a83f03
Sep 16 02:17:43.950827 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:43.950790    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-s5pxd" event={"ID":"e74b93a9-2fe3-43bd-81a4-b928b46db246","Type":"ContainerStarted","Data":"f92c7765431e8396f89c1e41406e3568b3fd449137d2d0568c6ad33de830fe9f"}
Sep 16 02:17:43.952157 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:43.952137    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-758df9dfb-pnlzd" event={"ID":"91fb29dc-53d2-4a7e-adc8-365d8791dfbc","Type":"ContainerStarted","Data":"aded7ae6ef25574f314ad59e410658b938d26e6bcb2e6192f51d5b21d6c9c780"}
Sep 16 02:17:43.952219 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:43.952171    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-758df9dfb-pnlzd" event={"ID":"91fb29dc-53d2-4a7e-adc8-365d8791dfbc","Type":"ContainerStarted","Data":"a3ccd0d1caad921d4b72bcbfd7ed00a11d21f9de89161f3f04aaab7772a83f03"}
Sep 16 02:17:43.970046 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:43.970003    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-758df9dfb-pnlzd" podStartSLOduration=33.969988215 podStartE2EDuration="33.969988215s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 02:17:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 02:17:43.968249749 +0000 UTC m=+284.351764993" watchObservedRunningTime="2026-09-16 02:17:43.969988215 +0000 UTC m=+284.353503457"
Sep 16 02:17:44.039571 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:44.039538    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-758df9dfb-pnlzd"
Sep 16 02:17:44.042113 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:44.042093    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-758df9dfb-pnlzd"
Sep 16 02:17:44.472261 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:44.472221    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/35bd8e04-3e55-44b6-b78e-83566dcf43cc-registry-tls\") pod \"image-registry-649f6698d9-cmzbc\" (UID: \"35bd8e04-3e55-44b6-b78e-83566dcf43cc\") " pod="openshift-image-registry/image-registry-649f6698d9-cmzbc"
Sep 16 02:17:44.472461 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:44.472281    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8e7a4943-f231-4531-9e5d-4bcf2a844cf9-metrics-tls\") pod \"dns-default-q6tmq\" (UID: \"8e7a4943-f231-4531-9e5d-4bcf2a844cf9\") " pod="openshift-dns/dns-default-q6tmq"
Sep 16 02:17:44.472461 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:44.472324    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8ff5e4be-2aa6-4203-83ac-e715f2f0f565-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-jtz8l\" (UID: \"8ff5e4be-2aa6-4203-83ac-e715f2f0f565\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-jtz8l"
Sep 16 02:17:44.475237 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:44.475210    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8e7a4943-f231-4531-9e5d-4bcf2a844cf9-metrics-tls\") pod \"dns-default-q6tmq\" (UID: \"8e7a4943-f231-4531-9e5d-4bcf2a844cf9\") " pod="openshift-dns/dns-default-q6tmq"
Sep 16 02:17:44.475330 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:44.475301    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/35bd8e04-3e55-44b6-b78e-83566dcf43cc-registry-tls\") pod \"image-registry-649f6698d9-cmzbc\" (UID: \"35bd8e04-3e55-44b6-b78e-83566dcf43cc\") " pod="openshift-image-registry/image-registry-649f6698d9-cmzbc"
Sep 16 02:17:44.475330 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:44.475315    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8ff5e4be-2aa6-4203-83ac-e715f2f0f565-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-jtz8l\" (UID: \"8ff5e4be-2aa6-4203-83ac-e715f2f0f565\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-jtz8l"
Sep 16 02:17:44.545311 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:44.545278    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-9j6zg\""
Sep 16 02:17:44.545311 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:44.545278    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-cfd5b\""
Sep 16 02:17:44.545549 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:44.545315    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-h6q9g\""
Sep 16 02:17:44.553289 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:44.553248    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-649f6698d9-cmzbc"
Sep 16 02:17:44.553289 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:44.553273    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-jtz8l"
Sep 16 02:17:44.553455 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:44.553292    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-q6tmq"
Sep 16 02:17:44.844273 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:44.844194    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-q6tmq"]
Sep 16 02:17:44.846978 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:44.846953    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-5dcbc7df6b-jtz8l"]
Sep 16 02:17:44.850195 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:44.850101    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-649f6698d9-cmzbc"]
Sep 16 02:17:44.956448 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:44.956419    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-758df9dfb-pnlzd"
Sep 16 02:17:44.957852 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:44.957828    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-758df9dfb-pnlzd"
Sep 16 02:17:45.258122 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:17:45.258085    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e7a4943_f231_4531_9e5d_4bcf2a844cf9.slice/crio-0de9be57fea5e91be447c9711a15723d2c464282be723b883f4d376a416d08ff WatchSource:0}: Error finding container 0de9be57fea5e91be447c9711a15723d2c464282be723b883f4d376a416d08ff: Status 404 returned error can't find the container with id 0de9be57fea5e91be447c9711a15723d2c464282be723b883f4d376a416d08ff
Sep 16 02:17:45.258563 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:17:45.258505    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ff5e4be_2aa6_4203_83ac_e715f2f0f565.slice/crio-d6b479c7741ec7b0d325af5a0eb405b855a1c0e9903d902b51ad1a29e5413218 WatchSource:0}: Error finding container d6b479c7741ec7b0d325af5a0eb405b855a1c0e9903d902b51ad1a29e5413218: Status 404 returned error can't find the container with id d6b479c7741ec7b0d325af5a0eb405b855a1c0e9903d902b51ad1a29e5413218
Sep 16 02:17:45.259386 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:17:45.259353    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35bd8e04_3e55_44b6_b78e_83566dcf43cc.slice/crio-4e63ff0b90b2fbaa15b20136b947ae18c6de5d3bf7418e5f8d70fda18a858480 WatchSource:0}: Error finding container 4e63ff0b90b2fbaa15b20136b947ae18c6de5d3bf7418e5f8d70fda18a858480: Status 404 returned error can't find the container with id 4e63ff0b90b2fbaa15b20136b947ae18c6de5d3bf7418e5f8d70fda18a858480
Sep 16 02:17:45.616093 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:45.615999    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:17:45.715686 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:45.715650    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:17:45.770139 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:45.770103    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:17:45.772778 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:45.772753    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-p2tqn"]
Sep 16 02:17:45.777078 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:45.777060    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-p2tqn"
Sep 16 02:17:45.779186 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:45.779164    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Sep 16 02:17:45.779186 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:45.779178    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-f8q9s\""
Sep 16 02:17:45.782287 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:45.782264    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-p2tqn"]
Sep 16 02:17:45.885758 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:45.885674    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/c46f41bf-3230-4da8-9ec8-45759ed3551d-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-p2tqn\" (UID: \"c46f41bf-3230-4da8-9ec8-45759ed3551d\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-p2tqn"
Sep 16 02:17:45.963711 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:45.963663    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-s5pxd" event={"ID":"e74b93a9-2fe3-43bd-81a4-b928b46db246","Type":"ContainerStarted","Data":"bf293f00de24524921be7f4033c80438707b08a9ab36a964db9bfe23cce3eb82"}
Sep 16 02:17:45.966934 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:45.966851    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-649f6698d9-cmzbc" event={"ID":"35bd8e04-3e55-44b6-b78e-83566dcf43cc","Type":"ContainerStarted","Data":"f236f6f23af08c0986c7b643d6084bdd009ba098fb5eacafe7d5aff80009ef0a"}
Sep 16 02:17:45.966934 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:45.966898    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-649f6698d9-cmzbc" event={"ID":"35bd8e04-3e55-44b6-b78e-83566dcf43cc","Type":"ContainerStarted","Data":"4e63ff0b90b2fbaa15b20136b947ae18c6de5d3bf7418e5f8d70fda18a858480"}
Sep 16 02:17:45.967157 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:45.967126    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-649f6698d9-cmzbc"
Sep 16 02:17:45.968497 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:45.968453    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-q6tmq" event={"ID":"8e7a4943-f231-4531-9e5d-4bcf2a844cf9","Type":"ContainerStarted","Data":"0de9be57fea5e91be447c9711a15723d2c464282be723b883f4d376a416d08ff"}
Sep 16 02:17:45.969993 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:45.969936    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-jtz8l" event={"ID":"8ff5e4be-2aa6-4203-83ac-e715f2f0f565","Type":"ContainerStarted","Data":"d6b479c7741ec7b0d325af5a0eb405b855a1c0e9903d902b51ad1a29e5413218"}
Sep 16 02:17:45.981376 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:45.981337    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-s5pxd" podStartSLOduration=33.983944173 podStartE2EDuration="35.981322686s" totalImagesPullingTime="1.997378513s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 02:17:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 02:17:45.98087825 +0000 UTC m=+286.364393511" watchObservedRunningTime="2026-09-16 02:17:45.981322686 +0000 UTC m=+286.364837933"
Sep 16 02:17:45.986448 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:45.986425    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/c46f41bf-3230-4da8-9ec8-45759ed3551d-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-p2tqn\" (UID: \"c46f41bf-3230-4da8-9ec8-45759ed3551d\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-p2tqn"
Sep 16 02:17:45.990288 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:45.989727    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/c46f41bf-3230-4da8-9ec8-45759ed3551d-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-p2tqn\" (UID: \"c46f41bf-3230-4da8-9ec8-45759ed3551d\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-p2tqn"
Sep 16 02:17:46.000117 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:46.000065    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-649f6698d9-cmzbc" podStartSLOduration=286.000048889 podStartE2EDuration="4m46.000048889s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 02:13:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 02:17:45.997864598 +0000 UTC m=+286.381379844" watchObservedRunningTime="2026-09-16 02:17:46.000048889 +0000 UTC m=+286.383564150"
Sep 16 02:17:46.004585 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:46.004552    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:17:46.087043 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:46.086765    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-p2tqn"
Sep 16 02:17:46.371456 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:46.371422    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-p2tqn"]
Sep 16 02:17:46.562581 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:17:46.562544    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc46f41bf_3230_4da8_9ec8_45759ed3551d.slice/crio-0f194593dfdf886031af60ffb824bb960f77299481e42b5b9cb4134a00ebff2f WatchSource:0}: Error finding container 0f194593dfdf886031af60ffb824bb960f77299481e42b5b9cb4134a00ebff2f: Status 404 returned error can't find the container with id 0f194593dfdf886031af60ffb824bb960f77299481e42b5b9cb4134a00ebff2f
Sep 16 02:17:46.975915 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:46.975874    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-p2tqn" event={"ID":"c46f41bf-3230-4da8-9ec8-45759ed3551d","Type":"ContainerStarted","Data":"0f194593dfdf886031af60ffb824bb960f77299481e42b5b9cb4134a00ebff2f"}
Sep 16 02:17:46.977750 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:46.977719    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-jtz8l" event={"ID":"8ff5e4be-2aa6-4203-83ac-e715f2f0f565","Type":"ContainerStarted","Data":"6f58f97aa34afa827a35c7d0eecf81b39b00a6e9a520bfaf38e8a9e6745e6f2e"}
Sep 16 02:17:46.992170 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:46.991992    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-jtz8l" podStartSLOduration=276.641456288 podStartE2EDuration="4m37.991972064s" totalImagesPullingTime="1.350515776s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 02:13:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 02:17:46.991852408 +0000 UTC m=+287.375367654" watchObservedRunningTime="2026-09-16 02:17:46.991972064 +0000 UTC m=+287.375487306"
Sep 16 02:17:47.109808 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:47.109774    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:17:47.164981 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:47.164952    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:17:47.982960 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:47.982923    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-q6tmq" event={"ID":"8e7a4943-f231-4531-9e5d-4bcf2a844cf9","Type":"ContainerStarted","Data":"a59459e494ff858ce75405299ceed86b4e997c7a5e5182d63ca63dfdaabe2c3e"}
Sep 16 02:17:47.983412 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:47.982974    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-q6tmq" event={"ID":"8e7a4943-f231-4531-9e5d-4bcf2a844cf9","Type":"ContainerStarted","Data":"abf717a976cd4b17e09e4f2d47b06920280e0bd6524073b499232999bcfb5669"}
Sep 16 02:17:47.999159 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:47.999107    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-q6tmq" podStartSLOduration=252.06961781 podStartE2EDuration="4m13.99909122s" totalImagesPullingTime="1.92947341s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 02:13:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 02:17:47.997702619 +0000 UTC m=+288.381217865" watchObservedRunningTime="2026-09-16 02:17:47.99909122 +0000 UTC m=+288.382606464"
Sep 16 02:17:48.105692 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:48.105655    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/aefd9717-5d49-4029-96d1-e195cfc682ff-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-84f2w\" (UID: \"aefd9717-5d49-4029-96d1-e195cfc682ff\") " pod="openshift-insights/insights-runtime-extractor-84f2w"
Sep 16 02:17:48.108053 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:48.108032    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/aefd9717-5d49-4029-96d1-e195cfc682ff-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-84f2w\" (UID: \"aefd9717-5d49-4029-96d1-e195cfc682ff\") " pod="openshift-insights/insights-runtime-extractor-84f2w"
Sep 16 02:17:48.303563 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:48.303500    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-84f2w"
Sep 16 02:17:48.652998 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:48.652926    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-84f2w"]
Sep 16 02:17:48.655801 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:17:48.655774    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaefd9717_5d49_4029_96d1_e195cfc682ff.slice/crio-45784658cfdeb668af6a385eed4f6fe545aad292bb2fca0d8a5e8d415d308790 WatchSource:0}: Error finding container 45784658cfdeb668af6a385eed4f6fe545aad292bb2fca0d8a5e8d415d308790: Status 404 returned error can't find the container with id 45784658cfdeb668af6a385eed4f6fe545aad292bb2fca0d8a5e8d415d308790
Sep 16 02:17:48.987918 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:48.987881    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-p2tqn" event={"ID":"c46f41bf-3230-4da8-9ec8-45759ed3551d","Type":"ContainerStarted","Data":"c1dc6175074fa3cee520063603ead97bb6d035f2489325a48246659992852ae0"}
Sep 16 02:17:48.988359 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:48.988010    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-p2tqn"
Sep 16 02:17:48.989520 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:48.989493    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-84f2w" event={"ID":"aefd9717-5d49-4029-96d1-e195cfc682ff","Type":"ContainerStarted","Data":"77510b7e1c79ef649e8b8a300a98ed188bd567d8811e73797c474e61c0085c24"}
Sep 16 02:17:48.989650 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:48.989540    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-84f2w" event={"ID":"aefd9717-5d49-4029-96d1-e195cfc682ff","Type":"ContainerStarted","Data":"45784658cfdeb668af6a385eed4f6fe545aad292bb2fca0d8a5e8d415d308790"}
Sep 16 02:17:48.989731 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:48.989707    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-q6tmq"
Sep 16 02:17:48.993068 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:48.993050    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-p2tqn"
Sep 16 02:17:49.002953 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:49.002895    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-p2tqn" podStartSLOduration=2.1762924 podStartE2EDuration="4.002883448s" totalImagesPullingTime="1.826591048s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 02:17:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 02:17:49.001948446 +0000 UTC m=+289.385463692" watchObservedRunningTime="2026-09-16 02:17:49.002883448 +0000 UTC m=+289.386398693"
Sep 16 02:17:49.793505 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:49.793447    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:17:49.827511 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:49.827483    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-68bcd7f987-kg6j9"]
Sep 16 02:17:49.830625 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:49.830609    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-68bcd7f987-kg6j9"
Sep 16 02:17:49.832936 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:49.832916    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Sep 16 02:17:49.833878 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:49.833858    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Sep 16 02:17:49.833968 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:49.833954    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-vggnw\""
Sep 16 02:17:49.834022 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:49.834007    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 16 02:17:49.840484 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:49.840444    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-68bcd7f987-kg6j9"]
Sep 16 02:17:49.858882 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:49.858854    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:17:49.920277 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:49.920252    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/ec276e5b-0368-4ee8-a433-b0941cf8bc16-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-kg6j9\" (UID: \"ec276e5b-0368-4ee8-a433-b0941cf8bc16\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-kg6j9"
Sep 16 02:17:49.920384 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:49.920277    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ec276e5b-0368-4ee8-a433-b0941cf8bc16-metrics-client-ca\") pod \"prometheus-operator-68bcd7f987-kg6j9\" (UID: \"ec276e5b-0368-4ee8-a433-b0941cf8bc16\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-kg6j9"
Sep 16 02:17:49.920384 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:49.920310    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qxdbm\" (UniqueName: \"kubernetes.io/projected/ec276e5b-0368-4ee8-a433-b0941cf8bc16-kube-api-access-qxdbm\") pod \"prometheus-operator-68bcd7f987-kg6j9\" (UID: \"ec276e5b-0368-4ee8-a433-b0941cf8bc16\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-kg6j9"
Sep 16 02:17:49.920384 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:49.920332    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ec276e5b-0368-4ee8-a433-b0941cf8bc16-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-68bcd7f987-kg6j9\" (UID: \"ec276e5b-0368-4ee8-a433-b0941cf8bc16\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-kg6j9"
Sep 16 02:17:49.961478 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:49.961450    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:17:49.994067 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:49.994043    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-84f2w" event={"ID":"aefd9717-5d49-4029-96d1-e195cfc682ff","Type":"ContainerStarted","Data":"97ab296fdec97fdc5ea9b358e7c266d367a9214ca239f8d993ca7eeb59ab72be"}
Sep 16 02:17:50.021495 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:50.021454    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/ec276e5b-0368-4ee8-a433-b0941cf8bc16-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-kg6j9\" (UID: \"ec276e5b-0368-4ee8-a433-b0941cf8bc16\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-kg6j9"
Sep 16 02:17:50.021558 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:50.021500    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ec276e5b-0368-4ee8-a433-b0941cf8bc16-metrics-client-ca\") pod \"prometheus-operator-68bcd7f987-kg6j9\" (UID: \"ec276e5b-0368-4ee8-a433-b0941cf8bc16\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-kg6j9"
Sep 16 02:17:50.021558 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:50.021530    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qxdbm\" (UniqueName: \"kubernetes.io/projected/ec276e5b-0368-4ee8-a433-b0941cf8bc16-kube-api-access-qxdbm\") pod \"prometheus-operator-68bcd7f987-kg6j9\" (UID: \"ec276e5b-0368-4ee8-a433-b0941cf8bc16\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-kg6j9"
Sep 16 02:17:50.021558 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:50.021554    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ec276e5b-0368-4ee8-a433-b0941cf8bc16-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-68bcd7f987-kg6j9\" (UID: \"ec276e5b-0368-4ee8-a433-b0941cf8bc16\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-kg6j9"
Sep 16 02:17:50.022307 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:50.022284    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ec276e5b-0368-4ee8-a433-b0941cf8bc16-metrics-client-ca\") pod \"prometheus-operator-68bcd7f987-kg6j9\" (UID: \"ec276e5b-0368-4ee8-a433-b0941cf8bc16\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-kg6j9"
Sep 16 02:17:50.024003 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:50.023983    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/ec276e5b-0368-4ee8-a433-b0941cf8bc16-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-kg6j9\" (UID: \"ec276e5b-0368-4ee8-a433-b0941cf8bc16\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-kg6j9"
Sep 16 02:17:50.024132 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:50.024112    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ec276e5b-0368-4ee8-a433-b0941cf8bc16-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-68bcd7f987-kg6j9\" (UID: \"ec276e5b-0368-4ee8-a433-b0941cf8bc16\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-kg6j9"
Sep 16 02:17:50.028776 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:50.028756    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qxdbm\" (UniqueName: \"kubernetes.io/projected/ec276e5b-0368-4ee8-a433-b0941cf8bc16-kube-api-access-qxdbm\") pod \"prometheus-operator-68bcd7f987-kg6j9\" (UID: \"ec276e5b-0368-4ee8-a433-b0941cf8bc16\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-kg6j9"
Sep 16 02:17:50.139049 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:50.138977    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-68bcd7f987-kg6j9"
Sep 16 02:17:50.413860 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:50.413520    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-68bcd7f987-kg6j9"]
Sep 16 02:17:50.416563 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:17:50.416532    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec276e5b_0368_4ee8_a433_b0941cf8bc16.slice/crio-62762d439bc9e9c400be9d2caa9f2787476fa2fdcab433315cdd9c42b11dfef9 WatchSource:0}: Error finding container 62762d439bc9e9c400be9d2caa9f2787476fa2fdcab433315cdd9c42b11dfef9: Status 404 returned error can't find the container with id 62762d439bc9e9c400be9d2caa9f2787476fa2fdcab433315cdd9c42b11dfef9
Sep 16 02:17:50.998619 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:50.998584    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-68bcd7f987-kg6j9" event={"ID":"ec276e5b-0368-4ee8-a433-b0941cf8bc16","Type":"ContainerStarted","Data":"62762d439bc9e9c400be9d2caa9f2787476fa2fdcab433315cdd9c42b11dfef9"}
Sep 16 02:17:52.004301 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:52.004272    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-84f2w" event={"ID":"aefd9717-5d49-4029-96d1-e195cfc682ff","Type":"ContainerStarted","Data":"8b161e5161e731e3d928d9a3ed4996cc396cbf71274287c9139644652d7f1aef"}
Sep 16 02:17:52.006187 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:52.006164    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-68bcd7f987-kg6j9" event={"ID":"ec276e5b-0368-4ee8-a433-b0941cf8bc16","Type":"ContainerStarted","Data":"a8871f62dac263d8fd63a3bd25a81d3c5ed7f7ef4efbd6d96f064d8111f4168e"}
Sep 16 02:17:52.020601 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:52.020554    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-84f2w" podStartSLOduration=33.890115257 podStartE2EDuration="36.020531999s" totalImagesPullingTime="2.130416742s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 02:17:16 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 02:17:52.019184766 +0000 UTC m=+292.402700012" watchObservedRunningTime="2026-09-16 02:17:52.020531999 +0000 UTC m=+292.404047263"
Sep 16 02:17:52.097573 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:17:52.097290    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-6lwr9" podUID="e3472392-891d-49a1-83a7-57f1f99f4b79"
Sep 16 02:17:52.167218 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:52.167129    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:17:53.011122 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:53.011084    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-68bcd7f987-kg6j9" event={"ID":"ec276e5b-0368-4ee8-a433-b0941cf8bc16","Type":"ContainerStarted","Data":"b3c5565e334c40c15beb741a09c453ea2e8bf8dcdb886aa5599d0cc2a03ae379"}
Sep 16 02:17:53.026095 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:53.026051    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-68bcd7f987-kg6j9" podStartSLOduration=2.536376568 podStartE2EDuration="4.026036298s" totalImagesPullingTime="1.48965973s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 02:17:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 02:17:53.024584781 +0000 UTC m=+293.408100026" watchObservedRunningTime="2026-09-16 02:17:53.026036298 +0000 UTC m=+293.409551542"
Sep 16 02:17:53.096576 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:53.096554    2579 scope.go:117] "RemoveContainer" containerID="af8faa2a6f130af678d8d07d5ba3021db5df3934a3b04a67786e3417884e5e0e"
Sep 16 02:17:53.096709 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:17:53.096692    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-696884fb9c-c8q8z_openshift-console-operator(ebf846fa-af7d-4fdd-9662-b7c533c36ddd)\"" pod="openshift-console-operator/console-operator-696884fb9c-c8q8z" podUID="ebf846fa-af7d-4fdd-9662-b7c533c36ddd"
Sep 16 02:17:54.915708 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:54.915674    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:17:55.012848 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:55.012812    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:17:55.133700 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:55.133652    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:17:55.188102 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:55.188075    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-59fbdd77d-sv5sm"]
Sep 16 02:17:55.191663 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:55.191640    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-sv5sm"
Sep 16 02:17:55.194253 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:55.194228    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Sep 16 02:17:55.194374 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:55.194277    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Sep 16 02:17:55.194503 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:55.194462    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Sep 16 02:17:55.195269 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:55.195252    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-jsd68\""
Sep 16 02:17:55.196501 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:55.196484    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-87q8v"]
Sep 16 02:17:55.199834 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:55.199820    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-87q8v"
Sep 16 02:17:55.203010 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:55.202987    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 16 02:17:55.203197 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:55.203183    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 16 02:17:55.203260 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:55.203233    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-qfp9h\""
Sep 16 02:17:55.205356 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:55.205338    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 16 02:17:55.210492 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:55.210452    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-59fbdd77d-sv5sm"]
Sep 16 02:17:55.248368 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:55.248343    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:17:55.260023 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:55.259996    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8h7bv\" (UniqueName: \"kubernetes.io/projected/3bd0fffa-d049-4ea6-ae0b-3e70c1645a12-kube-api-access-8h7bv\") pod \"node-exporter-87q8v\" (UID: \"3bd0fffa-d049-4ea6-ae0b-3e70c1645a12\") " pod="openshift-monitoring/node-exporter-87q8v"
Sep 16 02:17:55.260023 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:55.260022    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3bd0fffa-d049-4ea6-ae0b-3e70c1645a12-sys\") pod \"node-exporter-87q8v\" (UID: \"3bd0fffa-d049-4ea6-ae0b-3e70c1645a12\") " pod="openshift-monitoring/node-exporter-87q8v"
Sep 16 02:17:55.260168 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:55.260044    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/e75b8c92-b447-4b4a-847a-6529c04af4cb-volume-directive-shadow\") pod \"kube-state-metrics-59fbdd77d-sv5sm\" (UID: \"e75b8c92-b447-4b4a-847a-6529c04af4cb\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-sv5sm"
Sep 16 02:17:55.260168 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:55.260066    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3bd0fffa-d049-4ea6-ae0b-3e70c1645a12-node-exporter-wtmp\") pod \"node-exporter-87q8v\" (UID: \"3bd0fffa-d049-4ea6-ae0b-3e70c1645a12\") " pod="openshift-monitoring/node-exporter-87q8v"
Sep 16 02:17:55.260168 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:55.260137    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/e75b8c92-b447-4b4a-847a-6529c04af4cb-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-sv5sm\" (UID: \"e75b8c92-b447-4b4a-847a-6529c04af4cb\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-sv5sm"
Sep 16 02:17:55.260273 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:55.260217    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e75b8c92-b447-4b4a-847a-6529c04af4cb-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-59fbdd77d-sv5sm\" (UID: \"e75b8c92-b447-4b4a-847a-6529c04af4cb\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-sv5sm"
Sep 16 02:17:55.260273 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:55.260244    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3bd0fffa-d049-4ea6-ae0b-3e70c1645a12-root\") pod \"node-exporter-87q8v\" (UID: \"3bd0fffa-d049-4ea6-ae0b-3e70c1645a12\") " pod="openshift-monitoring/node-exporter-87q8v"
Sep 16 02:17:55.260273 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:55.260260    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3bd0fffa-d049-4ea6-ae0b-3e70c1645a12-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-87q8v\" (UID: \"3bd0fffa-d049-4ea6-ae0b-3e70c1645a12\") " pod="openshift-monitoring/node-exporter-87q8v"
Sep 16 02:17:55.260377 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:55.260278    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jxlw6\" (UniqueName: \"kubernetes.io/projected/e75b8c92-b447-4b4a-847a-6529c04af4cb-kube-api-access-jxlw6\") pod \"kube-state-metrics-59fbdd77d-sv5sm\" (UID: \"e75b8c92-b447-4b4a-847a-6529c04af4cb\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-sv5sm"
Sep 16 02:17:55.260377 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:55.260304    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e75b8c92-b447-4b4a-847a-6529c04af4cb-metrics-client-ca\") pod \"kube-state-metrics-59fbdd77d-sv5sm\" (UID: \"e75b8c92-b447-4b4a-847a-6529c04af4cb\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-sv5sm"
Sep 16 02:17:55.260377 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:55.260320    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3bd0fffa-d049-4ea6-ae0b-3e70c1645a12-node-exporter-textfile\") pod \"node-exporter-87q8v\" (UID: \"3bd0fffa-d049-4ea6-ae0b-3e70c1645a12\") " pod="openshift-monitoring/node-exporter-87q8v"
Sep 16 02:17:55.260377 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:55.260340    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/e75b8c92-b447-4b4a-847a-6529c04af4cb-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-59fbdd77d-sv5sm\" (UID: \"e75b8c92-b447-4b4a-847a-6529c04af4cb\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-sv5sm"
Sep 16 02:17:55.260377 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:55.260370    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3bd0fffa-d049-4ea6-ae0b-3e70c1645a12-metrics-client-ca\") pod \"node-exporter-87q8v\" (UID: \"3bd0fffa-d049-4ea6-ae0b-3e70c1645a12\") " pod="openshift-monitoring/node-exporter-87q8v"
Sep 16 02:17:55.260574 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:55.260450    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3bd0fffa-d049-4ea6-ae0b-3e70c1645a12-node-exporter-accelerators-collector-config\") pod \"node-exporter-87q8v\" (UID: \"3bd0fffa-d049-4ea6-ae0b-3e70c1645a12\") " pod="openshift-monitoring/node-exporter-87q8v"
Sep 16 02:17:55.260574 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:55.260490    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3bd0fffa-d049-4ea6-ae0b-3e70c1645a12-node-exporter-tls\") pod \"node-exporter-87q8v\" (UID: \"3bd0fffa-d049-4ea6-ae0b-3e70c1645a12\") " pod="openshift-monitoring/node-exporter-87q8v"
Sep 16 02:17:55.356826 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:55.356789    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:17:55.361761 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:55.361741    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/e75b8c92-b447-4b4a-847a-6529c04af4cb-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-59fbdd77d-sv5sm\" (UID: \"e75b8c92-b447-4b4a-847a-6529c04af4cb\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-sv5sm"
Sep 16 02:17:55.361857 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:55.361783    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3bd0fffa-d049-4ea6-ae0b-3e70c1645a12-metrics-client-ca\") pod \"node-exporter-87q8v\" (UID: \"3bd0fffa-d049-4ea6-ae0b-3e70c1645a12\") " pod="openshift-monitoring/node-exporter-87q8v"
Sep 16 02:17:55.361857 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:55.361810    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3bd0fffa-d049-4ea6-ae0b-3e70c1645a12-node-exporter-accelerators-collector-config\") pod \"node-exporter-87q8v\" (UID: \"3bd0fffa-d049-4ea6-ae0b-3e70c1645a12\") " pod="openshift-monitoring/node-exporter-87q8v"
Sep 16 02:17:55.361857 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:55.361837    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3bd0fffa-d049-4ea6-ae0b-3e70c1645a12-node-exporter-tls\") pod \"node-exporter-87q8v\" (UID: \"3bd0fffa-d049-4ea6-ae0b-3e70c1645a12\") " pod="openshift-monitoring/node-exporter-87q8v"
Sep 16 02:17:55.361992 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:55.361863    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8h7bv\" (UniqueName: \"kubernetes.io/projected/3bd0fffa-d049-4ea6-ae0b-3e70c1645a12-kube-api-access-8h7bv\") pod \"node-exporter-87q8v\" (UID: \"3bd0fffa-d049-4ea6-ae0b-3e70c1645a12\") " pod="openshift-monitoring/node-exporter-87q8v"
Sep 16 02:17:55.361992 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:55.361877    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3bd0fffa-d049-4ea6-ae0b-3e70c1645a12-sys\") pod \"node-exporter-87q8v\" (UID: \"3bd0fffa-d049-4ea6-ae0b-3e70c1645a12\") " pod="openshift-monitoring/node-exporter-87q8v"
Sep 16 02:17:55.361992 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:55.361897    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/e75b8c92-b447-4b4a-847a-6529c04af4cb-volume-directive-shadow\") pod \"kube-state-metrics-59fbdd77d-sv5sm\" (UID: \"e75b8c92-b447-4b4a-847a-6529c04af4cb\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-sv5sm"
Sep 16 02:17:55.361992 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:55.361914    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3bd0fffa-d049-4ea6-ae0b-3e70c1645a12-node-exporter-wtmp\") pod \"node-exporter-87q8v\" (UID: \"3bd0fffa-d049-4ea6-ae0b-3e70c1645a12\") " pod="openshift-monitoring/node-exporter-87q8v"
Sep 16 02:17:55.361992 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:55.361929    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/e75b8c92-b447-4b4a-847a-6529c04af4cb-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-sv5sm\" (UID: \"e75b8c92-b447-4b4a-847a-6529c04af4cb\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-sv5sm"
Sep 16 02:17:55.361992 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:55.361959    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e75b8c92-b447-4b4a-847a-6529c04af4cb-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-59fbdd77d-sv5sm\" (UID: \"e75b8c92-b447-4b4a-847a-6529c04af4cb\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-sv5sm"
Sep 16 02:17:55.361992 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:55.361982    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3bd0fffa-d049-4ea6-ae0b-3e70c1645a12-root\") pod \"node-exporter-87q8v\" (UID: \"3bd0fffa-d049-4ea6-ae0b-3e70c1645a12\") " pod="openshift-monitoring/node-exporter-87q8v"
Sep 16 02:17:55.362312 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:55.362004    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3bd0fffa-d049-4ea6-ae0b-3e70c1645a12-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-87q8v\" (UID: \"3bd0fffa-d049-4ea6-ae0b-3e70c1645a12\") " pod="openshift-monitoring/node-exporter-87q8v"
Sep 16 02:17:55.362312 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:55.362031    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jxlw6\" (UniqueName: \"kubernetes.io/projected/e75b8c92-b447-4b4a-847a-6529c04af4cb-kube-api-access-jxlw6\") pod \"kube-state-metrics-59fbdd77d-sv5sm\" (UID: \"e75b8c92-b447-4b4a-847a-6529c04af4cb\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-sv5sm"
Sep 16 02:17:55.362312 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:55.362065    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e75b8c92-b447-4b4a-847a-6529c04af4cb-metrics-client-ca\") pod \"kube-state-metrics-59fbdd77d-sv5sm\" (UID: \"e75b8c92-b447-4b4a-847a-6529c04af4cb\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-sv5sm"
Sep 16 02:17:55.362312 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:55.362086    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3bd0fffa-d049-4ea6-ae0b-3e70c1645a12-node-exporter-textfile\") pod \"node-exporter-87q8v\" (UID: \"3bd0fffa-d049-4ea6-ae0b-3e70c1645a12\") " pod="openshift-monitoring/node-exporter-87q8v"
Sep 16 02:17:55.362312 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:55.362157    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3bd0fffa-d049-4ea6-ae0b-3e70c1645a12-sys\") pod \"node-exporter-87q8v\" (UID: \"3bd0fffa-d049-4ea6-ae0b-3e70c1645a12\") " pod="openshift-monitoring/node-exporter-87q8v"
Sep 16 02:17:55.362312 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:55.362213    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3bd0fffa-d049-4ea6-ae0b-3e70c1645a12-root\") pod \"node-exporter-87q8v\" (UID: \"3bd0fffa-d049-4ea6-ae0b-3e70c1645a12\") " pod="openshift-monitoring/node-exporter-87q8v"
Sep 16 02:17:55.362312 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:55.362289    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/e75b8c92-b447-4b4a-847a-6529c04af4cb-volume-directive-shadow\") pod \"kube-state-metrics-59fbdd77d-sv5sm\" (UID: \"e75b8c92-b447-4b4a-847a-6529c04af4cb\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-sv5sm"
Sep 16 02:17:55.362674 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:55.362395    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3bd0fffa-d049-4ea6-ae0b-3e70c1645a12-node-exporter-wtmp\") pod \"node-exporter-87q8v\" (UID: \"3bd0fffa-d049-4ea6-ae0b-3e70c1645a12\") " pod="openshift-monitoring/node-exporter-87q8v"
Sep 16 02:17:55.362674 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:55.362443    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3bd0fffa-d049-4ea6-ae0b-3e70c1645a12-metrics-client-ca\") pod \"node-exporter-87q8v\" (UID: \"3bd0fffa-d049-4ea6-ae0b-3e70c1645a12\") " pod="openshift-monitoring/node-exporter-87q8v"
Sep 16 02:17:55.362674 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:55.362501    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3bd0fffa-d049-4ea6-ae0b-3e70c1645a12-node-exporter-accelerators-collector-config\") pod \"node-exporter-87q8v\" (UID: \"3bd0fffa-d049-4ea6-ae0b-3e70c1645a12\") " pod="openshift-monitoring/node-exporter-87q8v"
Sep 16 02:17:55.362674 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:55.362567    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/e75b8c92-b447-4b4a-847a-6529c04af4cb-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-59fbdd77d-sv5sm\" (UID: \"e75b8c92-b447-4b4a-847a-6529c04af4cb\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-sv5sm"
Sep 16 02:17:55.362976 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:55.362941    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3bd0fffa-d049-4ea6-ae0b-3e70c1645a12-node-exporter-textfile\") pod \"node-exporter-87q8v\" (UID: \"3bd0fffa-d049-4ea6-ae0b-3e70c1645a12\") " pod="openshift-monitoring/node-exporter-87q8v"
Sep 16 02:17:55.363273 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:55.363227    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e75b8c92-b447-4b4a-847a-6529c04af4cb-metrics-client-ca\") pod \"kube-state-metrics-59fbdd77d-sv5sm\" (UID: \"e75b8c92-b447-4b4a-847a-6529c04af4cb\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-sv5sm"
Sep 16 02:17:55.364779 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:55.364752    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3bd0fffa-d049-4ea6-ae0b-3e70c1645a12-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-87q8v\" (UID: \"3bd0fffa-d049-4ea6-ae0b-3e70c1645a12\") " pod="openshift-monitoring/node-exporter-87q8v"
Sep 16 02:17:55.364879 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:55.364853    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e75b8c92-b447-4b4a-847a-6529c04af4cb-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-59fbdd77d-sv5sm\" (UID: \"e75b8c92-b447-4b4a-847a-6529c04af4cb\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-sv5sm"
Sep 16 02:17:55.364930 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:55.364912    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3bd0fffa-d049-4ea6-ae0b-3e70c1645a12-node-exporter-tls\") pod \"node-exporter-87q8v\" (UID: \"3bd0fffa-d049-4ea6-ae0b-3e70c1645a12\") " pod="openshift-monitoring/node-exporter-87q8v"
Sep 16 02:17:55.364966 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:55.364908    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/e75b8c92-b447-4b4a-847a-6529c04af4cb-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-sv5sm\" (UID: \"e75b8c92-b447-4b4a-847a-6529c04af4cb\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-sv5sm"
Sep 16 02:17:55.370827 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:55.370805    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8h7bv\" (UniqueName: \"kubernetes.io/projected/3bd0fffa-d049-4ea6-ae0b-3e70c1645a12-kube-api-access-8h7bv\") pod \"node-exporter-87q8v\" (UID: \"3bd0fffa-d049-4ea6-ae0b-3e70c1645a12\") " pod="openshift-monitoring/node-exporter-87q8v"
Sep 16 02:17:55.371513 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:55.371485    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jxlw6\" (UniqueName: \"kubernetes.io/projected/e75b8c92-b447-4b4a-847a-6529c04af4cb-kube-api-access-jxlw6\") pod \"kube-state-metrics-59fbdd77d-sv5sm\" (UID: \"e75b8c92-b447-4b4a-847a-6529c04af4cb\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-sv5sm"
Sep 16 02:17:55.456970 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:55.456887    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:17:55.501210 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:55.501185    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-sv5sm"
Sep 16 02:17:55.507407 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:55.507392    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-87q8v"
Sep 16 02:17:55.519255 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:17:55.519231    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3bd0fffa_d049_4ea6_ae0b_3e70c1645a12.slice/crio-7dcf922f520aa80537acda4ef4f2ed1a0cd2703ffc1a7972dfec8b037c747856 WatchSource:0}: Error finding container 7dcf922f520aa80537acda4ef4f2ed1a0cd2703ffc1a7972dfec8b037c747856: Status 404 returned error can't find the container with id 7dcf922f520aa80537acda4ef4f2ed1a0cd2703ffc1a7972dfec8b037c747856
Sep 16 02:17:55.558036 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:55.558008    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:17:55.658280 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:55.658245    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:17:55.771406 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:55.771384    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-59fbdd77d-sv5sm"]
Sep 16 02:17:55.773773 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:17:55.773751    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode75b8c92_b447_4b4a_847a_6529c04af4cb.slice/crio-a6513f59fac7b85a89eb2b821a5eb56759364698ea2d70f51cb32872ae4b0d5d WatchSource:0}: Error finding container a6513f59fac7b85a89eb2b821a5eb56759364698ea2d70f51cb32872ae4b0d5d: Status 404 returned error can't find the container with id a6513f59fac7b85a89eb2b821a5eb56759364698ea2d70f51cb32872ae4b0d5d
Sep 16 02:17:56.023396 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:56.023324    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-87q8v" event={"ID":"3bd0fffa-d049-4ea6-ae0b-3e70c1645a12","Type":"ContainerStarted","Data":"7dcf922f520aa80537acda4ef4f2ed1a0cd2703ffc1a7972dfec8b037c747856"}
Sep 16 02:17:56.024748 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:56.024721    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-sv5sm" event={"ID":"e75b8c92-b447-4b4a-847a-6529c04af4cb","Type":"ContainerStarted","Data":"a6513f59fac7b85a89eb2b821a5eb56759364698ea2d70f51cb32872ae4b0d5d"}
Sep 16 02:17:56.152873 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:56.152831    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:17:56.223558 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:56.223526    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:17:56.282710 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:56.282630    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:17:57.031223 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:57.031191    2579 generic.go:356] "Generic (PLEG): container finished" podID="3bd0fffa-d049-4ea6-ae0b-3e70c1645a12" containerID="80bcfe212d3fb992c0a89f41c19ec145fd43c3bfcfe04981f42282dfcb1553b3" exitCode=0
Sep 16 02:17:57.031664 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:57.031238    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-87q8v" event={"ID":"3bd0fffa-d049-4ea6-ae0b-3e70c1645a12","Type":"ContainerDied","Data":"80bcfe212d3fb992c0a89f41c19ec145fd43c3bfcfe04981f42282dfcb1553b3"}
Sep 16 02:17:58.043411 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:58.043364    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-sv5sm" event={"ID":"e75b8c92-b447-4b4a-847a-6529c04af4cb","Type":"ContainerStarted","Data":"dadbab8b974044cfc013253b0e2adcf70a053fb4f7554f08a484f31f066b48b0"}
Sep 16 02:17:58.043881 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:58.043482    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-sv5sm" event={"ID":"e75b8c92-b447-4b4a-847a-6529c04af4cb","Type":"ContainerStarted","Data":"02bc0c68780d80a01ad8df0101bd547ae568384cea87028c47b6616498dd4b07"}
Sep 16 02:17:58.043881 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:58.043514    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-sv5sm" event={"ID":"e75b8c92-b447-4b4a-847a-6529c04af4cb","Type":"ContainerStarted","Data":"804b2804a135d9617a0d7b2e575aeaf3dffb41b66f7ecee15422b303172a0a38"}
Sep 16 02:17:58.046144 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:58.046109    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-87q8v" event={"ID":"3bd0fffa-d049-4ea6-ae0b-3e70c1645a12","Type":"ContainerStarted","Data":"66dbea01ddd379a38b93fdd4de1a7c0471368846a4f93b61716d5ccf3db2148e"}
Sep 16 02:17:58.046264 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:58.046159    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-87q8v" event={"ID":"3bd0fffa-d049-4ea6-ae0b-3e70c1645a12","Type":"ContainerStarted","Data":"0581aa9d0adab4d8364b18b7d89c40897e58114f3fde74cc64c6199b2009b7b7"}
Sep 16 02:17:58.061122 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:58.061078    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-sv5sm" podStartSLOduration=1.677093457 podStartE2EDuration="3.061063444s" totalImagesPullingTime="1.383969987s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 02:17:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 02:17:58.060292106 +0000 UTC m=+298.443807352" watchObservedRunningTime="2026-09-16 02:17:58.061063444 +0000 UTC m=+298.444578733"
Sep 16 02:17:58.081072 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:58.081029    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-87q8v" podStartSLOduration=1.940211795 podStartE2EDuration="3.081014672s" totalImagesPullingTime="1.065842507s" totalInitContainerRuntime="74.96037ms" isStatefulPod=true podCreationTimestamp="2026-09-16 02:17:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 02:17:58.079937994 +0000 UTC m=+298.463453240" watchObservedRunningTime="2026-09-16 02:17:58.081014672 +0000 UTC m=+298.464529917"
Sep 16 02:17:58.084278 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:58.084252    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:17:58.183899 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:58.183866    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:17:58.367509 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:58.367399    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:17:58.996610 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:58.996580    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-q6tmq"
Sep 16 02:17:59.382072 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:59.382003    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:17:59.466603 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:59.466574    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:17:59.518933 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:59.518903    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:17:59.930529 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:17:59.930497    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:18:00.041875 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:00.041844    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-c8q8z_ebf846fa-af7d-4fdd-9662-b7c533c36ddd/console-operator/2.log"
Sep 16 02:18:00.042118 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:00.042102    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-c8q8z_ebf846fa-af7d-4fdd-9662-b7c533c36ddd/console-operator/2.log"
Sep 16 02:18:00.053842 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:00.053811    2579 kubelet.go:1648] "Image garbage collection succeeded"
Sep 16 02:18:00.325300 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:00.325264    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-649f6698d9-cmzbc"]
Sep 16 02:18:00.329998 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:00.329971    2579 patch_prober.go:28] interesting pod/image-registry-649f6698d9-cmzbc container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Sep 16 02:18:00.330135 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:00.330019    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-649f6698d9-cmzbc" podUID="35bd8e04-3e55-44b6-b78e-83566dcf43cc" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 16 02:18:00.397272 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:00.397243    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:18:00.496508 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:00.496453    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:18:01.086746 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:01.086710    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:18:01.302574 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:01.302540    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:18:03.096066 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:03.096027    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6lwr9"
Sep 16 02:18:06.106208 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:06.106183    2579 scope.go:117] "RemoveContainer" containerID="af8faa2a6f130af678d8d07d5ba3021db5df3934a3b04a67786e3417884e5e0e"
Sep 16 02:18:06.766537 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:06.766503    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-76f99584f4-tknsv"]
Sep 16 02:18:06.769610 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:06.769589    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-76f99584f4-tknsv"
Sep 16 02:18:06.771898 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:06.771869    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 16 02:18:06.771898 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:06.771892    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 16 02:18:06.772162 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:06.772148    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-7hd5r\""
Sep 16 02:18:06.778080 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:06.778058    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-76f99584f4-tknsv"]
Sep 16 02:18:06.787240 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:06.787210    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:18:06.862969 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:06.862942    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vt6rz\" (UniqueName: \"kubernetes.io/projected/03b72ca3-a7e2-469f-85d4-281646ff1a2a-kube-api-access-vt6rz\") pod \"downloads-76f99584f4-tknsv\" (UID: \"03b72ca3-a7e2-469f-85d4-281646ff1a2a\") " pod="openshift-console/downloads-76f99584f4-tknsv"
Sep 16 02:18:06.963839 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:06.963804    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vt6rz\" (UniqueName: \"kubernetes.io/projected/03b72ca3-a7e2-469f-85d4-281646ff1a2a-kube-api-access-vt6rz\") pod \"downloads-76f99584f4-tknsv\" (UID: \"03b72ca3-a7e2-469f-85d4-281646ff1a2a\") " pod="openshift-console/downloads-76f99584f4-tknsv"
Sep 16 02:18:06.971296 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:06.971263    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vt6rz\" (UniqueName: \"kubernetes.io/projected/03b72ca3-a7e2-469f-85d4-281646ff1a2a-kube-api-access-vt6rz\") pod \"downloads-76f99584f4-tknsv\" (UID: \"03b72ca3-a7e2-469f-85d4-281646ff1a2a\") " pod="openshift-console/downloads-76f99584f4-tknsv"
Sep 16 02:18:07.079667 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:07.079529    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-76f99584f4-tknsv"
Sep 16 02:18:07.082421 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:07.082403    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-c8q8z_ebf846fa-af7d-4fdd-9662-b7c533c36ddd/console-operator/2.log"
Sep 16 02:18:07.082496 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:07.082446    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-c8q8z" event={"ID":"ebf846fa-af7d-4fdd-9662-b7c533c36ddd","Type":"ContainerStarted","Data":"787dbd97612ff1afc70f9be5aae7b0c5b0aa2a48ff3ddda3e3e0421019e2f4f5"}
Sep 16 02:18:07.082735 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:07.082719    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-696884fb9c-c8q8z"
Sep 16 02:18:07.087645 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:07.087625    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-696884fb9c-c8q8z"
Sep 16 02:18:07.098712 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:07.098669    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-696884fb9c-c8q8z" podStartSLOduration=53.795448117 podStartE2EDuration="57.098653565s" totalImagesPullingTime="3.303205448s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 02:17:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 02:18:07.098625814 +0000 UTC m=+307.482141057" watchObservedRunningTime="2026-09-16 02:18:07.098653565 +0000 UTC m=+307.482168811"
Sep 16 02:18:07.351102 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:07.351028    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-76f99584f4-tknsv"]
Sep 16 02:18:07.354203 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:18:07.354181    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03b72ca3_a7e2_469f_85d4_281646ff1a2a.slice/crio-d357a6a422b517a9ac89efc37d9fa585cf3473d6e96d60703c0ec781562bdc9c WatchSource:0}: Error finding container d357a6a422b517a9ac89efc37d9fa585cf3473d6e96d60703c0ec781562bdc9c: Status 404 returned error can't find the container with id d357a6a422b517a9ac89efc37d9fa585cf3473d6e96d60703c0ec781562bdc9c
Sep 16 02:18:07.356224 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:07.356205    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 16 02:18:08.002321 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:08.002285    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:18:08.088379 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:08.088344    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-76f99584f4-tknsv" event={"ID":"03b72ca3-a7e2-469f-85d4-281646ff1a2a","Type":"ContainerStarted","Data":"d357a6a422b517a9ac89efc37d9fa585cf3473d6e96d60703c0ec781562bdc9c"}
Sep 16 02:18:08.108547 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:08.108517    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:18:08.208780 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:08.208748    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:18:08.453159 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:08.453125    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:18:08.650687 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:08.650660    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:18:08.700240 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:08.700203    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:18:09.667776 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:09.667702    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:18:09.767919 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:09.767886    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:18:10.330855 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:10.330830    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-649f6698d9-cmzbc"
Sep 16 02:18:12.512546 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:12.512461    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:18:12.610689 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:12.610638    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:18:14.135633 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:14.135600    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:18:14.184844 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:14.184812    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:18:17.115931 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:17.115897    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:18:17.222269 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:17.222230    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:18:21.581625 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:21.581589    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:18:21.682308 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:21.682270    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:18:21.784255 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:21.784222    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:18:21.991521 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:21.991458    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:18:22.049986 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:22.049948    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:18:24.158854 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:24.158809    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-76f99584f4-tknsv" event={"ID":"03b72ca3-a7e2-469f-85d4-281646ff1a2a","Type":"ContainerStarted","Data":"22f8123341726e1e06a18a76225c62249dd607478403f2261affa6f98b989af9"}
Sep 16 02:18:24.159447 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:24.159418    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-76f99584f4-tknsv"
Sep 16 02:18:24.175593 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:24.175563    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-76f99584f4-tknsv"
Sep 16 02:18:24.176001 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:24.175962    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-76f99584f4-tknsv" podStartSLOduration=1.8457674210000001 podStartE2EDuration="18.175946394s" totalImagesPullingTime="16.330178973s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 02:18:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 02:18:24.17366811 +0000 UTC m=+324.557183357" watchObservedRunningTime="2026-09-16 02:18:24.175946394 +0000 UTC m=+324.559461641"
Sep 16 02:18:25.272743 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:25.272705    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:18:25.347443 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:25.347387    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-649f6698d9-cmzbc" podUID="35bd8e04-3e55-44b6-b78e-83566dcf43cc" containerName="registry" containerID="cri-o://f236f6f23af08c0986c7b643d6084bdd009ba098fb5eacafe7d5aff80009ef0a" gracePeriod=30
Sep 16 02:18:25.373147 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:25.373113    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:18:25.623121 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:25.623092    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-649f6698d9-cmzbc"
Sep 16 02:18:25.735006 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:25.734966    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-grvhj\" (UniqueName: \"kubernetes.io/projected/35bd8e04-3e55-44b6-b78e-83566dcf43cc-kube-api-access-grvhj\") pod \"35bd8e04-3e55-44b6-b78e-83566dcf43cc\" (UID: \"35bd8e04-3e55-44b6-b78e-83566dcf43cc\") "
Sep 16 02:18:25.735199 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:25.735020    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/35bd8e04-3e55-44b6-b78e-83566dcf43cc-image-registry-private-configuration\") pod \"35bd8e04-3e55-44b6-b78e-83566dcf43cc\" (UID: \"35bd8e04-3e55-44b6-b78e-83566dcf43cc\") "
Sep 16 02:18:25.735199 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:25.735049    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/35bd8e04-3e55-44b6-b78e-83566dcf43cc-installation-pull-secrets\") pod \"35bd8e04-3e55-44b6-b78e-83566dcf43cc\" (UID: \"35bd8e04-3e55-44b6-b78e-83566dcf43cc\") "
Sep 16 02:18:25.735199 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:25.735075    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/35bd8e04-3e55-44b6-b78e-83566dcf43cc-trusted-ca\") pod \"35bd8e04-3e55-44b6-b78e-83566dcf43cc\" (UID: \"35bd8e04-3e55-44b6-b78e-83566dcf43cc\") "
Sep 16 02:18:25.735199 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:25.735111    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/35bd8e04-3e55-44b6-b78e-83566dcf43cc-bound-sa-token\") pod \"35bd8e04-3e55-44b6-b78e-83566dcf43cc\" (UID: \"35bd8e04-3e55-44b6-b78e-83566dcf43cc\") "
Sep 16 02:18:25.735199 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:25.735158    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/35bd8e04-3e55-44b6-b78e-83566dcf43cc-registry-certificates\") pod \"35bd8e04-3e55-44b6-b78e-83566dcf43cc\" (UID: \"35bd8e04-3e55-44b6-b78e-83566dcf43cc\") "
Sep 16 02:18:25.735504 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:25.735211    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/35bd8e04-3e55-44b6-b78e-83566dcf43cc-ca-trust-extracted\") pod \"35bd8e04-3e55-44b6-b78e-83566dcf43cc\" (UID: \"35bd8e04-3e55-44b6-b78e-83566dcf43cc\") "
Sep 16 02:18:25.735504 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:25.735270    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/35bd8e04-3e55-44b6-b78e-83566dcf43cc-registry-tls\") pod \"35bd8e04-3e55-44b6-b78e-83566dcf43cc\" (UID: \"35bd8e04-3e55-44b6-b78e-83566dcf43cc\") "
Sep 16 02:18:25.739633 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:25.735909    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/35bd8e04-3e55-44b6-b78e-83566dcf43cc-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "35bd8e04-3e55-44b6-b78e-83566dcf43cc" (UID: "35bd8e04-3e55-44b6-b78e-83566dcf43cc"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 02:18:25.739633 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:25.735997    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/35bd8e04-3e55-44b6-b78e-83566dcf43cc-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "35bd8e04-3e55-44b6-b78e-83566dcf43cc" (UID: "35bd8e04-3e55-44b6-b78e-83566dcf43cc"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 02:18:25.739633 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:25.739199    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/35bd8e04-3e55-44b6-b78e-83566dcf43cc-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "35bd8e04-3e55-44b6-b78e-83566dcf43cc" (UID: "35bd8e04-3e55-44b6-b78e-83566dcf43cc"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 02:18:25.739633 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:25.739568    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/35bd8e04-3e55-44b6-b78e-83566dcf43cc-kube-api-access-grvhj" (OuterVolumeSpecName: "kube-api-access-grvhj") pod "35bd8e04-3e55-44b6-b78e-83566dcf43cc" (UID: "35bd8e04-3e55-44b6-b78e-83566dcf43cc"). InnerVolumeSpecName "kube-api-access-grvhj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 02:18:25.740070 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:25.740033    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/35bd8e04-3e55-44b6-b78e-83566dcf43cc-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "35bd8e04-3e55-44b6-b78e-83566dcf43cc" (UID: "35bd8e04-3e55-44b6-b78e-83566dcf43cc"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 02:18:25.740179 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:25.740125    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/35bd8e04-3e55-44b6-b78e-83566dcf43cc-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "35bd8e04-3e55-44b6-b78e-83566dcf43cc" (UID: "35bd8e04-3e55-44b6-b78e-83566dcf43cc"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 02:18:25.740238 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:25.740194    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/35bd8e04-3e55-44b6-b78e-83566dcf43cc-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "35bd8e04-3e55-44b6-b78e-83566dcf43cc" (UID: "35bd8e04-3e55-44b6-b78e-83566dcf43cc"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 02:18:25.749414 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:25.749391    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/35bd8e04-3e55-44b6-b78e-83566dcf43cc-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "35bd8e04-3e55-44b6-b78e-83566dcf43cc" (UID: "35bd8e04-3e55-44b6-b78e-83566dcf43cc"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 02:18:25.836432 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:25.836348    2579 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/35bd8e04-3e55-44b6-b78e-83566dcf43cc-ca-trust-extracted\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:18:25.836432 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:25.836384    2579 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/35bd8e04-3e55-44b6-b78e-83566dcf43cc-registry-tls\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:18:25.836432 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:25.836396    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-grvhj\" (UniqueName: \"kubernetes.io/projected/35bd8e04-3e55-44b6-b78e-83566dcf43cc-kube-api-access-grvhj\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:18:25.836432 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:25.836410    2579 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/35bd8e04-3e55-44b6-b78e-83566dcf43cc-image-registry-private-configuration\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:18:25.836432 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:25.836424    2579 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/35bd8e04-3e55-44b6-b78e-83566dcf43cc-installation-pull-secrets\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:18:25.836432 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:25.836435    2579 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/35bd8e04-3e55-44b6-b78e-83566dcf43cc-trusted-ca\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:18:25.836840 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:25.836447    2579 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/35bd8e04-3e55-44b6-b78e-83566dcf43cc-bound-sa-token\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:18:25.836840 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:25.836458    2579 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/35bd8e04-3e55-44b6-b78e-83566dcf43cc-registry-certificates\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:18:26.069042 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:26.069007    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-549b9d8f45-lbj98"]
Sep 16 02:18:26.069420 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:26.069401    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="35bd8e04-3e55-44b6-b78e-83566dcf43cc" containerName="registry"
Sep 16 02:18:26.069535 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:26.069423    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="35bd8e04-3e55-44b6-b78e-83566dcf43cc" containerName="registry"
Sep 16 02:18:26.069591 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:26.069534    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="35bd8e04-3e55-44b6-b78e-83566dcf43cc" containerName="registry"
Sep 16 02:18:26.087448 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:26.087294    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-549b9d8f45-lbj98"]
Sep 16 02:18:26.087448 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:26.087411    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-549b9d8f45-lbj98"
Sep 16 02:18:26.089689 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:26.089665    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 16 02:18:26.091489 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:26.089939    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 16 02:18:26.091489 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:26.089949    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 16 02:18:26.091489 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:26.090204    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 16 02:18:26.091489 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:26.090371    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 16 02:18:26.091489 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:26.090425    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-wrs7f\""
Sep 16 02:18:26.095599 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:26.095576    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 16 02:18:26.140091 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:26.140059    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/20f59f33-451c-4963-a96d-9f1081dd49dc-trusted-ca-bundle\") pod \"console-549b9d8f45-lbj98\" (UID: \"20f59f33-451c-4963-a96d-9f1081dd49dc\") " pod="openshift-console/console-549b9d8f45-lbj98"
Sep 16 02:18:26.140233 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:26.140175    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/20f59f33-451c-4963-a96d-9f1081dd49dc-console-config\") pod \"console-549b9d8f45-lbj98\" (UID: \"20f59f33-451c-4963-a96d-9f1081dd49dc\") " pod="openshift-console/console-549b9d8f45-lbj98"
Sep 16 02:18:26.140300 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:26.140233    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/20f59f33-451c-4963-a96d-9f1081dd49dc-service-ca\") pod \"console-549b9d8f45-lbj98\" (UID: \"20f59f33-451c-4963-a96d-9f1081dd49dc\") " pod="openshift-console/console-549b9d8f45-lbj98"
Sep 16 02:18:26.140300 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:26.140262    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pq59n\" (UniqueName: \"kubernetes.io/projected/20f59f33-451c-4963-a96d-9f1081dd49dc-kube-api-access-pq59n\") pod \"console-549b9d8f45-lbj98\" (UID: \"20f59f33-451c-4963-a96d-9f1081dd49dc\") " pod="openshift-console/console-549b9d8f45-lbj98"
Sep 16 02:18:26.140300 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:26.140294    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/20f59f33-451c-4963-a96d-9f1081dd49dc-console-oauth-config\") pod \"console-549b9d8f45-lbj98\" (UID: \"20f59f33-451c-4963-a96d-9f1081dd49dc\") " pod="openshift-console/console-549b9d8f45-lbj98"
Sep 16 02:18:26.140446 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:26.140359    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/20f59f33-451c-4963-a96d-9f1081dd49dc-console-serving-cert\") pod \"console-549b9d8f45-lbj98\" (UID: \"20f59f33-451c-4963-a96d-9f1081dd49dc\") " pod="openshift-console/console-549b9d8f45-lbj98"
Sep 16 02:18:26.140446 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:26.140394    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/20f59f33-451c-4963-a96d-9f1081dd49dc-oauth-serving-cert\") pod \"console-549b9d8f45-lbj98\" (UID: \"20f59f33-451c-4963-a96d-9f1081dd49dc\") " pod="openshift-console/console-549b9d8f45-lbj98"
Sep 16 02:18:26.141405 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:26.141380    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:18:26.168667 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:26.168640    2579 generic.go:356] "Generic (PLEG): container finished" podID="35bd8e04-3e55-44b6-b78e-83566dcf43cc" containerID="f236f6f23af08c0986c7b643d6084bdd009ba098fb5eacafe7d5aff80009ef0a" exitCode=0
Sep 16 02:18:26.168824 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:26.168683    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-649f6698d9-cmzbc"
Sep 16 02:18:26.168824 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:26.168735    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-649f6698d9-cmzbc" event={"ID":"35bd8e04-3e55-44b6-b78e-83566dcf43cc","Type":"ContainerDied","Data":"f236f6f23af08c0986c7b643d6084bdd009ba098fb5eacafe7d5aff80009ef0a"}
Sep 16 02:18:26.168824 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:26.168771    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-649f6698d9-cmzbc" event={"ID":"35bd8e04-3e55-44b6-b78e-83566dcf43cc","Type":"ContainerDied","Data":"4e63ff0b90b2fbaa15b20136b947ae18c6de5d3bf7418e5f8d70fda18a858480"}
Sep 16 02:18:26.168824 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:26.168812    2579 scope.go:117] "RemoveContainer" containerID="f236f6f23af08c0986c7b643d6084bdd009ba098fb5eacafe7d5aff80009ef0a"
Sep 16 02:18:26.184099 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:26.184078    2579 scope.go:117] "RemoveContainer" containerID="f236f6f23af08c0986c7b643d6084bdd009ba098fb5eacafe7d5aff80009ef0a"
Sep 16 02:18:26.184450 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:18:26.184426    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f236f6f23af08c0986c7b643d6084bdd009ba098fb5eacafe7d5aff80009ef0a\": container with ID starting with f236f6f23af08c0986c7b643d6084bdd009ba098fb5eacafe7d5aff80009ef0a not found: ID does not exist" containerID="f236f6f23af08c0986c7b643d6084bdd009ba098fb5eacafe7d5aff80009ef0a"
Sep 16 02:18:26.184563 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:26.184463    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f236f6f23af08c0986c7b643d6084bdd009ba098fb5eacafe7d5aff80009ef0a"} err="failed to get container status \"f236f6f23af08c0986c7b643d6084bdd009ba098fb5eacafe7d5aff80009ef0a\": rpc error: code = NotFound desc = could not find container \"f236f6f23af08c0986c7b643d6084bdd009ba098fb5eacafe7d5aff80009ef0a\": container with ID starting with f236f6f23af08c0986c7b643d6084bdd009ba098fb5eacafe7d5aff80009ef0a not found: ID does not exist"
Sep 16 02:18:26.184629 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:26.184607    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-649f6698d9-cmzbc"]
Sep 16 02:18:26.188222 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:26.188199    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-649f6698d9-cmzbc"]
Sep 16 02:18:26.193320 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:26.193298    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:18:26.241544 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:26.241507    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/20f59f33-451c-4963-a96d-9f1081dd49dc-console-config\") pod \"console-549b9d8f45-lbj98\" (UID: \"20f59f33-451c-4963-a96d-9f1081dd49dc\") " pod="openshift-console/console-549b9d8f45-lbj98"
Sep 16 02:18:26.241718 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:26.241573    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/20f59f33-451c-4963-a96d-9f1081dd49dc-service-ca\") pod \"console-549b9d8f45-lbj98\" (UID: \"20f59f33-451c-4963-a96d-9f1081dd49dc\") " pod="openshift-console/console-549b9d8f45-lbj98"
Sep 16 02:18:26.241718 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:26.241699    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pq59n\" (UniqueName: \"kubernetes.io/projected/20f59f33-451c-4963-a96d-9f1081dd49dc-kube-api-access-pq59n\") pod \"console-549b9d8f45-lbj98\" (UID: \"20f59f33-451c-4963-a96d-9f1081dd49dc\") " pod="openshift-console/console-549b9d8f45-lbj98"
Sep 16 02:18:26.241852 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:26.241752    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/20f59f33-451c-4963-a96d-9f1081dd49dc-console-oauth-config\") pod \"console-549b9d8f45-lbj98\" (UID: \"20f59f33-451c-4963-a96d-9f1081dd49dc\") " pod="openshift-console/console-549b9d8f45-lbj98"
Sep 16 02:18:26.241852 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:26.241781    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/20f59f33-451c-4963-a96d-9f1081dd49dc-console-serving-cert\") pod \"console-549b9d8f45-lbj98\" (UID: \"20f59f33-451c-4963-a96d-9f1081dd49dc\") " pod="openshift-console/console-549b9d8f45-lbj98"
Sep 16 02:18:26.241852 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:26.241802    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/20f59f33-451c-4963-a96d-9f1081dd49dc-oauth-serving-cert\") pod \"console-549b9d8f45-lbj98\" (UID: \"20f59f33-451c-4963-a96d-9f1081dd49dc\") " pod="openshift-console/console-549b9d8f45-lbj98"
Sep 16 02:18:26.241852 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:26.241837    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/20f59f33-451c-4963-a96d-9f1081dd49dc-trusted-ca-bundle\") pod \"console-549b9d8f45-lbj98\" (UID: \"20f59f33-451c-4963-a96d-9f1081dd49dc\") " pod="openshift-console/console-549b9d8f45-lbj98"
Sep 16 02:18:26.242401 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:26.242369    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/20f59f33-451c-4963-a96d-9f1081dd49dc-console-config\") pod \"console-549b9d8f45-lbj98\" (UID: \"20f59f33-451c-4963-a96d-9f1081dd49dc\") " pod="openshift-console/console-549b9d8f45-lbj98"
Sep 16 02:18:26.242765 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:26.242737    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/20f59f33-451c-4963-a96d-9f1081dd49dc-oauth-serving-cert\") pod \"console-549b9d8f45-lbj98\" (UID: \"20f59f33-451c-4963-a96d-9f1081dd49dc\") " pod="openshift-console/console-549b9d8f45-lbj98"
Sep 16 02:18:26.242881 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:26.242860    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/20f59f33-451c-4963-a96d-9f1081dd49dc-trusted-ca-bundle\") pod \"console-549b9d8f45-lbj98\" (UID: \"20f59f33-451c-4963-a96d-9f1081dd49dc\") " pod="openshift-console/console-549b9d8f45-lbj98"
Sep 16 02:18:26.243130 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:26.243110    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/20f59f33-451c-4963-a96d-9f1081dd49dc-service-ca\") pod \"console-549b9d8f45-lbj98\" (UID: \"20f59f33-451c-4963-a96d-9f1081dd49dc\") " pod="openshift-console/console-549b9d8f45-lbj98"
Sep 16 02:18:26.244837 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:26.244819    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/20f59f33-451c-4963-a96d-9f1081dd49dc-console-serving-cert\") pod \"console-549b9d8f45-lbj98\" (UID: \"20f59f33-451c-4963-a96d-9f1081dd49dc\") " pod="openshift-console/console-549b9d8f45-lbj98"
Sep 16 02:18:26.244929 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:26.244892    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/20f59f33-451c-4963-a96d-9f1081dd49dc-console-oauth-config\") pod \"console-549b9d8f45-lbj98\" (UID: \"20f59f33-451c-4963-a96d-9f1081dd49dc\") " pod="openshift-console/console-549b9d8f45-lbj98"
Sep 16 02:18:26.249143 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:26.249122    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pq59n\" (UniqueName: \"kubernetes.io/projected/20f59f33-451c-4963-a96d-9f1081dd49dc-kube-api-access-pq59n\") pod \"console-549b9d8f45-lbj98\" (UID: \"20f59f33-451c-4963-a96d-9f1081dd49dc\") " pod="openshift-console/console-549b9d8f45-lbj98"
Sep 16 02:18:26.402334 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:26.402260    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-549b9d8f45-lbj98"
Sep 16 02:18:26.704666 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:26.704577    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-549b9d8f45-lbj98"]
Sep 16 02:18:26.707135 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:18:26.707105    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20f59f33_451c_4963_a96d_9f1081dd49dc.slice/crio-8227e0415103a4b3a1825c396fec8badab14f8dc586ac19559bdcf22eb582254 WatchSource:0}: Error finding container 8227e0415103a4b3a1825c396fec8badab14f8dc586ac19559bdcf22eb582254: Status 404 returned error can't find the container with id 8227e0415103a4b3a1825c396fec8badab14f8dc586ac19559bdcf22eb582254
Sep 16 02:18:27.174681 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:27.174591    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-549b9d8f45-lbj98" event={"ID":"20f59f33-451c-4963-a96d-9f1081dd49dc","Type":"ContainerStarted","Data":"8227e0415103a4b3a1825c396fec8badab14f8dc586ac19559bdcf22eb582254"}
Sep 16 02:18:28.103444 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:28.103364    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="35bd8e04-3e55-44b6-b78e-83566dcf43cc" path="/var/lib/kubelet/pods/35bd8e04-3e55-44b6-b78e-83566dcf43cc/volumes"
Sep 16 02:18:30.480089 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:30.480047    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:18:30.580452 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:30.580427    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:18:31.197359 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:31.197316    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-549b9d8f45-lbj98" event={"ID":"20f59f33-451c-4963-a96d-9f1081dd49dc","Type":"ContainerStarted","Data":"5b38e2b3d366cc4b6fc95308ad669a6c97f08a51df2c9750228f0c4b38602946"}
Sep 16 02:18:31.213595 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:31.213539    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-549b9d8f45-lbj98" podStartSLOduration=1.434232193 podStartE2EDuration="5.213517824s" totalImagesPullingTime="3.779285631s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 02:18:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 02:18:31.212377583 +0000 UTC m=+331.595892829" watchObservedRunningTime="2026-09-16 02:18:31.213517824 +0000 UTC m=+331.597033070"
Sep 16 02:18:31.872362 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:31.872319    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:18:31.923121 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:31.923081    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:18:36.219347 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:36.219310    2579 generic.go:356] "Generic (PLEG): container finished" podID="111d4c95-703e-4afe-bfca-fbf61d55154f" containerID="e18c2ebfc4eb7b6ed7011b031f7f35449fcb4c4ec0049896c639fa8972643356" exitCode=0
Sep 16 02:18:36.219788 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:36.219385    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-bmj9c" event={"ID":"111d4c95-703e-4afe-bfca-fbf61d55154f","Type":"ContainerDied","Data":"e18c2ebfc4eb7b6ed7011b031f7f35449fcb4c4ec0049896c639fa8972643356"}
Sep 16 02:18:36.219788 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:36.219687    2579 scope.go:117] "RemoveContainer" containerID="e18c2ebfc4eb7b6ed7011b031f7f35449fcb4c4ec0049896c639fa8972643356"
Sep 16 02:18:36.402542 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:36.402507    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-549b9d8f45-lbj98"
Sep 16 02:18:36.402731 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:36.402570    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-549b9d8f45-lbj98"
Sep 16 02:18:36.407438 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:36.407415    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-549b9d8f45-lbj98"
Sep 16 02:18:36.981392 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:36.981352    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:18:37.028787 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:37.028756    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:18:37.225342 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:37.225307    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-bmj9c" event={"ID":"111d4c95-703e-4afe-bfca-fbf61d55154f","Type":"ContainerStarted","Data":"93c855567a38534bc463b4fc057377c541d2b6a221cb4404895ddbb582f9d87d"}
Sep 16 02:18:37.229428 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:37.229407    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-549b9d8f45-lbj98"
Sep 16 02:18:37.371855 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:37.371764    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:18:37.472352 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:37.472308    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:18:38.103376 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:38.103342    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:18:38.149976 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:38.149943    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:18:41.247761 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:41.247733    2579 generic.go:356] "Generic (PLEG): container finished" podID="d9ffb572-8b90-4503-82c1-84deef377ca7" containerID="ff61760bba35cf7fdbfa297a36f13fe8fadff711912d50acf4faebd39beca134" exitCode=0
Sep 16 02:18:41.248123 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:41.247789    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-hdtv6" event={"ID":"d9ffb572-8b90-4503-82c1-84deef377ca7","Type":"ContainerDied","Data":"ff61760bba35cf7fdbfa297a36f13fe8fadff711912d50acf4faebd39beca134"}
Sep 16 02:18:41.248123 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:41.248108    2579 scope.go:117] "RemoveContainer" containerID="ff61760bba35cf7fdbfa297a36f13fe8fadff711912d50acf4faebd39beca134"
Sep 16 02:18:42.254019 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:42.253981    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-hdtv6" event={"ID":"d9ffb572-8b90-4503-82c1-84deef377ca7","Type":"ContainerStarted","Data":"40c340727a6a7f1df468c7e5a4ac00f90ad33133e5d80ab06855fa3d3545ea3f"}
Sep 16 02:18:51.988939 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:51.988852    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:18:52.042815 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:18:52.042781    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:19:13.937943 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:19:13.937911    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3472392-891d-49a1-83a7-57f1f99f4b79-metrics-certs\") pod \"network-metrics-daemon-6lwr9\" (UID: \"e3472392-891d-49a1-83a7-57f1f99f4b79\") " pod="openshift-multus/network-metrics-daemon-6lwr9"
Sep 16 02:19:13.940379 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:19:13.940356    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3472392-891d-49a1-83a7-57f1f99f4b79-metrics-certs\") pod \"network-metrics-daemon-6lwr9\" (UID: \"e3472392-891d-49a1-83a7-57f1f99f4b79\") " pod="openshift-multus/network-metrics-daemon-6lwr9"
Sep 16 02:19:14.199146 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:19:14.199062    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-2ht6m\""
Sep 16 02:19:14.207786 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:19:14.207748    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6lwr9"
Sep 16 02:19:14.312307 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:19:14.312178    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:19:14.414426 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:19:14.414389    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:19:14.486997 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:19:14.486909    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-6lwr9"]
Sep 16 02:19:14.490661 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:19:14.490624    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3472392_891d_49a1_83a7_57f1f99f4b79.slice/crio-b41a93f621e607322272021f20b3976f45fa80c4e929c130f01e82294a473d95 WatchSource:0}: Error finding container b41a93f621e607322272021f20b3976f45fa80c4e929c130f01e82294a473d95: Status 404 returned error can't find the container with id b41a93f621e607322272021f20b3976f45fa80c4e929c130f01e82294a473d95
Sep 16 02:19:15.385025 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:19:15.384992    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-6lwr9" event={"ID":"e3472392-891d-49a1-83a7-57f1f99f4b79","Type":"ContainerStarted","Data":"b41a93f621e607322272021f20b3976f45fa80c4e929c130f01e82294a473d95"}
Sep 16 02:19:16.390769 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:19:16.390732    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-6lwr9" event={"ID":"e3472392-891d-49a1-83a7-57f1f99f4b79","Type":"ContainerStarted","Data":"a75437b9d8ab436fa8eff95a1e29c095239c3aa1215383df8152aec4cdc71985"}
Sep 16 02:19:16.391156 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:19:16.390796    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-6lwr9" event={"ID":"e3472392-891d-49a1-83a7-57f1f99f4b79","Type":"ContainerStarted","Data":"76d0fe953b66e5ca1842ebc779c265c88f4998a6aef85483ef3e53d2e0bd5f54"}
Sep 16 02:19:16.405830 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:19:16.405785    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-6lwr9" podStartSLOduration=375.274683519 podStartE2EDuration="6m16.405770676s" totalImagesPullingTime="1.131087157s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 02:13:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 02:19:16.404565558 +0000 UTC m=+376.788080803" watchObservedRunningTime="2026-09-16 02:19:16.405770676 +0000 UTC m=+376.789285943"
Sep 16 02:19:16.483219 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:19:16.483184    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:19:16.583962 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:19:16.583933    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:19:23.170017 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:19:23.169981    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:19:23.220402 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:19:23.220362    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:19:23.819674 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:19:23.819639    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:19:23.920139 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:19:23.920106    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:19:30.803274 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:19:30.803238    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:19:30.856458 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:19:30.856424    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:19:34.772112 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:19:34.772076    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-549b9d8f45-lbj98"]
Sep 16 02:19:34.865546 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:19:34.865511    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:19:34.965797 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:19:34.965762    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:19:35.015643 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:19:35.015611    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:19:59.793713 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:19:59.793665    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-549b9d8f45-lbj98" podUID="20f59f33-451c-4963-a96d-9f1081dd49dc" containerName="console" containerID="cri-o://5b38e2b3d366cc4b6fc95308ad669a6c97f08a51df2c9750228f0c4b38602946" gracePeriod=15
Sep 16 02:20:00.047520 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:20:00.047441    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-549b9d8f45-lbj98"
Sep 16 02:20:00.182785 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:20:00.182743    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/20f59f33-451c-4963-a96d-9f1081dd49dc-console-config\") pod \"20f59f33-451c-4963-a96d-9f1081dd49dc\" (UID: \"20f59f33-451c-4963-a96d-9f1081dd49dc\") "
Sep 16 02:20:00.182974 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:20:00.182803    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/20f59f33-451c-4963-a96d-9f1081dd49dc-console-serving-cert\") pod \"20f59f33-451c-4963-a96d-9f1081dd49dc\" (UID: \"20f59f33-451c-4963-a96d-9f1081dd49dc\") "
Sep 16 02:20:00.182974 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:20:00.182832    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/20f59f33-451c-4963-a96d-9f1081dd49dc-trusted-ca-bundle\") pod \"20f59f33-451c-4963-a96d-9f1081dd49dc\" (UID: \"20f59f33-451c-4963-a96d-9f1081dd49dc\") "
Sep 16 02:20:00.182974 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:20:00.182936    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/20f59f33-451c-4963-a96d-9f1081dd49dc-service-ca\") pod \"20f59f33-451c-4963-a96d-9f1081dd49dc\" (UID: \"20f59f33-451c-4963-a96d-9f1081dd49dc\") "
Sep 16 02:20:00.182974 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:20:00.182971    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/20f59f33-451c-4963-a96d-9f1081dd49dc-console-oauth-config\") pod \"20f59f33-451c-4963-a96d-9f1081dd49dc\" (UID: \"20f59f33-451c-4963-a96d-9f1081dd49dc\") "
Sep 16 02:20:00.183192 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:20:00.183015    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/20f59f33-451c-4963-a96d-9f1081dd49dc-oauth-serving-cert\") pod \"20f59f33-451c-4963-a96d-9f1081dd49dc\" (UID: \"20f59f33-451c-4963-a96d-9f1081dd49dc\") "
Sep 16 02:20:00.183192 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:20:00.183044    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pq59n\" (UniqueName: \"kubernetes.io/projected/20f59f33-451c-4963-a96d-9f1081dd49dc-kube-api-access-pq59n\") pod \"20f59f33-451c-4963-a96d-9f1081dd49dc\" (UID: \"20f59f33-451c-4963-a96d-9f1081dd49dc\") "
Sep 16 02:20:00.183323 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:20:00.183294    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20f59f33-451c-4963-a96d-9f1081dd49dc-console-config" (OuterVolumeSpecName: "console-config") pod "20f59f33-451c-4963-a96d-9f1081dd49dc" (UID: "20f59f33-451c-4963-a96d-9f1081dd49dc"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 02:20:00.183383 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:20:00.183323    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20f59f33-451c-4963-a96d-9f1081dd49dc-service-ca" (OuterVolumeSpecName: "service-ca") pod "20f59f33-451c-4963-a96d-9f1081dd49dc" (UID: "20f59f33-451c-4963-a96d-9f1081dd49dc"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 02:20:00.183383 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:20:00.183335    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20f59f33-451c-4963-a96d-9f1081dd49dc-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "20f59f33-451c-4963-a96d-9f1081dd49dc" (UID: "20f59f33-451c-4963-a96d-9f1081dd49dc"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 02:20:00.183457 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:20:00.183412    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20f59f33-451c-4963-a96d-9f1081dd49dc-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "20f59f33-451c-4963-a96d-9f1081dd49dc" (UID: "20f59f33-451c-4963-a96d-9f1081dd49dc"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 02:20:00.185283 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:20:00.185255    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/20f59f33-451c-4963-a96d-9f1081dd49dc-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "20f59f33-451c-4963-a96d-9f1081dd49dc" (UID: "20f59f33-451c-4963-a96d-9f1081dd49dc"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 02:20:00.185388 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:20:00.185338    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/20f59f33-451c-4963-a96d-9f1081dd49dc-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "20f59f33-451c-4963-a96d-9f1081dd49dc" (UID: "20f59f33-451c-4963-a96d-9f1081dd49dc"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 02:20:00.185388 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:20:00.185347    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20f59f33-451c-4963-a96d-9f1081dd49dc-kube-api-access-pq59n" (OuterVolumeSpecName: "kube-api-access-pq59n") pod "20f59f33-451c-4963-a96d-9f1081dd49dc" (UID: "20f59f33-451c-4963-a96d-9f1081dd49dc"). InnerVolumeSpecName "kube-api-access-pq59n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 02:20:00.283855 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:20:00.283827    2579 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/20f59f33-451c-4963-a96d-9f1081dd49dc-console-serving-cert\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:20:00.283855 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:20:00.283853    2579 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/20f59f33-451c-4963-a96d-9f1081dd49dc-trusted-ca-bundle\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:20:00.284012 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:20:00.283866    2579 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/20f59f33-451c-4963-a96d-9f1081dd49dc-service-ca\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:20:00.284012 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:20:00.283878    2579 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/20f59f33-451c-4963-a96d-9f1081dd49dc-console-oauth-config\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:20:00.284012 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:20:00.283890    2579 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/20f59f33-451c-4963-a96d-9f1081dd49dc-oauth-serving-cert\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:20:00.284012 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:20:00.283901    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pq59n\" (UniqueName: \"kubernetes.io/projected/20f59f33-451c-4963-a96d-9f1081dd49dc-kube-api-access-pq59n\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:20:00.284012 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:20:00.283913    2579 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/20f59f33-451c-4963-a96d-9f1081dd49dc-console-config\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:20:00.560850 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:20:00.560815    2579 generic.go:356] "Generic (PLEG): container finished" podID="20f59f33-451c-4963-a96d-9f1081dd49dc" containerID="5b38e2b3d366cc4b6fc95308ad669a6c97f08a51df2c9750228f0c4b38602946" exitCode=0
Sep 16 02:20:00.561029 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:20:00.560868    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-549b9d8f45-lbj98"
Sep 16 02:20:00.561029 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:20:00.560883    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-549b9d8f45-lbj98" event={"ID":"20f59f33-451c-4963-a96d-9f1081dd49dc","Type":"ContainerDied","Data":"5b38e2b3d366cc4b6fc95308ad669a6c97f08a51df2c9750228f0c4b38602946"}
Sep 16 02:20:00.561029 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:20:00.560930    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-549b9d8f45-lbj98" event={"ID":"20f59f33-451c-4963-a96d-9f1081dd49dc","Type":"ContainerDied","Data":"8227e0415103a4b3a1825c396fec8badab14f8dc586ac19559bdcf22eb582254"}
Sep 16 02:20:00.561029 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:20:00.560957    2579 scope.go:117] "RemoveContainer" containerID="5b38e2b3d366cc4b6fc95308ad669a6c97f08a51df2c9750228f0c4b38602946"
Sep 16 02:20:00.569985 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:20:00.569968    2579 scope.go:117] "RemoveContainer" containerID="5b38e2b3d366cc4b6fc95308ad669a6c97f08a51df2c9750228f0c4b38602946"
Sep 16 02:20:00.570228 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:20:00.570210    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5b38e2b3d366cc4b6fc95308ad669a6c97f08a51df2c9750228f0c4b38602946\": container with ID starting with 5b38e2b3d366cc4b6fc95308ad669a6c97f08a51df2c9750228f0c4b38602946 not found: ID does not exist" containerID="5b38e2b3d366cc4b6fc95308ad669a6c97f08a51df2c9750228f0c4b38602946"
Sep 16 02:20:00.570287 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:20:00.570241    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5b38e2b3d366cc4b6fc95308ad669a6c97f08a51df2c9750228f0c4b38602946"} err="failed to get container status \"5b38e2b3d366cc4b6fc95308ad669a6c97f08a51df2c9750228f0c4b38602946\": rpc error: code = NotFound desc = could not find container \"5b38e2b3d366cc4b6fc95308ad669a6c97f08a51df2c9750228f0c4b38602946\": container with ID starting with 5b38e2b3d366cc4b6fc95308ad669a6c97f08a51df2c9750228f0c4b38602946 not found: ID does not exist"
Sep 16 02:20:00.582334 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:20:00.582312    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-549b9d8f45-lbj98"]
Sep 16 02:20:00.585978 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:20:00.585957    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-549b9d8f45-lbj98"]
Sep 16 02:20:02.102243 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:20:02.102209    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="20f59f33-451c-4963-a96d-9f1081dd49dc" path="/var/lib/kubelet/pods/20f59f33-451c-4963-a96d-9f1081dd49dc/volumes"
Sep 16 02:20:12.116197 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:20:12.116160    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:20:12.163272 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:20:12.163242    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:20:25.550522 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:20:25.550436    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:20:29.523632 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:20:29.523591    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:20:29.688034 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:20:29.687986    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:20:33.864985 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:20:33.864951    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-76bc65b794-4z8k7"]
Sep 16 02:20:33.865394 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:20:33.865279    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20f59f33-451c-4963-a96d-9f1081dd49dc" containerName="console"
Sep 16 02:20:33.865394 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:20:33.865295    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="20f59f33-451c-4963-a96d-9f1081dd49dc" containerName="console"
Sep 16 02:20:33.865394 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:20:33.865359    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="20f59f33-451c-4963-a96d-9f1081dd49dc" containerName="console"
Sep 16 02:20:33.868425 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:20:33.868405    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-76bc65b794-4z8k7"
Sep 16 02:20:33.872399 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:20:33.872377    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 16 02:20:33.872592 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:20:33.872401    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 16 02:20:33.872670 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:20:33.872656    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 16 02:20:33.873226 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:20:33.873176    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 16 02:20:33.873444 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:20:33.873425    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-wrs7f\""
Sep 16 02:20:33.874401 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:20:33.874382    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 16 02:20:33.877180 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:20:33.877161    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 16 02:20:33.885306 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:20:33.885283    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-76bc65b794-4z8k7"]
Sep 16 02:20:33.938588 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:20:33.938538    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a972cd14-97a1-4889-aea5-ea1d4a18c861-oauth-serving-cert\") pod \"console-76bc65b794-4z8k7\" (UID: \"a972cd14-97a1-4889-aea5-ea1d4a18c861\") " pod="openshift-console/console-76bc65b794-4z8k7"
Sep 16 02:20:33.938763 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:20:33.938661    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a972cd14-97a1-4889-aea5-ea1d4a18c861-service-ca\") pod \"console-76bc65b794-4z8k7\" (UID: \"a972cd14-97a1-4889-aea5-ea1d4a18c861\") " pod="openshift-console/console-76bc65b794-4z8k7"
Sep 16 02:20:33.938763 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:20:33.938739    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a972cd14-97a1-4889-aea5-ea1d4a18c861-trusted-ca-bundle\") pod \"console-76bc65b794-4z8k7\" (UID: \"a972cd14-97a1-4889-aea5-ea1d4a18c861\") " pod="openshift-console/console-76bc65b794-4z8k7"
Sep 16 02:20:33.938880 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:20:33.938819    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a972cd14-97a1-4889-aea5-ea1d4a18c861-console-config\") pod \"console-76bc65b794-4z8k7\" (UID: \"a972cd14-97a1-4889-aea5-ea1d4a18c861\") " pod="openshift-console/console-76bc65b794-4z8k7"
Sep 16 02:20:33.938934 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:20:33.938884    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a972cd14-97a1-4889-aea5-ea1d4a18c861-console-serving-cert\") pod \"console-76bc65b794-4z8k7\" (UID: \"a972cd14-97a1-4889-aea5-ea1d4a18c861\") " pod="openshift-console/console-76bc65b794-4z8k7"
Sep 16 02:20:33.938934 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:20:33.938913    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kbfns\" (UniqueName: \"kubernetes.io/projected/a972cd14-97a1-4889-aea5-ea1d4a18c861-kube-api-access-kbfns\") pod \"console-76bc65b794-4z8k7\" (UID: \"a972cd14-97a1-4889-aea5-ea1d4a18c861\") " pod="openshift-console/console-76bc65b794-4z8k7"
Sep 16 02:20:33.939034 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:20:33.938981    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a972cd14-97a1-4889-aea5-ea1d4a18c861-console-oauth-config\") pod \"console-76bc65b794-4z8k7\" (UID: \"a972cd14-97a1-4889-aea5-ea1d4a18c861\") " pod="openshift-console/console-76bc65b794-4z8k7"
Sep 16 02:20:33.941385 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:20:33.941356    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:20:33.988418 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:20:33.988383    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:20:34.040097 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:20:34.040065    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a972cd14-97a1-4889-aea5-ea1d4a18c861-service-ca\") pod \"console-76bc65b794-4z8k7\" (UID: \"a972cd14-97a1-4889-aea5-ea1d4a18c861\") " pod="openshift-console/console-76bc65b794-4z8k7"
Sep 16 02:20:34.040253 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:20:34.040102    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a972cd14-97a1-4889-aea5-ea1d4a18c861-trusted-ca-bundle\") pod \"console-76bc65b794-4z8k7\" (UID: \"a972cd14-97a1-4889-aea5-ea1d4a18c861\") " pod="openshift-console/console-76bc65b794-4z8k7"
Sep 16 02:20:34.040253 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:20:34.040129    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a972cd14-97a1-4889-aea5-ea1d4a18c861-console-config\") pod \"console-76bc65b794-4z8k7\" (UID: \"a972cd14-97a1-4889-aea5-ea1d4a18c861\") " pod="openshift-console/console-76bc65b794-4z8k7"
Sep 16 02:20:34.040253 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:20:34.040157    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a972cd14-97a1-4889-aea5-ea1d4a18c861-console-serving-cert\") pod \"console-76bc65b794-4z8k7\" (UID: \"a972cd14-97a1-4889-aea5-ea1d4a18c861\") " pod="openshift-console/console-76bc65b794-4z8k7"
Sep 16 02:20:34.040253 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:20:34.040183    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kbfns\" (UniqueName: \"kubernetes.io/projected/a972cd14-97a1-4889-aea5-ea1d4a18c861-kube-api-access-kbfns\") pod \"console-76bc65b794-4z8k7\" (UID: \"a972cd14-97a1-4889-aea5-ea1d4a18c861\") " pod="openshift-console/console-76bc65b794-4z8k7"
Sep 16 02:20:34.040253 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:20:34.040216    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a972cd14-97a1-4889-aea5-ea1d4a18c861-console-oauth-config\") pod \"console-76bc65b794-4z8k7\" (UID: \"a972cd14-97a1-4889-aea5-ea1d4a18c861\") " pod="openshift-console/console-76bc65b794-4z8k7"
Sep 16 02:20:34.040253 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:20:34.040254    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a972cd14-97a1-4889-aea5-ea1d4a18c861-oauth-serving-cert\") pod \"console-76bc65b794-4z8k7\" (UID: \"a972cd14-97a1-4889-aea5-ea1d4a18c861\") " pod="openshift-console/console-76bc65b794-4z8k7"
Sep 16 02:20:34.040950 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:20:34.040878    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a972cd14-97a1-4889-aea5-ea1d4a18c861-service-ca\") pod \"console-76bc65b794-4z8k7\" (UID: \"a972cd14-97a1-4889-aea5-ea1d4a18c861\") " pod="openshift-console/console-76bc65b794-4z8k7"
Sep 16 02:20:34.040950 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:20:34.040947    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a972cd14-97a1-4889-aea5-ea1d4a18c861-console-config\") pod \"console-76bc65b794-4z8k7\" (UID: \"a972cd14-97a1-4889-aea5-ea1d4a18c861\") " pod="openshift-console/console-76bc65b794-4z8k7"
Sep 16 02:20:34.040950 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:20:34.040959    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a972cd14-97a1-4889-aea5-ea1d4a18c861-oauth-serving-cert\") pod \"console-76bc65b794-4z8k7\" (UID: \"a972cd14-97a1-4889-aea5-ea1d4a18c861\") " pod="openshift-console/console-76bc65b794-4z8k7"
Sep 16 02:20:34.040950 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:20:34.040986    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a972cd14-97a1-4889-aea5-ea1d4a18c861-trusted-ca-bundle\") pod \"console-76bc65b794-4z8k7\" (UID: \"a972cd14-97a1-4889-aea5-ea1d4a18c861\") " pod="openshift-console/console-76bc65b794-4z8k7"
Sep 16 02:20:34.042789 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:20:34.042760    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a972cd14-97a1-4889-aea5-ea1d4a18c861-console-oauth-config\") pod \"console-76bc65b794-4z8k7\" (UID: \"a972cd14-97a1-4889-aea5-ea1d4a18c861\") " pod="openshift-console/console-76bc65b794-4z8k7"
Sep 16 02:20:34.042872 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:20:34.042856    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a972cd14-97a1-4889-aea5-ea1d4a18c861-console-serving-cert\") pod \"console-76bc65b794-4z8k7\" (UID: \"a972cd14-97a1-4889-aea5-ea1d4a18c861\") " pod="openshift-console/console-76bc65b794-4z8k7"
Sep 16 02:20:34.047279 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:20:34.047258    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kbfns\" (UniqueName: \"kubernetes.io/projected/a972cd14-97a1-4889-aea5-ea1d4a18c861-kube-api-access-kbfns\") pod \"console-76bc65b794-4z8k7\" (UID: \"a972cd14-97a1-4889-aea5-ea1d4a18c861\") " pod="openshift-console/console-76bc65b794-4z8k7"
Sep 16 02:20:34.065653 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:20:34.065625    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:20:34.166878 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:20:34.166802    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:20:34.178705 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:20:34.178679    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-76bc65b794-4z8k7"
Sep 16 02:20:34.453445 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:20:34.453418    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-76bc65b794-4z8k7"]
Sep 16 02:20:34.456230 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:20:34.456204    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda972cd14_97a1_4889_aea5_ea1d4a18c861.slice/crio-efc029a3f3026e4caad7e47c58d39ae44f1477c2c4e5fadec4646e9bf08bc90f WatchSource:0}: Error finding container efc029a3f3026e4caad7e47c58d39ae44f1477c2c4e5fadec4646e9bf08bc90f: Status 404 returned error can't find the container with id efc029a3f3026e4caad7e47c58d39ae44f1477c2c4e5fadec4646e9bf08bc90f
Sep 16 02:20:34.662025 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:20:34.661990    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:20:34.690742 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:20:34.690707    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-76bc65b794-4z8k7" event={"ID":"a972cd14-97a1-4889-aea5-ea1d4a18c861","Type":"ContainerStarted","Data":"6d6505fad3a84b1611c89dfe782248ffe2c18f4f2dcdba11c45d0c2b447e1cba"}
Sep 16 02:20:34.690912 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:20:34.690753    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-76bc65b794-4z8k7" event={"ID":"a972cd14-97a1-4889-aea5-ea1d4a18c861","Type":"ContainerStarted","Data":"efc029a3f3026e4caad7e47c58d39ae44f1477c2c4e5fadec4646e9bf08bc90f"}
Sep 16 02:20:34.708524 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:20:34.708418    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-76bc65b794-4z8k7" podStartSLOduration=1.7084050579999999 podStartE2EDuration="1.708405058s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 02:20:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 02:20:34.707431026 +0000 UTC m=+455.090946271" watchObservedRunningTime="2026-09-16 02:20:34.708405058 +0000 UTC m=+455.091920303"
Sep 16 02:20:34.762282 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:20:34.762251    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:20:41.506874 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:20:41.506838    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:20:41.570447 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:20:41.570412    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:20:44.178759 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:20:44.178724    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-76bc65b794-4z8k7"
Sep 16 02:20:44.179124 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:20:44.178777    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-76bc65b794-4z8k7"
Sep 16 02:20:44.184217 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:20:44.184190    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-76bc65b794-4z8k7"
Sep 16 02:20:44.730792 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:20:44.730763    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-76bc65b794-4z8k7"
Sep 16 02:20:44.873805 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:20:44.873769    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:20:44.974853 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:20:44.974819    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:20:45.025447 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:20:45.025349    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:20:52.018026 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:20:52.017983    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:20:52.120462 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:20:52.120425    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:20:52.301989 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:20:52.301899    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:21:42.628397 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:21:42.628336    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:22:07.116268 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:22:07.116171    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:22:07.213725 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:22:07.213693    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:22:08.684705 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:22:08.684657    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:22:08.732626 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:22:08.732587    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:22:28.682179 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:22:28.682142    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:22:29.488125 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:22:29.488090    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:22:32.277266 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:22:32.277230    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:22:32.327174 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:22:32.327142    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:22:34.086403 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:22:34.086368    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:22:34.186627 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:22:34.186599    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:22:34.193363 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:22:34.193318    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-bth2m"]
Sep 16 02:22:34.198138 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:22:34.198109    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-bth2m"
Sep 16 02:22:34.200705 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:22:34.200685    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Sep 16 02:22:34.200705 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:22:34.200700    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Sep 16 02:22:34.206627 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:22:34.206606    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-bth2m"]
Sep 16 02:22:34.360367 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:22:34.360277    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m5jvp\" (UniqueName: \"kubernetes.io/projected/3a2ecdc8-5a59-4f5b-9100-6b9481ac83f4-kube-api-access-m5jvp\") pod \"cert-manager-webhook-6486ffd997-bth2m\" (UID: \"3a2ecdc8-5a59-4f5b-9100-6b9481ac83f4\") " pod="cert-manager/cert-manager-webhook-6486ffd997-bth2m"
Sep 16 02:22:34.360367 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:22:34.360317    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3a2ecdc8-5a59-4f5b-9100-6b9481ac83f4-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-bth2m\" (UID: \"3a2ecdc8-5a59-4f5b-9100-6b9481ac83f4\") " pod="cert-manager/cert-manager-webhook-6486ffd997-bth2m"
Sep 16 02:22:34.461160 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:22:34.461126    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m5jvp\" (UniqueName: \"kubernetes.io/projected/3a2ecdc8-5a59-4f5b-9100-6b9481ac83f4-kube-api-access-m5jvp\") pod \"cert-manager-webhook-6486ffd997-bth2m\" (UID: \"3a2ecdc8-5a59-4f5b-9100-6b9481ac83f4\") " pod="cert-manager/cert-manager-webhook-6486ffd997-bth2m"
Sep 16 02:22:34.461318 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:22:34.461161    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3a2ecdc8-5a59-4f5b-9100-6b9481ac83f4-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-bth2m\" (UID: \"3a2ecdc8-5a59-4f5b-9100-6b9481ac83f4\") " pod="cert-manager/cert-manager-webhook-6486ffd997-bth2m"
Sep 16 02:22:34.469825 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:22:34.469793    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3a2ecdc8-5a59-4f5b-9100-6b9481ac83f4-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-bth2m\" (UID: \"3a2ecdc8-5a59-4f5b-9100-6b9481ac83f4\") " pod="cert-manager/cert-manager-webhook-6486ffd997-bth2m"
Sep 16 02:22:34.470030 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:22:34.470010    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m5jvp\" (UniqueName: \"kubernetes.io/projected/3a2ecdc8-5a59-4f5b-9100-6b9481ac83f4-kube-api-access-m5jvp\") pod \"cert-manager-webhook-6486ffd997-bth2m\" (UID: \"3a2ecdc8-5a59-4f5b-9100-6b9481ac83f4\") " pod="cert-manager/cert-manager-webhook-6486ffd997-bth2m"
Sep 16 02:22:34.517937 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:22:34.517906    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-bth2m"
Sep 16 02:22:34.797450 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:22:34.797421    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-bth2m"]
Sep 16 02:22:34.801103 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:22:34.801073    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3a2ecdc8_5a59_4f5b_9100_6b9481ac83f4.slice/crio-ff5b6d60d80cbdf9ac26b0c8f949877ee59244d0e1fc8f246931d1074844e2f7 WatchSource:0}: Error finding container ff5b6d60d80cbdf9ac26b0c8f949877ee59244d0e1fc8f246931d1074844e2f7: Status 404 returned error can't find the container with id ff5b6d60d80cbdf9ac26b0c8f949877ee59244d0e1fc8f246931d1074844e2f7
Sep 16 02:22:34.825884 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:22:34.825840    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:22:34.927967 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:22:34.927925    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:22:35.129895 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:22:35.129801    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-bth2m" event={"ID":"3a2ecdc8-5a59-4f5b-9100-6b9481ac83f4","Type":"ContainerStarted","Data":"ff5b6d60d80cbdf9ac26b0c8f949877ee59244d0e1fc8f246931d1074844e2f7"}
Sep 16 02:22:36.294699 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:22:36.294665    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:22:36.494665 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:22:36.494628    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:22:39.292632 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:22:39.292598    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:22:39.392601 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:22:39.392569    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:22:40.152964 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:22:40.152931    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-bth2m" event={"ID":"3a2ecdc8-5a59-4f5b-9100-6b9481ac83f4","Type":"ContainerStarted","Data":"8c9a7e97512d359733bf60b80df5c2ddfa1f3259054ec76d066268f275bb42c2"}
Sep 16 02:22:40.153106 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:22:40.152988    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-bth2m"
Sep 16 02:22:40.168175 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:22:40.168129    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-bth2m" podStartSLOduration=1.763593995 podStartE2EDuration="6.168112965s" totalImagesPullingTime="4.40451897s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 02:22:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 02:22:40.167438514 +0000 UTC m=+580.550953824" watchObservedRunningTime="2026-09-16 02:22:40.168112965 +0000 UTC m=+580.551628209"
Sep 16 02:22:41.487176 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:22:41.487136    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:22:41.585039 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:22:41.585003    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:22:43.101981 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:22:43.101941    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:22:43.198519 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:22:43.198492    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:22:44.306257 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:22:44.306216    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:22:44.745282 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:22:44.745244    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:22:44.846065 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:22:44.846029    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:22:45.131379 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:22:45.131308    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jmk65"]
Sep 16 02:22:45.134626 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:22:45.134604    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jmk65"
Sep 16 02:22:45.137139 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:22:45.137119    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 16 02:22:45.137251 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:22:45.137141    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 16 02:22:45.137890 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:22:45.137876    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-7hgmr\""
Sep 16 02:22:45.143931 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:22:45.143910    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jmk65"]
Sep 16 02:22:45.145450 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:22:45.145430    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cdf71ee6-224a-42a8-a95a-58c7486d2241-tmp\") pod \"openshift-lws-operator-fd8ccff4c-jmk65\" (UID: \"cdf71ee6-224a-42a8-a95a-58c7486d2241\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jmk65"
Sep 16 02:22:45.145544 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:22:45.145528    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k2hbc\" (UniqueName: \"kubernetes.io/projected/cdf71ee6-224a-42a8-a95a-58c7486d2241-kube-api-access-k2hbc\") pod \"openshift-lws-operator-fd8ccff4c-jmk65\" (UID: \"cdf71ee6-224a-42a8-a95a-58c7486d2241\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jmk65"
Sep 16 02:22:45.246281 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:22:45.246242    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k2hbc\" (UniqueName: \"kubernetes.io/projected/cdf71ee6-224a-42a8-a95a-58c7486d2241-kube-api-access-k2hbc\") pod \"openshift-lws-operator-fd8ccff4c-jmk65\" (UID: \"cdf71ee6-224a-42a8-a95a-58c7486d2241\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jmk65"
Sep 16 02:22:45.246435 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:22:45.246293    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cdf71ee6-224a-42a8-a95a-58c7486d2241-tmp\") pod \"openshift-lws-operator-fd8ccff4c-jmk65\" (UID: \"cdf71ee6-224a-42a8-a95a-58c7486d2241\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jmk65"
Sep 16 02:22:45.246646 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:22:45.246630    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cdf71ee6-224a-42a8-a95a-58c7486d2241-tmp\") pod \"openshift-lws-operator-fd8ccff4c-jmk65\" (UID: \"cdf71ee6-224a-42a8-a95a-58c7486d2241\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jmk65"
Sep 16 02:22:45.254780 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:22:45.254753    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k2hbc\" (UniqueName: \"kubernetes.io/projected/cdf71ee6-224a-42a8-a95a-58c7486d2241-kube-api-access-k2hbc\") pod \"openshift-lws-operator-fd8ccff4c-jmk65\" (UID: \"cdf71ee6-224a-42a8-a95a-58c7486d2241\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jmk65"
Sep 16 02:22:45.443519 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:22:45.443492    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jmk65"
Sep 16 02:22:45.719267 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:22:45.719239    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jmk65"]
Sep 16 02:22:45.721627 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:22:45.721599    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcdf71ee6_224a_42a8_a95a_58c7486d2241.slice/crio-85b01084dd30319917defa52c84378a6c5f70db37b487c5c15e1bcbbf9e843c3 WatchSource:0}: Error finding container 85b01084dd30319917defa52c84378a6c5f70db37b487c5c15e1bcbbf9e843c3: Status 404 returned error can't find the container with id 85b01084dd30319917defa52c84378a6c5f70db37b487c5c15e1bcbbf9e843c3
Sep 16 02:22:46.158571 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:22:46.158494    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-bth2m"
Sep 16 02:22:46.176597 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:22:46.176563    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jmk65" event={"ID":"cdf71ee6-224a-42a8-a95a-58c7486d2241","Type":"ContainerStarted","Data":"85b01084dd30319917defa52c84378a6c5f70db37b487c5c15e1bcbbf9e843c3"}
Sep 16 02:22:49.192382 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:22:49.192351    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jmk65" event={"ID":"cdf71ee6-224a-42a8-a95a-58c7486d2241","Type":"ContainerStarted","Data":"ab439af632fa8a0c641ca055b3b6e214e1eb25e385d2ed9cdbefc8f7342bb7ce"}
Sep 16 02:22:49.212036 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:22:49.211984    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jmk65" podStartSLOduration=1.477071043 podStartE2EDuration="4.211970113s" totalImagesPullingTime="2.73489907s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 02:22:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 02:22:49.210273779 +0000 UTC m=+589.593789025" watchObservedRunningTime="2026-09-16 02:22:49.211970113 +0000 UTC m=+589.595485354"
Sep 16 02:22:50.064531 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:22:50.064490    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:22:50.162882 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:22:50.162850    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:22:52.225157 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:22:52.225106    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:23:00.072708 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:00.072676    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-c8q8z_ebf846fa-af7d-4fdd-9662-b7c533c36ddd/console-operator/2.log"
Sep 16 02:23:00.074528 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:00.074501    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-c8q8z_ebf846fa-af7d-4fdd-9662-b7c533c36ddd/console-operator/2.log"
Sep 16 02:23:07.317677 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:07.317644    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:23:08.108623 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:08.108586    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:23:08.818560 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:08.818529    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-79c8b99f94-2wfnm"]
Sep 16 02:23:08.821667 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:08.821634    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-79c8b99f94-2wfnm"
Sep 16 02:23:08.824537 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:08.824514    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Sep 16 02:23:08.825514 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:08.825497    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-t82b9\""
Sep 16 02:23:08.825629 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:08.825612    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Sep 16 02:23:08.825709 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:08.825695    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Sep 16 02:23:08.847157 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:08.847132    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-79c8b99f94-2wfnm"]
Sep 16 02:23:08.927631 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:08.927602    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/b7970ae7-f890-4baa-825b-8dedd185dce5-metrics-cert\") pod \"lws-controller-manager-79c8b99f94-2wfnm\" (UID: \"b7970ae7-f890-4baa-825b-8dedd185dce5\") " pod="openshift-lws-operator/lws-controller-manager-79c8b99f94-2wfnm"
Sep 16 02:23:08.927777 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:08.927642    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vq4nn\" (UniqueName: \"kubernetes.io/projected/b7970ae7-f890-4baa-825b-8dedd185dce5-kube-api-access-vq4nn\") pod \"lws-controller-manager-79c8b99f94-2wfnm\" (UID: \"b7970ae7-f890-4baa-825b-8dedd185dce5\") " pod="openshift-lws-operator/lws-controller-manager-79c8b99f94-2wfnm"
Sep 16 02:23:08.927777 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:08.927706    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b7970ae7-f890-4baa-825b-8dedd185dce5-cert\") pod \"lws-controller-manager-79c8b99f94-2wfnm\" (UID: \"b7970ae7-f890-4baa-825b-8dedd185dce5\") " pod="openshift-lws-operator/lws-controller-manager-79c8b99f94-2wfnm"
Sep 16 02:23:08.927848 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:08.927778    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/b7970ae7-f890-4baa-825b-8dedd185dce5-manager-config\") pod \"lws-controller-manager-79c8b99f94-2wfnm\" (UID: \"b7970ae7-f890-4baa-825b-8dedd185dce5\") " pod="openshift-lws-operator/lws-controller-manager-79c8b99f94-2wfnm"
Sep 16 02:23:09.028442 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:09.028410    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vq4nn\" (UniqueName: \"kubernetes.io/projected/b7970ae7-f890-4baa-825b-8dedd185dce5-kube-api-access-vq4nn\") pod \"lws-controller-manager-79c8b99f94-2wfnm\" (UID: \"b7970ae7-f890-4baa-825b-8dedd185dce5\") " pod="openshift-lws-operator/lws-controller-manager-79c8b99f94-2wfnm"
Sep 16 02:23:09.028620 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:09.028458    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b7970ae7-f890-4baa-825b-8dedd185dce5-cert\") pod \"lws-controller-manager-79c8b99f94-2wfnm\" (UID: \"b7970ae7-f890-4baa-825b-8dedd185dce5\") " pod="openshift-lws-operator/lws-controller-manager-79c8b99f94-2wfnm"
Sep 16 02:23:09.028620 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:09.028527    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/b7970ae7-f890-4baa-825b-8dedd185dce5-manager-config\") pod \"lws-controller-manager-79c8b99f94-2wfnm\" (UID: \"b7970ae7-f890-4baa-825b-8dedd185dce5\") " pod="openshift-lws-operator/lws-controller-manager-79c8b99f94-2wfnm"
Sep 16 02:23:09.028620 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:09.028561    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/b7970ae7-f890-4baa-825b-8dedd185dce5-metrics-cert\") pod \"lws-controller-manager-79c8b99f94-2wfnm\" (UID: \"b7970ae7-f890-4baa-825b-8dedd185dce5\") " pod="openshift-lws-operator/lws-controller-manager-79c8b99f94-2wfnm"
Sep 16 02:23:09.029322 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:09.029249    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/b7970ae7-f890-4baa-825b-8dedd185dce5-manager-config\") pod \"lws-controller-manager-79c8b99f94-2wfnm\" (UID: \"b7970ae7-f890-4baa-825b-8dedd185dce5\") " pod="openshift-lws-operator/lws-controller-manager-79c8b99f94-2wfnm"
Sep 16 02:23:09.031070 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:09.031052    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b7970ae7-f890-4baa-825b-8dedd185dce5-cert\") pod \"lws-controller-manager-79c8b99f94-2wfnm\" (UID: \"b7970ae7-f890-4baa-825b-8dedd185dce5\") " pod="openshift-lws-operator/lws-controller-manager-79c8b99f94-2wfnm"
Sep 16 02:23:09.031145 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:09.031129    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/b7970ae7-f890-4baa-825b-8dedd185dce5-metrics-cert\") pod \"lws-controller-manager-79c8b99f94-2wfnm\" (UID: \"b7970ae7-f890-4baa-825b-8dedd185dce5\") " pod="openshift-lws-operator/lws-controller-manager-79c8b99f94-2wfnm"
Sep 16 02:23:09.048643 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:09.048623    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vq4nn\" (UniqueName: \"kubernetes.io/projected/b7970ae7-f890-4baa-825b-8dedd185dce5-kube-api-access-vq4nn\") pod \"lws-controller-manager-79c8b99f94-2wfnm\" (UID: \"b7970ae7-f890-4baa-825b-8dedd185dce5\") " pod="openshift-lws-operator/lws-controller-manager-79c8b99f94-2wfnm"
Sep 16 02:23:09.130765 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:09.130680    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-79c8b99f94-2wfnm"
Sep 16 02:23:09.320709 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:09.320639    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:23:09.418322 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:09.418250    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-79c8b99f94-2wfnm"]
Sep 16 02:23:09.422006 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:23:09.421972    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7970ae7_f890_4baa_825b_8dedd185dce5.slice/crio-36fa0026c078cdcae99fc416e892c9e6d58055acc26161c9c7b4bfe7f371bbf4 WatchSource:0}: Error finding container 36fa0026c078cdcae99fc416e892c9e6d58055acc26161c9c7b4bfe7f371bbf4: Status 404 returned error can't find the container with id 36fa0026c078cdcae99fc416e892c9e6d58055acc26161c9c7b4bfe7f371bbf4
Sep 16 02:23:09.424913 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:09.424877    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 16 02:23:09.556266 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:09.556232    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:23:09.703622 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:09.703579    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:23:09.858954 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:09.858920    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-74fbd865d6-5jt27"]
Sep 16 02:23:09.863346 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:09.863326    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-74fbd865d6-5jt27"
Sep 16 02:23:09.885213 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:09.885184    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 16 02:23:09.885350 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:09.885236    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 16 02:23:09.885350 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:09.885273    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 16 02:23:09.886170 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:09.886151    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-xzpsv\""
Sep 16 02:23:09.886256 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:09.886158    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 16 02:23:09.905980 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:09.905954    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-74fbd865d6-5jt27"]
Sep 16 02:23:09.936580 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:09.936552    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f3048123-5d9f-4e0c-a3e6-56d2bbd04f5f-webhook-cert\") pod \"opendatahub-operator-controller-manager-74fbd865d6-5jt27\" (UID: \"f3048123-5d9f-4e0c-a3e6-56d2bbd04f5f\") " pod="opendatahub/opendatahub-operator-controller-manager-74fbd865d6-5jt27"
Sep 16 02:23:09.936679 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:09.936584    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f3048123-5d9f-4e0c-a3e6-56d2bbd04f5f-tmp\") pod \"opendatahub-operator-controller-manager-74fbd865d6-5jt27\" (UID: \"f3048123-5d9f-4e0c-a3e6-56d2bbd04f5f\") " pod="opendatahub/opendatahub-operator-controller-manager-74fbd865d6-5jt27"
Sep 16 02:23:09.936679 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:09.936618    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k6mw8\" (UniqueName: \"kubernetes.io/projected/f3048123-5d9f-4e0c-a3e6-56d2bbd04f5f-kube-api-access-k6mw8\") pod \"opendatahub-operator-controller-manager-74fbd865d6-5jt27\" (UID: \"f3048123-5d9f-4e0c-a3e6-56d2bbd04f5f\") " pod="opendatahub/opendatahub-operator-controller-manager-74fbd865d6-5jt27"
Sep 16 02:23:09.936771 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:09.936693    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f3048123-5d9f-4e0c-a3e6-56d2bbd04f5f-manifests\") pod \"opendatahub-operator-controller-manager-74fbd865d6-5jt27\" (UID: \"f3048123-5d9f-4e0c-a3e6-56d2bbd04f5f\") " pod="opendatahub/opendatahub-operator-controller-manager-74fbd865d6-5jt27"
Sep 16 02:23:09.936771 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:09.936707    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f3048123-5d9f-4e0c-a3e6-56d2bbd04f5f-apiservice-cert\") pod \"opendatahub-operator-controller-manager-74fbd865d6-5jt27\" (UID: \"f3048123-5d9f-4e0c-a3e6-56d2bbd04f5f\") " pod="opendatahub/opendatahub-operator-controller-manager-74fbd865d6-5jt27"
Sep 16 02:23:10.038035 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:10.037963    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f3048123-5d9f-4e0c-a3e6-56d2bbd04f5f-tmp\") pod \"opendatahub-operator-controller-manager-74fbd865d6-5jt27\" (UID: \"f3048123-5d9f-4e0c-a3e6-56d2bbd04f5f\") " pod="opendatahub/opendatahub-operator-controller-manager-74fbd865d6-5jt27"
Sep 16 02:23:10.038035 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:10.038012    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k6mw8\" (UniqueName: \"kubernetes.io/projected/f3048123-5d9f-4e0c-a3e6-56d2bbd04f5f-kube-api-access-k6mw8\") pod \"opendatahub-operator-controller-manager-74fbd865d6-5jt27\" (UID: \"f3048123-5d9f-4e0c-a3e6-56d2bbd04f5f\") " pod="opendatahub/opendatahub-operator-controller-manager-74fbd865d6-5jt27"
Sep 16 02:23:10.038229 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:10.038063    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f3048123-5d9f-4e0c-a3e6-56d2bbd04f5f-manifests\") pod \"opendatahub-operator-controller-manager-74fbd865d6-5jt27\" (UID: \"f3048123-5d9f-4e0c-a3e6-56d2bbd04f5f\") " pod="opendatahub/opendatahub-operator-controller-manager-74fbd865d6-5jt27"
Sep 16 02:23:10.038229 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:10.038080    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f3048123-5d9f-4e0c-a3e6-56d2bbd04f5f-apiservice-cert\") pod \"opendatahub-operator-controller-manager-74fbd865d6-5jt27\" (UID: \"f3048123-5d9f-4e0c-a3e6-56d2bbd04f5f\") " pod="opendatahub/opendatahub-operator-controller-manager-74fbd865d6-5jt27"
Sep 16 02:23:10.038229 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:10.038094    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f3048123-5d9f-4e0c-a3e6-56d2bbd04f5f-webhook-cert\") pod \"opendatahub-operator-controller-manager-74fbd865d6-5jt27\" (UID: \"f3048123-5d9f-4e0c-a3e6-56d2bbd04f5f\") " pod="opendatahub/opendatahub-operator-controller-manager-74fbd865d6-5jt27"
Sep 16 02:23:10.038482 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:10.038447    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f3048123-5d9f-4e0c-a3e6-56d2bbd04f5f-manifests\") pod \"opendatahub-operator-controller-manager-74fbd865d6-5jt27\" (UID: \"f3048123-5d9f-4e0c-a3e6-56d2bbd04f5f\") " pod="opendatahub/opendatahub-operator-controller-manager-74fbd865d6-5jt27"
Sep 16 02:23:10.040412 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:10.040384    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f3048123-5d9f-4e0c-a3e6-56d2bbd04f5f-tmp\") pod \"opendatahub-operator-controller-manager-74fbd865d6-5jt27\" (UID: \"f3048123-5d9f-4e0c-a3e6-56d2bbd04f5f\") " pod="opendatahub/opendatahub-operator-controller-manager-74fbd865d6-5jt27"
Sep 16 02:23:10.040546 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:10.040528    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f3048123-5d9f-4e0c-a3e6-56d2bbd04f5f-apiservice-cert\") pod \"opendatahub-operator-controller-manager-74fbd865d6-5jt27\" (UID: \"f3048123-5d9f-4e0c-a3e6-56d2bbd04f5f\") " pod="opendatahub/opendatahub-operator-controller-manager-74fbd865d6-5jt27"
Sep 16 02:23:10.040633 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:10.040614    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f3048123-5d9f-4e0c-a3e6-56d2bbd04f5f-webhook-cert\") pod \"opendatahub-operator-controller-manager-74fbd865d6-5jt27\" (UID: \"f3048123-5d9f-4e0c-a3e6-56d2bbd04f5f\") " pod="opendatahub/opendatahub-operator-controller-manager-74fbd865d6-5jt27"
Sep 16 02:23:10.063226 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:10.063196    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k6mw8\" (UniqueName: \"kubernetes.io/projected/f3048123-5d9f-4e0c-a3e6-56d2bbd04f5f-kube-api-access-k6mw8\") pod \"opendatahub-operator-controller-manager-74fbd865d6-5jt27\" (UID: \"f3048123-5d9f-4e0c-a3e6-56d2bbd04f5f\") " pod="opendatahub/opendatahub-operator-controller-manager-74fbd865d6-5jt27"
Sep 16 02:23:10.092364 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:10.092336    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:23:10.172796 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:10.172757    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-74fbd865d6-5jt27"
Sep 16 02:23:10.197331 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:10.197291    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:23:10.276452 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:10.276422    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-79c8b99f94-2wfnm" event={"ID":"b7970ae7-f890-4baa-825b-8dedd185dce5","Type":"ContainerStarted","Data":"36fa0026c078cdcae99fc416e892c9e6d58055acc26161c9c7b4bfe7f371bbf4"}
Sep 16 02:23:10.299559 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:10.299485    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:23:10.398728 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:10.398696    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:23:10.463927 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:10.463904    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-74fbd865d6-5jt27"]
Sep 16 02:23:10.466099 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:23:10.466069    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3048123_5d9f_4e0c_a3e6_56d2bbd04f5f.slice/crio-4d637f4c5a7e654c9ce8e55d87848ece2b95d4351986bb6da8369eb562ada64b WatchSource:0}: Error finding container 4d637f4c5a7e654c9ce8e55d87848ece2b95d4351986bb6da8369eb562ada64b: Status 404 returned error can't find the container with id 4d637f4c5a7e654c9ce8e55d87848ece2b95d4351986bb6da8369eb562ada64b
Sep 16 02:23:10.468199 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:10.468170    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:23:10.555293 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:10.555219    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:23:10.734795 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:10.734758    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:23:10.827423 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:10.827339    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:23:11.284676 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:11.284638    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-74fbd865d6-5jt27" event={"ID":"f3048123-5d9f-4e0c-a3e6-56d2bbd04f5f","Type":"ContainerStarted","Data":"4d637f4c5a7e654c9ce8e55d87848ece2b95d4351986bb6da8369eb562ada64b"}
Sep 16 02:23:14.300611 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:14.300575    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-74fbd865d6-5jt27" event={"ID":"f3048123-5d9f-4e0c-a3e6-56d2bbd04f5f","Type":"ContainerStarted","Data":"3cb7d6759814a29a3199faee3f0457988e1066f6a0ad8a1d6d6dfe31f465bc10"}
Sep 16 02:23:14.302156 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:14.302131    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-79c8b99f94-2wfnm" event={"ID":"b7970ae7-f890-4baa-825b-8dedd185dce5","Type":"ContainerStarted","Data":"db8a76a67fc0100785d47ba36456c527d65e2a3e46af385fab4bdcc4ef665088"}
Sep 16 02:23:14.302300 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:14.302186    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-79c8b99f94-2wfnm"
Sep 16 02:23:14.347145 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:14.347086    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-79c8b99f94-2wfnm" podStartSLOduration=2.356499443 podStartE2EDuration="6.347067691s" totalImagesPullingTime="3.990568248s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 02:23:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 02:23:14.345792061 +0000 UTC m=+614.729307307" watchObservedRunningTime="2026-09-16 02:23:14.347067691 +0000 UTC m=+614.730582937"
Sep 16 02:23:15.308296 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:15.308263    2579 generic.go:356] "Generic (PLEG): container finished" podID="f3048123-5d9f-4e0c-a3e6-56d2bbd04f5f" containerID="3cb7d6759814a29a3199faee3f0457988e1066f6a0ad8a1d6d6dfe31f465bc10" exitCode=0
Sep 16 02:23:15.308736 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:15.308328    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-74fbd865d6-5jt27" event={"ID":"f3048123-5d9f-4e0c-a3e6-56d2bbd04f5f","Type":"ContainerDied","Data":"3cb7d6759814a29a3199faee3f0457988e1066f6a0ad8a1d6d6dfe31f465bc10"}
Sep 16 02:23:16.315224 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:16.315187    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-74fbd865d6-5jt27" event={"ID":"f3048123-5d9f-4e0c-a3e6-56d2bbd04f5f","Type":"ContainerStarted","Data":"2e29926be78cc2c5fc62753fcc2165a3b1ade1780bbb1e5a09eeae71d9729013"}
Sep 16 02:23:16.315635 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:16.315269    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-74fbd865d6-5jt27"
Sep 16 02:23:16.342682 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:16.342638    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-74fbd865d6-5jt27" podStartSLOduration=3.00827058 podStartE2EDuration="7.342621714s" totalImagesPullingTime="3.300812377s" totalInitContainerRuntime="1.033538757s" isStatefulPod=false podCreationTimestamp="2026-09-16 02:23:09 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 02:23:16.342060683 +0000 UTC m=+616.725575940" watchObservedRunningTime="2026-09-16 02:23:16.342621714 +0000 UTC m=+616.726136958"
Sep 16 02:23:18.636905 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:18.636858    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:23:18.737865 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:18.737819    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:23:25.311334 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:25.311261    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-79c8b99f94-2wfnm"
Sep 16 02:23:27.322100 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:27.322069    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-74fbd865d6-5jt27"
Sep 16 02:23:28.998255 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:28.998210    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:23:29.734672 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:29.734632    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:23:29.769870 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:29.769831    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-76bc65b794-4z8k7"]
Sep 16 02:23:29.951636 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:29.951605    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:23:30.447145 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:30.447105    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:23:30.504137 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:30.504102    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-7cff6fbc6c-q989h"]
Sep 16 02:23:30.507425 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:30.507403    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7cff6fbc6c-q989h"
Sep 16 02:23:30.510983 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:30.510957    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 16 02:23:30.511110 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:30.511008    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-6b4pj\""
Sep 16 02:23:30.511784 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:30.511765    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Sep 16 02:23:30.551409 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:30.551383    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7cff6fbc6c-q989h"]
Sep 16 02:23:30.617798 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:30.617770    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/60cf8c38-8ee9-493d-a871-22678ad8b6d0-tls-certs\") pod \"kube-auth-proxy-7cff6fbc6c-q989h\" (UID: \"60cf8c38-8ee9-493d-a871-22678ad8b6d0\") " pod="openshift-ingress/kube-auth-proxy-7cff6fbc6c-q989h"
Sep 16 02:23:30.617943 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:30.617816    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/60cf8c38-8ee9-493d-a871-22678ad8b6d0-tmp\") pod \"kube-auth-proxy-7cff6fbc6c-q989h\" (UID: \"60cf8c38-8ee9-493d-a871-22678ad8b6d0\") " pod="openshift-ingress/kube-auth-proxy-7cff6fbc6c-q989h"
Sep 16 02:23:30.617943 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:30.617848    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qk9kf\" (UniqueName: \"kubernetes.io/projected/60cf8c38-8ee9-493d-a871-22678ad8b6d0-kube-api-access-qk9kf\") pod \"kube-auth-proxy-7cff6fbc6c-q989h\" (UID: \"60cf8c38-8ee9-493d-a871-22678ad8b6d0\") " pod="openshift-ingress/kube-auth-proxy-7cff6fbc6c-q989h"
Sep 16 02:23:30.694060 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:30.694028    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:23:30.719040 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:30.718986    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/60cf8c38-8ee9-493d-a871-22678ad8b6d0-tls-certs\") pod \"kube-auth-proxy-7cff6fbc6c-q989h\" (UID: \"60cf8c38-8ee9-493d-a871-22678ad8b6d0\") " pod="openshift-ingress/kube-auth-proxy-7cff6fbc6c-q989h"
Sep 16 02:23:30.719040 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:30.719021    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/60cf8c38-8ee9-493d-a871-22678ad8b6d0-tmp\") pod \"kube-auth-proxy-7cff6fbc6c-q989h\" (UID: \"60cf8c38-8ee9-493d-a871-22678ad8b6d0\") " pod="openshift-ingress/kube-auth-proxy-7cff6fbc6c-q989h"
Sep 16 02:23:30.719155 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:30.719051    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qk9kf\" (UniqueName: \"kubernetes.io/projected/60cf8c38-8ee9-493d-a871-22678ad8b6d0-kube-api-access-qk9kf\") pod \"kube-auth-proxy-7cff6fbc6c-q989h\" (UID: \"60cf8c38-8ee9-493d-a871-22678ad8b6d0\") " pod="openshift-ingress/kube-auth-proxy-7cff6fbc6c-q989h"
Sep 16 02:23:30.721425 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:30.721396    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/60cf8c38-8ee9-493d-a871-22678ad8b6d0-tmp\") pod \"kube-auth-proxy-7cff6fbc6c-q989h\" (UID: \"60cf8c38-8ee9-493d-a871-22678ad8b6d0\") " pod="openshift-ingress/kube-auth-proxy-7cff6fbc6c-q989h"
Sep 16 02:23:30.721631 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:30.721614    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/60cf8c38-8ee9-493d-a871-22678ad8b6d0-tls-certs\") pod \"kube-auth-proxy-7cff6fbc6c-q989h\" (UID: \"60cf8c38-8ee9-493d-a871-22678ad8b6d0\") " pod="openshift-ingress/kube-auth-proxy-7cff6fbc6c-q989h"
Sep 16 02:23:30.738330 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:30.738309    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qk9kf\" (UniqueName: \"kubernetes.io/projected/60cf8c38-8ee9-493d-a871-22678ad8b6d0-kube-api-access-qk9kf\") pod \"kube-auth-proxy-7cff6fbc6c-q989h\" (UID: \"60cf8c38-8ee9-493d-a871-22678ad8b6d0\") " pod="openshift-ingress/kube-auth-proxy-7cff6fbc6c-q989h"
Sep 16 02:23:30.818356 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:30.818317    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7cff6fbc6c-q989h"
Sep 16 02:23:31.134822 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:31.134753    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7cff6fbc6c-q989h"]
Sep 16 02:23:31.137601 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:23:31.137573    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60cf8c38_8ee9_493d_a871_22678ad8b6d0.slice/crio-a5f1ff56e066a6264de786aa671d3829a093e8efc1124575bd81610f31824875 WatchSource:0}: Error finding container a5f1ff56e066a6264de786aa671d3829a093e8efc1124575bd81610f31824875: Status 404 returned error can't find the container with id a5f1ff56e066a6264de786aa671d3829a093e8efc1124575bd81610f31824875
Sep 16 02:23:31.397818 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:31.397740    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7cff6fbc6c-q989h" event={"ID":"60cf8c38-8ee9-493d-a871-22678ad8b6d0","Type":"ContainerStarted","Data":"a5f1ff56e066a6264de786aa671d3829a093e8efc1124575bd81610f31824875"}
Sep 16 02:23:33.369218 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:33.369175    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:23:33.511622 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:33.511560    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:23:34.414633 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:34.414595    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7cff6fbc6c-q989h" event={"ID":"60cf8c38-8ee9-493d-a871-22678ad8b6d0","Type":"ContainerStarted","Data":"b3ee88ddec0014ba80d42af17922de1eb72adf4604312f07d5b481ba5347275d"}
Sep 16 02:23:34.439689 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:34.439645    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-7cff6fbc6c-q989h" podStartSLOduration=1.8955138219999998 podStartE2EDuration="4.439629122s" totalImagesPullingTime="2.5441153s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 02:23:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 02:23:34.437788082 +0000 UTC m=+634.821303328" watchObservedRunningTime="2026-09-16 02:23:34.439629122 +0000 UTC m=+634.823144397"
Sep 16 02:23:35.161366 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:35.161338    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:23:37.375317 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:37.375281    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:23:37.481486 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:37.481432    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:23:38.337374 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:38.337339    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:23:38.559153 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:38.559127    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:23:38.646756 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:38.646704    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:23:40.782241 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:40.782211    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:23:40.845565 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:40.845527    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:23:40.946696 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:40.946663    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:23:41.148530 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:41.148410    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:23:41.235462 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:41.235428    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:23:41.468353 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:41.468318    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:23:43.416095 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:43.416055    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:23:43.526810 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:43.526780    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:23:45.165841 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:45.165804    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:23:45.215825 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:45.215797    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:23:50.351961 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:50.351921    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:23:50.461834 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:50.461795    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:23:50.686951 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:50.686920    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:23:50.819538 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:50.819497    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:23:51.632836 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:51.632800    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:23:51.756266 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:51.756227    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:23:51.886821 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:51.886741    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:23:54.789389 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:54.789352    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-76bc65b794-4z8k7" podUID="a972cd14-97a1-4889-aea5-ea1d4a18c861" containerName="console" containerID="cri-o://6d6505fad3a84b1611c89dfe782248ffe2c18f4f2dcdba11c45d0c2b447e1cba" gracePeriod=15
Sep 16 02:23:55.035536 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:55.035511    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-76bc65b794-4z8k7"
Sep 16 02:23:55.122591 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:55.122520    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a972cd14-97a1-4889-aea5-ea1d4a18c861-console-config\") pod \"a972cd14-97a1-4889-aea5-ea1d4a18c861\" (UID: \"a972cd14-97a1-4889-aea5-ea1d4a18c861\") "
Sep 16 02:23:55.122591 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:55.122559    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a972cd14-97a1-4889-aea5-ea1d4a18c861-console-serving-cert\") pod \"a972cd14-97a1-4889-aea5-ea1d4a18c861\" (UID: \"a972cd14-97a1-4889-aea5-ea1d4a18c861\") "
Sep 16 02:23:55.122591 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:55.122581    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a972cd14-97a1-4889-aea5-ea1d4a18c861-trusted-ca-bundle\") pod \"a972cd14-97a1-4889-aea5-ea1d4a18c861\" (UID: \"a972cd14-97a1-4889-aea5-ea1d4a18c861\") "
Sep 16 02:23:55.122812 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:55.122617    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a972cd14-97a1-4889-aea5-ea1d4a18c861-console-oauth-config\") pod \"a972cd14-97a1-4889-aea5-ea1d4a18c861\" (UID: \"a972cd14-97a1-4889-aea5-ea1d4a18c861\") "
Sep 16 02:23:55.122812 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:55.122648    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kbfns\" (UniqueName: \"kubernetes.io/projected/a972cd14-97a1-4889-aea5-ea1d4a18c861-kube-api-access-kbfns\") pod \"a972cd14-97a1-4889-aea5-ea1d4a18c861\" (UID: \"a972cd14-97a1-4889-aea5-ea1d4a18c861\") "
Sep 16 02:23:55.122812 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:55.122673    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a972cd14-97a1-4889-aea5-ea1d4a18c861-service-ca\") pod \"a972cd14-97a1-4889-aea5-ea1d4a18c861\" (UID: \"a972cd14-97a1-4889-aea5-ea1d4a18c861\") "
Sep 16 02:23:55.122812 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:55.122693    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a972cd14-97a1-4889-aea5-ea1d4a18c861-oauth-serving-cert\") pod \"a972cd14-97a1-4889-aea5-ea1d4a18c861\" (UID: \"a972cd14-97a1-4889-aea5-ea1d4a18c861\") "
Sep 16 02:23:55.123008 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:55.122964    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a972cd14-97a1-4889-aea5-ea1d4a18c861-console-config" (OuterVolumeSpecName: "console-config") pod "a972cd14-97a1-4889-aea5-ea1d4a18c861" (UID: "a972cd14-97a1-4889-aea5-ea1d4a18c861"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 02:23:55.123110 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:55.123082    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a972cd14-97a1-4889-aea5-ea1d4a18c861-service-ca" (OuterVolumeSpecName: "service-ca") pod "a972cd14-97a1-4889-aea5-ea1d4a18c861" (UID: "a972cd14-97a1-4889-aea5-ea1d4a18c861"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 02:23:55.123279 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:55.123250    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a972cd14-97a1-4889-aea5-ea1d4a18c861-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "a972cd14-97a1-4889-aea5-ea1d4a18c861" (UID: "a972cd14-97a1-4889-aea5-ea1d4a18c861"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 02:23:55.123761 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:55.123737    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a972cd14-97a1-4889-aea5-ea1d4a18c861-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "a972cd14-97a1-4889-aea5-ea1d4a18c861" (UID: "a972cd14-97a1-4889-aea5-ea1d4a18c861"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 02:23:55.124975 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:55.124951    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a972cd14-97a1-4889-aea5-ea1d4a18c861-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "a972cd14-97a1-4889-aea5-ea1d4a18c861" (UID: "a972cd14-97a1-4889-aea5-ea1d4a18c861"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 02:23:55.125048 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:55.124968    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a972cd14-97a1-4889-aea5-ea1d4a18c861-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "a972cd14-97a1-4889-aea5-ea1d4a18c861" (UID: "a972cd14-97a1-4889-aea5-ea1d4a18c861"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 02:23:55.125048 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:55.124991    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a972cd14-97a1-4889-aea5-ea1d4a18c861-kube-api-access-kbfns" (OuterVolumeSpecName: "kube-api-access-kbfns") pod "a972cd14-97a1-4889-aea5-ea1d4a18c861" (UID: "a972cd14-97a1-4889-aea5-ea1d4a18c861"). InnerVolumeSpecName "kube-api-access-kbfns". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 02:23:55.223806 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:55.223781    2579 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a972cd14-97a1-4889-aea5-ea1d4a18c861-console-config\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:23:55.223806 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:55.223802    2579 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a972cd14-97a1-4889-aea5-ea1d4a18c861-console-serving-cert\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:23:55.223934 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:55.223812    2579 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a972cd14-97a1-4889-aea5-ea1d4a18c861-trusted-ca-bundle\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:23:55.223934 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:55.223821    2579 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a972cd14-97a1-4889-aea5-ea1d4a18c861-console-oauth-config\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:23:55.223934 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:55.223829    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kbfns\" (UniqueName: \"kubernetes.io/projected/a972cd14-97a1-4889-aea5-ea1d4a18c861-kube-api-access-kbfns\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:23:55.223934 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:55.223837    2579 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a972cd14-97a1-4889-aea5-ea1d4a18c861-service-ca\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:23:55.223934 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:55.223844    2579 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a972cd14-97a1-4889-aea5-ea1d4a18c861-oauth-serving-cert\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:23:55.517884 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:55.517840    2579 generic.go:356] "Generic (PLEG): container finished" podID="a972cd14-97a1-4889-aea5-ea1d4a18c861" containerID="6d6505fad3a84b1611c89dfe782248ffe2c18f4f2dcdba11c45d0c2b447e1cba" exitCode=0
Sep 16 02:23:55.518023 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:55.517891    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-76bc65b794-4z8k7" event={"ID":"a972cd14-97a1-4889-aea5-ea1d4a18c861","Type":"ContainerDied","Data":"6d6505fad3a84b1611c89dfe782248ffe2c18f4f2dcdba11c45d0c2b447e1cba"}
Sep 16 02:23:55.518023 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:55.517899    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-76bc65b794-4z8k7"
Sep 16 02:23:55.518023 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:55.517951    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-76bc65b794-4z8k7" event={"ID":"a972cd14-97a1-4889-aea5-ea1d4a18c861","Type":"ContainerDied","Data":"efc029a3f3026e4caad7e47c58d39ae44f1477c2c4e5fadec4646e9bf08bc90f"}
Sep 16 02:23:55.518023 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:55.517969    2579 scope.go:117] "RemoveContainer" containerID="6d6505fad3a84b1611c89dfe782248ffe2c18f4f2dcdba11c45d0c2b447e1cba"
Sep 16 02:23:55.526688 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:55.526671    2579 scope.go:117] "RemoveContainer" containerID="6d6505fad3a84b1611c89dfe782248ffe2c18f4f2dcdba11c45d0c2b447e1cba"
Sep 16 02:23:55.526932 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:23:55.526915    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6d6505fad3a84b1611c89dfe782248ffe2c18f4f2dcdba11c45d0c2b447e1cba\": container with ID starting with 6d6505fad3a84b1611c89dfe782248ffe2c18f4f2dcdba11c45d0c2b447e1cba not found: ID does not exist" containerID="6d6505fad3a84b1611c89dfe782248ffe2c18f4f2dcdba11c45d0c2b447e1cba"
Sep 16 02:23:55.527000 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:55.526942    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d6505fad3a84b1611c89dfe782248ffe2c18f4f2dcdba11c45d0c2b447e1cba"} err="failed to get container status \"6d6505fad3a84b1611c89dfe782248ffe2c18f4f2dcdba11c45d0c2b447e1cba\": rpc error: code = NotFound desc = could not find container \"6d6505fad3a84b1611c89dfe782248ffe2c18f4f2dcdba11c45d0c2b447e1cba\": container with ID starting with 6d6505fad3a84b1611c89dfe782248ffe2c18f4f2dcdba11c45d0c2b447e1cba not found: ID does not exist"
Sep 16 02:23:55.539874 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:55.539853    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-76bc65b794-4z8k7"]
Sep 16 02:23:55.546798 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:55.546777    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-76bc65b794-4z8k7"]
Sep 16 02:23:56.101735 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:23:56.101702    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a972cd14-97a1-4889-aea5-ea1d4a18c861" path="/var/lib/kubelet/pods/a972cd14-97a1-4889-aea5-ea1d4a18c861/volumes"
Sep 16 02:24:19.976800 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:24:19.976755    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:24:42.001281 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:24:42.001242    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:24:42.101795 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:24:42.101755    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:24:55.582988 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:24:55.582903    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:24:57.970762 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:24:57.970721    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:24:58.387576 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:24:58.387495    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:24:59.793516 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:24:59.793483    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:25:00.972947 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:25:00.972916    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:25:01.211095 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:25:01.211065    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:25:02.684730 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:25:02.684701    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:25:03.170419 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:25:03.170340    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:25:03.409603 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:25:03.409566    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:25:03.575050 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:25:03.575014    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:25:03.678374 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:25:03.678333    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:25:03.972031 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:25:03.971995    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:25:04.083159 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:25:04.083107    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:25:09.193579 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:25:09.193541    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:25:09.300692 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:25:09.300656    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:25:14.269628 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:25:14.269585    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:25:14.502297 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:25:14.502267    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:25:14.851304 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:25:14.851269    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:25:14.938147 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:25:14.938114    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:25:15.182994 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:25:15.182909    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:25:15.287532 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:25:15.287458    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:25:16.815614 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:25:16.815577    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:25:16.908050 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:25:16.908020    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:25:21.528223 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:25:21.528183    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:25:21.626820 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:25:21.626786    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:25:21.725945 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:25:21.725917    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:25:28.184581 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:25:28.184545    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:25:28.289971 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:25:28.289934    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:25:33.557708 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:25:33.557657    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:25:33.609199 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:25:33.609146    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:25:33.958625 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:25:33.958593    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:25:34.012066 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:25:34.012036    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:25:37.365286 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:25:37.365253    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:25:37.445196 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:25:37.445160    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:25:41.127364 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:25:41.127327    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:25:41.225290 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:25:41.225265    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:25:41.280676 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:25:41.280649    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:25:44.504959 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:25:44.504922    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:25:53.942946 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:25:53.942903    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:25:54.528131 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:25:54.528093    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:26:06.765260 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:06.765229    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:26:06.814678 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:06.814643    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:26:08.554355 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:08.554316    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:26:08.563932 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:08.563905    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-hv8dm"]
Sep 16 02:26:08.564214 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:08.564203    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a972cd14-97a1-4889-aea5-ea1d4a18c861" containerName="console"
Sep 16 02:26:08.564255 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:08.564216    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="a972cd14-97a1-4889-aea5-ea1d4a18c861" containerName="console"
Sep 16 02:26:08.564288 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:08.564269    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="a972cd14-97a1-4889-aea5-ea1d4a18c861" containerName="console"
Sep 16 02:26:08.567237 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:08.567220    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-hv8dm"
Sep 16 02:26:08.569480 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:08.569451    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-2t5hl\""
Sep 16 02:26:08.569551 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:08.569510    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Sep 16 02:26:08.569551 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:08.569535    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 16 02:26:08.570200 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:08.570165    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 16 02:26:08.575626 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:08.575607    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-hv8dm"]
Sep 16 02:26:08.659411 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:08.659382    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-hv8dm"]
Sep 16 02:26:08.672754 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:08.672722    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:26:08.698267 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:08.698242    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2vxmv\" (UniqueName: \"kubernetes.io/projected/e621e7bf-5585-408b-bdb2-79221a9d202a-kube-api-access-2vxmv\") pod \"limitador-limitador-8588749977-hv8dm\" (UID: \"e621e7bf-5585-408b-bdb2-79221a9d202a\") " pod="kuadrant-system/limitador-limitador-8588749977-hv8dm"
Sep 16 02:26:08.698387 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:08.698283    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/e621e7bf-5585-408b-bdb2-79221a9d202a-config-file\") pod \"limitador-limitador-8588749977-hv8dm\" (UID: \"e621e7bf-5585-408b-bdb2-79221a9d202a\") " pod="kuadrant-system/limitador-limitador-8588749977-hv8dm"
Sep 16 02:26:08.773399 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:08.773366    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:26:08.799311 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:08.799283    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/e621e7bf-5585-408b-bdb2-79221a9d202a-config-file\") pod \"limitador-limitador-8588749977-hv8dm\" (UID: \"e621e7bf-5585-408b-bdb2-79221a9d202a\") " pod="kuadrant-system/limitador-limitador-8588749977-hv8dm"
Sep 16 02:26:08.799537 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:08.799347    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2vxmv\" (UniqueName: \"kubernetes.io/projected/e621e7bf-5585-408b-bdb2-79221a9d202a-kube-api-access-2vxmv\") pod \"limitador-limitador-8588749977-hv8dm\" (UID: \"e621e7bf-5585-408b-bdb2-79221a9d202a\") " pod="kuadrant-system/limitador-limitador-8588749977-hv8dm"
Sep 16 02:26:08.799986 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:08.799967    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/e621e7bf-5585-408b-bdb2-79221a9d202a-config-file\") pod \"limitador-limitador-8588749977-hv8dm\" (UID: \"e621e7bf-5585-408b-bdb2-79221a9d202a\") " pod="kuadrant-system/limitador-limitador-8588749977-hv8dm"
Sep 16 02:26:08.806880 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:08.806828    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2vxmv\" (UniqueName: \"kubernetes.io/projected/e621e7bf-5585-408b-bdb2-79221a9d202a-kube-api-access-2vxmv\") pod \"limitador-limitador-8588749977-hv8dm\" (UID: \"e621e7bf-5585-408b-bdb2-79221a9d202a\") " pod="kuadrant-system/limitador-limitador-8588749977-hv8dm"
Sep 16 02:26:08.877544 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:08.877512    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-hv8dm"
Sep 16 02:26:09.146252 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:09.146175    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-hv8dm"]
Sep 16 02:26:09.150078 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:26:09.150053    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode621e7bf_5585_408b_bdb2_79221a9d202a.slice/crio-f78eeb179962d2708d3b32755bdf5d163bc759730b2736b81940ac568664a531 WatchSource:0}: Error finding container f78eeb179962d2708d3b32755bdf5d163bc759730b2736b81940ac568664a531: Status 404 returned error can't find the container with id f78eeb179962d2708d3b32755bdf5d163bc759730b2736b81940ac568664a531
Sep 16 02:26:09.352371 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:09.352330    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-rlqtq"]
Sep 16 02:26:09.356641 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:09.356612    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-rlqtq"
Sep 16 02:26:09.358802 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:09.358781    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-7xl4z\""
Sep 16 02:26:09.365230 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:09.365199    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-rlqtq"]
Sep 16 02:26:09.388977 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:09.388951    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:26:09.404699 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:09.404641    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2w8rn\" (UniqueName: \"kubernetes.io/projected/9a3d4d75-a880-4dc7-a861-a79e1d51bacf-kube-api-access-2w8rn\") pod \"authorino-7bcfd8f9cd-rlqtq\" (UID: \"9a3d4d75-a880-4dc7-a861-a79e1d51bacf\") " pod="kuadrant-system/authorino-7bcfd8f9cd-rlqtq"
Sep 16 02:26:09.490513 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:09.490456    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:26:09.505619 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:09.505592    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2w8rn\" (UniqueName: \"kubernetes.io/projected/9a3d4d75-a880-4dc7-a861-a79e1d51bacf-kube-api-access-2w8rn\") pod \"authorino-7bcfd8f9cd-rlqtq\" (UID: \"9a3d4d75-a880-4dc7-a861-a79e1d51bacf\") " pod="kuadrant-system/authorino-7bcfd8f9cd-rlqtq"
Sep 16 02:26:09.513267 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:09.513235    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2w8rn\" (UniqueName: \"kubernetes.io/projected/9a3d4d75-a880-4dc7-a861-a79e1d51bacf-kube-api-access-2w8rn\") pod \"authorino-7bcfd8f9cd-rlqtq\" (UID: \"9a3d4d75-a880-4dc7-a861-a79e1d51bacf\") " pod="kuadrant-system/authorino-7bcfd8f9cd-rlqtq"
Sep 16 02:26:09.666420 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:09.666344    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-rlqtq"
Sep 16 02:26:09.934091 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:09.933719    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-rlqtq"]
Sep 16 02:26:09.936533 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:26:09.936502    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9a3d4d75_a880_4dc7_a861_a79e1d51bacf.slice/crio-f5aa14125b36d39174966e3689e8b38e04bffb45391640ae052bf204470d0cda WatchSource:0}: Error finding container f5aa14125b36d39174966e3689e8b38e04bffb45391640ae052bf204470d0cda: Status 404 returned error can't find the container with id f5aa14125b36d39174966e3689e8b38e04bffb45391640ae052bf204470d0cda
Sep 16 02:26:10.108824 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:10.108757    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-hv8dm" event={"ID":"e621e7bf-5585-408b-bdb2-79221a9d202a","Type":"ContainerStarted","Data":"f78eeb179962d2708d3b32755bdf5d163bc759730b2736b81940ac568664a531"}
Sep 16 02:26:10.110167 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:10.110132    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-rlqtq" event={"ID":"9a3d4d75-a880-4dc7-a861-a79e1d51bacf","Type":"ContainerStarted","Data":"f5aa14125b36d39174966e3689e8b38e04bffb45391640ae052bf204470d0cda"}
Sep 16 02:26:12.122388 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:12.122350    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-hv8dm" event={"ID":"e621e7bf-5585-408b-bdb2-79221a9d202a","Type":"ContainerStarted","Data":"6236420483f1bb80b50515640d6eac99cf6cabc6233dc269639e88dbf0fa7368"}
Sep 16 02:26:12.122710 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:12.122441    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-8588749977-hv8dm"
Sep 16 02:26:12.137114 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:12.137071    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-8588749977-hv8dm" podStartSLOduration=1.9569732869999998 podStartE2EDuration="4.137057076s" totalImagesPullingTime="2.180083789s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 02:26:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 02:26:12.135917577 +0000 UTC m=+792.519432841" watchObservedRunningTime="2026-09-16 02:26:12.137057076 +0000 UTC m=+792.520572321"
Sep 16 02:26:12.674182 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:12.674146    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-rlqtq"]
Sep 16 02:26:13.128208 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:13.128167    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-rlqtq" event={"ID":"9a3d4d75-a880-4dc7-a861-a79e1d51bacf","Type":"ContainerStarted","Data":"9d399677644160c173427f4a697fe125a417a8aa41fa7ad23506115e890ec9e8"}
Sep 16 02:26:13.141878 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:13.141820    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7bcfd8f9cd-rlqtq" podStartSLOduration=1.96791422 podStartE2EDuration="4.141803179s" totalImagesPullingTime="2.173888959s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 02:26:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 02:26:13.140979745 +0000 UTC m=+793.524494991" watchObservedRunningTime="2026-09-16 02:26:13.141803179 +0000 UTC m=+793.525318425"
Sep 16 02:26:14.132371 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:14.132312    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-7bcfd8f9cd-rlqtq" podUID="9a3d4d75-a880-4dc7-a861-a79e1d51bacf" containerName="authorino" containerID="cri-o://9d399677644160c173427f4a697fe125a417a8aa41fa7ad23506115e890ec9e8" gracePeriod=30
Sep 16 02:26:14.383417 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:14.383365    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-rlqtq"
Sep 16 02:26:14.448432 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:14.448406    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2w8rn\" (UniqueName: \"kubernetes.io/projected/9a3d4d75-a880-4dc7-a861-a79e1d51bacf-kube-api-access-2w8rn\") pod \"9a3d4d75-a880-4dc7-a861-a79e1d51bacf\" (UID: \"9a3d4d75-a880-4dc7-a861-a79e1d51bacf\") "
Sep 16 02:26:14.450724 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:14.450701    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9a3d4d75-a880-4dc7-a861-a79e1d51bacf-kube-api-access-2w8rn" (OuterVolumeSpecName: "kube-api-access-2w8rn") pod "9a3d4d75-a880-4dc7-a861-a79e1d51bacf" (UID: "9a3d4d75-a880-4dc7-a861-a79e1d51bacf"). InnerVolumeSpecName "kube-api-access-2w8rn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 02:26:14.549211 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:14.549186    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2w8rn\" (UniqueName: \"kubernetes.io/projected/9a3d4d75-a880-4dc7-a861-a79e1d51bacf-kube-api-access-2w8rn\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:26:15.138696 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:15.138663    2579 generic.go:356] "Generic (PLEG): container finished" podID="9a3d4d75-a880-4dc7-a861-a79e1d51bacf" containerID="9d399677644160c173427f4a697fe125a417a8aa41fa7ad23506115e890ec9e8" exitCode=0
Sep 16 02:26:15.139071 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:15.138700    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-rlqtq"
Sep 16 02:26:15.139071 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:15.138706    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-rlqtq" event={"ID":"9a3d4d75-a880-4dc7-a861-a79e1d51bacf","Type":"ContainerDied","Data":"9d399677644160c173427f4a697fe125a417a8aa41fa7ad23506115e890ec9e8"}
Sep 16 02:26:15.139071 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:15.138732    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-rlqtq" event={"ID":"9a3d4d75-a880-4dc7-a861-a79e1d51bacf","Type":"ContainerDied","Data":"f5aa14125b36d39174966e3689e8b38e04bffb45391640ae052bf204470d0cda"}
Sep 16 02:26:15.139071 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:15.138754    2579 scope.go:117] "RemoveContainer" containerID="9d399677644160c173427f4a697fe125a417a8aa41fa7ad23506115e890ec9e8"
Sep 16 02:26:15.147757 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:15.147742    2579 scope.go:117] "RemoveContainer" containerID="9d399677644160c173427f4a697fe125a417a8aa41fa7ad23506115e890ec9e8"
Sep 16 02:26:15.148039 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:26:15.148020    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9d399677644160c173427f4a697fe125a417a8aa41fa7ad23506115e890ec9e8\": container with ID starting with 9d399677644160c173427f4a697fe125a417a8aa41fa7ad23506115e890ec9e8 not found: ID does not exist" containerID="9d399677644160c173427f4a697fe125a417a8aa41fa7ad23506115e890ec9e8"
Sep 16 02:26:15.148086 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:15.148049    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d399677644160c173427f4a697fe125a417a8aa41fa7ad23506115e890ec9e8"} err="failed to get container status \"9d399677644160c173427f4a697fe125a417a8aa41fa7ad23506115e890ec9e8\": rpc error: code = NotFound desc = could not find container \"9d399677644160c173427f4a697fe125a417a8aa41fa7ad23506115e890ec9e8\": container with ID starting with 9d399677644160c173427f4a697fe125a417a8aa41fa7ad23506115e890ec9e8 not found: ID does not exist"
Sep 16 02:26:15.157917 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:15.157897    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-rlqtq"]
Sep 16 02:26:15.161213 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:15.161194    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-rlqtq"]
Sep 16 02:26:16.108291 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:16.108256    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9a3d4d75-a880-4dc7-a861-a79e1d51bacf" path="/var/lib/kubelet/pods/9a3d4d75-a880-4dc7-a861-a79e1d51bacf/volumes"
Sep 16 02:26:22.681043 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:22.680959    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-hv8dm"]
Sep 16 02:26:22.681505 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:22.681229    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-8588749977-hv8dm" podUID="e621e7bf-5585-408b-bdb2-79221a9d202a" containerName="limitador" containerID="cri-o://6236420483f1bb80b50515640d6eac99cf6cabc6233dc269639e88dbf0fa7368" gracePeriod=30
Sep 16 02:26:22.681988 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:22.681968    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-8588749977-hv8dm"
Sep 16 02:26:23.173827 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:23.173795    2579 generic.go:356] "Generic (PLEG): container finished" podID="e621e7bf-5585-408b-bdb2-79221a9d202a" containerID="6236420483f1bb80b50515640d6eac99cf6cabc6233dc269639e88dbf0fa7368" exitCode=0
Sep 16 02:26:23.173999 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:23.173872    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-hv8dm" event={"ID":"e621e7bf-5585-408b-bdb2-79221a9d202a","Type":"ContainerDied","Data":"6236420483f1bb80b50515640d6eac99cf6cabc6233dc269639e88dbf0fa7368"}
Sep 16 02:26:23.623968 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:23.623946    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-hv8dm"
Sep 16 02:26:23.720717 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:23.720687    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2vxmv\" (UniqueName: \"kubernetes.io/projected/e621e7bf-5585-408b-bdb2-79221a9d202a-kube-api-access-2vxmv\") pod \"e621e7bf-5585-408b-bdb2-79221a9d202a\" (UID: \"e621e7bf-5585-408b-bdb2-79221a9d202a\") "
Sep 16 02:26:23.721083 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:23.720732    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/e621e7bf-5585-408b-bdb2-79221a9d202a-config-file\") pod \"e621e7bf-5585-408b-bdb2-79221a9d202a\" (UID: \"e621e7bf-5585-408b-bdb2-79221a9d202a\") "
Sep 16 02:26:23.721083 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:23.721065    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e621e7bf-5585-408b-bdb2-79221a9d202a-config-file" (OuterVolumeSpecName: "config-file") pod "e621e7bf-5585-408b-bdb2-79221a9d202a" (UID: "e621e7bf-5585-408b-bdb2-79221a9d202a"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 02:26:23.722762 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:23.722732    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e621e7bf-5585-408b-bdb2-79221a9d202a-kube-api-access-2vxmv" (OuterVolumeSpecName: "kube-api-access-2vxmv") pod "e621e7bf-5585-408b-bdb2-79221a9d202a" (UID: "e621e7bf-5585-408b-bdb2-79221a9d202a"). InnerVolumeSpecName "kube-api-access-2vxmv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 02:26:23.821435 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:23.821371    2579 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/e621e7bf-5585-408b-bdb2-79221a9d202a-config-file\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:26:23.821435 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:23.821394    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2vxmv\" (UniqueName: \"kubernetes.io/projected/e621e7bf-5585-408b-bdb2-79221a9d202a-kube-api-access-2vxmv\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:26:24.179566 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:24.179541    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-hv8dm"
Sep 16 02:26:24.179710 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:24.179539    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-hv8dm" event={"ID":"e621e7bf-5585-408b-bdb2-79221a9d202a","Type":"ContainerDied","Data":"f78eeb179962d2708d3b32755bdf5d163bc759730b2736b81940ac568664a531"}
Sep 16 02:26:24.179710 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:24.179672    2579 scope.go:117] "RemoveContainer" containerID="6236420483f1bb80b50515640d6eac99cf6cabc6233dc269639e88dbf0fa7368"
Sep 16 02:26:24.194351 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:24.194327    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-hv8dm"]
Sep 16 02:26:24.197889 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:24.197859    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-hv8dm"]
Sep 16 02:26:26.102049 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:26.102018    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e621e7bf-5585-408b-bdb2-79221a9d202a" path="/var/lib/kubelet/pods/e621e7bf-5585-408b-bdb2-79221a9d202a/volumes"
Sep 16 02:26:27.446531 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:27.446491    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:26:27.545773 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:27.545733    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:26:27.909575 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:27.909500    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-rk2wx"]
Sep 16 02:26:27.909835 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:27.909823    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a3d4d75-a880-4dc7-a861-a79e1d51bacf" containerName="authorino"
Sep 16 02:26:27.909889 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:27.909846    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a3d4d75-a880-4dc7-a861-a79e1d51bacf" containerName="authorino"
Sep 16 02:26:27.909889 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:27.909859    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e621e7bf-5585-408b-bdb2-79221a9d202a" containerName="limitador"
Sep 16 02:26:27.909889 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:27.909864    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="e621e7bf-5585-408b-bdb2-79221a9d202a" containerName="limitador"
Sep 16 02:26:27.909979 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:27.909909    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="9a3d4d75-a880-4dc7-a861-a79e1d51bacf" containerName="authorino"
Sep 16 02:26:27.909979 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:27.909921    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="e621e7bf-5585-408b-bdb2-79221a9d202a" containerName="limitador"
Sep 16 02:26:27.912955 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:27.912935    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-rk2wx"
Sep 16 02:26:27.915552 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:27.915530    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Sep 16 02:26:27.915791 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:27.915770    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-vcc26\""
Sep 16 02:26:27.915933 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:27.915836    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 16 02:26:27.916400 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:27.916384    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 16 02:26:27.920879 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:27.920861    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-rk2wx"]
Sep 16 02:26:27.943562 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:27.943532    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:26:28.057772 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:28.057741    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bn8dw\" (UniqueName: \"kubernetes.io/projected/26df93f5-3477-4d86-a02b-1c4a8ac8a2a4-kube-api-access-bn8dw\") pod \"postgres-69fb4594cf-rk2wx\" (UID: \"26df93f5-3477-4d86-a02b-1c4a8ac8a2a4\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-rk2wx"
Sep 16 02:26:28.057944 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:28.057779    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/26df93f5-3477-4d86-a02b-1c4a8ac8a2a4-data\") pod \"postgres-69fb4594cf-rk2wx\" (UID: \"26df93f5-3477-4d86-a02b-1c4a8ac8a2a4\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-rk2wx"
Sep 16 02:26:28.158712 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:28.158679    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/26df93f5-3477-4d86-a02b-1c4a8ac8a2a4-data\") pod \"postgres-69fb4594cf-rk2wx\" (UID: \"26df93f5-3477-4d86-a02b-1c4a8ac8a2a4\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-rk2wx"
Sep 16 02:26:28.158872 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:28.158746    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bn8dw\" (UniqueName: \"kubernetes.io/projected/26df93f5-3477-4d86-a02b-1c4a8ac8a2a4-kube-api-access-bn8dw\") pod \"postgres-69fb4594cf-rk2wx\" (UID: \"26df93f5-3477-4d86-a02b-1c4a8ac8a2a4\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-rk2wx"
Sep 16 02:26:28.159081 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:28.159062    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/26df93f5-3477-4d86-a02b-1c4a8ac8a2a4-data\") pod \"postgres-69fb4594cf-rk2wx\" (UID: \"26df93f5-3477-4d86-a02b-1c4a8ac8a2a4\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-rk2wx"
Sep 16 02:26:28.167354 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:28.167293    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bn8dw\" (UniqueName: \"kubernetes.io/projected/26df93f5-3477-4d86-a02b-1c4a8ac8a2a4-kube-api-access-bn8dw\") pod \"postgres-69fb4594cf-rk2wx\" (UID: \"26df93f5-3477-4d86-a02b-1c4a8ac8a2a4\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-rk2wx"
Sep 16 02:26:28.225294 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:28.225265    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-rk2wx"
Sep 16 02:26:28.491730 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:28.491645    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-rk2wx"]
Sep 16 02:26:28.494264 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:26:28.494237    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26df93f5_3477_4d86_a02b_1c4a8ac8a2a4.slice/crio-9beca456b295dddea525f6de6768f5ad5aee79bc40740180994271cbad6046eb WatchSource:0}: Error finding container 9beca456b295dddea525f6de6768f5ad5aee79bc40740180994271cbad6046eb: Status 404 returned error can't find the container with id 9beca456b295dddea525f6de6768f5ad5aee79bc40740180994271cbad6046eb
Sep 16 02:26:29.206505 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:29.206450    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-rk2wx" event={"ID":"26df93f5-3477-4d86-a02b-1c4a8ac8a2a4","Type":"ContainerStarted","Data":"9beca456b295dddea525f6de6768f5ad5aee79bc40740180994271cbad6046eb"}
Sep 16 02:26:33.228698 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:33.228665    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-rk2wx" event={"ID":"26df93f5-3477-4d86-a02b-1c4a8ac8a2a4","Type":"ContainerStarted","Data":"8cf01abdc91dee7aaec9192a80fe545f7fb3478a664eefa00a25a09d25a75080"}
Sep 16 02:26:33.229013 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:33.228781    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-rk2wx"
Sep 16 02:26:33.243980 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:33.243934    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-rk2wx" podStartSLOduration=1.598498108 podStartE2EDuration="6.243918301s" totalImagesPullingTime="4.645420193s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 02:26:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 02:26:33.242325578 +0000 UTC m=+813.625840825" watchObservedRunningTime="2026-09-16 02:26:33.243918301 +0000 UTC m=+813.627433546"
Sep 16 02:26:39.262074 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:39.262047    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-rk2wx"
Sep 16 02:26:39.586285 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:39.586197    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:26:39.870218 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:39.870130    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-557f876f69-462nb"]
Sep 16 02:26:39.876514 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:39.876498    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-462nb"
Sep 16 02:26:39.878803 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:39.878775    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-7xl4z\""
Sep 16 02:26:39.878922 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:39.878823    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 16 02:26:39.879310 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:39.879285    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-557f876f69-462nb"]
Sep 16 02:26:39.879503 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:39.879486    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 16 02:26:40.003899 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:40.003862    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:26:40.060142 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:40.060116    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cv24t\" (UniqueName: \"kubernetes.io/projected/2b322541-b6af-4637-9d10-730bee14ba60-kube-api-access-cv24t\") pod \"authorino-557f876f69-462nb\" (UID: \"2b322541-b6af-4637-9d10-730bee14ba60\") " pod="kuadrant-system/authorino-557f876f69-462nb"
Sep 16 02:26:40.071502 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:40.071480    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-462nb"]
Sep 16 02:26:40.071705 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:26:40.071688    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-cv24t], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-557f876f69-462nb" podUID="2b322541-b6af-4637-9d10-730bee14ba60"
Sep 16 02:26:40.095020 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:40.094996    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-m2jvs"]
Sep 16 02:26:40.099142 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:40.099125    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-m2jvs"
Sep 16 02:26:40.101263 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:40.101245    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Sep 16 02:26:40.108154 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:40.108131    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-m2jvs"]
Sep 16 02:26:40.150156 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:40.150096    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-m2jvs"]
Sep 16 02:26:40.150367 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:26:40.150346    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-j4t68 tls-cert], unattached volumes=[], failed to process volumes=[kube-api-access-j4t68 tls-cert]: context canceled" pod="kuadrant-system/authorino-6d4dfc649f-m2jvs" podUID="54043da2-36be-42db-9de7-405c04467e52"
Sep 16 02:26:40.161005 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:40.160970    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cv24t\" (UniqueName: \"kubernetes.io/projected/2b322541-b6af-4637-9d10-730bee14ba60-kube-api-access-cv24t\") pod \"authorino-557f876f69-462nb\" (UID: \"2b322541-b6af-4637-9d10-730bee14ba60\") " pod="kuadrant-system/authorino-557f876f69-462nb"
Sep 16 02:26:40.171683 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:40.171655    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cv24t\" (UniqueName: \"kubernetes.io/projected/2b322541-b6af-4637-9d10-730bee14ba60-kube-api-access-cv24t\") pod \"authorino-557f876f69-462nb\" (UID: \"2b322541-b6af-4637-9d10-730bee14ba60\") " pod="kuadrant-system/authorino-557f876f69-462nb"
Sep 16 02:26:40.176111 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:40.176090    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6fd859b494-xjvtn"]
Sep 16 02:26:40.180623 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:40.180606    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6fd859b494-xjvtn"
Sep 16 02:26:40.184805 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:40.184781    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6fd859b494-xjvtn"]
Sep 16 02:26:40.260308 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:40.260287    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-462nb"
Sep 16 02:26:40.260458 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:40.260289    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-m2jvs"
Sep 16 02:26:40.261716 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:40.261435    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j4t68\" (UniqueName: \"kubernetes.io/projected/54043da2-36be-42db-9de7-405c04467e52-kube-api-access-j4t68\") pod \"authorino-6d4dfc649f-m2jvs\" (UID: \"54043da2-36be-42db-9de7-405c04467e52\") " pod="kuadrant-system/authorino-6d4dfc649f-m2jvs"
Sep 16 02:26:40.261716 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:40.261587    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/54043da2-36be-42db-9de7-405c04467e52-tls-cert\") pod \"authorino-6d4dfc649f-m2jvs\" (UID: \"54043da2-36be-42db-9de7-405c04467e52\") " pod="kuadrant-system/authorino-6d4dfc649f-m2jvs"
Sep 16 02:26:40.268231 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:40.268214    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-462nb"
Sep 16 02:26:40.271865 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:40.271848    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-m2jvs"
Sep 16 02:26:40.362802 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:40.362772    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f24407e5-008c-48b4-8709-44cd316947da-tls-cert\") pod \"authorino-6fd859b494-xjvtn\" (UID: \"f24407e5-008c-48b4-8709-44cd316947da\") " pod="kuadrant-system/authorino-6fd859b494-xjvtn"
Sep 16 02:26:40.362974 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:40.362812    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j4t68\" (UniqueName: \"kubernetes.io/projected/54043da2-36be-42db-9de7-405c04467e52-kube-api-access-j4t68\") pod \"authorino-6d4dfc649f-m2jvs\" (UID: \"54043da2-36be-42db-9de7-405c04467e52\") " pod="kuadrant-system/authorino-6d4dfc649f-m2jvs"
Sep 16 02:26:40.362974 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:40.362835    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/54043da2-36be-42db-9de7-405c04467e52-tls-cert\") pod \"authorino-6d4dfc649f-m2jvs\" (UID: \"54043da2-36be-42db-9de7-405c04467e52\") " pod="kuadrant-system/authorino-6d4dfc649f-m2jvs"
Sep 16 02:26:40.362974 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:40.362851    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vjlh4\" (UniqueName: \"kubernetes.io/projected/f24407e5-008c-48b4-8709-44cd316947da-kube-api-access-vjlh4\") pod \"authorino-6fd859b494-xjvtn\" (UID: \"f24407e5-008c-48b4-8709-44cd316947da\") " pod="kuadrant-system/authorino-6fd859b494-xjvtn"
Sep 16 02:26:40.365360 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:40.365332    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/54043da2-36be-42db-9de7-405c04467e52-tls-cert\") pod \"authorino-6d4dfc649f-m2jvs\" (UID: \"54043da2-36be-42db-9de7-405c04467e52\") " pod="kuadrant-system/authorino-6d4dfc649f-m2jvs"
Sep 16 02:26:40.370487 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:40.370449    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j4t68\" (UniqueName: \"kubernetes.io/projected/54043da2-36be-42db-9de7-405c04467e52-kube-api-access-j4t68\") pod \"authorino-6d4dfc649f-m2jvs\" (UID: \"54043da2-36be-42db-9de7-405c04467e52\") " pod="kuadrant-system/authorino-6d4dfc649f-m2jvs"
Sep 16 02:26:40.464214 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:40.464182    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cv24t\" (UniqueName: \"kubernetes.io/projected/2b322541-b6af-4637-9d10-730bee14ba60-kube-api-access-cv24t\") pod \"2b322541-b6af-4637-9d10-730bee14ba60\" (UID: \"2b322541-b6af-4637-9d10-730bee14ba60\") "
Sep 16 02:26:40.464359 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:40.464234    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/54043da2-36be-42db-9de7-405c04467e52-tls-cert\") pod \"54043da2-36be-42db-9de7-405c04467e52\" (UID: \"54043da2-36be-42db-9de7-405c04467e52\") "
Sep 16 02:26:40.464406 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:40.464379    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vjlh4\" (UniqueName: \"kubernetes.io/projected/f24407e5-008c-48b4-8709-44cd316947da-kube-api-access-vjlh4\") pod \"authorino-6fd859b494-xjvtn\" (UID: \"f24407e5-008c-48b4-8709-44cd316947da\") " pod="kuadrant-system/authorino-6fd859b494-xjvtn"
Sep 16 02:26:40.464494 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:40.464455    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f24407e5-008c-48b4-8709-44cd316947da-tls-cert\") pod \"authorino-6fd859b494-xjvtn\" (UID: \"f24407e5-008c-48b4-8709-44cd316947da\") " pod="kuadrant-system/authorino-6fd859b494-xjvtn"
Sep 16 02:26:40.466340 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:40.466310    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/54043da2-36be-42db-9de7-405c04467e52-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "54043da2-36be-42db-9de7-405c04467e52" (UID: "54043da2-36be-42db-9de7-405c04467e52"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 02:26:40.466437 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:40.466389    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2b322541-b6af-4637-9d10-730bee14ba60-kube-api-access-cv24t" (OuterVolumeSpecName: "kube-api-access-cv24t") pod "2b322541-b6af-4637-9d10-730bee14ba60" (UID: "2b322541-b6af-4637-9d10-730bee14ba60"). InnerVolumeSpecName "kube-api-access-cv24t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 02:26:40.466885 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:40.466861    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f24407e5-008c-48b4-8709-44cd316947da-tls-cert\") pod \"authorino-6fd859b494-xjvtn\" (UID: \"f24407e5-008c-48b4-8709-44cd316947da\") " pod="kuadrant-system/authorino-6fd859b494-xjvtn"
Sep 16 02:26:40.473111 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:40.473092    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vjlh4\" (UniqueName: \"kubernetes.io/projected/f24407e5-008c-48b4-8709-44cd316947da-kube-api-access-vjlh4\") pod \"authorino-6fd859b494-xjvtn\" (UID: \"f24407e5-008c-48b4-8709-44cd316947da\") " pod="kuadrant-system/authorino-6fd859b494-xjvtn"
Sep 16 02:26:40.490829 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:40.490809    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6fd859b494-xjvtn"
Sep 16 02:26:40.564959 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:40.564931    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j4t68\" (UniqueName: \"kubernetes.io/projected/54043da2-36be-42db-9de7-405c04467e52-kube-api-access-j4t68\") pod \"54043da2-36be-42db-9de7-405c04467e52\" (UID: \"54043da2-36be-42db-9de7-405c04467e52\") "
Sep 16 02:26:40.565078 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:40.565064    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cv24t\" (UniqueName: \"kubernetes.io/projected/2b322541-b6af-4637-9d10-730bee14ba60-kube-api-access-cv24t\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:26:40.565078 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:40.565075    2579 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/54043da2-36be-42db-9de7-405c04467e52-tls-cert\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:26:40.566959 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:40.566932    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/54043da2-36be-42db-9de7-405c04467e52-kube-api-access-j4t68" (OuterVolumeSpecName: "kube-api-access-j4t68") pod "54043da2-36be-42db-9de7-405c04467e52" (UID: "54043da2-36be-42db-9de7-405c04467e52"). InnerVolumeSpecName "kube-api-access-j4t68". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 02:26:40.665587 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:40.665557    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j4t68\" (UniqueName: \"kubernetes.io/projected/54043da2-36be-42db-9de7-405c04467e52-kube-api-access-j4t68\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:26:40.759017 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:40.758903    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6fd859b494-xjvtn"]
Sep 16 02:26:40.761539 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:26:40.761513    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf24407e5_008c_48b4_8709_44cd316947da.slice/crio-7d5fe44073d7a728172cc08525dbc5d0f5257ed379c27e66e1edf2f0505559e7 WatchSource:0}: Error finding container 7d5fe44073d7a728172cc08525dbc5d0f5257ed379c27e66e1edf2f0505559e7: Status 404 returned error can't find the container with id 7d5fe44073d7a728172cc08525dbc5d0f5257ed379c27e66e1edf2f0505559e7
Sep 16 02:26:41.266231 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:41.266200    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-m2jvs"
Sep 16 02:26:41.266231 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:41.266209    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6fd859b494-xjvtn" event={"ID":"f24407e5-008c-48b4-8709-44cd316947da","Type":"ContainerStarted","Data":"7d5fe44073d7a728172cc08525dbc5d0f5257ed379c27e66e1edf2f0505559e7"}
Sep 16 02:26:41.266231 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:41.266223    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-462nb"
Sep 16 02:26:41.295984 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:41.295960    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-m2jvs"]
Sep 16 02:26:41.302510 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:41.302454    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-m2jvs"]
Sep 16 02:26:41.319571 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:41.319540    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-462nb"]
Sep 16 02:26:41.322605 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:41.322585    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-557f876f69-462nb"]
Sep 16 02:26:42.102146 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:42.102117    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2b322541-b6af-4637-9d10-730bee14ba60" path="/var/lib/kubelet/pods/2b322541-b6af-4637-9d10-730bee14ba60/volumes"
Sep 16 02:26:42.102403 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:42.102390    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="54043da2-36be-42db-9de7-405c04467e52" path="/var/lib/kubelet/pods/54043da2-36be-42db-9de7-405c04467e52/volumes"
Sep 16 02:26:42.272015 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:42.271980    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6fd859b494-xjvtn" event={"ID":"f24407e5-008c-48b4-8709-44cd316947da","Type":"ContainerStarted","Data":"d965ee52f9213f353f7318f8f5b8247b2c443f73e9d92fd116278476032cbd68"}
Sep 16 02:26:42.286333 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:42.286275    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-6fd859b494-xjvtn" podStartSLOduration=1.60655163 podStartE2EDuration="2.286256069s" totalImagesPullingTime="679.704439ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 02:26:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 02:26:42.285737138 +0000 UTC m=+822.669252384" watchObservedRunningTime="2026-09-16 02:26:42.286256069 +0000 UTC m=+822.669771315"
Sep 16 02:26:47.221318 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:47.221281    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:26:47.225734 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:47.225708    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-75dc6cbdf-5snl2"]
Sep 16 02:26:47.330136 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:47.330108    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-75dc6cbdf-5snl2"]
Sep 16 02:26:47.330279 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:47.330218    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-75dc6cbdf-5snl2"
Sep 16 02:26:47.334503 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:47.334481    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 16 02:26:47.334677 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:47.334665    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-cs9kk\""
Sep 16 02:26:47.335265 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:47.335250    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 16 02:26:47.335333 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:47.335318    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 16 02:26:47.394540 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:47.394510    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:26:47.416601 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:47.416571    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c0c2fbdc-b54e-44ae-b6d7-8379ce50aa3a-metrics-certs\") pod \"maas-controller-75dc6cbdf-5snl2\" (UID: \"c0c2fbdc-b54e-44ae-b6d7-8379ce50aa3a\") " pod="opendatahub/maas-controller-75dc6cbdf-5snl2"
Sep 16 02:26:47.416728 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:47.416621    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c0c2fbdc-b54e-44ae-b6d7-8379ce50aa3a-webhook-cert\") pod \"maas-controller-75dc6cbdf-5snl2\" (UID: \"c0c2fbdc-b54e-44ae-b6d7-8379ce50aa3a\") " pod="opendatahub/maas-controller-75dc6cbdf-5snl2"
Sep 16 02:26:47.416728 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:47.416642    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bs92l\" (UniqueName: \"kubernetes.io/projected/c0c2fbdc-b54e-44ae-b6d7-8379ce50aa3a-kube-api-access-bs92l\") pod \"maas-controller-75dc6cbdf-5snl2\" (UID: \"c0c2fbdc-b54e-44ae-b6d7-8379ce50aa3a\") " pod="opendatahub/maas-controller-75dc6cbdf-5snl2"
Sep 16 02:26:47.421658 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:47.421639    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-b878dcb6c-wrqd4"]
Sep 16 02:26:47.455369 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:47.455328    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:26:47.458939 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:47.458922    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-b878dcb6c-wrqd4"]
Sep 16 02:26:47.459024 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:47.459003    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-b878dcb6c-wrqd4"
Sep 16 02:26:47.517015 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:47.516961    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/168a7399-5259-4faf-93c1-2b1e5b9af5f3-metrics-certs\") pod \"maas-controller-b878dcb6c-wrqd4\" (UID: \"168a7399-5259-4faf-93c1-2b1e5b9af5f3\") " pod="opendatahub/maas-controller-b878dcb6c-wrqd4"
Sep 16 02:26:47.517015 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:47.516990    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k7lm7\" (UniqueName: \"kubernetes.io/projected/168a7399-5259-4faf-93c1-2b1e5b9af5f3-kube-api-access-k7lm7\") pod \"maas-controller-b878dcb6c-wrqd4\" (UID: \"168a7399-5259-4faf-93c1-2b1e5b9af5f3\") " pod="opendatahub/maas-controller-b878dcb6c-wrqd4"
Sep 16 02:26:47.517152 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:47.517018    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c0c2fbdc-b54e-44ae-b6d7-8379ce50aa3a-webhook-cert\") pod \"maas-controller-75dc6cbdf-5snl2\" (UID: \"c0c2fbdc-b54e-44ae-b6d7-8379ce50aa3a\") " pod="opendatahub/maas-controller-75dc6cbdf-5snl2"
Sep 16 02:26:47.517152 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:47.517037    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bs92l\" (UniqueName: \"kubernetes.io/projected/c0c2fbdc-b54e-44ae-b6d7-8379ce50aa3a-kube-api-access-bs92l\") pod \"maas-controller-75dc6cbdf-5snl2\" (UID: \"c0c2fbdc-b54e-44ae-b6d7-8379ce50aa3a\") " pod="opendatahub/maas-controller-75dc6cbdf-5snl2"
Sep 16 02:26:47.517247 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:47.517145    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c0c2fbdc-b54e-44ae-b6d7-8379ce50aa3a-metrics-certs\") pod \"maas-controller-75dc6cbdf-5snl2\" (UID: \"c0c2fbdc-b54e-44ae-b6d7-8379ce50aa3a\") " pod="opendatahub/maas-controller-75dc6cbdf-5snl2"
Sep 16 02:26:47.517247 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:47.517181    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/168a7399-5259-4faf-93c1-2b1e5b9af5f3-webhook-cert\") pod \"maas-controller-b878dcb6c-wrqd4\" (UID: \"168a7399-5259-4faf-93c1-2b1e5b9af5f3\") " pod="opendatahub/maas-controller-b878dcb6c-wrqd4"
Sep 16 02:26:47.519509 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:47.519486    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c0c2fbdc-b54e-44ae-b6d7-8379ce50aa3a-metrics-certs\") pod \"maas-controller-75dc6cbdf-5snl2\" (UID: \"c0c2fbdc-b54e-44ae-b6d7-8379ce50aa3a\") " pod="opendatahub/maas-controller-75dc6cbdf-5snl2"
Sep 16 02:26:47.519618 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:47.519524    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c0c2fbdc-b54e-44ae-b6d7-8379ce50aa3a-webhook-cert\") pod \"maas-controller-75dc6cbdf-5snl2\" (UID: \"c0c2fbdc-b54e-44ae-b6d7-8379ce50aa3a\") " pod="opendatahub/maas-controller-75dc6cbdf-5snl2"
Sep 16 02:26:47.525653 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:47.525632    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bs92l\" (UniqueName: \"kubernetes.io/projected/c0c2fbdc-b54e-44ae-b6d7-8379ce50aa3a-kube-api-access-bs92l\") pod \"maas-controller-75dc6cbdf-5snl2\" (UID: \"c0c2fbdc-b54e-44ae-b6d7-8379ce50aa3a\") " pod="opendatahub/maas-controller-75dc6cbdf-5snl2"
Sep 16 02:26:47.556691 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:47.556651    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:26:47.618455 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:47.618430    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/168a7399-5259-4faf-93c1-2b1e5b9af5f3-webhook-cert\") pod \"maas-controller-b878dcb6c-wrqd4\" (UID: \"168a7399-5259-4faf-93c1-2b1e5b9af5f3\") " pod="opendatahub/maas-controller-b878dcb6c-wrqd4"
Sep 16 02:26:47.618580 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:47.618460    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/168a7399-5259-4faf-93c1-2b1e5b9af5f3-metrics-certs\") pod \"maas-controller-b878dcb6c-wrqd4\" (UID: \"168a7399-5259-4faf-93c1-2b1e5b9af5f3\") " pod="opendatahub/maas-controller-b878dcb6c-wrqd4"
Sep 16 02:26:47.618580 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:47.618504    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k7lm7\" (UniqueName: \"kubernetes.io/projected/168a7399-5259-4faf-93c1-2b1e5b9af5f3-kube-api-access-k7lm7\") pod \"maas-controller-b878dcb6c-wrqd4\" (UID: \"168a7399-5259-4faf-93c1-2b1e5b9af5f3\") " pod="opendatahub/maas-controller-b878dcb6c-wrqd4"
Sep 16 02:26:47.621127 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:47.621105    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/168a7399-5259-4faf-93c1-2b1e5b9af5f3-webhook-cert\") pod \"maas-controller-b878dcb6c-wrqd4\" (UID: \"168a7399-5259-4faf-93c1-2b1e5b9af5f3\") " pod="opendatahub/maas-controller-b878dcb6c-wrqd4"
Sep 16 02:26:47.621236 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:47.621105    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/168a7399-5259-4faf-93c1-2b1e5b9af5f3-metrics-certs\") pod \"maas-controller-b878dcb6c-wrqd4\" (UID: \"168a7399-5259-4faf-93c1-2b1e5b9af5f3\") " pod="opendatahub/maas-controller-b878dcb6c-wrqd4"
Sep 16 02:26:47.626111 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:47.626083    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k7lm7\" (UniqueName: \"kubernetes.io/projected/168a7399-5259-4faf-93c1-2b1e5b9af5f3-kube-api-access-k7lm7\") pod \"maas-controller-b878dcb6c-wrqd4\" (UID: \"168a7399-5259-4faf-93c1-2b1e5b9af5f3\") " pod="opendatahub/maas-controller-b878dcb6c-wrqd4"
Sep 16 02:26:47.640825 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:47.640807    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-75dc6cbdf-5snl2"
Sep 16 02:26:47.768647 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:47.768584    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-b878dcb6c-wrqd4"
Sep 16 02:26:47.955508 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:47.955485    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-75dc6cbdf-5snl2"]
Sep 16 02:26:47.957560 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:26:47.957538    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0c2fbdc_b54e_44ae_b6d7_8379ce50aa3a.slice/crio-9757fa1166d96292463ebfb8046175bc68cb4b035923bc03fb46f7255ae5f834 WatchSource:0}: Error finding container 9757fa1166d96292463ebfb8046175bc68cb4b035923bc03fb46f7255ae5f834: Status 404 returned error can't find the container with id 9757fa1166d96292463ebfb8046175bc68cb4b035923bc03fb46f7255ae5f834
Sep 16 02:26:48.059664 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:48.059643    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-b878dcb6c-wrqd4"]
Sep 16 02:26:48.061865 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:26:48.061831    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod168a7399_5259_4faf_93c1_2b1e5b9af5f3.slice/crio-fa08c29a6edc3f3cd49dfd6cb656363ceabc88473f41dd33a88cba14572441fb WatchSource:0}: Error finding container fa08c29a6edc3f3cd49dfd6cb656363ceabc88473f41dd33a88cba14572441fb: Status 404 returned error can't find the container with id fa08c29a6edc3f3cd49dfd6cb656363ceabc88473f41dd33a88cba14572441fb
Sep 16 02:26:48.306793 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:48.306758    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-75dc6cbdf-5snl2" event={"ID":"c0c2fbdc-b54e-44ae-b6d7-8379ce50aa3a","Type":"ContainerStarted","Data":"9757fa1166d96292463ebfb8046175bc68cb4b035923bc03fb46f7255ae5f834"}
Sep 16 02:26:48.308024 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:48.308000    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-b878dcb6c-wrqd4" event={"ID":"168a7399-5259-4faf-93c1-2b1e5b9af5f3","Type":"ContainerStarted","Data":"fa08c29a6edc3f3cd49dfd6cb656363ceabc88473f41dd33a88cba14572441fb"}
Sep 16 02:26:50.763967 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:50.763916    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:26:50.859853 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:50.859761    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:26:51.324647 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:51.324602    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-b878dcb6c-wrqd4" event={"ID":"168a7399-5259-4faf-93c1-2b1e5b9af5f3","Type":"ContainerStarted","Data":"d24414f5231ca1d47600a7acbe72b36c4ed3cff04be64e5cea149f3e3c081ddc"}
Sep 16 02:26:51.324841 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:51.324754    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-b878dcb6c-wrqd4"
Sep 16 02:26:51.326095 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:51.326076    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-75dc6cbdf-5snl2" event={"ID":"c0c2fbdc-b54e-44ae-b6d7-8379ce50aa3a","Type":"ContainerStarted","Data":"9f78bc7f3992242e507a74522f69f391749c3e092a397d1f4f816bf9ed44715a"}
Sep 16 02:26:51.326214 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:51.326201    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-75dc6cbdf-5snl2"
Sep 16 02:26:51.341365 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:51.341326    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-b878dcb6c-wrqd4" podStartSLOduration=1.812553759 podStartE2EDuration="4.341313161s" totalImagesPullingTime="2.528759402s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 02:26:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 02:26:51.339726704 +0000 UTC m=+831.723241962" watchObservedRunningTime="2026-09-16 02:26:51.341313161 +0000 UTC m=+831.724828406"
Sep 16 02:26:51.356262 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:26:51.356221    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-75dc6cbdf-5snl2" podStartSLOduration=1.7299245810000001 podStartE2EDuration="4.35620931s" totalImagesPullingTime="2.626284729s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 02:26:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 02:26:51.354756017 +0000 UTC m=+831.738271285" watchObservedRunningTime="2026-09-16 02:26:51.35620931 +0000 UTC m=+831.739724554"
Sep 16 02:27:02.333522 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:27:02.333485    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-b878dcb6c-wrqd4"
Sep 16 02:27:02.334057 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:27:02.333583    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-75dc6cbdf-5snl2"
Sep 16 02:27:02.377464 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:27:02.377432    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-75dc6cbdf-5snl2"]
Sep 16 02:27:02.378434 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:27:02.377976    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-75dc6cbdf-5snl2" podUID="c0c2fbdc-b54e-44ae-b6d7-8379ce50aa3a" containerName="manager" containerID="cri-o://9f78bc7f3992242e507a74522f69f391749c3e092a397d1f4f816bf9ed44715a" gracePeriod=30
Sep 16 02:27:02.617496 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:27:02.617457    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-75dc6cbdf-5snl2"
Sep 16 02:27:02.639167 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:27:02.639143    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bs92l\" (UniqueName: \"kubernetes.io/projected/c0c2fbdc-b54e-44ae-b6d7-8379ce50aa3a-kube-api-access-bs92l\") pod \"c0c2fbdc-b54e-44ae-b6d7-8379ce50aa3a\" (UID: \"c0c2fbdc-b54e-44ae-b6d7-8379ce50aa3a\") "
Sep 16 02:27:02.639309 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:27:02.639193    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c0c2fbdc-b54e-44ae-b6d7-8379ce50aa3a-metrics-certs\") pod \"c0c2fbdc-b54e-44ae-b6d7-8379ce50aa3a\" (UID: \"c0c2fbdc-b54e-44ae-b6d7-8379ce50aa3a\") "
Sep 16 02:27:02.639309 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:27:02.639246    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c0c2fbdc-b54e-44ae-b6d7-8379ce50aa3a-webhook-cert\") pod \"c0c2fbdc-b54e-44ae-b6d7-8379ce50aa3a\" (UID: \"c0c2fbdc-b54e-44ae-b6d7-8379ce50aa3a\") "
Sep 16 02:27:02.641792 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:27:02.641757    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c0c2fbdc-b54e-44ae-b6d7-8379ce50aa3a-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "c0c2fbdc-b54e-44ae-b6d7-8379ce50aa3a" (UID: "c0c2fbdc-b54e-44ae-b6d7-8379ce50aa3a"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 02:27:02.641901 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:27:02.641856    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c0c2fbdc-b54e-44ae-b6d7-8379ce50aa3a-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "c0c2fbdc-b54e-44ae-b6d7-8379ce50aa3a" (UID: "c0c2fbdc-b54e-44ae-b6d7-8379ce50aa3a"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 02:27:02.641901 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:27:02.641880    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c0c2fbdc-b54e-44ae-b6d7-8379ce50aa3a-kube-api-access-bs92l" (OuterVolumeSpecName: "kube-api-access-bs92l") pod "c0c2fbdc-b54e-44ae-b6d7-8379ce50aa3a" (UID: "c0c2fbdc-b54e-44ae-b6d7-8379ce50aa3a"). InnerVolumeSpecName "kube-api-access-bs92l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 02:27:02.740269 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:27:02.740226    2579 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c0c2fbdc-b54e-44ae-b6d7-8379ce50aa3a-webhook-cert\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:27:02.740269 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:27:02.740265    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bs92l\" (UniqueName: \"kubernetes.io/projected/c0c2fbdc-b54e-44ae-b6d7-8379ce50aa3a-kube-api-access-bs92l\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:27:02.740269 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:27:02.740278    2579 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c0c2fbdc-b54e-44ae-b6d7-8379ce50aa3a-metrics-certs\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:27:03.382260 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:27:03.382225    2579 generic.go:356] "Generic (PLEG): container finished" podID="c0c2fbdc-b54e-44ae-b6d7-8379ce50aa3a" containerID="9f78bc7f3992242e507a74522f69f391749c3e092a397d1f4f816bf9ed44715a" exitCode=0
Sep 16 02:27:03.382697 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:27:03.382260    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-75dc6cbdf-5snl2" event={"ID":"c0c2fbdc-b54e-44ae-b6d7-8379ce50aa3a","Type":"ContainerDied","Data":"9f78bc7f3992242e507a74522f69f391749c3e092a397d1f4f816bf9ed44715a"}
Sep 16 02:27:03.382697 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:27:03.382317    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-75dc6cbdf-5snl2" event={"ID":"c0c2fbdc-b54e-44ae-b6d7-8379ce50aa3a","Type":"ContainerDied","Data":"9757fa1166d96292463ebfb8046175bc68cb4b035923bc03fb46f7255ae5f834"}
Sep 16 02:27:03.382697 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:27:03.382338    2579 scope.go:117] "RemoveContainer" containerID="9f78bc7f3992242e507a74522f69f391749c3e092a397d1f4f816bf9ed44715a"
Sep 16 02:27:03.382697 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:27:03.382272    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-75dc6cbdf-5snl2"
Sep 16 02:27:03.393106 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:27:03.393088    2579 scope.go:117] "RemoveContainer" containerID="9f78bc7f3992242e507a74522f69f391749c3e092a397d1f4f816bf9ed44715a"
Sep 16 02:27:03.393389 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:27:03.393371    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9f78bc7f3992242e507a74522f69f391749c3e092a397d1f4f816bf9ed44715a\": container with ID starting with 9f78bc7f3992242e507a74522f69f391749c3e092a397d1f4f816bf9ed44715a not found: ID does not exist" containerID="9f78bc7f3992242e507a74522f69f391749c3e092a397d1f4f816bf9ed44715a"
Sep 16 02:27:03.393437 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:27:03.393398    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f78bc7f3992242e507a74522f69f391749c3e092a397d1f4f816bf9ed44715a"} err="failed to get container status \"9f78bc7f3992242e507a74522f69f391749c3e092a397d1f4f816bf9ed44715a\": rpc error: code = NotFound desc = could not find container \"9f78bc7f3992242e507a74522f69f391749c3e092a397d1f4f816bf9ed44715a\": container with ID starting with 9f78bc7f3992242e507a74522f69f391749c3e092a397d1f4f816bf9ed44715a not found: ID does not exist"
Sep 16 02:27:03.404576 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:27:03.404552    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-75dc6cbdf-5snl2"]
Sep 16 02:27:03.408004 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:27:03.407984    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-75dc6cbdf-5snl2"]
Sep 16 02:27:04.102640 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:27:04.102609    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c0c2fbdc-b54e-44ae-b6d7-8379ce50aa3a" path="/var/lib/kubelet/pods/c0c2fbdc-b54e-44ae-b6d7-8379ce50aa3a/volumes"
Sep 16 02:27:23.231480 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:27:23.231423    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:27:23.332432 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:27:23.332397    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:27:24.650528 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:27:24.650487    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:27:24.716033 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:27:24.715997    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:27:24.953640 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:27:24.953608    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:27:25.111836 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:27:25.111803    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:27:25.209040 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:27:25.208949    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:27:25.225922 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:27:25.225889    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-qjdcd"]
Sep 16 02:27:25.226231 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:27:25.226219    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0c2fbdc-b54e-44ae-b6d7-8379ce50aa3a" containerName="manager"
Sep 16 02:27:25.226277 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:27:25.226233    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0c2fbdc-b54e-44ae-b6d7-8379ce50aa3a" containerName="manager"
Sep 16 02:27:25.226313 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:27:25.226284    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="c0c2fbdc-b54e-44ae-b6d7-8379ce50aa3a" containerName="manager"
Sep 16 02:27:25.229089 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:27:25.229074    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-qjdcd"
Sep 16 02:27:25.231516 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:27:25.231491    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Sep 16 02:27:25.231629 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:27:25.231516    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-zzx6j\""
Sep 16 02:27:25.239406 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:27:25.239379    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-qjdcd"]
Sep 16 02:27:25.327304 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:27:25.327271    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ba48ec30-ade2-477f-9486-c2fd16d1caae-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-qjdcd\" (UID: \"ba48ec30-ade2-477f-9486-c2fd16d1caae\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-qjdcd"
Sep 16 02:27:25.327493 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:27:25.327337    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5qfhs\" (UniqueName: \"kubernetes.io/projected/ba48ec30-ade2-477f-9486-c2fd16d1caae-kube-api-access-5qfhs\") pod \"payload-pre-processing-59fbc47bf8-qjdcd\" (UID: \"ba48ec30-ade2-477f-9486-c2fd16d1caae\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-qjdcd"
Sep 16 02:27:25.428503 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:27:25.428450    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ba48ec30-ade2-477f-9486-c2fd16d1caae-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-qjdcd\" (UID: \"ba48ec30-ade2-477f-9486-c2fd16d1caae\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-qjdcd"
Sep 16 02:27:25.428677 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:27:25.428525    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qfhs\" (UniqueName: \"kubernetes.io/projected/ba48ec30-ade2-477f-9486-c2fd16d1caae-kube-api-access-5qfhs\") pod \"payload-pre-processing-59fbc47bf8-qjdcd\" (UID: \"ba48ec30-ade2-477f-9486-c2fd16d1caae\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-qjdcd"
Sep 16 02:27:25.429343 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:27:25.429314    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ba48ec30-ade2-477f-9486-c2fd16d1caae-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-qjdcd\" (UID: \"ba48ec30-ade2-477f-9486-c2fd16d1caae\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-qjdcd"
Sep 16 02:27:25.439345 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:27:25.439319    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5qfhs\" (UniqueName: \"kubernetes.io/projected/ba48ec30-ade2-477f-9486-c2fd16d1caae-kube-api-access-5qfhs\") pod \"payload-pre-processing-59fbc47bf8-qjdcd\" (UID: \"ba48ec30-ade2-477f-9486-c2fd16d1caae\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-qjdcd"
Sep 16 02:27:25.541147 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:27:25.541060    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-qjdcd"
Sep 16 02:27:25.813358 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:27:25.813280    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-qjdcd"]
Sep 16 02:27:25.816880 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:27:25.816854    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba48ec30_ade2_477f_9486_c2fd16d1caae.slice/crio-27d597e0c1e7c04e2e437beac929601c0f691dd94d8b367ec89f08d12fc33fac WatchSource:0}: Error finding container 27d597e0c1e7c04e2e437beac929601c0f691dd94d8b367ec89f08d12fc33fac: Status 404 returned error can't find the container with id 27d597e0c1e7c04e2e437beac929601c0f691dd94d8b367ec89f08d12fc33fac
Sep 16 02:27:26.491903 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:27:26.491870    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-qjdcd" event={"ID":"ba48ec30-ade2-477f-9486-c2fd16d1caae","Type":"ContainerStarted","Data":"27d597e0c1e7c04e2e437beac929601c0f691dd94d8b367ec89f08d12fc33fac"}
Sep 16 02:27:27.498713 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:27:27.498675    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-qjdcd" event={"ID":"ba48ec30-ade2-477f-9486-c2fd16d1caae","Type":"ContainerStarted","Data":"4d6b2da1644a31d28f6bfb42d4497df6ebc37866aab178955064ee527fd09b0a"}
Sep 16 02:27:27.499086 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:27:27.498753    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-qjdcd"
Sep 16 02:27:27.513993 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:27:27.513944    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-qjdcd" podStartSLOduration=1.521202116 podStartE2EDuration="2.513929518s" totalImagesPullingTime="992.727402ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 02:27:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 02:27:27.51254542 +0000 UTC m=+867.896060664" watchObservedRunningTime="2026-09-16 02:27:27.513929518 +0000 UTC m=+867.897444763"
Sep 16 02:27:36.048945 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:27:36.048905    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:27:36.186216 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:27:36.186184    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:27:38.505657 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:27:38.505624    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-qjdcd"
Sep 16 02:27:47.076115 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:27:47.076083    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bqp4b"]
Sep 16 02:27:47.084021 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:27:47.083978    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bqp4b"
Sep 16 02:27:47.086745 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:27:47.086717    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 16 02:27:47.087799 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:27:47.087777    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-lrwjw\""
Sep 16 02:27:47.088273 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:27:47.088257    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Sep 16 02:27:47.088744 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:27:47.088718    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 16 02:27:47.090162 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:27:47.090142    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bqp4b"]
Sep 16 02:27:47.097842 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:27:47.097812    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:27:47.208760 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:27:47.208730    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h5gzc\" (UniqueName: \"kubernetes.io/projected/5f1dc5bb-e330-4f02-bc3b-d39868a8250b-kube-api-access-h5gzc\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bqp4b\" (UID: \"5f1dc5bb-e330-4f02-bc3b-d39868a8250b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bqp4b"
Sep 16 02:27:47.208888 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:27:47.208791    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5f1dc5bb-e330-4f02-bc3b-d39868a8250b-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bqp4b\" (UID: \"5f1dc5bb-e330-4f02-bc3b-d39868a8250b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bqp4b"
Sep 16 02:27:47.208888 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:27:47.208822    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5f1dc5bb-e330-4f02-bc3b-d39868a8250b-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bqp4b\" (UID: \"5f1dc5bb-e330-4f02-bc3b-d39868a8250b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bqp4b"
Sep 16 02:27:47.208975 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:27:47.208941    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5f1dc5bb-e330-4f02-bc3b-d39868a8250b-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bqp4b\" (UID: \"5f1dc5bb-e330-4f02-bc3b-d39868a8250b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bqp4b"
Sep 16 02:27:47.209022 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:27:47.208990    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5f1dc5bb-e330-4f02-bc3b-d39868a8250b-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bqp4b\" (UID: \"5f1dc5bb-e330-4f02-bc3b-d39868a8250b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bqp4b"
Sep 16 02:27:47.209072 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:27:47.209058    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5f1dc5bb-e330-4f02-bc3b-d39868a8250b-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bqp4b\" (UID: \"5f1dc5bb-e330-4f02-bc3b-d39868a8250b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bqp4b"
Sep 16 02:27:47.309824 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:27:47.309795    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5f1dc5bb-e330-4f02-bc3b-d39868a8250b-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bqp4b\" (UID: \"5f1dc5bb-e330-4f02-bc3b-d39868a8250b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bqp4b"
Sep 16 02:27:47.309967 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:27:47.309836    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5f1dc5bb-e330-4f02-bc3b-d39868a8250b-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bqp4b\" (UID: \"5f1dc5bb-e330-4f02-bc3b-d39868a8250b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bqp4b"
Sep 16 02:27:47.309967 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:27:47.309881    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5f1dc5bb-e330-4f02-bc3b-d39868a8250b-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bqp4b\" (UID: \"5f1dc5bb-e330-4f02-bc3b-d39868a8250b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bqp4b"
Sep 16 02:27:47.309967 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:27:47.309919    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5gzc\" (UniqueName: \"kubernetes.io/projected/5f1dc5bb-e330-4f02-bc3b-d39868a8250b-kube-api-access-h5gzc\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bqp4b\" (UID: \"5f1dc5bb-e330-4f02-bc3b-d39868a8250b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bqp4b"
Sep 16 02:27:47.309967 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:27:47.309954    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5f1dc5bb-e330-4f02-bc3b-d39868a8250b-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bqp4b\" (UID: \"5f1dc5bb-e330-4f02-bc3b-d39868a8250b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bqp4b"
Sep 16 02:27:47.310170 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:27:47.309976    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5f1dc5bb-e330-4f02-bc3b-d39868a8250b-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bqp4b\" (UID: \"5f1dc5bb-e330-4f02-bc3b-d39868a8250b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bqp4b"
Sep 16 02:27:47.310271 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:27:47.310249    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5f1dc5bb-e330-4f02-bc3b-d39868a8250b-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bqp4b\" (UID: \"5f1dc5bb-e330-4f02-bc3b-d39868a8250b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bqp4b"
Sep 16 02:27:47.310346 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:27:47.310285    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5f1dc5bb-e330-4f02-bc3b-d39868a8250b-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bqp4b\" (UID: \"5f1dc5bb-e330-4f02-bc3b-d39868a8250b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bqp4b"
Sep 16 02:27:47.310346 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:27:47.310324    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5f1dc5bb-e330-4f02-bc3b-d39868a8250b-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bqp4b\" (UID: \"5f1dc5bb-e330-4f02-bc3b-d39868a8250b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bqp4b"
Sep 16 02:27:47.312337 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:27:47.312316    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5f1dc5bb-e330-4f02-bc3b-d39868a8250b-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bqp4b\" (UID: \"5f1dc5bb-e330-4f02-bc3b-d39868a8250b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bqp4b"
Sep 16 02:27:47.312650 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:27:47.312633    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5f1dc5bb-e330-4f02-bc3b-d39868a8250b-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bqp4b\" (UID: \"5f1dc5bb-e330-4f02-bc3b-d39868a8250b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bqp4b"
Sep 16 02:27:47.316986 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:27:47.316960    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h5gzc\" (UniqueName: \"kubernetes.io/projected/5f1dc5bb-e330-4f02-bc3b-d39868a8250b-kube-api-access-h5gzc\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bqp4b\" (UID: \"5f1dc5bb-e330-4f02-bc3b-d39868a8250b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bqp4b"
Sep 16 02:27:47.415808 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:27:47.415731    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bqp4b"
Sep 16 02:27:47.686492 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:27:47.686399    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bqp4b"]
Sep 16 02:27:47.688626 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:27:47.688600    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f1dc5bb_e330_4f02_bc3b_d39868a8250b.slice/crio-457eede62f135e89491253867fb7f0119ba905af9903f342cd6902d196338860 WatchSource:0}: Error finding container 457eede62f135e89491253867fb7f0119ba905af9903f342cd6902d196338860: Status 404 returned error can't find the container with id 457eede62f135e89491253867fb7f0119ba905af9903f342cd6902d196338860
Sep 16 02:27:48.594735 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:27:48.594686    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bqp4b" event={"ID":"5f1dc5bb-e330-4f02-bc3b-d39868a8250b","Type":"ContainerStarted","Data":"457eede62f135e89491253867fb7f0119ba905af9903f342cd6902d196338860"}
Sep 16 02:27:49.602164 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:27:49.602104    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bqp4b" event={"ID":"5f1dc5bb-e330-4f02-bc3b-d39868a8250b","Type":"ContainerStarted","Data":"72e26e142494c274af79d4dbbd1d8d50f19fa279f651017b196acead2c398083"}
Sep 16 02:27:49.617624 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:27:49.617571    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bqp4b" podStartSLOduration=1.56865676 podStartE2EDuration="2.617539876s" totalImagesPullingTime="1.048883116s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 02:27:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 02:27:49.617070437 +0000 UTC m=+890.000585683" watchObservedRunningTime="2026-09-16 02:27:49.617539876 +0000 UTC m=+890.001055125"
Sep 16 02:27:55.619330 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:27:55.619252    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:27:57.416323 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:27:57.416288    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bqp4b"
Sep 16 02:27:57.416875 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:27:57.416343    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bqp4b"
Sep 16 02:27:57.428981 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:27:57.428954    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bqp4b"
Sep 16 02:27:57.649417 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:27:57.649390    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bqp4b"
Sep 16 02:28:00.105079 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:28:00.105054    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-c8q8z_ebf846fa-af7d-4fdd-9662-b7c533c36ddd/console-operator/2.log"
Sep 16 02:28:00.105560 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:28:00.105539    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-c8q8z_ebf846fa-af7d-4fdd-9662-b7c533c36ddd/console-operator/2.log"
Sep 16 02:28:06.430419 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:28:06.430388    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:28:15.303884 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:28:15.303843    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:28:24.097797 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:28:24.097765    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:28:24.198727 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:28:24.198683    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:28:26.268654 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:28:26.268624    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-zq4lj"]
Sep 16 02:28:26.272484 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:28:26.272451    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zq4lj"
Sep 16 02:28:26.274746 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:28:26.274720    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Sep 16 02:28:26.280602 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:28:26.280579    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-zq4lj"]
Sep 16 02:28:26.297969 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:28:26.297939    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:28:26.324697 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:28:26.324671    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b9a06f09-f326-4b7b-b74f-de8fb1a35187-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-zq4lj\" (UID: \"b9a06f09-f326-4b7b-b74f-de8fb1a35187\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zq4lj"
Sep 16 02:28:26.324812 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:28:26.324713    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4z42x\" (UniqueName: \"kubernetes.io/projected/b9a06f09-f326-4b7b-b74f-de8fb1a35187-kube-api-access-4z42x\") pod \"e2e-trlp-test-simulated-kserve-598466548d-zq4lj\" (UID: \"b9a06f09-f326-4b7b-b74f-de8fb1a35187\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zq4lj"
Sep 16 02:28:26.324812 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:28:26.324738    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b9a06f09-f326-4b7b-b74f-de8fb1a35187-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-zq4lj\" (UID: \"b9a06f09-f326-4b7b-b74f-de8fb1a35187\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zq4lj"
Sep 16 02:28:26.324812 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:28:26.324779    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b9a06f09-f326-4b7b-b74f-de8fb1a35187-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-zq4lj\" (UID: \"b9a06f09-f326-4b7b-b74f-de8fb1a35187\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zq4lj"
Sep 16 02:28:26.324812 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:28:26.324804    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b9a06f09-f326-4b7b-b74f-de8fb1a35187-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-zq4lj\" (UID: \"b9a06f09-f326-4b7b-b74f-de8fb1a35187\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zq4lj"
Sep 16 02:28:26.324950 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:28:26.324844    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b9a06f09-f326-4b7b-b74f-de8fb1a35187-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-zq4lj\" (UID: \"b9a06f09-f326-4b7b-b74f-de8fb1a35187\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zq4lj"
Sep 16 02:28:26.426038 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:28:26.426009    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b9a06f09-f326-4b7b-b74f-de8fb1a35187-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-zq4lj\" (UID: \"b9a06f09-f326-4b7b-b74f-de8fb1a35187\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zq4lj"
Sep 16 02:28:26.426163 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:28:26.426050    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4z42x\" (UniqueName: \"kubernetes.io/projected/b9a06f09-f326-4b7b-b74f-de8fb1a35187-kube-api-access-4z42x\") pod \"e2e-trlp-test-simulated-kserve-598466548d-zq4lj\" (UID: \"b9a06f09-f326-4b7b-b74f-de8fb1a35187\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zq4lj"
Sep 16 02:28:26.426163 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:28:26.426075    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b9a06f09-f326-4b7b-b74f-de8fb1a35187-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-zq4lj\" (UID: \"b9a06f09-f326-4b7b-b74f-de8fb1a35187\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zq4lj"
Sep 16 02:28:26.426163 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:28:26.426098    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b9a06f09-f326-4b7b-b74f-de8fb1a35187-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-zq4lj\" (UID: \"b9a06f09-f326-4b7b-b74f-de8fb1a35187\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zq4lj"
Sep 16 02:28:26.426163 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:28:26.426130    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b9a06f09-f326-4b7b-b74f-de8fb1a35187-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-zq4lj\" (UID: \"b9a06f09-f326-4b7b-b74f-de8fb1a35187\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zq4lj"
Sep 16 02:28:26.426355 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:28:26.426179    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b9a06f09-f326-4b7b-b74f-de8fb1a35187-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-zq4lj\" (UID: \"b9a06f09-f326-4b7b-b74f-de8fb1a35187\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zq4lj"
Sep 16 02:28:26.426568 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:28:26.426544    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b9a06f09-f326-4b7b-b74f-de8fb1a35187-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-zq4lj\" (UID: \"b9a06f09-f326-4b7b-b74f-de8fb1a35187\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zq4lj"
Sep 16 02:28:26.426646 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:28:26.426576    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b9a06f09-f326-4b7b-b74f-de8fb1a35187-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-zq4lj\" (UID: \"b9a06f09-f326-4b7b-b74f-de8fb1a35187\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zq4lj"
Sep 16 02:28:26.426646 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:28:26.426605    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b9a06f09-f326-4b7b-b74f-de8fb1a35187-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-zq4lj\" (UID: \"b9a06f09-f326-4b7b-b74f-de8fb1a35187\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zq4lj"
Sep 16 02:28:26.428515 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:28:26.428484    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b9a06f09-f326-4b7b-b74f-de8fb1a35187-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-zq4lj\" (UID: \"b9a06f09-f326-4b7b-b74f-de8fb1a35187\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zq4lj"
Sep 16 02:28:26.428679 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:28:26.428663    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b9a06f09-f326-4b7b-b74f-de8fb1a35187-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-zq4lj\" (UID: \"b9a06f09-f326-4b7b-b74f-de8fb1a35187\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zq4lj"
Sep 16 02:28:26.433022 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:28:26.433006    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4z42x\" (UniqueName: \"kubernetes.io/projected/b9a06f09-f326-4b7b-b74f-de8fb1a35187-kube-api-access-4z42x\") pod \"e2e-trlp-test-simulated-kserve-598466548d-zq4lj\" (UID: \"b9a06f09-f326-4b7b-b74f-de8fb1a35187\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zq4lj"
Sep 16 02:28:26.585587 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:28:26.585506    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zq4lj"
Sep 16 02:28:26.858639 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:28:26.858601    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-zq4lj"]
Sep 16 02:28:26.860919 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:28:26.860891    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9a06f09_f326_4b7b_b74f_de8fb1a35187.slice/crio-d4b391890c7a5af546180bfd33453b138ac54c1fff56dedab0b9f5c1beb6985e WatchSource:0}: Error finding container d4b391890c7a5af546180bfd33453b138ac54c1fff56dedab0b9f5c1beb6985e: Status 404 returned error can't find the container with id d4b391890c7a5af546180bfd33453b138ac54c1fff56dedab0b9f5c1beb6985e
Sep 16 02:28:26.862164 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:28:26.862148    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 16 02:28:27.773256 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:28:27.773221    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zq4lj" event={"ID":"b9a06f09-f326-4b7b-b74f-de8fb1a35187","Type":"ContainerStarted","Data":"c38dcad88f0ba78484a4890addea589b012c0801e192da3c8ff495e4aaf634be"}
Sep 16 02:28:27.773674 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:28:27.773272    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zq4lj" event={"ID":"b9a06f09-f326-4b7b-b74f-de8fb1a35187","Type":"ContainerStarted","Data":"d4b391890c7a5af546180bfd33453b138ac54c1fff56dedab0b9f5c1beb6985e"}
Sep 16 02:28:27.794249 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:28:27.794192    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zq4lj" podStartSLOduration=1.610253242 podStartE2EDuration="1.794176071s" totalImagesPullingTime="183.922829ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 02:28:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 02:28:27.792561799 +0000 UTC m=+928.176077044" watchObservedRunningTime="2026-09-16 02:28:27.794176071 +0000 UTC m=+928.177691315"
Sep 16 02:28:29.912340 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:28:29.912305    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:28:36.586402 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:28:36.586358    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zq4lj"
Sep 16 02:28:36.586812 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:28:36.586439    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zq4lj"
Sep 16 02:28:36.599072 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:28:36.599050    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zq4lj"
Sep 16 02:28:36.813827 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:28:36.813789    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:28:36.827985 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:28:36.827960    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zq4lj"
Sep 16 02:28:42.676187 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:28:42.676154    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-kt5tg"]
Sep 16 02:28:42.679613 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:28:42.679596    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-kt5tg"
Sep 16 02:28:42.681786 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:28:42.681763    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Sep 16 02:28:42.687237 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:28:42.687212    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-kt5tg"]
Sep 16 02:28:42.764497 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:28:42.764451    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/14f09d5e-725c-43d5-a17f-7183e94c2d44-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-kt5tg\" (UID: \"14f09d5e-725c-43d5-a17f-7183e94c2d44\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-kt5tg"
Sep 16 02:28:42.764682 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:28:42.764507    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k5899\" (UniqueName: \"kubernetes.io/projected/14f09d5e-725c-43d5-a17f-7183e94c2d44-kube-api-access-k5899\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-kt5tg\" (UID: \"14f09d5e-725c-43d5-a17f-7183e94c2d44\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-kt5tg"
Sep 16 02:28:42.764682 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:28:42.764552    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/14f09d5e-725c-43d5-a17f-7183e94c2d44-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-kt5tg\" (UID: \"14f09d5e-725c-43d5-a17f-7183e94c2d44\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-kt5tg"
Sep 16 02:28:42.764803 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:28:42.764674    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/14f09d5e-725c-43d5-a17f-7183e94c2d44-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-kt5tg\" (UID: \"14f09d5e-725c-43d5-a17f-7183e94c2d44\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-kt5tg"
Sep 16 02:28:42.764803 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:28:42.764758    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/14f09d5e-725c-43d5-a17f-7183e94c2d44-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-kt5tg\" (UID: \"14f09d5e-725c-43d5-a17f-7183e94c2d44\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-kt5tg"
Sep 16 02:28:42.764892 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:28:42.764831    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/14f09d5e-725c-43d5-a17f-7183e94c2d44-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-kt5tg\" (UID: \"14f09d5e-725c-43d5-a17f-7183e94c2d44\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-kt5tg"
Sep 16 02:28:42.776163 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:28:42.776132    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:28:42.865894 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:28:42.865854    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/14f09d5e-725c-43d5-a17f-7183e94c2d44-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-kt5tg\" (UID: \"14f09d5e-725c-43d5-a17f-7183e94c2d44\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-kt5tg"
Sep 16 02:28:42.866059 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:28:42.865905    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/14f09d5e-725c-43d5-a17f-7183e94c2d44-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-kt5tg\" (UID: \"14f09d5e-725c-43d5-a17f-7183e94c2d44\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-kt5tg"
Sep 16 02:28:42.866059 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:28:42.865939    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/14f09d5e-725c-43d5-a17f-7183e94c2d44-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-kt5tg\" (UID: \"14f09d5e-725c-43d5-a17f-7183e94c2d44\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-kt5tg"
Sep 16 02:28:42.866059 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:28:42.865982    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/14f09d5e-725c-43d5-a17f-7183e94c2d44-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-kt5tg\" (UID: \"14f09d5e-725c-43d5-a17f-7183e94c2d44\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-kt5tg"
Sep 16 02:28:42.866059 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:28:42.866005    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k5899\" (UniqueName: \"kubernetes.io/projected/14f09d5e-725c-43d5-a17f-7183e94c2d44-kube-api-access-k5899\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-kt5tg\" (UID: \"14f09d5e-725c-43d5-a17f-7183e94c2d44\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-kt5tg"
Sep 16 02:28:42.866059 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:28:42.866032    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/14f09d5e-725c-43d5-a17f-7183e94c2d44-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-kt5tg\" (UID: \"14f09d5e-725c-43d5-a17f-7183e94c2d44\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-kt5tg"
Sep 16 02:28:42.866491 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:28:42.866449    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/14f09d5e-725c-43d5-a17f-7183e94c2d44-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-kt5tg\" (UID: \"14f09d5e-725c-43d5-a17f-7183e94c2d44\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-kt5tg"
Sep 16 02:28:42.866630 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:28:42.866538    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/14f09d5e-725c-43d5-a17f-7183e94c2d44-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-kt5tg\" (UID: \"14f09d5e-725c-43d5-a17f-7183e94c2d44\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-kt5tg"
Sep 16 02:28:42.866703 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:28:42.866670    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/14f09d5e-725c-43d5-a17f-7183e94c2d44-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-kt5tg\" (UID: \"14f09d5e-725c-43d5-a17f-7183e94c2d44\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-kt5tg"
Sep 16 02:28:42.868461 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:28:42.868438    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/14f09d5e-725c-43d5-a17f-7183e94c2d44-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-kt5tg\" (UID: \"14f09d5e-725c-43d5-a17f-7183e94c2d44\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-kt5tg"
Sep 16 02:28:42.868948 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:28:42.868932    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/14f09d5e-725c-43d5-a17f-7183e94c2d44-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-kt5tg\" (UID: \"14f09d5e-725c-43d5-a17f-7183e94c2d44\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-kt5tg"
Sep 16 02:28:42.873594 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:28:42.873574    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k5899\" (UniqueName: \"kubernetes.io/projected/14f09d5e-725c-43d5-a17f-7183e94c2d44-kube-api-access-k5899\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-kt5tg\" (UID: \"14f09d5e-725c-43d5-a17f-7183e94c2d44\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-kt5tg"
Sep 16 02:28:42.989300 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:28:42.989270    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-kt5tg"
Sep 16 02:28:43.257820 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:28:43.257797    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-kt5tg"]
Sep 16 02:28:43.259954 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:28:43.259928    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod14f09d5e_725c_43d5_a17f_7183e94c2d44.slice/crio-d3fb57857d0b7ecc40b4b22ccfa4b45e269b7b4a3fbb60bae43d5c409dabd2d6 WatchSource:0}: Error finding container d3fb57857d0b7ecc40b4b22ccfa4b45e269b7b4a3fbb60bae43d5c409dabd2d6: Status 404 returned error can't find the container with id d3fb57857d0b7ecc40b4b22ccfa4b45e269b7b4a3fbb60bae43d5c409dabd2d6
Sep 16 02:28:43.854539 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:28:43.854499    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-kt5tg" event={"ID":"14f09d5e-725c-43d5-a17f-7183e94c2d44","Type":"ContainerStarted","Data":"ab9389d17b50c04b764041b0c93769f8e39042c8a8511704a9566a00169d473c"}
Sep 16 02:28:43.854539 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:28:43.854542    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-kt5tg" event={"ID":"14f09d5e-725c-43d5-a17f-7183e94c2d44","Type":"ContainerStarted","Data":"d3fb57857d0b7ecc40b4b22ccfa4b45e269b7b4a3fbb60bae43d5c409dabd2d6"}
Sep 16 02:28:43.874067 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:28:43.873987    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-kt5tg" podStartSLOduration=1.698696198 podStartE2EDuration="1.873967024s" totalImagesPullingTime="175.270826ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 02:28:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 02:28:43.871816476 +0000 UTC m=+944.255331720" watchObservedRunningTime="2026-09-16 02:28:43.873967024 +0000 UTC m=+944.257482270"
Sep 16 02:28:52.989861 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:28:52.989824    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-kt5tg"
Sep 16 02:28:52.990346 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:28:52.990127    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-kt5tg"
Sep 16 02:28:53.002596 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:28:53.002564    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-kt5tg"
Sep 16 02:28:53.910827 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:28:53.910798    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-kt5tg"
Sep 16 02:29:44.005743 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:29:44.005707    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:29:44.058986 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:29:44.058956    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:29:57.931228 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:29:57.931189    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6784f86975-8tkcv"]
Sep 16 02:29:57.934656 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:29:57.934633    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6784f86975-8tkcv"
Sep 16 02:29:57.941183 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:29:57.941160    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6784f86975-8tkcv"]
Sep 16 02:29:57.982420 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:29:57.982390    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8wxcr\" (UniqueName: \"kubernetes.io/projected/b05ade60-6d3a-493e-a720-923ee22c57cb-kube-api-access-8wxcr\") pod \"authorino-6784f86975-8tkcv\" (UID: \"b05ade60-6d3a-493e-a720-923ee22c57cb\") " pod="kuadrant-system/authorino-6784f86975-8tkcv"
Sep 16 02:29:57.982577 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:29:57.982457    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/b05ade60-6d3a-493e-a720-923ee22c57cb-tls-cert\") pod \"authorino-6784f86975-8tkcv\" (UID: \"b05ade60-6d3a-493e-a720-923ee22c57cb\") " pod="kuadrant-system/authorino-6784f86975-8tkcv"
Sep 16 02:29:58.083913 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:29:58.083881    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8wxcr\" (UniqueName: \"kubernetes.io/projected/b05ade60-6d3a-493e-a720-923ee22c57cb-kube-api-access-8wxcr\") pod \"authorino-6784f86975-8tkcv\" (UID: \"b05ade60-6d3a-493e-a720-923ee22c57cb\") " pod="kuadrant-system/authorino-6784f86975-8tkcv"
Sep 16 02:29:58.084067 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:29:58.083937    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/b05ade60-6d3a-493e-a720-923ee22c57cb-tls-cert\") pod \"authorino-6784f86975-8tkcv\" (UID: \"b05ade60-6d3a-493e-a720-923ee22c57cb\") " pod="kuadrant-system/authorino-6784f86975-8tkcv"
Sep 16 02:29:58.086618 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:29:58.086600    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/b05ade60-6d3a-493e-a720-923ee22c57cb-tls-cert\") pod \"authorino-6784f86975-8tkcv\" (UID: \"b05ade60-6d3a-493e-a720-923ee22c57cb\") " pod="kuadrant-system/authorino-6784f86975-8tkcv"
Sep 16 02:29:58.091514 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:29:58.091488    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8wxcr\" (UniqueName: \"kubernetes.io/projected/b05ade60-6d3a-493e-a720-923ee22c57cb-kube-api-access-8wxcr\") pod \"authorino-6784f86975-8tkcv\" (UID: \"b05ade60-6d3a-493e-a720-923ee22c57cb\") " pod="kuadrant-system/authorino-6784f86975-8tkcv"
Sep 16 02:29:58.244309 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:29:58.244277    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6784f86975-8tkcv"
Sep 16 02:29:58.513617 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:29:58.513154    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6784f86975-8tkcv"]
Sep 16 02:29:58.517964 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:29:58.517938    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb05ade60_6d3a_493e_a720_923ee22c57cb.slice/crio-427ac4437a496135bf50998f4f075832b697a1954ba638470aa1950909978aa6 WatchSource:0}: Error finding container 427ac4437a496135bf50998f4f075832b697a1954ba638470aa1950909978aa6: Status 404 returned error can't find the container with id 427ac4437a496135bf50998f4f075832b697a1954ba638470aa1950909978aa6
Sep 16 02:29:59.190091 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:29:59.190049    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6784f86975-8tkcv" event={"ID":"b05ade60-6d3a-493e-a720-923ee22c57cb","Type":"ContainerStarted","Data":"09210addd775886a79fccf3cee971fc823f933210c57c0ca1a5790d7e6fc1e77"}
Sep 16 02:29:59.190555 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:29:59.190111    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6784f86975-8tkcv" event={"ID":"b05ade60-6d3a-493e-a720-923ee22c57cb","Type":"ContainerStarted","Data":"427ac4437a496135bf50998f4f075832b697a1954ba638470aa1950909978aa6"}
Sep 16 02:29:59.205422 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:29:59.205348    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-6784f86975-8tkcv" podStartSLOduration=1.6591117020000001 podStartE2EDuration="2.205320077s" totalImagesPullingTime="546.208375ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 02:29:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 02:29:59.205070885 +0000 UTC m=+1019.588586131" watchObservedRunningTime="2026-09-16 02:29:59.205320077 +0000 UTC m=+1019.588835322"
Sep 16 02:29:59.232331 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:29:59.232285    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6fd859b494-xjvtn"]
Sep 16 02:29:59.232588 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:29:59.232545    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-6fd859b494-xjvtn" podUID="f24407e5-008c-48b4-8709-44cd316947da" containerName="authorino" containerID="cri-o://d965ee52f9213f353f7318f8f5b8247b2c443f73e9d92fd116278476032cbd68" gracePeriod=30
Sep 16 02:29:59.497534 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:29:59.497508    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6fd859b494-xjvtn"
Sep 16 02:29:59.584239 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:29:59.584201    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:29:59.594534 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:29:59.594506    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vjlh4\" (UniqueName: \"kubernetes.io/projected/f24407e5-008c-48b4-8709-44cd316947da-kube-api-access-vjlh4\") pod \"f24407e5-008c-48b4-8709-44cd316947da\" (UID: \"f24407e5-008c-48b4-8709-44cd316947da\") "
Sep 16 02:29:59.594694 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:29:59.594585    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f24407e5-008c-48b4-8709-44cd316947da-tls-cert\") pod \"f24407e5-008c-48b4-8709-44cd316947da\" (UID: \"f24407e5-008c-48b4-8709-44cd316947da\") "
Sep 16 02:29:59.596849 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:29:59.596813    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f24407e5-008c-48b4-8709-44cd316947da-kube-api-access-vjlh4" (OuterVolumeSpecName: "kube-api-access-vjlh4") pod "f24407e5-008c-48b4-8709-44cd316947da" (UID: "f24407e5-008c-48b4-8709-44cd316947da"). InnerVolumeSpecName "kube-api-access-vjlh4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 02:29:59.605175 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:29:59.605145    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f24407e5-008c-48b4-8709-44cd316947da-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "f24407e5-008c-48b4-8709-44cd316947da" (UID: "f24407e5-008c-48b4-8709-44cd316947da"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 02:29:59.695359 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:29:59.695323    2579 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f24407e5-008c-48b4-8709-44cd316947da-tls-cert\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:29:59.695359 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:29:59.695361    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vjlh4\" (UniqueName: \"kubernetes.io/projected/f24407e5-008c-48b4-8709-44cd316947da-kube-api-access-vjlh4\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:29:59.787408 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:29:59.787373    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-b878dcb6c-wrqd4"]
Sep 16 02:29:59.787728 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:29:59.787675    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-b878dcb6c-wrqd4" podUID="168a7399-5259-4faf-93c1-2b1e5b9af5f3" containerName="manager" containerID="cri-o://d24414f5231ca1d47600a7acbe72b36c4ed3cff04be64e5cea149f3e3c081ddc" gracePeriod=30
Sep 16 02:29:59.806827 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:29:59.806796    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-qjdcd"]
Sep 16 02:29:59.807017 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:29:59.806978    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-qjdcd" podUID="ba48ec30-ade2-477f-9486-c2fd16d1caae" containerName="payload-pre-processing" containerID="cri-o://4d6b2da1644a31d28f6bfb42d4497df6ebc37866aab178955064ee527fd09b0a" gracePeriod=30
Sep 16 02:29:59.830629 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:29:59.830590    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:30:00.086895 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:30:00.086862    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-qjdcd"
Sep 16 02:30:00.097590 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:30:00.097498    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ba48ec30-ade2-477f-9486-c2fd16d1caae-plugins-config-volume\") pod \"ba48ec30-ade2-477f-9486-c2fd16d1caae\" (UID: \"ba48ec30-ade2-477f-9486-c2fd16d1caae\") "
Sep 16 02:30:00.097590 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:30:00.097565    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5qfhs\" (UniqueName: \"kubernetes.io/projected/ba48ec30-ade2-477f-9486-c2fd16d1caae-kube-api-access-5qfhs\") pod \"ba48ec30-ade2-477f-9486-c2fd16d1caae\" (UID: \"ba48ec30-ade2-477f-9486-c2fd16d1caae\") "
Sep 16 02:30:00.098269 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:30:00.098192    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ba48ec30-ade2-477f-9486-c2fd16d1caae-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ba48ec30-ade2-477f-9486-c2fd16d1caae" (UID: "ba48ec30-ade2-477f-9486-c2fd16d1caae"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 02:30:00.102489 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:30:00.102446    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ba48ec30-ade2-477f-9486-c2fd16d1caae-kube-api-access-5qfhs" (OuterVolumeSpecName: "kube-api-access-5qfhs") pod "ba48ec30-ade2-477f-9486-c2fd16d1caae" (UID: "ba48ec30-ade2-477f-9486-c2fd16d1caae"). InnerVolumeSpecName "kube-api-access-5qfhs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 02:30:00.156254 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:30:00.156221    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-b878dcb6c-wrqd4"
Sep 16 02:30:00.176986 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:30:00.176926    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:30:00.198221 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:30:00.198087    2579 generic.go:356] "Generic (PLEG): container finished" podID="ba48ec30-ade2-477f-9486-c2fd16d1caae" containerID="4d6b2da1644a31d28f6bfb42d4497df6ebc37866aab178955064ee527fd09b0a" exitCode=0
Sep 16 02:30:00.198221 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:30:00.198110    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k7lm7\" (UniqueName: \"kubernetes.io/projected/168a7399-5259-4faf-93c1-2b1e5b9af5f3-kube-api-access-k7lm7\") pod \"168a7399-5259-4faf-93c1-2b1e5b9af5f3\" (UID: \"168a7399-5259-4faf-93c1-2b1e5b9af5f3\") "
Sep 16 02:30:00.198221 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:30:00.198161    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/168a7399-5259-4faf-93c1-2b1e5b9af5f3-metrics-certs\") pod \"168a7399-5259-4faf-93c1-2b1e5b9af5f3\" (UID: \"168a7399-5259-4faf-93c1-2b1e5b9af5f3\") "
Sep 16 02:30:00.198221 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:30:00.198169    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-qjdcd"
Sep 16 02:30:00.198221 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:30:00.198176    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-qjdcd" event={"ID":"ba48ec30-ade2-477f-9486-c2fd16d1caae","Type":"ContainerDied","Data":"4d6b2da1644a31d28f6bfb42d4497df6ebc37866aab178955064ee527fd09b0a"}
Sep 16 02:30:00.198905 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:30:00.198241    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-qjdcd" event={"ID":"ba48ec30-ade2-477f-9486-c2fd16d1caae","Type":"ContainerDied","Data":"27d597e0c1e7c04e2e437beac929601c0f691dd94d8b367ec89f08d12fc33fac"}
Sep 16 02:30:00.198905 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:30:00.198272    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/168a7399-5259-4faf-93c1-2b1e5b9af5f3-webhook-cert\") pod \"168a7399-5259-4faf-93c1-2b1e5b9af5f3\" (UID: \"168a7399-5259-4faf-93c1-2b1e5b9af5f3\") "
Sep 16 02:30:00.198905 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:30:00.198504    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5qfhs\" (UniqueName: \"kubernetes.io/projected/ba48ec30-ade2-477f-9486-c2fd16d1caae-kube-api-access-5qfhs\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:30:00.198905 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:30:00.198523    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ba48ec30-ade2-477f-9486-c2fd16d1caae-plugins-config-volume\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:30:00.198905 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:30:00.198274    2579 scope.go:117] "RemoveContainer" containerID="4d6b2da1644a31d28f6bfb42d4497df6ebc37866aab178955064ee527fd09b0a"
Sep 16 02:30:00.202160 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:30:00.200975    2579 generic.go:356] "Generic (PLEG): container finished" podID="f24407e5-008c-48b4-8709-44cd316947da" containerID="d965ee52f9213f353f7318f8f5b8247b2c443f73e9d92fd116278476032cbd68" exitCode=0
Sep 16 02:30:00.202160 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:30:00.201080    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6fd859b494-xjvtn" event={"ID":"f24407e5-008c-48b4-8709-44cd316947da","Type":"ContainerDied","Data":"d965ee52f9213f353f7318f8f5b8247b2c443f73e9d92fd116278476032cbd68"}
Sep 16 02:30:00.202160 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:30:00.201109    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6fd859b494-xjvtn" event={"ID":"f24407e5-008c-48b4-8709-44cd316947da","Type":"ContainerDied","Data":"7d5fe44073d7a728172cc08525dbc5d0f5257ed379c27e66e1edf2f0505559e7"}
Sep 16 02:30:00.202160 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:30:00.201188    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6fd859b494-xjvtn"
Sep 16 02:30:00.202160 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:30:00.201690    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/168a7399-5259-4faf-93c1-2b1e5b9af5f3-kube-api-access-k7lm7" (OuterVolumeSpecName: "kube-api-access-k7lm7") pod "168a7399-5259-4faf-93c1-2b1e5b9af5f3" (UID: "168a7399-5259-4faf-93c1-2b1e5b9af5f3"). InnerVolumeSpecName "kube-api-access-k7lm7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 02:30:00.202160 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:30:00.202091    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/168a7399-5259-4faf-93c1-2b1e5b9af5f3-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "168a7399-5259-4faf-93c1-2b1e5b9af5f3" (UID: "168a7399-5259-4faf-93c1-2b1e5b9af5f3"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 02:30:00.203941 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:30:00.203875    2579 generic.go:356] "Generic (PLEG): container finished" podID="168a7399-5259-4faf-93c1-2b1e5b9af5f3" containerID="d24414f5231ca1d47600a7acbe72b36c4ed3cff04be64e5cea149f3e3c081ddc" exitCode=0
Sep 16 02:30:00.204290 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:30:00.204266    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/168a7399-5259-4faf-93c1-2b1e5b9af5f3-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "168a7399-5259-4faf-93c1-2b1e5b9af5f3" (UID: "168a7399-5259-4faf-93c1-2b1e5b9af5f3"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 02:30:00.204397 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:30:00.204354    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-b878dcb6c-wrqd4"
Sep 16 02:30:00.204397 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:30:00.204383    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-b878dcb6c-wrqd4" event={"ID":"168a7399-5259-4faf-93c1-2b1e5b9af5f3","Type":"ContainerDied","Data":"d24414f5231ca1d47600a7acbe72b36c4ed3cff04be64e5cea149f3e3c081ddc"}
Sep 16 02:30:00.204570 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:30:00.204426    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-b878dcb6c-wrqd4" event={"ID":"168a7399-5259-4faf-93c1-2b1e5b9af5f3","Type":"ContainerDied","Data":"fa08c29a6edc3f3cd49dfd6cb656363ceabc88473f41dd33a88cba14572441fb"}
Sep 16 02:30:00.223400 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:30:00.223377    2579 scope.go:117] "RemoveContainer" containerID="4d6b2da1644a31d28f6bfb42d4497df6ebc37866aab178955064ee527fd09b0a"
Sep 16 02:30:00.223797 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:30:00.223776    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4d6b2da1644a31d28f6bfb42d4497df6ebc37866aab178955064ee527fd09b0a\": container with ID starting with 4d6b2da1644a31d28f6bfb42d4497df6ebc37866aab178955064ee527fd09b0a not found: ID does not exist" containerID="4d6b2da1644a31d28f6bfb42d4497df6ebc37866aab178955064ee527fd09b0a"
Sep 16 02:30:00.223903 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:30:00.223804    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4d6b2da1644a31d28f6bfb42d4497df6ebc37866aab178955064ee527fd09b0a"} err="failed to get container status \"4d6b2da1644a31d28f6bfb42d4497df6ebc37866aab178955064ee527fd09b0a\": rpc error: code = NotFound desc = could not find container \"4d6b2da1644a31d28f6bfb42d4497df6ebc37866aab178955064ee527fd09b0a\": container with ID starting with 4d6b2da1644a31d28f6bfb42d4497df6ebc37866aab178955064ee527fd09b0a not found: ID does not exist"
Sep 16 02:30:00.223903 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:30:00.223819    2579 scope.go:117] "RemoveContainer" containerID="d965ee52f9213f353f7318f8f5b8247b2c443f73e9d92fd116278476032cbd68"
Sep 16 02:30:00.233439 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:30:00.233416    2579 scope.go:117] "RemoveContainer" containerID="d965ee52f9213f353f7318f8f5b8247b2c443f73e9d92fd116278476032cbd68"
Sep 16 02:30:00.233702 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:30:00.233681    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-qjdcd"]
Sep 16 02:30:00.233771 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:30:00.233753    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d965ee52f9213f353f7318f8f5b8247b2c443f73e9d92fd116278476032cbd68\": container with ID starting with d965ee52f9213f353f7318f8f5b8247b2c443f73e9d92fd116278476032cbd68 not found: ID does not exist" containerID="d965ee52f9213f353f7318f8f5b8247b2c443f73e9d92fd116278476032cbd68"
Sep 16 02:30:00.233832 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:30:00.233773    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d965ee52f9213f353f7318f8f5b8247b2c443f73e9d92fd116278476032cbd68"} err="failed to get container status \"d965ee52f9213f353f7318f8f5b8247b2c443f73e9d92fd116278476032cbd68\": rpc error: code = NotFound desc = could not find container \"d965ee52f9213f353f7318f8f5b8247b2c443f73e9d92fd116278476032cbd68\": container with ID starting with d965ee52f9213f353f7318f8f5b8247b2c443f73e9d92fd116278476032cbd68 not found: ID does not exist"
Sep 16 02:30:00.233832 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:30:00.233789    2579 scope.go:117] "RemoveContainer" containerID="d24414f5231ca1d47600a7acbe72b36c4ed3cff04be64e5cea149f3e3c081ddc"
Sep 16 02:30:00.239734 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:30:00.239695    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-qjdcd"]
Sep 16 02:30:00.243048 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:30:00.243030    2579 scope.go:117] "RemoveContainer" containerID="d24414f5231ca1d47600a7acbe72b36c4ed3cff04be64e5cea149f3e3c081ddc"
Sep 16 02:30:00.243322 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:30:00.243301    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d24414f5231ca1d47600a7acbe72b36c4ed3cff04be64e5cea149f3e3c081ddc\": container with ID starting with d24414f5231ca1d47600a7acbe72b36c4ed3cff04be64e5cea149f3e3c081ddc not found: ID does not exist" containerID="d24414f5231ca1d47600a7acbe72b36c4ed3cff04be64e5cea149f3e3c081ddc"
Sep 16 02:30:00.243413 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:30:00.243324    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d24414f5231ca1d47600a7acbe72b36c4ed3cff04be64e5cea149f3e3c081ddc"} err="failed to get container status \"d24414f5231ca1d47600a7acbe72b36c4ed3cff04be64e5cea149f3e3c081ddc\": rpc error: code = NotFound desc = could not find container \"d24414f5231ca1d47600a7acbe72b36c4ed3cff04be64e5cea149f3e3c081ddc\": container with ID starting with d24414f5231ca1d47600a7acbe72b36c4ed3cff04be64e5cea149f3e3c081ddc not found: ID does not exist"
Sep 16 02:30:00.248635 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:30:00.248612    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6fd859b494-xjvtn"]
Sep 16 02:30:00.252551 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:30:00.252532    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6fd859b494-xjvtn"]
Sep 16 02:30:00.262717 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:30:00.262690    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-b878dcb6c-wrqd4"]
Sep 16 02:30:00.266245 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:30:00.266221    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-b878dcb6c-wrqd4"]
Sep 16 02:30:00.299547 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:30:00.299520    2579 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/168a7399-5259-4faf-93c1-2b1e5b9af5f3-webhook-cert\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:30:00.299547 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:30:00.299541    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k7lm7\" (UniqueName: \"kubernetes.io/projected/168a7399-5259-4faf-93c1-2b1e5b9af5f3-kube-api-access-k7lm7\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:30:00.299547 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:30:00.299551    2579 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/168a7399-5259-4faf-93c1-2b1e5b9af5f3-metrics-certs\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:30:00.835963 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:30:00.835925    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-b878dcb6c-6kg9c"]
Sep 16 02:30:00.836256 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:30:00.836244    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ba48ec30-ade2-477f-9486-c2fd16d1caae" containerName="payload-pre-processing"
Sep 16 02:30:00.836295 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:30:00.836257    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="ba48ec30-ade2-477f-9486-c2fd16d1caae" containerName="payload-pre-processing"
Sep 16 02:30:00.836295 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:30:00.836272    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f24407e5-008c-48b4-8709-44cd316947da" containerName="authorino"
Sep 16 02:30:00.836295 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:30:00.836277    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="f24407e5-008c-48b4-8709-44cd316947da" containerName="authorino"
Sep 16 02:30:00.836295 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:30:00.836284    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="168a7399-5259-4faf-93c1-2b1e5b9af5f3" containerName="manager"
Sep 16 02:30:00.836295 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:30:00.836289    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="168a7399-5259-4faf-93c1-2b1e5b9af5f3" containerName="manager"
Sep 16 02:30:00.836495 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:30:00.836353    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="f24407e5-008c-48b4-8709-44cd316947da" containerName="authorino"
Sep 16 02:30:00.836495 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:30:00.836363    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="168a7399-5259-4faf-93c1-2b1e5b9af5f3" containerName="manager"
Sep 16 02:30:00.836495 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:30:00.836371    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="ba48ec30-ade2-477f-9486-c2fd16d1caae" containerName="payload-pre-processing"
Sep 16 02:30:00.840935 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:30:00.840908    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-b878dcb6c-6kg9c"
Sep 16 02:30:00.843830 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:30:00.843803    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 16 02:30:00.843956 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:30:00.843839    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-cs9kk\""
Sep 16 02:30:00.843956 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:30:00.843852    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 16 02:30:00.843956 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:30:00.843874    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 16 02:30:00.848180 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:30:00.848119    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-b878dcb6c-6kg9c"]
Sep 16 02:30:00.904971 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:30:00.904938    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8e36faf3-4f71-4f46-8928-45795654eea8-webhook-cert\") pod \"maas-controller-b878dcb6c-6kg9c\" (UID: \"8e36faf3-4f71-4f46-8928-45795654eea8\") " pod="opendatahub/maas-controller-b878dcb6c-6kg9c"
Sep 16 02:30:00.904971 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:30:00.904981    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8e36faf3-4f71-4f46-8928-45795654eea8-metrics-certs\") pod \"maas-controller-b878dcb6c-6kg9c\" (UID: \"8e36faf3-4f71-4f46-8928-45795654eea8\") " pod="opendatahub/maas-controller-b878dcb6c-6kg9c"
Sep 16 02:30:00.905184 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:30:00.905076    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bzlg5\" (UniqueName: \"kubernetes.io/projected/8e36faf3-4f71-4f46-8928-45795654eea8-kube-api-access-bzlg5\") pod \"maas-controller-b878dcb6c-6kg9c\" (UID: \"8e36faf3-4f71-4f46-8928-45795654eea8\") " pod="opendatahub/maas-controller-b878dcb6c-6kg9c"
Sep 16 02:30:01.006114 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:30:01.006064    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8e36faf3-4f71-4f46-8928-45795654eea8-metrics-certs\") pod \"maas-controller-b878dcb6c-6kg9c\" (UID: \"8e36faf3-4f71-4f46-8928-45795654eea8\") " pod="opendatahub/maas-controller-b878dcb6c-6kg9c"
Sep 16 02:30:01.006321 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:30:01.006176    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bzlg5\" (UniqueName: \"kubernetes.io/projected/8e36faf3-4f71-4f46-8928-45795654eea8-kube-api-access-bzlg5\") pod \"maas-controller-b878dcb6c-6kg9c\" (UID: \"8e36faf3-4f71-4f46-8928-45795654eea8\") " pod="opendatahub/maas-controller-b878dcb6c-6kg9c"
Sep 16 02:30:01.006321 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:30:01.006208    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8e36faf3-4f71-4f46-8928-45795654eea8-webhook-cert\") pod \"maas-controller-b878dcb6c-6kg9c\" (UID: \"8e36faf3-4f71-4f46-8928-45795654eea8\") " pod="opendatahub/maas-controller-b878dcb6c-6kg9c"
Sep 16 02:30:01.008755 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:30:01.008723    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8e36faf3-4f71-4f46-8928-45795654eea8-webhook-cert\") pod \"maas-controller-b878dcb6c-6kg9c\" (UID: \"8e36faf3-4f71-4f46-8928-45795654eea8\") " pod="opendatahub/maas-controller-b878dcb6c-6kg9c"
Sep 16 02:30:01.008966 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:30:01.008945    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8e36faf3-4f71-4f46-8928-45795654eea8-metrics-certs\") pod \"maas-controller-b878dcb6c-6kg9c\" (UID: \"8e36faf3-4f71-4f46-8928-45795654eea8\") " pod="opendatahub/maas-controller-b878dcb6c-6kg9c"
Sep 16 02:30:01.013936 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:30:01.013911    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bzlg5\" (UniqueName: \"kubernetes.io/projected/8e36faf3-4f71-4f46-8928-45795654eea8-kube-api-access-bzlg5\") pod \"maas-controller-b878dcb6c-6kg9c\" (UID: \"8e36faf3-4f71-4f46-8928-45795654eea8\") " pod="opendatahub/maas-controller-b878dcb6c-6kg9c"
Sep 16 02:30:01.153420 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:30:01.153340    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-b878dcb6c-6kg9c"
Sep 16 02:30:01.427897 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:30:01.427873    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-b878dcb6c-6kg9c"]
Sep 16 02:30:01.430316 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:30:01.430289    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e36faf3_4f71_4f46_8928_45795654eea8.slice/crio-e2b1defc93cb0a6520f2148ae924ef032ad6965ddf9a1e3a5978d39f49d72944 WatchSource:0}: Error finding container e2b1defc93cb0a6520f2148ae924ef032ad6965ddf9a1e3a5978d39f49d72944: Status 404 returned error can't find the container with id e2b1defc93cb0a6520f2148ae924ef032ad6965ddf9a1e3a5978d39f49d72944
Sep 16 02:30:02.105541 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:30:02.105512    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="168a7399-5259-4faf-93c1-2b1e5b9af5f3" path="/var/lib/kubelet/pods/168a7399-5259-4faf-93c1-2b1e5b9af5f3/volumes"
Sep 16 02:30:02.105895 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:30:02.105873    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ba48ec30-ade2-477f-9486-c2fd16d1caae" path="/var/lib/kubelet/pods/ba48ec30-ade2-477f-9486-c2fd16d1caae/volumes"
Sep 16 02:30:02.106182 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:30:02.106171    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f24407e5-008c-48b4-8709-44cd316947da" path="/var/lib/kubelet/pods/f24407e5-008c-48b4-8709-44cd316947da/volumes"
Sep 16 02:30:02.129849 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:30:02.129822    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:30:02.222436 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:30:02.222402    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-b878dcb6c-6kg9c" event={"ID":"8e36faf3-4f71-4f46-8928-45795654eea8","Type":"ContainerStarted","Data":"e2b1defc93cb0a6520f2148ae924ef032ad6965ddf9a1e3a5978d39f49d72944"}
Sep 16 02:30:03.228753 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:30:03.228720    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-b878dcb6c-6kg9c" event={"ID":"8e36faf3-4f71-4f46-8928-45795654eea8","Type":"ContainerStarted","Data":"c53b9cd4475773d7cb95f4d0c9f8caa3c387555c88e24a769c7bd86557b0709f"}
Sep 16 02:30:03.229155 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:30:03.228981    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-b878dcb6c-6kg9c"
Sep 16 02:30:03.245906 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:30:03.245856    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-b878dcb6c-6kg9c" podStartSLOduration=2.128729501 podStartE2EDuration="3.245841212s" totalImagesPullingTime="1.117111711s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 02:30:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 02:30:03.244572026 +0000 UTC m=+1023.628087274" watchObservedRunningTime="2026-09-16 02:30:03.245841212 +0000 UTC m=+1023.629356457"
Sep 16 02:30:14.235283 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:30:14.235241    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-b878dcb6c-6kg9c"
Sep 16 02:30:17.824977 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:30:17.824933    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:30:17.828572 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:30:17.828545    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-6cd864bc95-mh4jh"]
Sep 16 02:30:17.832124 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:30:17.832105    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-6cd864bc95-mh4jh"
Sep 16 02:30:17.834275 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:30:17.834258    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-zzx6j\""
Sep 16 02:30:17.834386 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:30:17.834337    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Sep 16 02:30:17.840222 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:30:17.840200    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-6cd864bc95-mh4jh"]
Sep 16 02:30:17.920785 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:30:17.920747    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:30:17.953650 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:30:17.953622    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8c3aa767-5631-4c03-bbe4-9cb9fa96b7f8-plugins-config-volume\") pod \"payload-pre-processing-6cd864bc95-mh4jh\" (UID: \"8c3aa767-5631-4c03-bbe4-9cb9fa96b7f8\") " pod="openshift-ingress/payload-pre-processing-6cd864bc95-mh4jh"
Sep 16 02:30:17.953803 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:30:17.953670    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zzw2h\" (UniqueName: \"kubernetes.io/projected/8c3aa767-5631-4c03-bbe4-9cb9fa96b7f8-kube-api-access-zzw2h\") pod \"payload-pre-processing-6cd864bc95-mh4jh\" (UID: \"8c3aa767-5631-4c03-bbe4-9cb9fa96b7f8\") " pod="openshift-ingress/payload-pre-processing-6cd864bc95-mh4jh"
Sep 16 02:30:18.055063 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:30:18.055026    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8c3aa767-5631-4c03-bbe4-9cb9fa96b7f8-plugins-config-volume\") pod \"payload-pre-processing-6cd864bc95-mh4jh\" (UID: \"8c3aa767-5631-4c03-bbe4-9cb9fa96b7f8\") " pod="openshift-ingress/payload-pre-processing-6cd864bc95-mh4jh"
Sep 16 02:30:18.055238 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:30:18.055076    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zzw2h\" (UniqueName: \"kubernetes.io/projected/8c3aa767-5631-4c03-bbe4-9cb9fa96b7f8-kube-api-access-zzw2h\") pod \"payload-pre-processing-6cd864bc95-mh4jh\" (UID: \"8c3aa767-5631-4c03-bbe4-9cb9fa96b7f8\") " pod="openshift-ingress/payload-pre-processing-6cd864bc95-mh4jh"
Sep 16 02:30:18.055955 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:30:18.055934    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8c3aa767-5631-4c03-bbe4-9cb9fa96b7f8-plugins-config-volume\") pod \"payload-pre-processing-6cd864bc95-mh4jh\" (UID: \"8c3aa767-5631-4c03-bbe4-9cb9fa96b7f8\") " pod="openshift-ingress/payload-pre-processing-6cd864bc95-mh4jh"
Sep 16 02:30:18.062796 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:30:18.062777    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zzw2h\" (UniqueName: \"kubernetes.io/projected/8c3aa767-5631-4c03-bbe4-9cb9fa96b7f8-kube-api-access-zzw2h\") pod \"payload-pre-processing-6cd864bc95-mh4jh\" (UID: \"8c3aa767-5631-4c03-bbe4-9cb9fa96b7f8\") " pod="openshift-ingress/payload-pre-processing-6cd864bc95-mh4jh"
Sep 16 02:30:18.143727 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:30:18.143666    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-6cd864bc95-mh4jh"
Sep 16 02:30:18.414373 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:30:18.414344    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-6cd864bc95-mh4jh"]
Sep 16 02:30:18.415878 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:30:18.415849    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c3aa767_5631_4c03_bbe4_9cb9fa96b7f8.slice/crio-b7f253bb56d5ad90d4a3fdb4265a711168defe7ac523b4a9f8dd409e9c77cd85 WatchSource:0}: Error finding container b7f253bb56d5ad90d4a3fdb4265a711168defe7ac523b4a9f8dd409e9c77cd85: Status 404 returned error can't find the container with id b7f253bb56d5ad90d4a3fdb4265a711168defe7ac523b4a9f8dd409e9c77cd85
Sep 16 02:30:19.304160 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:30:19.304124    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-6cd864bc95-mh4jh" event={"ID":"8c3aa767-5631-4c03-bbe4-9cb9fa96b7f8","Type":"ContainerStarted","Data":"b7f253bb56d5ad90d4a3fdb4265a711168defe7ac523b4a9f8dd409e9c77cd85"}
Sep 16 02:30:20.309840 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:30:20.309804    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-6cd864bc95-mh4jh" event={"ID":"8c3aa767-5631-4c03-bbe4-9cb9fa96b7f8","Type":"ContainerStarted","Data":"a11c600d2440b2808316b07144240f22dc6ded64504a954eb8aee457193f2626"}
Sep 16 02:30:20.310244 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:30:20.309899    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-6cd864bc95-mh4jh"
Sep 16 02:30:20.325896 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:30:20.325842    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-6cd864bc95-mh4jh" podStartSLOduration=2.396083136 podStartE2EDuration="3.32582839s" totalImagesPullingTime="929.745254ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 02:30:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 02:30:20.324492786 +0000 UTC m=+1040.708008027" watchObservedRunningTime="2026-09-16 02:30:20.32582839 +0000 UTC m=+1040.709343635"
Sep 16 02:30:31.317387 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:30:31.317349    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-6cd864bc95-mh4jh"
Sep 16 02:30:31.317869 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:30:31.317776    2579 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-pre-processing-6cd864bc95-mh4jh" secret="" err="secret \"payload-processing-dockercfg-zzx6j\" not found"
Sep 16 02:30:43.039396 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:30:43.039352    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:30:43.407775 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:30:43.407684    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:30:44.485449 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:30:44.485415    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:30:44.596124 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:30:44.596084    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:30:45.599613 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:30:45.599579    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:30:45.701833 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:30:45.701802    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:31:01.290633 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:01.290594    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:31:01.391499 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:01.391443    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:31:01.591141 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:01.591050    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:31:01.684806 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:01.684769    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:31:01.752705 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:01.752665    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:31:01.850371 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:01.850288    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:31:02.066783 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:02.066736    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:31:04.930529 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:04.930497    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:31:05.034286 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:05.034248    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:31:05.141995 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:05.141948    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:31:05.278391 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:05.278336    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:31:05.378429 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:05.378398    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:31:06.249791 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:06.249753    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:31:06.349563 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:06.349524    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:31:06.607293 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:06.607191    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:31:06.724098 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:06.724063    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:31:06.821929 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:06.821899    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:31:06.838838 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:06.838811    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-60674e-69589755b6-wpf7z"]
Sep 16 02:31:06.844838 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:06.844817    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-60674e-69589755b6-wpf7z"
Sep 16 02:31:06.847628 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:06.847603    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-60674e\""
Sep 16 02:31:06.848114 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:06.847925    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-60674e-dockercfg-fsb5f\""
Sep 16 02:31:06.852774 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:06.852750    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-60674e-69589755b6-wpf7z"]
Sep 16 02:31:06.855538 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:06.855519    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-60674e-f95445b5c-x5gpp"]
Sep 16 02:31:06.858987 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:06.858939    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-60674e-f95445b5c-x5gpp"
Sep 16 02:31:06.869356 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:06.869336    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-60674e-f95445b5c-x5gpp"]
Sep 16 02:31:06.945831 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:06.945800    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cbq4b\" (UniqueName: \"kubernetes.io/projected/7e347e28-a5be-4179-8f6e-9e8107b979a2-kube-api-access-cbq4b\") pod \"payload-pre-processing-e2e-inf-a-60674e-69589755b6-wpf7z\" (UID: \"7e347e28-a5be-4179-8f6e-9e8107b979a2\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-60674e-69589755b6-wpf7z"
Sep 16 02:31:06.945993 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:06.945857    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7e347e28-a5be-4179-8f6e-9e8107b979a2-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-60674e-69589755b6-wpf7z\" (UID: \"7e347e28-a5be-4179-8f6e-9e8107b979a2\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-60674e-69589755b6-wpf7z"
Sep 16 02:31:06.945993 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:06.945881    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/34acbdfe-3681-424d-b1f1-ecb47fde6a7d-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-60674e-f95445b5c-x5gpp\" (UID: \"34acbdfe-3681-424d-b1f1-ecb47fde6a7d\") " pod="openshift-ingress/payload-processing-e2e-inf-a-60674e-f95445b5c-x5gpp"
Sep 16 02:31:06.945993 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:06.945906    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c9ttp\" (UniqueName: \"kubernetes.io/projected/34acbdfe-3681-424d-b1f1-ecb47fde6a7d-kube-api-access-c9ttp\") pod \"payload-processing-e2e-inf-a-60674e-f95445b5c-x5gpp\" (UID: \"34acbdfe-3681-424d-b1f1-ecb47fde6a7d\") " pod="openshift-ingress/payload-processing-e2e-inf-a-60674e-f95445b5c-x5gpp"
Sep 16 02:31:07.046626 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:07.046582    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7e347e28-a5be-4179-8f6e-9e8107b979a2-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-60674e-69589755b6-wpf7z\" (UID: \"7e347e28-a5be-4179-8f6e-9e8107b979a2\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-60674e-69589755b6-wpf7z"
Sep 16 02:31:07.046836 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:07.046643    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/34acbdfe-3681-424d-b1f1-ecb47fde6a7d-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-60674e-f95445b5c-x5gpp\" (UID: \"34acbdfe-3681-424d-b1f1-ecb47fde6a7d\") " pod="openshift-ingress/payload-processing-e2e-inf-a-60674e-f95445b5c-x5gpp"
Sep 16 02:31:07.046836 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:07.046671    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c9ttp\" (UniqueName: \"kubernetes.io/projected/34acbdfe-3681-424d-b1f1-ecb47fde6a7d-kube-api-access-c9ttp\") pod \"payload-processing-e2e-inf-a-60674e-f95445b5c-x5gpp\" (UID: \"34acbdfe-3681-424d-b1f1-ecb47fde6a7d\") " pod="openshift-ingress/payload-processing-e2e-inf-a-60674e-f95445b5c-x5gpp"
Sep 16 02:31:07.046836 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:07.046765    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cbq4b\" (UniqueName: \"kubernetes.io/projected/7e347e28-a5be-4179-8f6e-9e8107b979a2-kube-api-access-cbq4b\") pod \"payload-pre-processing-e2e-inf-a-60674e-69589755b6-wpf7z\" (UID: \"7e347e28-a5be-4179-8f6e-9e8107b979a2\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-60674e-69589755b6-wpf7z"
Sep 16 02:31:07.047554 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:07.047529    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7e347e28-a5be-4179-8f6e-9e8107b979a2-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-60674e-69589755b6-wpf7z\" (UID: \"7e347e28-a5be-4179-8f6e-9e8107b979a2\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-60674e-69589755b6-wpf7z"
Sep 16 02:31:07.047748 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:07.047730    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/34acbdfe-3681-424d-b1f1-ecb47fde6a7d-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-60674e-f95445b5c-x5gpp\" (UID: \"34acbdfe-3681-424d-b1f1-ecb47fde6a7d\") " pod="openshift-ingress/payload-processing-e2e-inf-a-60674e-f95445b5c-x5gpp"
Sep 16 02:31:07.055073 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:07.055049    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cbq4b\" (UniqueName: \"kubernetes.io/projected/7e347e28-a5be-4179-8f6e-9e8107b979a2-kube-api-access-cbq4b\") pod \"payload-pre-processing-e2e-inf-a-60674e-69589755b6-wpf7z\" (UID: \"7e347e28-a5be-4179-8f6e-9e8107b979a2\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-60674e-69589755b6-wpf7z"
Sep 16 02:31:07.055189 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:07.055102    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c9ttp\" (UniqueName: \"kubernetes.io/projected/34acbdfe-3681-424d-b1f1-ecb47fde6a7d-kube-api-access-c9ttp\") pod \"payload-processing-e2e-inf-a-60674e-f95445b5c-x5gpp\" (UID: \"34acbdfe-3681-424d-b1f1-ecb47fde6a7d\") " pod="openshift-ingress/payload-processing-e2e-inf-a-60674e-f95445b5c-x5gpp"
Sep 16 02:31:07.139014 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:07.138928    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:31:07.158850 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:07.158822    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-60674e-69589755b6-wpf7z"
Sep 16 02:31:07.169991 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:07.169967    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-60674e-f95445b5c-x5gpp"
Sep 16 02:31:07.242795 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:07.242733    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:31:07.432289 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:07.432266    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-60674e-69589755b6-wpf7z"]
Sep 16 02:31:07.434203 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:31:07.434178    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e347e28_a5be_4179_8f6e_9e8107b979a2.slice/crio-794b5c550af327573728d73e34ae99f5de56f2f6dba359eb21189b05ba47800b WatchSource:0}: Error finding container 794b5c550af327573728d73e34ae99f5de56f2f6dba359eb21189b05ba47800b: Status 404 returned error can't find the container with id 794b5c550af327573728d73e34ae99f5de56f2f6dba359eb21189b05ba47800b
Sep 16 02:31:07.462231 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:07.462207    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-60674e-f95445b5c-x5gpp"]
Sep 16 02:31:07.464531 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:31:07.464506    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34acbdfe_3681_424d_b1f1_ecb47fde6a7d.slice/crio-a5e3866f8594c97e3df5b590a15184116c815ab61aadd3aac8de177425a35b3d WatchSource:0}: Error finding container a5e3866f8594c97e3df5b590a15184116c815ab61aadd3aac8de177425a35b3d: Status 404 returned error can't find the container with id a5e3866f8594c97e3df5b590a15184116c815ab61aadd3aac8de177425a35b3d
Sep 16 02:31:07.531973 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:07.531945    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-60674e-f95445b5c-x5gpp" event={"ID":"34acbdfe-3681-424d-b1f1-ecb47fde6a7d","Type":"ContainerStarted","Data":"77ebcd2a77c28de1318d306c4c946275ede245c102b7ce5d0ebb0b9f433267c9"}
Sep 16 02:31:07.532302 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:07.531999    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-60674e-f95445b5c-x5gpp" event={"ID":"34acbdfe-3681-424d-b1f1-ecb47fde6a7d","Type":"ContainerStarted","Data":"a5e3866f8594c97e3df5b590a15184116c815ab61aadd3aac8de177425a35b3d"}
Sep 16 02:31:07.532302 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:07.532044    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-60674e-f95445b5c-x5gpp"
Sep 16 02:31:07.533574 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:07.533555    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-60674e-69589755b6-wpf7z" event={"ID":"7e347e28-a5be-4179-8f6e-9e8107b979a2","Type":"ContainerStarted","Data":"57f4b7db4002e499803dfe1f22543619457d17b19c6412333ac30679fa0b277e"}
Sep 16 02:31:07.533669 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:07.533592    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-60674e-69589755b6-wpf7z" event={"ID":"7e347e28-a5be-4179-8f6e-9e8107b979a2","Type":"ContainerStarted","Data":"794b5c550af327573728d73e34ae99f5de56f2f6dba359eb21189b05ba47800b"}
Sep 16 02:31:07.533669 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:07.533658    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-60674e-69589755b6-wpf7z"
Sep 16 02:31:07.549069 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:07.549010    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-60674e-f95445b5c-x5gpp" podStartSLOduration=1.5489911090000001 podStartE2EDuration="1.548991109s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 02:31:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 02:31:07.547638527 +0000 UTC m=+1087.931153773" watchObservedRunningTime="2026-09-16 02:31:07.548991109 +0000 UTC m=+1087.932506357"
Sep 16 02:31:07.566366 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:07.566308    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-60674e-69589755b6-wpf7z" podStartSLOduration=1.566285243 podStartE2EDuration="1.566285243s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 02:31:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 02:31:07.563258195 +0000 UTC m=+1087.946773444" watchObservedRunningTime="2026-09-16 02:31:07.566285243 +0000 UTC m=+1087.949800505"
Sep 16 02:31:07.589811 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:07.589778    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:31:07.695326 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:07.695295    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:31:07.787893 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:07.787864    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:31:07.791808 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:07.791754    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-619550-597b7898-4xksl"]
Sep 16 02:31:07.795345 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:07.795324    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-619550-597b7898-4xksl"
Sep 16 02:31:07.797658 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:07.797639    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w0-619550-dockercfg-wnxkr\""
Sep 16 02:31:07.797764 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:07.797676    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w0-619550\""
Sep 16 02:31:07.805398 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:07.805376    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-619550-597b7898-4xksl"]
Sep 16 02:31:07.834446 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:07.834422    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-619550-5984cc5ccf-58g44"]
Sep 16 02:31:07.837946 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:07.837928    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-619550-5984cc5ccf-58g44"
Sep 16 02:31:07.847652 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:07.847630    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-619550-5984cc5ccf-58g44"]
Sep 16 02:31:07.855418 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:07.855396    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t848s\" (UniqueName: \"kubernetes.io/projected/f84078e6-c7af-4767-8fd4-c68bb7d27bfa-kube-api-access-t848s\") pod \"payload-processing-e2e-worker-w0-619550-5984cc5ccf-58g44\" (UID: \"f84078e6-c7af-4767-8fd4-c68bb7d27bfa\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-619550-5984cc5ccf-58g44"
Sep 16 02:31:07.855566 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:07.855486    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/452d6b9b-47b4-458f-a332-9c2ed48c3266-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-619550-597b7898-4xksl\" (UID: \"452d6b9b-47b4-458f-a332-9c2ed48c3266\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-619550-597b7898-4xksl"
Sep 16 02:31:07.855566 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:07.855533    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z9s2t\" (UniqueName: \"kubernetes.io/projected/452d6b9b-47b4-458f-a332-9c2ed48c3266-kube-api-access-z9s2t\") pod \"payload-pre-processing-e2e-worker-w0-619550-597b7898-4xksl\" (UID: \"452d6b9b-47b4-458f-a332-9c2ed48c3266\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-619550-597b7898-4xksl"
Sep 16 02:31:07.855699 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:07.855591    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f84078e6-c7af-4767-8fd4-c68bb7d27bfa-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-619550-5984cc5ccf-58g44\" (UID: \"f84078e6-c7af-4767-8fd4-c68bb7d27bfa\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-619550-5984cc5ccf-58g44"
Sep 16 02:31:07.956448 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:07.956357    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z9s2t\" (UniqueName: \"kubernetes.io/projected/452d6b9b-47b4-458f-a332-9c2ed48c3266-kube-api-access-z9s2t\") pod \"payload-pre-processing-e2e-worker-w0-619550-597b7898-4xksl\" (UID: \"452d6b9b-47b4-458f-a332-9c2ed48c3266\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-619550-597b7898-4xksl"
Sep 16 02:31:07.956448 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:07.956409    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f84078e6-c7af-4767-8fd4-c68bb7d27bfa-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-619550-5984cc5ccf-58g44\" (UID: \"f84078e6-c7af-4767-8fd4-c68bb7d27bfa\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-619550-5984cc5ccf-58g44"
Sep 16 02:31:07.956448 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:07.956442    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t848s\" (UniqueName: \"kubernetes.io/projected/f84078e6-c7af-4767-8fd4-c68bb7d27bfa-kube-api-access-t848s\") pod \"payload-processing-e2e-worker-w0-619550-5984cc5ccf-58g44\" (UID: \"f84078e6-c7af-4767-8fd4-c68bb7d27bfa\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-619550-5984cc5ccf-58g44"
Sep 16 02:31:07.956764 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:07.956525    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/452d6b9b-47b4-458f-a332-9c2ed48c3266-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-619550-597b7898-4xksl\" (UID: \"452d6b9b-47b4-458f-a332-9c2ed48c3266\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-619550-597b7898-4xksl"
Sep 16 02:31:07.957247 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:07.957227    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/452d6b9b-47b4-458f-a332-9c2ed48c3266-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-619550-597b7898-4xksl\" (UID: \"452d6b9b-47b4-458f-a332-9c2ed48c3266\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-619550-597b7898-4xksl"
Sep 16 02:31:07.957370 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:07.957346    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f84078e6-c7af-4767-8fd4-c68bb7d27bfa-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-619550-5984cc5ccf-58g44\" (UID: \"f84078e6-c7af-4767-8fd4-c68bb7d27bfa\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-619550-5984cc5ccf-58g44"
Sep 16 02:31:07.964859 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:07.964832    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z9s2t\" (UniqueName: \"kubernetes.io/projected/452d6b9b-47b4-458f-a332-9c2ed48c3266-kube-api-access-z9s2t\") pod \"payload-pre-processing-e2e-worker-w0-619550-597b7898-4xksl\" (UID: \"452d6b9b-47b4-458f-a332-9c2ed48c3266\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-619550-597b7898-4xksl"
Sep 16 02:31:07.964967 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:07.964933    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t848s\" (UniqueName: \"kubernetes.io/projected/f84078e6-c7af-4767-8fd4-c68bb7d27bfa-kube-api-access-t848s\") pod \"payload-processing-e2e-worker-w0-619550-5984cc5ccf-58g44\" (UID: \"f84078e6-c7af-4767-8fd4-c68bb7d27bfa\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-619550-5984cc5ccf-58g44"
Sep 16 02:31:08.106937 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:08.106908    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-619550-597b7898-4xksl"
Sep 16 02:31:08.150401 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:08.150367    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-619550-5984cc5ccf-58g44"
Sep 16 02:31:08.231675 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:08.231648    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:31:08.292137 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:08.292105    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:31:08.382924 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:08.382895    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-619550-597b7898-4xksl"]
Sep 16 02:31:08.384307 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:31:08.384275    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod452d6b9b_47b4_458f_a332_9c2ed48c3266.slice/crio-90d644c6f542ba17f289ac4193db6a71f904420011676fb40adf85949c2aab82 WatchSource:0}: Error finding container 90d644c6f542ba17f289ac4193db6a71f904420011676fb40adf85949c2aab82: Status 404 returned error can't find the container with id 90d644c6f542ba17f289ac4193db6a71f904420011676fb40adf85949c2aab82
Sep 16 02:31:08.435978 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:08.435955    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-619550-5984cc5ccf-58g44"]
Sep 16 02:31:08.437550 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:31:08.437522    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf84078e6_c7af_4767_8fd4_c68bb7d27bfa.slice/crio-c956be1654b3d392f6c9e5cc4e0d9373377cae334a7434476fe7c9b519e68bfc WatchSource:0}: Error finding container c956be1654b3d392f6c9e5cc4e0d9373377cae334a7434476fe7c9b519e68bfc: Status 404 returned error can't find the container with id c956be1654b3d392f6c9e5cc4e0d9373377cae334a7434476fe7c9b519e68bfc
Sep 16 02:31:08.448941 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:08.448917    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:31:08.540372 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:08.540337    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-619550-5984cc5ccf-58g44" event={"ID":"f84078e6-c7af-4767-8fd4-c68bb7d27bfa","Type":"ContainerStarted","Data":"1e5ad5ca25ab553a3afc3e699a07eb2f3018d8759632959c358c1fa0ce951221"}
Sep 16 02:31:08.540553 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:08.540393    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-619550-5984cc5ccf-58g44" event={"ID":"f84078e6-c7af-4767-8fd4-c68bb7d27bfa","Type":"ContainerStarted","Data":"c956be1654b3d392f6c9e5cc4e0d9373377cae334a7434476fe7c9b519e68bfc"}
Sep 16 02:31:08.540553 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:08.540451    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w0-619550-5984cc5ccf-58g44"
Sep 16 02:31:08.542268 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:08.542240    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-619550-597b7898-4xksl" event={"ID":"452d6b9b-47b4-458f-a332-9c2ed48c3266","Type":"ContainerStarted","Data":"f369ea688d4aa23f2b39a11da4f87356b339216a551f9fdeeea6433f43d78d65"}
Sep 16 02:31:08.542408 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:08.542292    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-619550-597b7898-4xksl" event={"ID":"452d6b9b-47b4-458f-a332-9c2ed48c3266","Type":"ContainerStarted","Data":"90d644c6f542ba17f289ac4193db6a71f904420011676fb40adf85949c2aab82"}
Sep 16 02:31:08.542408 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:08.542359    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-619550-597b7898-4xksl"
Sep 16 02:31:08.561725 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:08.561666    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w0-619550-5984cc5ccf-58g44" podStartSLOduration=1.561645043 podStartE2EDuration="1.561645043s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 02:31:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 02:31:08.560572666 +0000 UTC m=+1088.944087921" watchObservedRunningTime="2026-09-16 02:31:08.561645043 +0000 UTC m=+1088.945160289"
Sep 16 02:31:08.901940 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:08.901835    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:31:08.971611 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:08.971549    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-619550-597b7898-4xksl" podStartSLOduration=1.9715281999999998 podStartE2EDuration="1.9715282s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 02:31:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 02:31:08.581847055 +0000 UTC m=+1088.965362301" watchObservedRunningTime="2026-09-16 02:31:08.9715282 +0000 UTC m=+1089.355043462"
Sep 16 02:31:08.973307 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:08.973280    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-542ed5-84b56598c8-pcz52"]
Sep 16 02:31:08.975958 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:08.975929    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:31:08.977036 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:08.977021    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-542ed5-84b56598c8-pcz52"
Sep 16 02:31:08.979245 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:08.979226    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w3-542ed5-dockercfg-xknsv\""
Sep 16 02:31:08.979365 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:08.979293    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w3-542ed5\""
Sep 16 02:31:08.987705 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:08.987681    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-542ed5-84b56598c8-pcz52"]
Sep 16 02:31:09.067577 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:09.067543    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2169a66a-2ca0-4cb1-81f8-6dcc588e9652-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w3-542ed5-84b56598c8-pcz52\" (UID: \"2169a66a-2ca0-4cb1-81f8-6dcc588e9652\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-542ed5-84b56598c8-pcz52"
Sep 16 02:31:09.067748 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:09.067597    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fg62z\" (UniqueName: \"kubernetes.io/projected/2169a66a-2ca0-4cb1-81f8-6dcc588e9652-kube-api-access-fg62z\") pod \"payload-pre-processing-e2e-worker-w3-542ed5-84b56598c8-pcz52\" (UID: \"2169a66a-2ca0-4cb1-81f8-6dcc588e9652\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-542ed5-84b56598c8-pcz52"
Sep 16 02:31:09.168826 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:09.168743    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2169a66a-2ca0-4cb1-81f8-6dcc588e9652-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w3-542ed5-84b56598c8-pcz52\" (UID: \"2169a66a-2ca0-4cb1-81f8-6dcc588e9652\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-542ed5-84b56598c8-pcz52"
Sep 16 02:31:09.168826 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:09.168792    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fg62z\" (UniqueName: \"kubernetes.io/projected/2169a66a-2ca0-4cb1-81f8-6dcc588e9652-kube-api-access-fg62z\") pod \"payload-pre-processing-e2e-worker-w3-542ed5-84b56598c8-pcz52\" (UID: \"2169a66a-2ca0-4cb1-81f8-6dcc588e9652\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-542ed5-84b56598c8-pcz52"
Sep 16 02:31:09.169420 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:09.169399    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2169a66a-2ca0-4cb1-81f8-6dcc588e9652-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w3-542ed5-84b56598c8-pcz52\" (UID: \"2169a66a-2ca0-4cb1-81f8-6dcc588e9652\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-542ed5-84b56598c8-pcz52"
Sep 16 02:31:09.176544 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:09.176521    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fg62z\" (UniqueName: \"kubernetes.io/projected/2169a66a-2ca0-4cb1-81f8-6dcc588e9652-kube-api-access-fg62z\") pod \"payload-pre-processing-e2e-worker-w3-542ed5-84b56598c8-pcz52\" (UID: \"2169a66a-2ca0-4cb1-81f8-6dcc588e9652\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-542ed5-84b56598c8-pcz52"
Sep 16 02:31:09.289276 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:09.289230    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-542ed5-84b56598c8-pcz52"
Sep 16 02:31:09.359286 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:09.359247    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:31:09.568048 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:09.567682    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-542ed5-84b56598c8-pcz52"]
Sep 16 02:31:09.570229 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:31:09.570178    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2169a66a_2ca0_4cb1_81f8_6dcc588e9652.slice/crio-5984ac8df234170a0b2e503de3be48ac35c93b5bd382ec314e7faed0891f816b WatchSource:0}: Error finding container 5984ac8df234170a0b2e503de3be48ac35c93b5bd382ec314e7faed0891f816b: Status 404 returned error can't find the container with id 5984ac8df234170a0b2e503de3be48ac35c93b5bd382ec314e7faed0891f816b
Sep 16 02:31:10.158928 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:10.158895    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:31:10.553083 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:10.553038    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-542ed5-84b56598c8-pcz52" event={"ID":"2169a66a-2ca0-4cb1-81f8-6dcc588e9652","Type":"ContainerStarted","Data":"835a1b970d3712febafe0d52175ae00b74911bd1887a25afc65733ae7fd4f200"}
Sep 16 02:31:10.553083 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:10.553087    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-542ed5-84b56598c8-pcz52" event={"ID":"2169a66a-2ca0-4cb1-81f8-6dcc588e9652","Type":"ContainerStarted","Data":"5984ac8df234170a0b2e503de3be48ac35c93b5bd382ec314e7faed0891f816b"}
Sep 16 02:31:10.553363 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:10.553182    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-542ed5-84b56598c8-pcz52"
Sep 16 02:31:10.569530 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:10.569461    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-542ed5-84b56598c8-pcz52" podStartSLOduration=2.569448415 podStartE2EDuration="2.569448415s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 02:31:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 02:31:10.567804769 +0000 UTC m=+1090.951320014" watchObservedRunningTime="2026-09-16 02:31:10.569448415 +0000 UTC m=+1090.952963702"
Sep 16 02:31:10.898975 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:10.898882    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:31:18.333689 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:18.333653    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:31:18.434661 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:18.434621    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:31:18.544502 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:18.544450    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-60674e-f95445b5c-x5gpp"
Sep 16 02:31:18.544684 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:18.544669    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-60674e-69589755b6-wpf7z"
Sep 16 02:31:18.971226 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:18.971120    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:31:19.065690 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:19.065656    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:31:19.169200 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:19.169147    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:31:19.548908 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:19.548878    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-619550-597b7898-4xksl"
Sep 16 02:31:19.549334 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:19.548954    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w0-619550-5984cc5ccf-58g44"
Sep 16 02:31:21.561009 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:21.560981    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-542ed5-84b56598c8-pcz52"
Sep 16 02:31:24.970576 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:24.970544    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:31:25.078286 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:25.078253    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:31:25.173112 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:25.173081    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:31:25.291952 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:25.291869    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:31:27.085073 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:27.085032    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:31:28.717906 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:28.717863    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:31:29.716993 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:29.716961    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:31:29.880085 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:29.880049    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:31:29.980775 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:29.980666    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:31:30.213294 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:30.213262    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:31:30.317384 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:30.317280    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:31:30.417509 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:30.416969    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:31:30.522293 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:30.522250    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:31:30.613946 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:30.613859    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:31:30.713780 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:30.713751    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:31:30.814410 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:30.814368    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:31:31.375328 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:31.375293    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:31:32.143759 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:32.143726    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:31:32.205259 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:32.205214    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:31:32.337372 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:32.337339    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:31:33.010214 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:33.010165    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:31:33.938220 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:33.938186    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:31:34.187601 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:34.187572    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:31:34.281621 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:34.281524    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:31:34.644557 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:34.644457    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:31:34.746431 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:34.746397    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:31:35.013168 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:35.013133    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:31:35.481340 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:35.481283    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:31:35.632714 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:35.632678    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:31:35.728411 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:35.728380    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:31:36.356654 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:36.356619    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:31:36.418825 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:36.418780    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:31:36.664063 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:36.663971    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:31:36.891539 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:36.891499    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:31:36.993254 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:36.993215    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:31:37.002367 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:37.002341    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-1a323c-dc54d6f4-c5k58"]
Sep 16 02:31:37.006050 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:37.006034    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-1a323c-dc54d6f4-c5k58"
Sep 16 02:31:37.008453 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:37.008436    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-1a323c-dockercfg-lbltj\""
Sep 16 02:31:37.008545 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:37.008451    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-1a323c\""
Sep 16 02:31:37.015342 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:37.015317    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-1a323c-dc54d6f4-c5k58"]
Sep 16 02:31:37.036556 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:37.036529    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-1a323c-5d57f9fdfc-fcm9x"]
Sep 16 02:31:37.040721 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:37.040703    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-1a323c-5d57f9fdfc-fcm9x"
Sep 16 02:31:37.049056 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:37.049031    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-1a323c-5d57f9fdfc-fcm9x"]
Sep 16 02:31:37.117827 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:37.117800    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c03ab1c3-17db-4555-99aa-f3ab4a2f11f0-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-1a323c-5d57f9fdfc-fcm9x\" (UID: \"c03ab1c3-17db-4555-99aa-f3ab4a2f11f0\") " pod="openshift-ingress/payload-processing-e2e-inf-b-1a323c-5d57f9fdfc-fcm9x"
Sep 16 02:31:37.118017 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:37.117846    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j5v9m\" (UniqueName: \"kubernetes.io/projected/c03ab1c3-17db-4555-99aa-f3ab4a2f11f0-kube-api-access-j5v9m\") pod \"payload-processing-e2e-inf-b-1a323c-5d57f9fdfc-fcm9x\" (UID: \"c03ab1c3-17db-4555-99aa-f3ab4a2f11f0\") " pod="openshift-ingress/payload-processing-e2e-inf-b-1a323c-5d57f9fdfc-fcm9x"
Sep 16 02:31:37.118017 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:37.117946    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6f71a2e1-266a-40ff-b337-7822a912a8b0-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-1a323c-dc54d6f4-c5k58\" (UID: \"6f71a2e1-266a-40ff-b337-7822a912a8b0\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-1a323c-dc54d6f4-c5k58"
Sep 16 02:31:37.118017 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:37.117995    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xxrxz\" (UniqueName: \"kubernetes.io/projected/6f71a2e1-266a-40ff-b337-7822a912a8b0-kube-api-access-xxrxz\") pod \"payload-pre-processing-e2e-inf-b-1a323c-dc54d6f4-c5k58\" (UID: \"6f71a2e1-266a-40ff-b337-7822a912a8b0\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-1a323c-dc54d6f4-c5k58"
Sep 16 02:31:37.219185 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:37.219146    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j5v9m\" (UniqueName: \"kubernetes.io/projected/c03ab1c3-17db-4555-99aa-f3ab4a2f11f0-kube-api-access-j5v9m\") pod \"payload-processing-e2e-inf-b-1a323c-5d57f9fdfc-fcm9x\" (UID: \"c03ab1c3-17db-4555-99aa-f3ab4a2f11f0\") " pod="openshift-ingress/payload-processing-e2e-inf-b-1a323c-5d57f9fdfc-fcm9x"
Sep 16 02:31:37.219358 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:37.219233    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6f71a2e1-266a-40ff-b337-7822a912a8b0-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-1a323c-dc54d6f4-c5k58\" (UID: \"6f71a2e1-266a-40ff-b337-7822a912a8b0\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-1a323c-dc54d6f4-c5k58"
Sep 16 02:31:37.219358 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:37.219272    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xxrxz\" (UniqueName: \"kubernetes.io/projected/6f71a2e1-266a-40ff-b337-7822a912a8b0-kube-api-access-xxrxz\") pod \"payload-pre-processing-e2e-inf-b-1a323c-dc54d6f4-c5k58\" (UID: \"6f71a2e1-266a-40ff-b337-7822a912a8b0\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-1a323c-dc54d6f4-c5k58"
Sep 16 02:31:37.219358 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:37.219349    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c03ab1c3-17db-4555-99aa-f3ab4a2f11f0-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-1a323c-5d57f9fdfc-fcm9x\" (UID: \"c03ab1c3-17db-4555-99aa-f3ab4a2f11f0\") " pod="openshift-ingress/payload-processing-e2e-inf-b-1a323c-5d57f9fdfc-fcm9x"
Sep 16 02:31:37.220073 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:37.220048    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6f71a2e1-266a-40ff-b337-7822a912a8b0-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-1a323c-dc54d6f4-c5k58\" (UID: \"6f71a2e1-266a-40ff-b337-7822a912a8b0\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-1a323c-dc54d6f4-c5k58"
Sep 16 02:31:37.220326 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:37.220304    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c03ab1c3-17db-4555-99aa-f3ab4a2f11f0-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-1a323c-5d57f9fdfc-fcm9x\" (UID: \"c03ab1c3-17db-4555-99aa-f3ab4a2f11f0\") " pod="openshift-ingress/payload-processing-e2e-inf-b-1a323c-5d57f9fdfc-fcm9x"
Sep 16 02:31:37.226932 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:37.226903    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j5v9m\" (UniqueName: \"kubernetes.io/projected/c03ab1c3-17db-4555-99aa-f3ab4a2f11f0-kube-api-access-j5v9m\") pod \"payload-processing-e2e-inf-b-1a323c-5d57f9fdfc-fcm9x\" (UID: \"c03ab1c3-17db-4555-99aa-f3ab4a2f11f0\") " pod="openshift-ingress/payload-processing-e2e-inf-b-1a323c-5d57f9fdfc-fcm9x"
Sep 16 02:31:37.227073 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:37.227054    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xxrxz\" (UniqueName: \"kubernetes.io/projected/6f71a2e1-266a-40ff-b337-7822a912a8b0-kube-api-access-xxrxz\") pod \"payload-pre-processing-e2e-inf-b-1a323c-dc54d6f4-c5k58\" (UID: \"6f71a2e1-266a-40ff-b337-7822a912a8b0\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-1a323c-dc54d6f4-c5k58"
Sep 16 02:31:37.319259 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:37.319162    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-1a323c-dc54d6f4-c5k58"
Sep 16 02:31:37.352040 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:37.352007    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-1a323c-5d57f9fdfc-fcm9x"
Sep 16 02:31:37.528534 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:37.528497    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:31:37.598137 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:37.598110    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-1a323c-dc54d6f4-c5k58"]
Sep 16 02:31:37.599968 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:31:37.599936    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f71a2e1_266a_40ff_b337_7822a912a8b0.slice/crio-4353a57f59e4a60ba3c90992fdf86343e2a2fde9f0c2ec6f0d29cdd7228f6e52 WatchSource:0}: Error finding container 4353a57f59e4a60ba3c90992fdf86343e2a2fde9f0c2ec6f0d29cdd7228f6e52: Status 404 returned error can't find the container with id 4353a57f59e4a60ba3c90992fdf86343e2a2fde9f0c2ec6f0d29cdd7228f6e52
Sep 16 02:31:37.632967 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:37.632867    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-1a323c-5d57f9fdfc-fcm9x"]
Sep 16 02:31:37.634417 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:31:37.634393    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc03ab1c3_17db_4555_99aa_f3ab4a2f11f0.slice/crio-82c107635a35c875ef07b7593f9a260c12391087fb53d3f3b105ca7ea4a2b383 WatchSource:0}: Error finding container 82c107635a35c875ef07b7593f9a260c12391087fb53d3f3b105ca7ea4a2b383: Status 404 returned error can't find the container with id 82c107635a35c875ef07b7593f9a260c12391087fb53d3f3b105ca7ea4a2b383
Sep 16 02:31:37.689410 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:37.689374    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-1a323c-5d57f9fdfc-fcm9x" event={"ID":"c03ab1c3-17db-4555-99aa-f3ab4a2f11f0","Type":"ContainerStarted","Data":"82c107635a35c875ef07b7593f9a260c12391087fb53d3f3b105ca7ea4a2b383"}
Sep 16 02:31:37.691146 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:37.691123    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-1a323c-dc54d6f4-c5k58" event={"ID":"6f71a2e1-266a-40ff-b337-7822a912a8b0","Type":"ContainerStarted","Data":"168bdcfb558c3ccad944b529e6b9145c8c41a7c7b32d1a301102b3d622364ec2"}
Sep 16 02:31:37.691266 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:37.691167    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-1a323c-dc54d6f4-c5k58" event={"ID":"6f71a2e1-266a-40ff-b337-7822a912a8b0","Type":"ContainerStarted","Data":"4353a57f59e4a60ba3c90992fdf86343e2a2fde9f0c2ec6f0d29cdd7228f6e52"}
Sep 16 02:31:37.691266 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:37.691200    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-1a323c-dc54d6f4-c5k58"
Sep 16 02:31:37.708736 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:37.708687    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-1a323c-dc54d6f4-c5k58" podStartSLOduration=1.7086729649999999 podStartE2EDuration="1.708672965s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 02:31:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 02:31:37.706090047 +0000 UTC m=+1118.089605292" watchObservedRunningTime="2026-09-16 02:31:37.708672965 +0000 UTC m=+1118.092188209"
Sep 16 02:31:38.233418 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:38.233365    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:31:38.697697 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:38.697658    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-1a323c-5d57f9fdfc-fcm9x" event={"ID":"c03ab1c3-17db-4555-99aa-f3ab4a2f11f0","Type":"ContainerStarted","Data":"176f9e73189f12784e44ca0ad56cb90df6c3aa84fe24f8525b9ba6438d418497"}
Sep 16 02:31:38.698169 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:38.697830    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-1a323c-5d57f9fdfc-fcm9x"
Sep 16 02:31:38.712879 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:38.712834    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-1a323c-5d57f9fdfc-fcm9x" podStartSLOduration=1.712819631 podStartE2EDuration="1.712819631s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 02:31:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 02:31:38.711910905 +0000 UTC m=+1119.095426154" watchObservedRunningTime="2026-09-16 02:31:38.712819631 +0000 UTC m=+1119.096334875"
Sep 16 02:31:48.700123 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:48.700093    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-1a323c-dc54d6f4-c5k58"
Sep 16 02:31:49.376968 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:49.376935    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-619550/simulator-w0-619550-kserve-5958b89dff-t28jc"]
Sep 16 02:31:49.380980 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:49.380957    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-619550/simulator-w0-619550-kserve-5958b89dff-t28jc"
Sep 16 02:31:49.383120 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:49.383097    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-619550\"/\"kube-root-ca.crt\""
Sep 16 02:31:49.384071 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:49.384047    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-619550\"/\"default-dockercfg-s94hk\""
Sep 16 02:31:49.384071 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:49.384065    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-619550\"/\"openshift-service-ca.crt\""
Sep 16 02:31:49.384248 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:49.384095    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-619550\"/\"simulator-w0-619550-kserve-self-signed-certs\""
Sep 16 02:31:49.389514 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:49.389451    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-619550/simulator-w0-619550-kserve-5958b89dff-t28jc"]
Sep 16 02:31:49.408781 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:49.408741    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:31:49.431950 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:49.431921    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b6e49a5a-de18-4720-aaac-b1b3b1c617b7-dshm\") pod \"simulator-w0-619550-kserve-5958b89dff-t28jc\" (UID: \"b6e49a5a-de18-4720-aaac-b1b3b1c617b7\") " pod="e2e-models-e2e-worker-w0-619550/simulator-w0-619550-kserve-5958b89dff-t28jc"
Sep 16 02:31:49.432064 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:49.431970    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b6e49a5a-de18-4720-aaac-b1b3b1c617b7-tmp-dir\") pod \"simulator-w0-619550-kserve-5958b89dff-t28jc\" (UID: \"b6e49a5a-de18-4720-aaac-b1b3b1c617b7\") " pod="e2e-models-e2e-worker-w0-619550/simulator-w0-619550-kserve-5958b89dff-t28jc"
Sep 16 02:31:49.432064 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:49.432008    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4mggc\" (UniqueName: \"kubernetes.io/projected/b6e49a5a-de18-4720-aaac-b1b3b1c617b7-kube-api-access-4mggc\") pod \"simulator-w0-619550-kserve-5958b89dff-t28jc\" (UID: \"b6e49a5a-de18-4720-aaac-b1b3b1c617b7\") " pod="e2e-models-e2e-worker-w0-619550/simulator-w0-619550-kserve-5958b89dff-t28jc"
Sep 16 02:31:49.432064 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:49.432027    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b6e49a5a-de18-4720-aaac-b1b3b1c617b7-home\") pod \"simulator-w0-619550-kserve-5958b89dff-t28jc\" (UID: \"b6e49a5a-de18-4720-aaac-b1b3b1c617b7\") " pod="e2e-models-e2e-worker-w0-619550/simulator-w0-619550-kserve-5958b89dff-t28jc"
Sep 16 02:31:49.432064 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:49.432048    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b6e49a5a-de18-4720-aaac-b1b3b1c617b7-tls-certs\") pod \"simulator-w0-619550-kserve-5958b89dff-t28jc\" (UID: \"b6e49a5a-de18-4720-aaac-b1b3b1c617b7\") " pod="e2e-models-e2e-worker-w0-619550/simulator-w0-619550-kserve-5958b89dff-t28jc"
Sep 16 02:31:49.432225 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:49.432116    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b6e49a5a-de18-4720-aaac-b1b3b1c617b7-model-cache\") pod \"simulator-w0-619550-kserve-5958b89dff-t28jc\" (UID: \"b6e49a5a-de18-4720-aaac-b1b3b1c617b7\") " pod="e2e-models-e2e-worker-w0-619550/simulator-w0-619550-kserve-5958b89dff-t28jc"
Sep 16 02:31:49.533222 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:49.533185    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b6e49a5a-de18-4720-aaac-b1b3b1c617b7-dshm\") pod \"simulator-w0-619550-kserve-5958b89dff-t28jc\" (UID: \"b6e49a5a-de18-4720-aaac-b1b3b1c617b7\") " pod="e2e-models-e2e-worker-w0-619550/simulator-w0-619550-kserve-5958b89dff-t28jc"
Sep 16 02:31:49.533357 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:49.533240    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b6e49a5a-de18-4720-aaac-b1b3b1c617b7-tmp-dir\") pod \"simulator-w0-619550-kserve-5958b89dff-t28jc\" (UID: \"b6e49a5a-de18-4720-aaac-b1b3b1c617b7\") " pod="e2e-models-e2e-worker-w0-619550/simulator-w0-619550-kserve-5958b89dff-t28jc"
Sep 16 02:31:49.533396 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:49.533364    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4mggc\" (UniqueName: \"kubernetes.io/projected/b6e49a5a-de18-4720-aaac-b1b3b1c617b7-kube-api-access-4mggc\") pod \"simulator-w0-619550-kserve-5958b89dff-t28jc\" (UID: \"b6e49a5a-de18-4720-aaac-b1b3b1c617b7\") " pod="e2e-models-e2e-worker-w0-619550/simulator-w0-619550-kserve-5958b89dff-t28jc"
Sep 16 02:31:49.533429 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:49.533403    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b6e49a5a-de18-4720-aaac-b1b3b1c617b7-home\") pod \"simulator-w0-619550-kserve-5958b89dff-t28jc\" (UID: \"b6e49a5a-de18-4720-aaac-b1b3b1c617b7\") " pod="e2e-models-e2e-worker-w0-619550/simulator-w0-619550-kserve-5958b89dff-t28jc"
Sep 16 02:31:49.533548 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:49.533527    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b6e49a5a-de18-4720-aaac-b1b3b1c617b7-tls-certs\") pod \"simulator-w0-619550-kserve-5958b89dff-t28jc\" (UID: \"b6e49a5a-de18-4720-aaac-b1b3b1c617b7\") " pod="e2e-models-e2e-worker-w0-619550/simulator-w0-619550-kserve-5958b89dff-t28jc"
Sep 16 02:31:49.533614 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:49.533554    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b6e49a5a-de18-4720-aaac-b1b3b1c617b7-tmp-dir\") pod \"simulator-w0-619550-kserve-5958b89dff-t28jc\" (UID: \"b6e49a5a-de18-4720-aaac-b1b3b1c617b7\") " pod="e2e-models-e2e-worker-w0-619550/simulator-w0-619550-kserve-5958b89dff-t28jc"
Sep 16 02:31:49.533614 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:49.533578    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b6e49a5a-de18-4720-aaac-b1b3b1c617b7-model-cache\") pod \"simulator-w0-619550-kserve-5958b89dff-t28jc\" (UID: \"b6e49a5a-de18-4720-aaac-b1b3b1c617b7\") " pod="e2e-models-e2e-worker-w0-619550/simulator-w0-619550-kserve-5958b89dff-t28jc"
Sep 16 02:31:49.533722 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:49.533701    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b6e49a5a-de18-4720-aaac-b1b3b1c617b7-home\") pod \"simulator-w0-619550-kserve-5958b89dff-t28jc\" (UID: \"b6e49a5a-de18-4720-aaac-b1b3b1c617b7\") " pod="e2e-models-e2e-worker-w0-619550/simulator-w0-619550-kserve-5958b89dff-t28jc"
Sep 16 02:31:49.533930 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:49.533909    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b6e49a5a-de18-4720-aaac-b1b3b1c617b7-model-cache\") pod \"simulator-w0-619550-kserve-5958b89dff-t28jc\" (UID: \"b6e49a5a-de18-4720-aaac-b1b3b1c617b7\") " pod="e2e-models-e2e-worker-w0-619550/simulator-w0-619550-kserve-5958b89dff-t28jc"
Sep 16 02:31:49.535735 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:49.535711    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b6e49a5a-de18-4720-aaac-b1b3b1c617b7-dshm\") pod \"simulator-w0-619550-kserve-5958b89dff-t28jc\" (UID: \"b6e49a5a-de18-4720-aaac-b1b3b1c617b7\") " pod="e2e-models-e2e-worker-w0-619550/simulator-w0-619550-kserve-5958b89dff-t28jc"
Sep 16 02:31:49.535976 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:49.535960    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b6e49a5a-de18-4720-aaac-b1b3b1c617b7-tls-certs\") pod \"simulator-w0-619550-kserve-5958b89dff-t28jc\" (UID: \"b6e49a5a-de18-4720-aaac-b1b3b1c617b7\") " pod="e2e-models-e2e-worker-w0-619550/simulator-w0-619550-kserve-5958b89dff-t28jc"
Sep 16 02:31:49.541356 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:49.541339    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4mggc\" (UniqueName: \"kubernetes.io/projected/b6e49a5a-de18-4720-aaac-b1b3b1c617b7-kube-api-access-4mggc\") pod \"simulator-w0-619550-kserve-5958b89dff-t28jc\" (UID: \"b6e49a5a-de18-4720-aaac-b1b3b1c617b7\") " pod="e2e-models-e2e-worker-w0-619550/simulator-w0-619550-kserve-5958b89dff-t28jc"
Sep 16 02:31:49.693033 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:49.692998    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-619550/simulator-w0-619550-kserve-5958b89dff-t28jc"
Sep 16 02:31:49.705432 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:49.705410    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-1a323c-5d57f9fdfc-fcm9x"
Sep 16 02:31:49.965920 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:49.965893    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-619550/simulator-w0-619550-kserve-5958b89dff-t28jc"]
Sep 16 02:31:49.967713 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:31:49.967685    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb6e49a5a_de18_4720_aaac_b1b3b1c617b7.slice/crio-31aa4d2c5db9b93de40e53455bd96863a724c5ba7b2d7fe9b5ab89ba78f067c7 WatchSource:0}: Error finding container 31aa4d2c5db9b93de40e53455bd96863a724c5ba7b2d7fe9b5ab89ba78f067c7: Status 404 returned error can't find the container with id 31aa4d2c5db9b93de40e53455bd96863a724c5ba7b2d7fe9b5ab89ba78f067c7
Sep 16 02:31:50.758533 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:50.758500    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-619550/simulator-w0-619550-kserve-5958b89dff-t28jc" event={"ID":"b6e49a5a-de18-4720-aaac-b1b3b1c617b7","Type":"ContainerStarted","Data":"4bdcd06293f14fdfee8e2881fb8c0c4b4bdf4143016db237779f96adb5cc2a26"}
Sep 16 02:31:50.758912 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:50.758543    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-619550/simulator-w0-619550-kserve-5958b89dff-t28jc" event={"ID":"b6e49a5a-de18-4720-aaac-b1b3b1c617b7","Type":"ContainerStarted","Data":"31aa4d2c5db9b93de40e53455bd96863a724c5ba7b2d7fe9b5ab89ba78f067c7"}
Sep 16 02:31:50.774234 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:50.774186    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-619550/simulator-w0-619550-kserve-5958b89dff-t28jc" podStartSLOduration=1.7741719649999999 podStartE2EDuration="1.774171965s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 02:31:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 02:31:50.772076086 +0000 UTC m=+1131.155591331" watchObservedRunningTime="2026-09-16 02:31:50.774171965 +0000 UTC m=+1131.157687214"
Sep 16 02:31:53.097204 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:53.097170    2579 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-pre-processing-6cd864bc95-mh4jh" secret="" err="secret \"payload-processing-dockercfg-zzx6j\" not found"
Sep 16 02:31:58.271639 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:58.271594    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:31:59.025097 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:59.025056    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:31:59.132566 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:59.132534    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:31:59.693183 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:59.693150    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-619550/simulator-w0-619550-kserve-5958b89dff-t28jc"
Sep 16 02:31:59.693708 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:59.693200    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-619550/simulator-w0-619550-kserve-5958b89dff-t28jc"
Sep 16 02:31:59.706445 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:59.706417    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-619550/simulator-w0-619550-kserve-5958b89dff-t28jc"
Sep 16 02:31:59.820169 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:59.820141    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-619550/simulator-w0-619550-kserve-5958b89dff-t28jc"
Sep 16 02:31:59.920729 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:31:59.920647    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:32:00.541262 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:32:00.541222    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:32:05.578627 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:32:05.578580    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w3-542ed5/simulator-w3-542ed5-kserve-77ff54b49-nwttm"]
Sep 16 02:32:05.582493 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:32:05.582456    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-542ed5/simulator-w3-542ed5-kserve-77ff54b49-nwttm"
Sep 16 02:32:05.585623 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:32:05.585582    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-542ed5\"/\"kube-root-ca.crt\""
Sep 16 02:32:05.585623 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:32:05.585598    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-542ed5\"/\"openshift-service-ca.crt\""
Sep 16 02:32:05.585623 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:32:05.585613    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-542ed5\"/\"default-dockercfg-vtfl8\""
Sep 16 02:32:05.585822 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:32:05.585597    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-542ed5\"/\"simulator-w3-542ed5-kserve-self-signed-certs\""
Sep 16 02:32:05.593596 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:32:05.593575    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-542ed5/simulator-w3-542ed5-kserve-77ff54b49-nwttm"]
Sep 16 02:32:05.614690 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:32:05.614660    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:32:05.670047 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:32:05.670015    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/99191806-ff18-4c21-afa7-701c9c9a85fa-home\") pod \"simulator-w3-542ed5-kserve-77ff54b49-nwttm\" (UID: \"99191806-ff18-4c21-afa7-701c9c9a85fa\") " pod="e2e-models-e2e-worker-w3-542ed5/simulator-w3-542ed5-kserve-77ff54b49-nwttm"
Sep 16 02:32:05.670198 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:32:05.670051    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/99191806-ff18-4c21-afa7-701c9c9a85fa-dshm\") pod \"simulator-w3-542ed5-kserve-77ff54b49-nwttm\" (UID: \"99191806-ff18-4c21-afa7-701c9c9a85fa\") " pod="e2e-models-e2e-worker-w3-542ed5/simulator-w3-542ed5-kserve-77ff54b49-nwttm"
Sep 16 02:32:05.670198 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:32:05.670078    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b8qks\" (UniqueName: \"kubernetes.io/projected/99191806-ff18-4c21-afa7-701c9c9a85fa-kube-api-access-b8qks\") pod \"simulator-w3-542ed5-kserve-77ff54b49-nwttm\" (UID: \"99191806-ff18-4c21-afa7-701c9c9a85fa\") " pod="e2e-models-e2e-worker-w3-542ed5/simulator-w3-542ed5-kserve-77ff54b49-nwttm"
Sep 16 02:32:05.670198 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:32:05.670152    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/99191806-ff18-4c21-afa7-701c9c9a85fa-model-cache\") pod \"simulator-w3-542ed5-kserve-77ff54b49-nwttm\" (UID: \"99191806-ff18-4c21-afa7-701c9c9a85fa\") " pod="e2e-models-e2e-worker-w3-542ed5/simulator-w3-542ed5-kserve-77ff54b49-nwttm"
Sep 16 02:32:05.670198 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:32:05.670183    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/99191806-ff18-4c21-afa7-701c9c9a85fa-tmp-dir\") pod \"simulator-w3-542ed5-kserve-77ff54b49-nwttm\" (UID: \"99191806-ff18-4c21-afa7-701c9c9a85fa\") " pod="e2e-models-e2e-worker-w3-542ed5/simulator-w3-542ed5-kserve-77ff54b49-nwttm"
Sep 16 02:32:05.670446 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:32:05.670320    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/99191806-ff18-4c21-afa7-701c9c9a85fa-tls-certs\") pod \"simulator-w3-542ed5-kserve-77ff54b49-nwttm\" (UID: \"99191806-ff18-4c21-afa7-701c9c9a85fa\") " pod="e2e-models-e2e-worker-w3-542ed5/simulator-w3-542ed5-kserve-77ff54b49-nwttm"
Sep 16 02:32:05.771588 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:32:05.771561    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/99191806-ff18-4c21-afa7-701c9c9a85fa-tls-certs\") pod \"simulator-w3-542ed5-kserve-77ff54b49-nwttm\" (UID: \"99191806-ff18-4c21-afa7-701c9c9a85fa\") " pod="e2e-models-e2e-worker-w3-542ed5/simulator-w3-542ed5-kserve-77ff54b49-nwttm"
Sep 16 02:32:05.771761 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:32:05.771616    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/99191806-ff18-4c21-afa7-701c9c9a85fa-home\") pod \"simulator-w3-542ed5-kserve-77ff54b49-nwttm\" (UID: \"99191806-ff18-4c21-afa7-701c9c9a85fa\") " pod="e2e-models-e2e-worker-w3-542ed5/simulator-w3-542ed5-kserve-77ff54b49-nwttm"
Sep 16 02:32:05.771761 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:32:05.771643    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/99191806-ff18-4c21-afa7-701c9c9a85fa-dshm\") pod \"simulator-w3-542ed5-kserve-77ff54b49-nwttm\" (UID: \"99191806-ff18-4c21-afa7-701c9c9a85fa\") " pod="e2e-models-e2e-worker-w3-542ed5/simulator-w3-542ed5-kserve-77ff54b49-nwttm"
Sep 16 02:32:05.771761 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:32:05.771665    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b8qks\" (UniqueName: \"kubernetes.io/projected/99191806-ff18-4c21-afa7-701c9c9a85fa-kube-api-access-b8qks\") pod \"simulator-w3-542ed5-kserve-77ff54b49-nwttm\" (UID: \"99191806-ff18-4c21-afa7-701c9c9a85fa\") " pod="e2e-models-e2e-worker-w3-542ed5/simulator-w3-542ed5-kserve-77ff54b49-nwttm"
Sep 16 02:32:05.771761 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:32:05.771701    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/99191806-ff18-4c21-afa7-701c9c9a85fa-model-cache\") pod \"simulator-w3-542ed5-kserve-77ff54b49-nwttm\" (UID: \"99191806-ff18-4c21-afa7-701c9c9a85fa\") " pod="e2e-models-e2e-worker-w3-542ed5/simulator-w3-542ed5-kserve-77ff54b49-nwttm"
Sep 16 02:32:05.771761 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:32:05.771724    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/99191806-ff18-4c21-afa7-701c9c9a85fa-tmp-dir\") pod \"simulator-w3-542ed5-kserve-77ff54b49-nwttm\" (UID: \"99191806-ff18-4c21-afa7-701c9c9a85fa\") " pod="e2e-models-e2e-worker-w3-542ed5/simulator-w3-542ed5-kserve-77ff54b49-nwttm"
Sep 16 02:32:05.772181 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:32:05.772159    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/99191806-ff18-4c21-afa7-701c9c9a85fa-model-cache\") pod \"simulator-w3-542ed5-kserve-77ff54b49-nwttm\" (UID: \"99191806-ff18-4c21-afa7-701c9c9a85fa\") " pod="e2e-models-e2e-worker-w3-542ed5/simulator-w3-542ed5-kserve-77ff54b49-nwttm"
Sep 16 02:32:05.772246 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:32:05.772163    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/99191806-ff18-4c21-afa7-701c9c9a85fa-tmp-dir\") pod \"simulator-w3-542ed5-kserve-77ff54b49-nwttm\" (UID: \"99191806-ff18-4c21-afa7-701c9c9a85fa\") " pod="e2e-models-e2e-worker-w3-542ed5/simulator-w3-542ed5-kserve-77ff54b49-nwttm"
Sep 16 02:32:05.772246 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:32:05.772220    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/99191806-ff18-4c21-afa7-701c9c9a85fa-home\") pod \"simulator-w3-542ed5-kserve-77ff54b49-nwttm\" (UID: \"99191806-ff18-4c21-afa7-701c9c9a85fa\") " pod="e2e-models-e2e-worker-w3-542ed5/simulator-w3-542ed5-kserve-77ff54b49-nwttm"
Sep 16 02:32:05.774167 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:32:05.774137    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/99191806-ff18-4c21-afa7-701c9c9a85fa-dshm\") pod \"simulator-w3-542ed5-kserve-77ff54b49-nwttm\" (UID: \"99191806-ff18-4c21-afa7-701c9c9a85fa\") " pod="e2e-models-e2e-worker-w3-542ed5/simulator-w3-542ed5-kserve-77ff54b49-nwttm"
Sep 16 02:32:05.774374 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:32:05.774355    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/99191806-ff18-4c21-afa7-701c9c9a85fa-tls-certs\") pod \"simulator-w3-542ed5-kserve-77ff54b49-nwttm\" (UID: \"99191806-ff18-4c21-afa7-701c9c9a85fa\") " pod="e2e-models-e2e-worker-w3-542ed5/simulator-w3-542ed5-kserve-77ff54b49-nwttm"
Sep 16 02:32:05.779340 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:32:05.779319    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b8qks\" (UniqueName: \"kubernetes.io/projected/99191806-ff18-4c21-afa7-701c9c9a85fa-kube-api-access-b8qks\") pod \"simulator-w3-542ed5-kserve-77ff54b49-nwttm\" (UID: \"99191806-ff18-4c21-afa7-701c9c9a85fa\") " pod="e2e-models-e2e-worker-w3-542ed5/simulator-w3-542ed5-kserve-77ff54b49-nwttm"
Sep 16 02:32:05.893658 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:32:05.893583    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-542ed5/simulator-w3-542ed5-kserve-77ff54b49-nwttm"
Sep 16 02:32:06.162517 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:32:06.162459    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-542ed5/simulator-w3-542ed5-kserve-77ff54b49-nwttm"]
Sep 16 02:32:06.164902 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:32:06.164873    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod99191806_ff18_4c21_afa7_701c9c9a85fa.slice/crio-2f9f486eadf2682f2cbabe56581e050a665ef9e5ef1eeea139ba307f09117b34 WatchSource:0}: Error finding container 2f9f486eadf2682f2cbabe56581e050a665ef9e5ef1eeea139ba307f09117b34: Status 404 returned error can't find the container with id 2f9f486eadf2682f2cbabe56581e050a665ef9e5ef1eeea139ba307f09117b34
Sep 16 02:32:06.842684 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:32:06.842653    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-542ed5/simulator-w3-542ed5-kserve-77ff54b49-nwttm" event={"ID":"99191806-ff18-4c21-afa7-701c9c9a85fa","Type":"ContainerStarted","Data":"91b0943e1e232cfcd0e07cb06546019798067ce42f9fb2626df3b9b1a16d90a1"}
Sep 16 02:32:06.842684 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:32:06.842691    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-542ed5/simulator-w3-542ed5-kserve-77ff54b49-nwttm" event={"ID":"99191806-ff18-4c21-afa7-701c9c9a85fa","Type":"ContainerStarted","Data":"2f9f486eadf2682f2cbabe56581e050a665ef9e5ef1eeea139ba307f09117b34"}
Sep 16 02:32:06.858701 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:32:06.858655    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w3-542ed5/simulator-w3-542ed5-kserve-77ff54b49-nwttm" podStartSLOduration=1.858640305 podStartE2EDuration="1.858640305s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 02:32:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 02:32:06.857004294 +0000 UTC m=+1147.240519539" watchObservedRunningTime="2026-09-16 02:32:06.858640305 +0000 UTC m=+1147.242155549"
Sep 16 02:32:15.894110 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:32:15.894073    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w3-542ed5/simulator-w3-542ed5-kserve-77ff54b49-nwttm"
Sep 16 02:32:15.894110 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:32:15.894119    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-542ed5/simulator-w3-542ed5-kserve-77ff54b49-nwttm"
Sep 16 02:32:15.906705 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:32:15.906683    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w3-542ed5/simulator-w3-542ed5-kserve-77ff54b49-nwttm"
Sep 16 02:32:16.906315 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:32:16.906281    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-542ed5/simulator-w3-542ed5-kserve-77ff54b49-nwttm"
Sep 16 02:32:31.782753 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:32:31.782665    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-60674e/test-model-60674e-kserve-7d5c4c75b5-8k68k"]
Sep 16 02:32:31.786175 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:32:31.786151    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-60674e/test-model-60674e-kserve-7d5c4c75b5-8k68k"
Sep 16 02:32:31.789452 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:32:31.789413    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-60674e\"/\"openshift-service-ca.crt\""
Sep 16 02:32:31.789452 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:32:31.789425    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-60674e\"/\"test-model-60674e-kserve-self-signed-certs\""
Sep 16 02:32:31.789654 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:32:31.789460    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-60674e\"/\"default-dockercfg-xdwlj\""
Sep 16 02:32:31.789654 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:32:31.789419    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-60674e\"/\"kube-root-ca.crt\""
Sep 16 02:32:31.795763 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:32:31.795738    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-60674e/test-model-60674e-kserve-7d5c4c75b5-8k68k"]
Sep 16 02:32:31.808332 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:32:31.808307    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nzdzc\" (UniqueName: \"kubernetes.io/projected/16bc04a7-12ff-437a-8cc9-65c213ad457e-kube-api-access-nzdzc\") pod \"test-model-60674e-kserve-7d5c4c75b5-8k68k\" (UID: \"16bc04a7-12ff-437a-8cc9-65c213ad457e\") " pod="ai-tenant-e2e-inf-a-60674e/test-model-60674e-kserve-7d5c4c75b5-8k68k"
Sep 16 02:32:31.808514 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:32:31.808353    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/16bc04a7-12ff-437a-8cc9-65c213ad457e-tmp-dir\") pod \"test-model-60674e-kserve-7d5c4c75b5-8k68k\" (UID: \"16bc04a7-12ff-437a-8cc9-65c213ad457e\") " pod="ai-tenant-e2e-inf-a-60674e/test-model-60674e-kserve-7d5c4c75b5-8k68k"
Sep 16 02:32:31.808514 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:32:31.808408    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/16bc04a7-12ff-437a-8cc9-65c213ad457e-home\") pod \"test-model-60674e-kserve-7d5c4c75b5-8k68k\" (UID: \"16bc04a7-12ff-437a-8cc9-65c213ad457e\") " pod="ai-tenant-e2e-inf-a-60674e/test-model-60674e-kserve-7d5c4c75b5-8k68k"
Sep 16 02:32:31.808514 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:32:31.808451    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/16bc04a7-12ff-437a-8cc9-65c213ad457e-dshm\") pod \"test-model-60674e-kserve-7d5c4c75b5-8k68k\" (UID: \"16bc04a7-12ff-437a-8cc9-65c213ad457e\") " pod="ai-tenant-e2e-inf-a-60674e/test-model-60674e-kserve-7d5c4c75b5-8k68k"
Sep 16 02:32:31.808648 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:32:31.808519    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/16bc04a7-12ff-437a-8cc9-65c213ad457e-model-cache\") pod \"test-model-60674e-kserve-7d5c4c75b5-8k68k\" (UID: \"16bc04a7-12ff-437a-8cc9-65c213ad457e\") " pod="ai-tenant-e2e-inf-a-60674e/test-model-60674e-kserve-7d5c4c75b5-8k68k"
Sep 16 02:32:31.808648 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:32:31.808576    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/16bc04a7-12ff-437a-8cc9-65c213ad457e-tls-certs\") pod \"test-model-60674e-kserve-7d5c4c75b5-8k68k\" (UID: \"16bc04a7-12ff-437a-8cc9-65c213ad457e\") " pod="ai-tenant-e2e-inf-a-60674e/test-model-60674e-kserve-7d5c4c75b5-8k68k"
Sep 16 02:32:31.813052 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:32:31.813012    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:32:31.909888 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:32:31.909851    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/16bc04a7-12ff-437a-8cc9-65c213ad457e-home\") pod \"test-model-60674e-kserve-7d5c4c75b5-8k68k\" (UID: \"16bc04a7-12ff-437a-8cc9-65c213ad457e\") " pod="ai-tenant-e2e-inf-a-60674e/test-model-60674e-kserve-7d5c4c75b5-8k68k"
Sep 16 02:32:31.909888 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:32:31.909882    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/16bc04a7-12ff-437a-8cc9-65c213ad457e-dshm\") pod \"test-model-60674e-kserve-7d5c4c75b5-8k68k\" (UID: \"16bc04a7-12ff-437a-8cc9-65c213ad457e\") " pod="ai-tenant-e2e-inf-a-60674e/test-model-60674e-kserve-7d5c4c75b5-8k68k"
Sep 16 02:32:31.910105 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:32:31.910019    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/16bc04a7-12ff-437a-8cc9-65c213ad457e-model-cache\") pod \"test-model-60674e-kserve-7d5c4c75b5-8k68k\" (UID: \"16bc04a7-12ff-437a-8cc9-65c213ad457e\") " pod="ai-tenant-e2e-inf-a-60674e/test-model-60674e-kserve-7d5c4c75b5-8k68k"
Sep 16 02:32:31.910167 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:32:31.910140    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/16bc04a7-12ff-437a-8cc9-65c213ad457e-tls-certs\") pod \"test-model-60674e-kserve-7d5c4c75b5-8k68k\" (UID: \"16bc04a7-12ff-437a-8cc9-65c213ad457e\") " pod="ai-tenant-e2e-inf-a-60674e/test-model-60674e-kserve-7d5c4c75b5-8k68k"
Sep 16 02:32:31.910226 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:32:31.910210    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nzdzc\" (UniqueName: \"kubernetes.io/projected/16bc04a7-12ff-437a-8cc9-65c213ad457e-kube-api-access-nzdzc\") pod \"test-model-60674e-kserve-7d5c4c75b5-8k68k\" (UID: \"16bc04a7-12ff-437a-8cc9-65c213ad457e\") " pod="ai-tenant-e2e-inf-a-60674e/test-model-60674e-kserve-7d5c4c75b5-8k68k"
Sep 16 02:32:31.910276 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:32:31.910255    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/16bc04a7-12ff-437a-8cc9-65c213ad457e-home\") pod \"test-model-60674e-kserve-7d5c4c75b5-8k68k\" (UID: \"16bc04a7-12ff-437a-8cc9-65c213ad457e\") " pod="ai-tenant-e2e-inf-a-60674e/test-model-60674e-kserve-7d5c4c75b5-8k68k"
Sep 16 02:32:31.910276 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:32:31.910269    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/16bc04a7-12ff-437a-8cc9-65c213ad457e-tmp-dir\") pod \"test-model-60674e-kserve-7d5c4c75b5-8k68k\" (UID: \"16bc04a7-12ff-437a-8cc9-65c213ad457e\") " pod="ai-tenant-e2e-inf-a-60674e/test-model-60674e-kserve-7d5c4c75b5-8k68k"
Sep 16 02:32:31.910380 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:32:31.910323    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/16bc04a7-12ff-437a-8cc9-65c213ad457e-model-cache\") pod \"test-model-60674e-kserve-7d5c4c75b5-8k68k\" (UID: \"16bc04a7-12ff-437a-8cc9-65c213ad457e\") " pod="ai-tenant-e2e-inf-a-60674e/test-model-60674e-kserve-7d5c4c75b5-8k68k"
Sep 16 02:32:31.910608 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:32:31.910589    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/16bc04a7-12ff-437a-8cc9-65c213ad457e-tmp-dir\") pod \"test-model-60674e-kserve-7d5c4c75b5-8k68k\" (UID: \"16bc04a7-12ff-437a-8cc9-65c213ad457e\") " pod="ai-tenant-e2e-inf-a-60674e/test-model-60674e-kserve-7d5c4c75b5-8k68k"
Sep 16 02:32:31.912199 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:32:31.912174    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/16bc04a7-12ff-437a-8cc9-65c213ad457e-dshm\") pod \"test-model-60674e-kserve-7d5c4c75b5-8k68k\" (UID: \"16bc04a7-12ff-437a-8cc9-65c213ad457e\") " pod="ai-tenant-e2e-inf-a-60674e/test-model-60674e-kserve-7d5c4c75b5-8k68k"
Sep 16 02:32:31.912572 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:32:31.912555    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/16bc04a7-12ff-437a-8cc9-65c213ad457e-tls-certs\") pod \"test-model-60674e-kserve-7d5c4c75b5-8k68k\" (UID: \"16bc04a7-12ff-437a-8cc9-65c213ad457e\") " pod="ai-tenant-e2e-inf-a-60674e/test-model-60674e-kserve-7d5c4c75b5-8k68k"
Sep 16 02:32:31.917609 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:32:31.917588    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nzdzc\" (UniqueName: \"kubernetes.io/projected/16bc04a7-12ff-437a-8cc9-65c213ad457e-kube-api-access-nzdzc\") pod \"test-model-60674e-kserve-7d5c4c75b5-8k68k\" (UID: \"16bc04a7-12ff-437a-8cc9-65c213ad457e\") " pod="ai-tenant-e2e-inf-a-60674e/test-model-60674e-kserve-7d5c4c75b5-8k68k"
Sep 16 02:32:32.098865 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:32:32.098765    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-60674e/test-model-60674e-kserve-7d5c4c75b5-8k68k"
Sep 16 02:32:32.370385 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:32:32.370359    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-60674e/test-model-60674e-kserve-7d5c4c75b5-8k68k"]
Sep 16 02:32:32.372971 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:32:32.372940    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod16bc04a7_12ff_437a_8cc9_65c213ad457e.slice/crio-d00b91699eaef50807409b9eaff8c127a2fe3baa02d17044bf52627f594ec6ae WatchSource:0}: Error finding container d00b91699eaef50807409b9eaff8c127a2fe3baa02d17044bf52627f594ec6ae: Status 404 returned error can't find the container with id d00b91699eaef50807409b9eaff8c127a2fe3baa02d17044bf52627f594ec6ae
Sep 16 02:32:32.979850 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:32:32.979810    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-60674e/test-model-60674e-kserve-7d5c4c75b5-8k68k" event={"ID":"16bc04a7-12ff-437a-8cc9-65c213ad457e","Type":"ContainerStarted","Data":"4d4a5ae785f733cab01b145dc1032ee21dcd13609b7bada921baf5ec9e1c47de"}
Sep 16 02:32:32.980293 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:32:32.979854    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-60674e/test-model-60674e-kserve-7d5c4c75b5-8k68k" event={"ID":"16bc04a7-12ff-437a-8cc9-65c213ad457e","Type":"ContainerStarted","Data":"d00b91699eaef50807409b9eaff8c127a2fe3baa02d17044bf52627f594ec6ae"}
Sep 16 02:32:33.000276 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:32:33.000219    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-60674e/test-model-60674e-kserve-7d5c4c75b5-8k68k" podStartSLOduration=2.000204641 podStartE2EDuration="2.000204641s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 02:32:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 02:32:32.997814668 +0000 UTC m=+1173.381329915" watchObservedRunningTime="2026-09-16 02:32:33.000204641 +0000 UTC m=+1173.383719885"
Sep 16 02:32:42.103223 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:32:42.103193    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-60674e/test-model-60674e-kserve-7d5c4c75b5-8k68k"
Sep 16 02:32:42.103778 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:32:42.103238    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-60674e/test-model-60674e-kserve-7d5c4c75b5-8k68k"
Sep 16 02:32:42.111743 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:32:42.111723    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-60674e/test-model-60674e-kserve-7d5c4c75b5-8k68k"
Sep 16 02:32:43.039460 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:32:43.039425    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-60674e/test-model-60674e-kserve-7d5c4c75b5-8k68k"
Sep 16 02:32:58.705597 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:32:58.705561    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:33:00.138328 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:33:00.138301    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-c8q8z_ebf846fa-af7d-4fdd-9662-b7c533c36ddd/console-operator/2.log"
Sep 16 02:33:00.141108 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:33:00.141084    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-c8q8z_ebf846fa-af7d-4fdd-9662-b7c533c36ddd/console-operator/2.log"
Sep 16 02:33:05.097023 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:33:05.096991    2579 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-pre-processing-6cd864bc95-mh4jh" secret="" err="secret \"payload-processing-dockercfg-zzx6j\" not found"
Sep 16 02:33:07.450363 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:33:07.450332    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:33:07.500893 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:33:07.500858    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:33:22.379068 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:33:22.379036    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w3-542ed5/premium-w3-542ed5-kserve-56d8467ff6-mqzvc"]
Sep 16 02:33:22.382358 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:33:22.382342    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-542ed5/premium-w3-542ed5-kserve-56d8467ff6-mqzvc"
Sep 16 02:33:22.384531 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:33:22.384511    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-542ed5\"/\"premium-w3-542ed5-kserve-self-signed-certs\""
Sep 16 02:33:22.392333 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:33:22.392309    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-542ed5/premium-w3-542ed5-kserve-56d8467ff6-mqzvc"]
Sep 16 02:33:22.465496 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:33:22.465433    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/01ea8c54-637f-488b-b324-2eba77119902-tmp-dir\") pod \"premium-w3-542ed5-kserve-56d8467ff6-mqzvc\" (UID: \"01ea8c54-637f-488b-b324-2eba77119902\") " pod="e2e-models-e2e-worker-w3-542ed5/premium-w3-542ed5-kserve-56d8467ff6-mqzvc"
Sep 16 02:33:22.465650 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:33:22.465626    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/01ea8c54-637f-488b-b324-2eba77119902-home\") pod \"premium-w3-542ed5-kserve-56d8467ff6-mqzvc\" (UID: \"01ea8c54-637f-488b-b324-2eba77119902\") " pod="e2e-models-e2e-worker-w3-542ed5/premium-w3-542ed5-kserve-56d8467ff6-mqzvc"
Sep 16 02:33:22.465710 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:33:22.465684    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/01ea8c54-637f-488b-b324-2eba77119902-tls-certs\") pod \"premium-w3-542ed5-kserve-56d8467ff6-mqzvc\" (UID: \"01ea8c54-637f-488b-b324-2eba77119902\") " pod="e2e-models-e2e-worker-w3-542ed5/premium-w3-542ed5-kserve-56d8467ff6-mqzvc"
Sep 16 02:33:22.465797 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:33:22.465769    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bf89z\" (UniqueName: \"kubernetes.io/projected/01ea8c54-637f-488b-b324-2eba77119902-kube-api-access-bf89z\") pod \"premium-w3-542ed5-kserve-56d8467ff6-mqzvc\" (UID: \"01ea8c54-637f-488b-b324-2eba77119902\") " pod="e2e-models-e2e-worker-w3-542ed5/premium-w3-542ed5-kserve-56d8467ff6-mqzvc"
Sep 16 02:33:22.465863 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:33:22.465844    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/01ea8c54-637f-488b-b324-2eba77119902-dshm\") pod \"premium-w3-542ed5-kserve-56d8467ff6-mqzvc\" (UID: \"01ea8c54-637f-488b-b324-2eba77119902\") " pod="e2e-models-e2e-worker-w3-542ed5/premium-w3-542ed5-kserve-56d8467ff6-mqzvc"
Sep 16 02:33:22.465929 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:33:22.465878    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/01ea8c54-637f-488b-b324-2eba77119902-model-cache\") pod \"premium-w3-542ed5-kserve-56d8467ff6-mqzvc\" (UID: \"01ea8c54-637f-488b-b324-2eba77119902\") " pod="e2e-models-e2e-worker-w3-542ed5/premium-w3-542ed5-kserve-56d8467ff6-mqzvc"
Sep 16 02:33:22.476899 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:33:22.476868    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:33:22.567010 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:33:22.566975    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/01ea8c54-637f-488b-b324-2eba77119902-home\") pod \"premium-w3-542ed5-kserve-56d8467ff6-mqzvc\" (UID: \"01ea8c54-637f-488b-b324-2eba77119902\") " pod="e2e-models-e2e-worker-w3-542ed5/premium-w3-542ed5-kserve-56d8467ff6-mqzvc"
Sep 16 02:33:22.567186 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:33:22.567028    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/01ea8c54-637f-488b-b324-2eba77119902-tls-certs\") pod \"premium-w3-542ed5-kserve-56d8467ff6-mqzvc\" (UID: \"01ea8c54-637f-488b-b324-2eba77119902\") " pod="e2e-models-e2e-worker-w3-542ed5/premium-w3-542ed5-kserve-56d8467ff6-mqzvc"
Sep 16 02:33:22.567186 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:33:22.567058    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bf89z\" (UniqueName: \"kubernetes.io/projected/01ea8c54-637f-488b-b324-2eba77119902-kube-api-access-bf89z\") pod \"premium-w3-542ed5-kserve-56d8467ff6-mqzvc\" (UID: \"01ea8c54-637f-488b-b324-2eba77119902\") " pod="e2e-models-e2e-worker-w3-542ed5/premium-w3-542ed5-kserve-56d8467ff6-mqzvc"
Sep 16 02:33:22.567186 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:33:22.567087    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/01ea8c54-637f-488b-b324-2eba77119902-dshm\") pod \"premium-w3-542ed5-kserve-56d8467ff6-mqzvc\" (UID: \"01ea8c54-637f-488b-b324-2eba77119902\") " pod="e2e-models-e2e-worker-w3-542ed5/premium-w3-542ed5-kserve-56d8467ff6-mqzvc"
Sep 16 02:33:22.567186 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:33:22.567105    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/01ea8c54-637f-488b-b324-2eba77119902-model-cache\") pod \"premium-w3-542ed5-kserve-56d8467ff6-mqzvc\" (UID: \"01ea8c54-637f-488b-b324-2eba77119902\") " pod="e2e-models-e2e-worker-w3-542ed5/premium-w3-542ed5-kserve-56d8467ff6-mqzvc"
Sep 16 02:33:22.567186 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:33:22.567142    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/01ea8c54-637f-488b-b324-2eba77119902-tmp-dir\") pod \"premium-w3-542ed5-kserve-56d8467ff6-mqzvc\" (UID: \"01ea8c54-637f-488b-b324-2eba77119902\") " pod="e2e-models-e2e-worker-w3-542ed5/premium-w3-542ed5-kserve-56d8467ff6-mqzvc"
Sep 16 02:33:22.567456 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:33:22.567417    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/01ea8c54-637f-488b-b324-2eba77119902-home\") pod \"premium-w3-542ed5-kserve-56d8467ff6-mqzvc\" (UID: \"01ea8c54-637f-488b-b324-2eba77119902\") " pod="e2e-models-e2e-worker-w3-542ed5/premium-w3-542ed5-kserve-56d8467ff6-mqzvc"
Sep 16 02:33:22.567547 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:33:22.567527    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/01ea8c54-637f-488b-b324-2eba77119902-model-cache\") pod \"premium-w3-542ed5-kserve-56d8467ff6-mqzvc\" (UID: \"01ea8c54-637f-488b-b324-2eba77119902\") " pod="e2e-models-e2e-worker-w3-542ed5/premium-w3-542ed5-kserve-56d8467ff6-mqzvc"
Sep 16 02:33:22.567606 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:33:22.567536    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/01ea8c54-637f-488b-b324-2eba77119902-tmp-dir\") pod \"premium-w3-542ed5-kserve-56d8467ff6-mqzvc\" (UID: \"01ea8c54-637f-488b-b324-2eba77119902\") " pod="e2e-models-e2e-worker-w3-542ed5/premium-w3-542ed5-kserve-56d8467ff6-mqzvc"
Sep 16 02:33:22.569737 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:33:22.569713    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/01ea8c54-637f-488b-b324-2eba77119902-dshm\") pod \"premium-w3-542ed5-kserve-56d8467ff6-mqzvc\" (UID: \"01ea8c54-637f-488b-b324-2eba77119902\") " pod="e2e-models-e2e-worker-w3-542ed5/premium-w3-542ed5-kserve-56d8467ff6-mqzvc"
Sep 16 02:33:22.569920 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:33:22.569897    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/01ea8c54-637f-488b-b324-2eba77119902-tls-certs\") pod \"premium-w3-542ed5-kserve-56d8467ff6-mqzvc\" (UID: \"01ea8c54-637f-488b-b324-2eba77119902\") " pod="e2e-models-e2e-worker-w3-542ed5/premium-w3-542ed5-kserve-56d8467ff6-mqzvc"
Sep 16 02:33:22.574428 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:33:22.574402    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bf89z\" (UniqueName: \"kubernetes.io/projected/01ea8c54-637f-488b-b324-2eba77119902-kube-api-access-bf89z\") pod \"premium-w3-542ed5-kserve-56d8467ff6-mqzvc\" (UID: \"01ea8c54-637f-488b-b324-2eba77119902\") " pod="e2e-models-e2e-worker-w3-542ed5/premium-w3-542ed5-kserve-56d8467ff6-mqzvc"
Sep 16 02:33:22.692321 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:33:22.692295    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-542ed5/premium-w3-542ed5-kserve-56d8467ff6-mqzvc"
Sep 16 02:33:22.964009 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:33:22.963986    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-542ed5/premium-w3-542ed5-kserve-56d8467ff6-mqzvc"]
Sep 16 02:33:22.966484 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:33:22.966450    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod01ea8c54_637f_488b_b324_2eba77119902.slice/crio-dc93abd435d484faddd3743aac0fa37119449e6b7e3e2943a9d075e2fa1b2362 WatchSource:0}: Error finding container dc93abd435d484faddd3743aac0fa37119449e6b7e3e2943a9d075e2fa1b2362: Status 404 returned error can't find the container with id dc93abd435d484faddd3743aac0fa37119449e6b7e3e2943a9d075e2fa1b2362
Sep 16 02:33:23.230559 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:33:23.230450    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-542ed5/premium-w3-542ed5-kserve-56d8467ff6-mqzvc" event={"ID":"01ea8c54-637f-488b-b324-2eba77119902","Type":"ContainerStarted","Data":"8ffc3fbf2a3f909e816bb92b5ced4352afc3c33a96fed59f56009d15a09cfba3"}
Sep 16 02:33:23.230559 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:33:23.230524    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-542ed5/premium-w3-542ed5-kserve-56d8467ff6-mqzvc" event={"ID":"01ea8c54-637f-488b-b324-2eba77119902","Type":"ContainerStarted","Data":"dc93abd435d484faddd3743aac0fa37119449e6b7e3e2943a9d075e2fa1b2362"}
Sep 16 02:33:23.247238 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:33:23.247190    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w3-542ed5/premium-w3-542ed5-kserve-56d8467ff6-mqzvc" podStartSLOduration=1.247174821 podStartE2EDuration="1.247174821s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 02:33:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 02:33:23.245053868 +0000 UTC m=+1223.628569108" watchObservedRunningTime="2026-09-16 02:33:23.247174821 +0000 UTC m=+1223.630690065"
Sep 16 02:33:32.693278 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:33:32.693247    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w3-542ed5/premium-w3-542ed5-kserve-56d8467ff6-mqzvc"
Sep 16 02:33:32.693766 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:33:32.693294    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-542ed5/premium-w3-542ed5-kserve-56d8467ff6-mqzvc"
Sep 16 02:33:32.705992 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:33:32.705972    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w3-542ed5/premium-w3-542ed5-kserve-56d8467ff6-mqzvc"
Sep 16 02:33:33.292577 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:33:33.292548    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-542ed5/premium-w3-542ed5-kserve-56d8467ff6-mqzvc"
Sep 16 02:33:44.106587 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:33:44.106542    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:34:35.097171 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:34:35.097138    2579 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-pre-processing-6cd864bc95-mh4jh" secret="" err="secret \"payload-processing-dockercfg-zzx6j\" not found"
Sep 16 02:34:39.702125 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:34:39.702085    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:34:48.476914 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:34:48.476880    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w3-542ed5/distinct-w3-542ed5-kserve-64fdd5d5f-jjxfp"]
Sep 16 02:34:48.480761 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:34:48.480741    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-542ed5/distinct-w3-542ed5-kserve-64fdd5d5f-jjxfp"
Sep 16 02:34:48.482968 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:34:48.482946    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-542ed5\"/\"distinct-w3-542ed5-kserve-self-signed-certs\""
Sep 16 02:34:48.489432 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:34:48.489406    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-542ed5/distinct-w3-542ed5-kserve-64fdd5d5f-jjxfp"]
Sep 16 02:34:48.551381 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:34:48.551349    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/44750d09-f1af-47cb-9a01-ce4b408f092f-dshm\") pod \"distinct-w3-542ed5-kserve-64fdd5d5f-jjxfp\" (UID: \"44750d09-f1af-47cb-9a01-ce4b408f092f\") " pod="e2e-models-e2e-worker-w3-542ed5/distinct-w3-542ed5-kserve-64fdd5d5f-jjxfp"
Sep 16 02:34:48.551581 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:34:48.551397    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/44750d09-f1af-47cb-9a01-ce4b408f092f-model-cache\") pod \"distinct-w3-542ed5-kserve-64fdd5d5f-jjxfp\" (UID: \"44750d09-f1af-47cb-9a01-ce4b408f092f\") " pod="e2e-models-e2e-worker-w3-542ed5/distinct-w3-542ed5-kserve-64fdd5d5f-jjxfp"
Sep 16 02:34:48.551581 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:34:48.551497    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/44750d09-f1af-47cb-9a01-ce4b408f092f-tls-certs\") pod \"distinct-w3-542ed5-kserve-64fdd5d5f-jjxfp\" (UID: \"44750d09-f1af-47cb-9a01-ce4b408f092f\") " pod="e2e-models-e2e-worker-w3-542ed5/distinct-w3-542ed5-kserve-64fdd5d5f-jjxfp"
Sep 16 02:34:48.551581 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:34:48.551538    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/44750d09-f1af-47cb-9a01-ce4b408f092f-home\") pod \"distinct-w3-542ed5-kserve-64fdd5d5f-jjxfp\" (UID: \"44750d09-f1af-47cb-9a01-ce4b408f092f\") " pod="e2e-models-e2e-worker-w3-542ed5/distinct-w3-542ed5-kserve-64fdd5d5f-jjxfp"
Sep 16 02:34:48.551581 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:34:48.551560    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/44750d09-f1af-47cb-9a01-ce4b408f092f-tmp-dir\") pod \"distinct-w3-542ed5-kserve-64fdd5d5f-jjxfp\" (UID: \"44750d09-f1af-47cb-9a01-ce4b408f092f\") " pod="e2e-models-e2e-worker-w3-542ed5/distinct-w3-542ed5-kserve-64fdd5d5f-jjxfp"
Sep 16 02:34:48.551817 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:34:48.551686    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sswhj\" (UniqueName: \"kubernetes.io/projected/44750d09-f1af-47cb-9a01-ce4b408f092f-kube-api-access-sswhj\") pod \"distinct-w3-542ed5-kserve-64fdd5d5f-jjxfp\" (UID: \"44750d09-f1af-47cb-9a01-ce4b408f092f\") " pod="e2e-models-e2e-worker-w3-542ed5/distinct-w3-542ed5-kserve-64fdd5d5f-jjxfp"
Sep 16 02:34:48.580456 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:34:48.580422    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:34:48.652143 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:34:48.652112    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/44750d09-f1af-47cb-9a01-ce4b408f092f-home\") pod \"distinct-w3-542ed5-kserve-64fdd5d5f-jjxfp\" (UID: \"44750d09-f1af-47cb-9a01-ce4b408f092f\") " pod="e2e-models-e2e-worker-w3-542ed5/distinct-w3-542ed5-kserve-64fdd5d5f-jjxfp"
Sep 16 02:34:48.652143 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:34:48.652143    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/44750d09-f1af-47cb-9a01-ce4b408f092f-tmp-dir\") pod \"distinct-w3-542ed5-kserve-64fdd5d5f-jjxfp\" (UID: \"44750d09-f1af-47cb-9a01-ce4b408f092f\") " pod="e2e-models-e2e-worker-w3-542ed5/distinct-w3-542ed5-kserve-64fdd5d5f-jjxfp"
Sep 16 02:34:48.652341 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:34:48.652215    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sswhj\" (UniqueName: \"kubernetes.io/projected/44750d09-f1af-47cb-9a01-ce4b408f092f-kube-api-access-sswhj\") pod \"distinct-w3-542ed5-kserve-64fdd5d5f-jjxfp\" (UID: \"44750d09-f1af-47cb-9a01-ce4b408f092f\") " pod="e2e-models-e2e-worker-w3-542ed5/distinct-w3-542ed5-kserve-64fdd5d5f-jjxfp"
Sep 16 02:34:48.652341 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:34:48.652269    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/44750d09-f1af-47cb-9a01-ce4b408f092f-dshm\") pod \"distinct-w3-542ed5-kserve-64fdd5d5f-jjxfp\" (UID: \"44750d09-f1af-47cb-9a01-ce4b408f092f\") " pod="e2e-models-e2e-worker-w3-542ed5/distinct-w3-542ed5-kserve-64fdd5d5f-jjxfp"
Sep 16 02:34:48.652341 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:34:48.652300    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/44750d09-f1af-47cb-9a01-ce4b408f092f-model-cache\") pod \"distinct-w3-542ed5-kserve-64fdd5d5f-jjxfp\" (UID: \"44750d09-f1af-47cb-9a01-ce4b408f092f\") " pod="e2e-models-e2e-worker-w3-542ed5/distinct-w3-542ed5-kserve-64fdd5d5f-jjxfp"
Sep 16 02:34:48.652341 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:34:48.652332    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/44750d09-f1af-47cb-9a01-ce4b408f092f-tls-certs\") pod \"distinct-w3-542ed5-kserve-64fdd5d5f-jjxfp\" (UID: \"44750d09-f1af-47cb-9a01-ce4b408f092f\") " pod="e2e-models-e2e-worker-w3-542ed5/distinct-w3-542ed5-kserve-64fdd5d5f-jjxfp"
Sep 16 02:34:48.652577 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:34:48.652550    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/44750d09-f1af-47cb-9a01-ce4b408f092f-home\") pod \"distinct-w3-542ed5-kserve-64fdd5d5f-jjxfp\" (UID: \"44750d09-f1af-47cb-9a01-ce4b408f092f\") " pod="e2e-models-e2e-worker-w3-542ed5/distinct-w3-542ed5-kserve-64fdd5d5f-jjxfp"
Sep 16 02:34:48.652644 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:34:48.652594    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/44750d09-f1af-47cb-9a01-ce4b408f092f-tmp-dir\") pod \"distinct-w3-542ed5-kserve-64fdd5d5f-jjxfp\" (UID: \"44750d09-f1af-47cb-9a01-ce4b408f092f\") " pod="e2e-models-e2e-worker-w3-542ed5/distinct-w3-542ed5-kserve-64fdd5d5f-jjxfp"
Sep 16 02:34:48.652702 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:34:48.652690    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/44750d09-f1af-47cb-9a01-ce4b408f092f-model-cache\") pod \"distinct-w3-542ed5-kserve-64fdd5d5f-jjxfp\" (UID: \"44750d09-f1af-47cb-9a01-ce4b408f092f\") " pod="e2e-models-e2e-worker-w3-542ed5/distinct-w3-542ed5-kserve-64fdd5d5f-jjxfp"
Sep 16 02:34:48.654758 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:34:48.654738    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/44750d09-f1af-47cb-9a01-ce4b408f092f-dshm\") pod \"distinct-w3-542ed5-kserve-64fdd5d5f-jjxfp\" (UID: \"44750d09-f1af-47cb-9a01-ce4b408f092f\") " pod="e2e-models-e2e-worker-w3-542ed5/distinct-w3-542ed5-kserve-64fdd5d5f-jjxfp"
Sep 16 02:34:48.654888 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:34:48.654873    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/44750d09-f1af-47cb-9a01-ce4b408f092f-tls-certs\") pod \"distinct-w3-542ed5-kserve-64fdd5d5f-jjxfp\" (UID: \"44750d09-f1af-47cb-9a01-ce4b408f092f\") " pod="e2e-models-e2e-worker-w3-542ed5/distinct-w3-542ed5-kserve-64fdd5d5f-jjxfp"
Sep 16 02:34:48.662599 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:34:48.662576    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sswhj\" (UniqueName: \"kubernetes.io/projected/44750d09-f1af-47cb-9a01-ce4b408f092f-kube-api-access-sswhj\") pod \"distinct-w3-542ed5-kserve-64fdd5d5f-jjxfp\" (UID: \"44750d09-f1af-47cb-9a01-ce4b408f092f\") " pod="e2e-models-e2e-worker-w3-542ed5/distinct-w3-542ed5-kserve-64fdd5d5f-jjxfp"
Sep 16 02:34:48.791016 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:34:48.790938    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-542ed5/distinct-w3-542ed5-kserve-64fdd5d5f-jjxfp"
Sep 16 02:34:49.061332 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:34:49.061305    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-542ed5/distinct-w3-542ed5-kserve-64fdd5d5f-jjxfp"]
Sep 16 02:34:49.063161 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:34:49.063133    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod44750d09_f1af_47cb_9a01_ce4b408f092f.slice/crio-8ed3e033c8fd0ca35b23f214af1e18d5f9d5a93f4a1f60e98b382fef7fc629bd WatchSource:0}: Error finding container 8ed3e033c8fd0ca35b23f214af1e18d5f9d5a93f4a1f60e98b382fef7fc629bd: Status 404 returned error can't find the container with id 8ed3e033c8fd0ca35b23f214af1e18d5f9d5a93f4a1f60e98b382fef7fc629bd
Sep 16 02:34:49.676035 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:34:49.675997    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-542ed5/distinct-w3-542ed5-kserve-64fdd5d5f-jjxfp" event={"ID":"44750d09-f1af-47cb-9a01-ce4b408f092f","Type":"ContainerStarted","Data":"e71e2b36976904eff4f8c247ea886a51f195b055480dde7644a5567ec4be98d8"}
Sep 16 02:34:49.676459 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:34:49.676046    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-542ed5/distinct-w3-542ed5-kserve-64fdd5d5f-jjxfp" event={"ID":"44750d09-f1af-47cb-9a01-ce4b408f092f","Type":"ContainerStarted","Data":"8ed3e033c8fd0ca35b23f214af1e18d5f9d5a93f4a1f60e98b382fef7fc629bd"}
Sep 16 02:34:49.692108 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:34:49.692059    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w3-542ed5/distinct-w3-542ed5-kserve-64fdd5d5f-jjxfp" podStartSLOduration=1.6920436749999999 podStartE2EDuration="1.692043675s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 02:34:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 02:34:49.691155503 +0000 UTC m=+1310.074670750" watchObservedRunningTime="2026-09-16 02:34:49.692043675 +0000 UTC m=+1310.075558919"
Sep 16 02:34:58.791655 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:34:58.791616    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w3-542ed5/distinct-w3-542ed5-kserve-64fdd5d5f-jjxfp"
Sep 16 02:34:58.792042 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:34:58.791676    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-542ed5/distinct-w3-542ed5-kserve-64fdd5d5f-jjxfp"
Sep 16 02:34:58.804303 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:34:58.804281    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w3-542ed5/distinct-w3-542ed5-kserve-64fdd5d5f-jjxfp"
Sep 16 02:34:59.739254 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:34:59.739228    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-542ed5/distinct-w3-542ed5-kserve-64fdd5d5f-jjxfp"
Sep 16 02:35:01.805152 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:35:01.805121    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:35:01.909711 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:35:01.909679    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:35:02.239857 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:35:02.239821    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:35:08.009308 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:35:08.009277    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:35:08.064630 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:35:08.064598    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:35:08.248712 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:35:08.248677    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:35:08.545772 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:35:08.545720    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:35:09.242490 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:35:09.242440    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:35:10.138211 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:35:10.138173    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:35:10.736640 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:35:10.736599    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:35:10.830427 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:35:10.830399    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:35:11.001179 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:35:11.001089    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:35:11.734890 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:35:11.734837    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:35:12.529327 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:35:12.529290    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:35:13.434268 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:35:13.434233    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:35:13.536011 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:35:13.535972    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:35:13.593665 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:35:13.593628    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:35:13.867670 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:35:13.867592    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:35:13.970459 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:35:13.970424    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:35:14.231699 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:35:14.231666    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:35:14.882337 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:35:14.882307    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:35:35.301919 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:35:35.301822    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:35:35.924933 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:35:35.924891    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:35:36.829710 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:35:36.829676    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:35:37.435795 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:35:37.435757    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:35:38.340906 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:35:38.340872    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:35:38.958596 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:35:38.958560    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:35:39.734687 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:35:39.734657    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:35:41.998997 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:35:41.998960    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:35:42.654542 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:35:42.654500    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:35:42.724652 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:35:42.724620    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:35:42.784223 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:35:42.784188    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:35:42.842957 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:35:42.842917    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:35:42.911060 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:35:42.910979    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:35:43.008885 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:35:43.008851    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:35:43.111050 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:35:43.111016    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:35:43.232489 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:35:43.232437    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:35:43.428462 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:35:43.428429    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:35:43.522782 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:35:43.522702    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:35:43.625049 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:35:43.625013    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:35:43.937448 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:35:43.937417    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:35:44.063928 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:35:44.063892    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:35:44.225679 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:35:44.225585    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:35:44.863041 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:35:44.863007    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:35:45.469224 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:35:45.469184    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:35:46.407079 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:35:46.407042    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:35:47.134854 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:35:47.134816    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:35:47.234930 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:35:47.234899    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:35:48.018529 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:35:48.018461    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:35:48.940332 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:35:48.940292    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:35:49.571687 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:35:49.571654    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:35:49.631581 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:35:49.631543    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:35:49.732535 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:35:49.732506    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:35:50.015886 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:35:50.015858    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:35:50.099639 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:35:50.099599    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:35:50.148869 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:35:50.148839    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-0bdfbb54-f5857fc4-787jq"]
Sep 16 02:35:50.152401 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:35:50.152375    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-0bdfbb54-f5857fc4-787jq"
Sep 16 02:35:50.154437 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:35:50.154404    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-0bdfbb54\""
Sep 16 02:35:50.154565 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:35:50.154514    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-0bdfbb54-dockercfg-c5d9n\""
Sep 16 02:35:50.161587 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:35:50.161567    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-0bdfbb54-f5857fc4-787jq"]
Sep 16 02:35:50.253852 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:35:50.253819    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ltb59\" (UniqueName: \"kubernetes.io/projected/6eb34ba4-487a-4c81-9e60-470c8faa2795-kube-api-access-ltb59\") pod \"payload-processing-e2e-migrate-0bdfbb54-f5857fc4-787jq\" (UID: \"6eb34ba4-487a-4c81-9e60-470c8faa2795\") " pod="openshift-ingress/payload-processing-e2e-migrate-0bdfbb54-f5857fc4-787jq"
Sep 16 02:35:50.254028 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:35:50.253891    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6eb34ba4-487a-4c81-9e60-470c8faa2795-plugins-config-volume\") pod \"payload-processing-e2e-migrate-0bdfbb54-f5857fc4-787jq\" (UID: \"6eb34ba4-487a-4c81-9e60-470c8faa2795\") " pod="openshift-ingress/payload-processing-e2e-migrate-0bdfbb54-f5857fc4-787jq"
Sep 16 02:35:50.354729 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:35:50.354655    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6eb34ba4-487a-4c81-9e60-470c8faa2795-plugins-config-volume\") pod \"payload-processing-e2e-migrate-0bdfbb54-f5857fc4-787jq\" (UID: \"6eb34ba4-487a-4c81-9e60-470c8faa2795\") " pod="openshift-ingress/payload-processing-e2e-migrate-0bdfbb54-f5857fc4-787jq"
Sep 16 02:35:50.354870 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:35:50.354764    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ltb59\" (UniqueName: \"kubernetes.io/projected/6eb34ba4-487a-4c81-9e60-470c8faa2795-kube-api-access-ltb59\") pod \"payload-processing-e2e-migrate-0bdfbb54-f5857fc4-787jq\" (UID: \"6eb34ba4-487a-4c81-9e60-470c8faa2795\") " pod="openshift-ingress/payload-processing-e2e-migrate-0bdfbb54-f5857fc4-787jq"
Sep 16 02:35:50.355312 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:35:50.355294    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6eb34ba4-487a-4c81-9e60-470c8faa2795-plugins-config-volume\") pod \"payload-processing-e2e-migrate-0bdfbb54-f5857fc4-787jq\" (UID: \"6eb34ba4-487a-4c81-9e60-470c8faa2795\") " pod="openshift-ingress/payload-processing-e2e-migrate-0bdfbb54-f5857fc4-787jq"
Sep 16 02:35:50.362767 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:35:50.362739    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ltb59\" (UniqueName: \"kubernetes.io/projected/6eb34ba4-487a-4c81-9e60-470c8faa2795-kube-api-access-ltb59\") pod \"payload-processing-e2e-migrate-0bdfbb54-f5857fc4-787jq\" (UID: \"6eb34ba4-487a-4c81-9e60-470c8faa2795\") " pod="openshift-ingress/payload-processing-e2e-migrate-0bdfbb54-f5857fc4-787jq"
Sep 16 02:35:50.465123 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:35:50.465089    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-0bdfbb54-f5857fc4-787jq"
Sep 16 02:35:50.526316 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:35:50.526078    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:35:50.738384 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:35:50.738360    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-0bdfbb54-f5857fc4-787jq"]
Sep 16 02:35:50.742084 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:35:50.742059    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6eb34ba4_487a_4c81_9e60_470c8faa2795.slice/crio-216c547bce40ce60520e93a089eadc3d58c2615340bf699fa86aed58646b1bb3 WatchSource:0}: Error finding container 216c547bce40ce60520e93a089eadc3d58c2615340bf699fa86aed58646b1bb3: Status 404 returned error can't find the container with id 216c547bce40ce60520e93a089eadc3d58c2615340bf699fa86aed58646b1bb3
Sep 16 02:35:50.998657 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:35:50.998570    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-0bdfbb54-f5857fc4-787jq" event={"ID":"6eb34ba4-487a-4c81-9e60-470c8faa2795","Type":"ContainerStarted","Data":"8321e28543a23cd0866bf0f1cff47ca9c4d726e6ee913a970343d8d990371c79"}
Sep 16 02:35:50.998657 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:35:50.998619    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-0bdfbb54-f5857fc4-787jq" event={"ID":"6eb34ba4-487a-4c81-9e60-470c8faa2795","Type":"ContainerStarted","Data":"216c547bce40ce60520e93a089eadc3d58c2615340bf699fa86aed58646b1bb3"}
Sep 16 02:35:50.998877 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:35:50.998726    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-0bdfbb54-f5857fc4-787jq"
Sep 16 02:35:51.016233 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:35:51.016188    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-0bdfbb54-f5857fc4-787jq" podStartSLOduration=1.016172729 podStartE2EDuration="1.016172729s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 02:35:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 02:35:51.014355965 +0000 UTC m=+1371.397871213" watchObservedRunningTime="2026-09-16 02:35:51.016172729 +0000 UTC m=+1371.399687974"
Sep 16 02:35:51.258643 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:35:51.258525    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:35:51.966863 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:35:51.966829    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:35:52.062984 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:35:52.062949    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-619550/distinct-2-w0-619550-kserve-6949cd8666-knndp"]
Sep 16 02:35:52.066616 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:35:52.066590    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-619550/distinct-2-w0-619550-kserve-6949cd8666-knndp"
Sep 16 02:35:52.068558 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:35:52.068537    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-619550\"/\"distinct-2-w0-619550-kserve-self-signed-certs\""
Sep 16 02:35:52.075529 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:35:52.075506    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-619550/distinct-2-w0-619550-kserve-6949cd8666-knndp"]
Sep 16 02:35:52.096813 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:35:52.096792    2579 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-pre-processing-6cd864bc95-mh4jh" secret="" err="secret \"payload-processing-dockercfg-zzx6j\" not found"
Sep 16 02:35:52.174183 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:35:52.174145    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/72a6b825-de7c-4806-8db6-f116f9cd615b-dshm\") pod \"distinct-2-w0-619550-kserve-6949cd8666-knndp\" (UID: \"72a6b825-de7c-4806-8db6-f116f9cd615b\") " pod="e2e-models-e2e-worker-w0-619550/distinct-2-w0-619550-kserve-6949cd8666-knndp"
Sep 16 02:35:52.174372 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:35:52.174201    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5csn9\" (UniqueName: \"kubernetes.io/projected/72a6b825-de7c-4806-8db6-f116f9cd615b-kube-api-access-5csn9\") pod \"distinct-2-w0-619550-kserve-6949cd8666-knndp\" (UID: \"72a6b825-de7c-4806-8db6-f116f9cd615b\") " pod="e2e-models-e2e-worker-w0-619550/distinct-2-w0-619550-kserve-6949cd8666-knndp"
Sep 16 02:35:52.174372 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:35:52.174293    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/72a6b825-de7c-4806-8db6-f116f9cd615b-home\") pod \"distinct-2-w0-619550-kserve-6949cd8666-knndp\" (UID: \"72a6b825-de7c-4806-8db6-f116f9cd615b\") " pod="e2e-models-e2e-worker-w0-619550/distinct-2-w0-619550-kserve-6949cd8666-knndp"
Sep 16 02:35:52.174372 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:35:52.174321    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/72a6b825-de7c-4806-8db6-f116f9cd615b-tls-certs\") pod \"distinct-2-w0-619550-kserve-6949cd8666-knndp\" (UID: \"72a6b825-de7c-4806-8db6-f116f9cd615b\") " pod="e2e-models-e2e-worker-w0-619550/distinct-2-w0-619550-kserve-6949cd8666-knndp"
Sep 16 02:35:52.174574 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:35:52.174412    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/72a6b825-de7c-4806-8db6-f116f9cd615b-tmp-dir\") pod \"distinct-2-w0-619550-kserve-6949cd8666-knndp\" (UID: \"72a6b825-de7c-4806-8db6-f116f9cd615b\") " pod="e2e-models-e2e-worker-w0-619550/distinct-2-w0-619550-kserve-6949cd8666-knndp"
Sep 16 02:35:52.174574 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:35:52.174444    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/72a6b825-de7c-4806-8db6-f116f9cd615b-model-cache\") pod \"distinct-2-w0-619550-kserve-6949cd8666-knndp\" (UID: \"72a6b825-de7c-4806-8db6-f116f9cd615b\") " pod="e2e-models-e2e-worker-w0-619550/distinct-2-w0-619550-kserve-6949cd8666-knndp"
Sep 16 02:35:52.275246 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:35:52.275158    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/72a6b825-de7c-4806-8db6-f116f9cd615b-dshm\") pod \"distinct-2-w0-619550-kserve-6949cd8666-knndp\" (UID: \"72a6b825-de7c-4806-8db6-f116f9cd615b\") " pod="e2e-models-e2e-worker-w0-619550/distinct-2-w0-619550-kserve-6949cd8666-knndp"
Sep 16 02:35:52.275246 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:35:52.275207    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5csn9\" (UniqueName: \"kubernetes.io/projected/72a6b825-de7c-4806-8db6-f116f9cd615b-kube-api-access-5csn9\") pod \"distinct-2-w0-619550-kserve-6949cd8666-knndp\" (UID: \"72a6b825-de7c-4806-8db6-f116f9cd615b\") " pod="e2e-models-e2e-worker-w0-619550/distinct-2-w0-619550-kserve-6949cd8666-knndp"
Sep 16 02:35:52.275514 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:35:52.275261    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/72a6b825-de7c-4806-8db6-f116f9cd615b-home\") pod \"distinct-2-w0-619550-kserve-6949cd8666-knndp\" (UID: \"72a6b825-de7c-4806-8db6-f116f9cd615b\") " pod="e2e-models-e2e-worker-w0-619550/distinct-2-w0-619550-kserve-6949cd8666-knndp"
Sep 16 02:35:52.275514 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:35:52.275279    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/72a6b825-de7c-4806-8db6-f116f9cd615b-tls-certs\") pod \"distinct-2-w0-619550-kserve-6949cd8666-knndp\" (UID: \"72a6b825-de7c-4806-8db6-f116f9cd615b\") " pod="e2e-models-e2e-worker-w0-619550/distinct-2-w0-619550-kserve-6949cd8666-knndp"
Sep 16 02:35:52.275514 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:35:52.275325    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/72a6b825-de7c-4806-8db6-f116f9cd615b-tmp-dir\") pod \"distinct-2-w0-619550-kserve-6949cd8666-knndp\" (UID: \"72a6b825-de7c-4806-8db6-f116f9cd615b\") " pod="e2e-models-e2e-worker-w0-619550/distinct-2-w0-619550-kserve-6949cd8666-knndp"
Sep 16 02:35:52.275514 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:35:52.275346    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/72a6b825-de7c-4806-8db6-f116f9cd615b-model-cache\") pod \"distinct-2-w0-619550-kserve-6949cd8666-knndp\" (UID: \"72a6b825-de7c-4806-8db6-f116f9cd615b\") " pod="e2e-models-e2e-worker-w0-619550/distinct-2-w0-619550-kserve-6949cd8666-knndp"
Sep 16 02:35:52.275748 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:35:52.275692    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/72a6b825-de7c-4806-8db6-f116f9cd615b-home\") pod \"distinct-2-w0-619550-kserve-6949cd8666-knndp\" (UID: \"72a6b825-de7c-4806-8db6-f116f9cd615b\") " pod="e2e-models-e2e-worker-w0-619550/distinct-2-w0-619550-kserve-6949cd8666-knndp"
Sep 16 02:35:52.275806 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:35:52.275734    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/72a6b825-de7c-4806-8db6-f116f9cd615b-tmp-dir\") pod \"distinct-2-w0-619550-kserve-6949cd8666-knndp\" (UID: \"72a6b825-de7c-4806-8db6-f116f9cd615b\") " pod="e2e-models-e2e-worker-w0-619550/distinct-2-w0-619550-kserve-6949cd8666-knndp"
Sep 16 02:35:52.275806 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:35:52.275782    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/72a6b825-de7c-4806-8db6-f116f9cd615b-model-cache\") pod \"distinct-2-w0-619550-kserve-6949cd8666-knndp\" (UID: \"72a6b825-de7c-4806-8db6-f116f9cd615b\") " pod="e2e-models-e2e-worker-w0-619550/distinct-2-w0-619550-kserve-6949cd8666-knndp"
Sep 16 02:35:52.278156 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:35:52.278132    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/72a6b825-de7c-4806-8db6-f116f9cd615b-dshm\") pod \"distinct-2-w0-619550-kserve-6949cd8666-knndp\" (UID: \"72a6b825-de7c-4806-8db6-f116f9cd615b\") " pod="e2e-models-e2e-worker-w0-619550/distinct-2-w0-619550-kserve-6949cd8666-knndp"
Sep 16 02:35:52.278438 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:35:52.278419    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/72a6b825-de7c-4806-8db6-f116f9cd615b-tls-certs\") pod \"distinct-2-w0-619550-kserve-6949cd8666-knndp\" (UID: \"72a6b825-de7c-4806-8db6-f116f9cd615b\") " pod="e2e-models-e2e-worker-w0-619550/distinct-2-w0-619550-kserve-6949cd8666-knndp"
Sep 16 02:35:52.283574 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:35:52.283546    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5csn9\" (UniqueName: \"kubernetes.io/projected/72a6b825-de7c-4806-8db6-f116f9cd615b-kube-api-access-5csn9\") pod \"distinct-2-w0-619550-kserve-6949cd8666-knndp\" (UID: \"72a6b825-de7c-4806-8db6-f116f9cd615b\") " pod="e2e-models-e2e-worker-w0-619550/distinct-2-w0-619550-kserve-6949cd8666-knndp"
Sep 16 02:35:52.363539 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:35:52.363508    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:35:52.376387 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:35:52.376335    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-619550/distinct-2-w0-619550-kserve-6949cd8666-knndp"
Sep 16 02:35:52.652923 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:35:52.652901    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-619550/distinct-2-w0-619550-kserve-6949cd8666-knndp"]
Sep 16 02:35:52.655226 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:35:52.655189    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod72a6b825_de7c_4806_8db6_f116f9cd615b.slice/crio-5a5b891957a7cc788945e7586188be3b54a2f4585cfca7e0404e44e76b2ba69a WatchSource:0}: Error finding container 5a5b891957a7cc788945e7586188be3b54a2f4585cfca7e0404e44e76b2ba69a: Status 404 returned error can't find the container with id 5a5b891957a7cc788945e7586188be3b54a2f4585cfca7e0404e44e76b2ba69a
Sep 16 02:35:53.010531 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:35:53.010496    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-619550/distinct-2-w0-619550-kserve-6949cd8666-knndp" event={"ID":"72a6b825-de7c-4806-8db6-f116f9cd615b","Type":"ContainerStarted","Data":"d4aaad0ca86a465ba9be107d4334c51a7278b10c12f93acc0e67c40247429b75"}
Sep 16 02:35:53.010680 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:35:53.010543    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-619550/distinct-2-w0-619550-kserve-6949cd8666-knndp" event={"ID":"72a6b825-de7c-4806-8db6-f116f9cd615b","Type":"ContainerStarted","Data":"5a5b891957a7cc788945e7586188be3b54a2f4585cfca7e0404e44e76b2ba69a"}
Sep 16 02:35:53.030114 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:35:53.030069    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-619550/distinct-2-w0-619550-kserve-6949cd8666-knndp" podStartSLOduration=1.030055122 podStartE2EDuration="1.030055122s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 02:35:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 02:35:53.027123561 +0000 UTC m=+1373.410638806" watchObservedRunningTime="2026-09-16 02:35:53.030055122 +0000 UTC m=+1373.413570366"
Sep 16 02:35:56.853528 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:35:56.853440    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-60674e/test-model-60674e-kserve-7d5c4c75b5-8k68k"]
Sep 16 02:35:56.873673 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:35:56.873642    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:36:02.005464 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:02.005429    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-0bdfbb54-f5857fc4-787jq"
Sep 16 02:36:02.377035 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:02.376935    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-619550/distinct-2-w0-619550-kserve-6949cd8666-knndp"
Sep 16 02:36:02.377035 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:02.376998    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-619550/distinct-2-w0-619550-kserve-6949cd8666-knndp"
Sep 16 02:36:02.389563 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:02.389539    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-619550/distinct-2-w0-619550-kserve-6949cd8666-knndp"
Sep 16 02:36:03.074114 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:03.074086    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-619550/distinct-2-w0-619550-kserve-6949cd8666-knndp"
Sep 16 02:36:08.419759 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:08.419708    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:36:11.476292 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:11.476255    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:36:11.567157 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:11.567126    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:36:11.876163 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:11.876062    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-60674e/test-model-60674e-kserve-7d5c4c75b5-8k68k" podUID="16bc04a7-12ff-437a-8cc9-65c213ad457e" containerName="main" containerID="cri-o://4d4a5ae785f733cab01b145dc1032ee21dcd13609b7bada921baf5ec9e1c47de" gracePeriod=45
Sep 16 02:36:11.943924 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:11.943893    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0bdfbb54-q5j4n"]
Sep 16 02:36:11.947583 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:11.947564    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0bdfbb54-q5j4n"
Sep 16 02:36:11.949781 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:11.949764    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-jpd4f\""
Sep 16 02:36:11.953914 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:11.953888    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0bdfbb54-q5j4n"]
Sep 16 02:36:12.057193 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:12.057090    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/be849b25-20ad-4ea1-8a08-c0c1d35c46bb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-0bdfbb54-q5j4n\" (UID: \"be849b25-20ad-4ea1-8a08-c0c1d35c46bb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0bdfbb54-q5j4n"
Sep 16 02:36:12.114726 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:12.114695    2579 generic.go:356] "Generic (PLEG): container finished" podID="16bc04a7-12ff-437a-8cc9-65c213ad457e" containerID="4d4a5ae785f733cab01b145dc1032ee21dcd13609b7bada921baf5ec9e1c47de" exitCode=0
Sep 16 02:36:12.114874 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:12.114769    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-60674e/test-model-60674e-kserve-7d5c4c75b5-8k68k" event={"ID":"16bc04a7-12ff-437a-8cc9-65c213ad457e","Type":"ContainerDied","Data":"4d4a5ae785f733cab01b145dc1032ee21dcd13609b7bada921baf5ec9e1c47de"}
Sep 16 02:36:12.133106 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:12.133051    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-60674e/test-model-60674e-kserve-7d5c4c75b5-8k68k"
Sep 16 02:36:12.158327 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:12.158299    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/16bc04a7-12ff-437a-8cc9-65c213ad457e-tmp-dir\") pod \"16bc04a7-12ff-437a-8cc9-65c213ad457e\" (UID: \"16bc04a7-12ff-437a-8cc9-65c213ad457e\") "
Sep 16 02:36:12.158460 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:12.158350    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nzdzc\" (UniqueName: \"kubernetes.io/projected/16bc04a7-12ff-437a-8cc9-65c213ad457e-kube-api-access-nzdzc\") pod \"16bc04a7-12ff-437a-8cc9-65c213ad457e\" (UID: \"16bc04a7-12ff-437a-8cc9-65c213ad457e\") "
Sep 16 02:36:12.158460 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:12.158431    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/16bc04a7-12ff-437a-8cc9-65c213ad457e-tls-certs\") pod \"16bc04a7-12ff-437a-8cc9-65c213ad457e\" (UID: \"16bc04a7-12ff-437a-8cc9-65c213ad457e\") "
Sep 16 02:36:12.158601 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:12.158515    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/16bc04a7-12ff-437a-8cc9-65c213ad457e-home\") pod \"16bc04a7-12ff-437a-8cc9-65c213ad457e\" (UID: \"16bc04a7-12ff-437a-8cc9-65c213ad457e\") "
Sep 16 02:36:12.158601 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:12.158570    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/16bc04a7-12ff-437a-8cc9-65c213ad457e-dshm\") pod \"16bc04a7-12ff-437a-8cc9-65c213ad457e\" (UID: \"16bc04a7-12ff-437a-8cc9-65c213ad457e\") "
Sep 16 02:36:12.158713 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:12.158607    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/16bc04a7-12ff-437a-8cc9-65c213ad457e-model-cache\") pod \"16bc04a7-12ff-437a-8cc9-65c213ad457e\" (UID: \"16bc04a7-12ff-437a-8cc9-65c213ad457e\") "
Sep 16 02:36:12.158713 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:12.158655    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/16bc04a7-12ff-437a-8cc9-65c213ad457e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "16bc04a7-12ff-437a-8cc9-65c213ad457e" (UID: "16bc04a7-12ff-437a-8cc9-65c213ad457e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 02:36:12.158840 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:12.158812    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/16bc04a7-12ff-437a-8cc9-65c213ad457e-home" (OuterVolumeSpecName: "home") pod "16bc04a7-12ff-437a-8cc9-65c213ad457e" (UID: "16bc04a7-12ff-437a-8cc9-65c213ad457e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 02:36:12.158959 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:12.158932    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/16bc04a7-12ff-437a-8cc9-65c213ad457e-model-cache" (OuterVolumeSpecName: "model-cache") pod "16bc04a7-12ff-437a-8cc9-65c213ad457e" (UID: "16bc04a7-12ff-437a-8cc9-65c213ad457e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 02:36:12.159103 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:12.159035    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/be849b25-20ad-4ea1-8a08-c0c1d35c46bb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-0bdfbb54-q5j4n\" (UID: \"be849b25-20ad-4ea1-8a08-c0c1d35c46bb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0bdfbb54-q5j4n"
Sep 16 02:36:12.159342 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:12.159172    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/16bc04a7-12ff-437a-8cc9-65c213ad457e-model-cache\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:36:12.159342 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:12.159184    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/16bc04a7-12ff-437a-8cc9-65c213ad457e-tmp-dir\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:36:12.159342 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:12.159192    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/16bc04a7-12ff-437a-8cc9-65c213ad457e-home\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:36:12.159756 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:12.159716    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/be849b25-20ad-4ea1-8a08-c0c1d35c46bb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-0bdfbb54-q5j4n\" (UID: \"be849b25-20ad-4ea1-8a08-c0c1d35c46bb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0bdfbb54-q5j4n"
Sep 16 02:36:12.161387 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:12.161355    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/16bc04a7-12ff-437a-8cc9-65c213ad457e-kube-api-access-nzdzc" (OuterVolumeSpecName: "kube-api-access-nzdzc") pod "16bc04a7-12ff-437a-8cc9-65c213ad457e" (UID: "16bc04a7-12ff-437a-8cc9-65c213ad457e"). InnerVolumeSpecName "kube-api-access-nzdzc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 02:36:12.161792 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:12.161771    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/16bc04a7-12ff-437a-8cc9-65c213ad457e-dshm" (OuterVolumeSpecName: "dshm") pod "16bc04a7-12ff-437a-8cc9-65c213ad457e" (UID: "16bc04a7-12ff-437a-8cc9-65c213ad457e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 02:36:12.162036 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:12.162016    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/16bc04a7-12ff-437a-8cc9-65c213ad457e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "16bc04a7-12ff-437a-8cc9-65c213ad457e" (UID: "16bc04a7-12ff-437a-8cc9-65c213ad457e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 02:36:12.259248 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:12.259216    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0bdfbb54-q5j4n"
Sep 16 02:36:12.260759 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:12.260736    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nzdzc\" (UniqueName: \"kubernetes.io/projected/16bc04a7-12ff-437a-8cc9-65c213ad457e-kube-api-access-nzdzc\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:36:12.260868 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:12.260763    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/16bc04a7-12ff-437a-8cc9-65c213ad457e-tls-certs\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:36:12.260868 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:12.260773    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/16bc04a7-12ff-437a-8cc9-65c213ad457e-dshm\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:36:12.435152 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:12.435115    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:36:12.536077 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:12.536053    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0bdfbb54-q5j4n"]
Sep 16 02:36:12.537770 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:36:12.537746    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe849b25_20ad_4ea1_8a08_c0c1d35c46bb.slice/crio-12f9833f8a027add3190ca48180f6461cb33894561ec7768941d2a6d8f137180 WatchSource:0}: Error finding container 12f9833f8a027add3190ca48180f6461cb33894561ec7768941d2a6d8f137180: Status 404 returned error can't find the container with id 12f9833f8a027add3190ca48180f6461cb33894561ec7768941d2a6d8f137180
Sep 16 02:36:12.539785 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:12.539768    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 16 02:36:13.120730 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:13.120695    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0bdfbb54-q5j4n" event={"ID":"be849b25-20ad-4ea1-8a08-c0c1d35c46bb","Type":"ContainerStarted","Data":"12f9833f8a027add3190ca48180f6461cb33894561ec7768941d2a6d8f137180"}
Sep 16 02:36:13.122076 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:13.122052    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-60674e/test-model-60674e-kserve-7d5c4c75b5-8k68k"
Sep 16 02:36:13.122197 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:13.122083    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-60674e/test-model-60674e-kserve-7d5c4c75b5-8k68k" event={"ID":"16bc04a7-12ff-437a-8cc9-65c213ad457e","Type":"ContainerDied","Data":"d00b91699eaef50807409b9eaff8c127a2fe3baa02d17044bf52627f594ec6ae"}
Sep 16 02:36:13.122197 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:13.122126    2579 scope.go:117] "RemoveContainer" containerID="4d4a5ae785f733cab01b145dc1032ee21dcd13609b7bada921baf5ec9e1c47de"
Sep 16 02:36:13.143358 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:13.143336    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-60674e/test-model-60674e-kserve-7d5c4c75b5-8k68k"]
Sep 16 02:36:13.146778 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:13.146755    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-60674e/test-model-60674e-kserve-7d5c4c75b5-8k68k"]
Sep 16 02:36:13.307632 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:13.307590    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:36:14.053347 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:14.053312    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:36:14.103434 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:14.103399    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="16bc04a7-12ff-437a-8cc9-65c213ad457e" path="/var/lib/kubelet/pods/16bc04a7-12ff-437a-8cc9-65c213ad457e/volumes"
Sep 16 02:36:14.934105 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:14.934023    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:36:15.634539 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:15.634509    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:36:16.139934 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:16.139895    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0bdfbb54-q5j4n" event={"ID":"be849b25-20ad-4ea1-8a08-c0c1d35c46bb","Type":"ContainerStarted","Data":"dd7267484e97e8deb82d4f5e2a83998ce34af57a9a408c30d8283536e938bfc6"}
Sep 16 02:36:16.155591 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:16.155543    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0bdfbb54-q5j4n" podStartSLOduration=1.991125458 podStartE2EDuration="5.155528152s" totalImagesPullingTime="3.164402694s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 02:36:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 02:36:16.153128496 +0000 UTC m=+1396.536643742" watchObservedRunningTime="2026-09-16 02:36:16.155528152 +0000 UTC m=+1396.539043397"
Sep 16 02:36:17.146542 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:17.146507    2579 generic.go:356] "Generic (PLEG): container finished" podID="be849b25-20ad-4ea1-8a08-c0c1d35c46bb" containerID="dd7267484e97e8deb82d4f5e2a83998ce34af57a9a408c30d8283536e938bfc6" exitCode=0
Sep 16 02:36:17.146910 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:17.146590    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0bdfbb54-q5j4n" event={"ID":"be849b25-20ad-4ea1-8a08-c0c1d35c46bb","Type":"ContainerDied","Data":"dd7267484e97e8deb82d4f5e2a83998ce34af57a9a408c30d8283536e938bfc6"}
Sep 16 02:36:18.296242 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:18.296215    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0bdfbb54-q5j4n"
Sep 16 02:36:18.419077 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:18.418996    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/be849b25-20ad-4ea1-8a08-c0c1d35c46bb-maas-api-service-ca\") pod \"be849b25-20ad-4ea1-8a08-c0c1d35c46bb\" (UID: \"be849b25-20ad-4ea1-8a08-c0c1d35c46bb\") "
Sep 16 02:36:18.419324 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:18.419304    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/be849b25-20ad-4ea1-8a08-c0c1d35c46bb-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "be849b25-20ad-4ea1-8a08-c0c1d35c46bb" (UID: "be849b25-20ad-4ea1-8a08-c0c1d35c46bb"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 02:36:18.519925 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:18.519896    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/be849b25-20ad-4ea1-8a08-c0c1d35c46bb-maas-api-service-ca\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:36:19.158658 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:19.158623    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0bdfbb54-q5j4n"
Sep 16 02:36:19.158658 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:19.158653    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0bdfbb54-q5j4n" event={"ID":"be849b25-20ad-4ea1-8a08-c0c1d35c46bb","Type":"ContainerDied","Data":"12f9833f8a027add3190ca48180f6461cb33894561ec7768941d2a6d8f137180"}
Sep 16 02:36:19.158882 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:19.158707    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="12f9833f8a027add3190ca48180f6461cb33894561ec7768941d2a6d8f137180"
Sep 16 02:36:21.819529 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:21.819464    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:36:22.360944 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:22.360905    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:36:22.389081 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:22.389051    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-60674e-f95445b5c-x5gpp"]
Sep 16 02:36:22.389363 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:22.389333    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-60674e-f95445b5c-x5gpp" podUID="34acbdfe-3681-424d-b1f1-ecb47fde6a7d" containerName="payload-processing" containerID="cri-o://77ebcd2a77c28de1318d306c4c946275ede245c102b7ce5d0ebb0b9f433267c9" gracePeriod=30
Sep 16 02:36:22.395005 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:22.394823    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-60674e-69589755b6-wpf7z"]
Sep 16 02:36:22.395164 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:22.395098    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-60674e-69589755b6-wpf7z" podUID="7e347e28-a5be-4179-8f6e-9e8107b979a2" containerName="payload-pre-processing" containerID="cri-o://57f4b7db4002e499803dfe1f22543619457d17b19c6412333ac30679fa0b277e" gracePeriod=30
Sep 16 02:36:22.428145 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:22.428114    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:36:22.625165 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:22.625075    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:36:22.678498 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:22.678449    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:36:22.695800 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:22.695765    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-60674e-f95445b5c-x5gpp"
Sep 16 02:36:22.699268 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:22.699251    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-60674e-69589755b6-wpf7z"
Sep 16 02:36:22.762665 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:22.762624    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c9ttp\" (UniqueName: \"kubernetes.io/projected/34acbdfe-3681-424d-b1f1-ecb47fde6a7d-kube-api-access-c9ttp\") pod \"34acbdfe-3681-424d-b1f1-ecb47fde6a7d\" (UID: \"34acbdfe-3681-424d-b1f1-ecb47fde6a7d\") "
Sep 16 02:36:22.762853 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:22.762718    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/34acbdfe-3681-424d-b1f1-ecb47fde6a7d-plugins-config-volume\") pod \"34acbdfe-3681-424d-b1f1-ecb47fde6a7d\" (UID: \"34acbdfe-3681-424d-b1f1-ecb47fde6a7d\") "
Sep 16 02:36:22.762853 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:22.762762    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cbq4b\" (UniqueName: \"kubernetes.io/projected/7e347e28-a5be-4179-8f6e-9e8107b979a2-kube-api-access-cbq4b\") pod \"7e347e28-a5be-4179-8f6e-9e8107b979a2\" (UID: \"7e347e28-a5be-4179-8f6e-9e8107b979a2\") "
Sep 16 02:36:22.762853 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:22.762808    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7e347e28-a5be-4179-8f6e-9e8107b979a2-plugins-config-volume\") pod \"7e347e28-a5be-4179-8f6e-9e8107b979a2\" (UID: \"7e347e28-a5be-4179-8f6e-9e8107b979a2\") "
Sep 16 02:36:22.763126 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:22.763097    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/34acbdfe-3681-424d-b1f1-ecb47fde6a7d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "34acbdfe-3681-424d-b1f1-ecb47fde6a7d" (UID: "34acbdfe-3681-424d-b1f1-ecb47fde6a7d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 02:36:22.763379 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:22.763352    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7e347e28-a5be-4179-8f6e-9e8107b979a2-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "7e347e28-a5be-4179-8f6e-9e8107b979a2" (UID: "7e347e28-a5be-4179-8f6e-9e8107b979a2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 02:36:22.765156 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:22.765132    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e347e28-a5be-4179-8f6e-9e8107b979a2-kube-api-access-cbq4b" (OuterVolumeSpecName: "kube-api-access-cbq4b") pod "7e347e28-a5be-4179-8f6e-9e8107b979a2" (UID: "7e347e28-a5be-4179-8f6e-9e8107b979a2"). InnerVolumeSpecName "kube-api-access-cbq4b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 02:36:22.765296 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:22.765161    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/34acbdfe-3681-424d-b1f1-ecb47fde6a7d-kube-api-access-c9ttp" (OuterVolumeSpecName: "kube-api-access-c9ttp") pod "34acbdfe-3681-424d-b1f1-ecb47fde6a7d" (UID: "34acbdfe-3681-424d-b1f1-ecb47fde6a7d"). InnerVolumeSpecName "kube-api-access-c9ttp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 02:36:22.863682 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:22.863647    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/34acbdfe-3681-424d-b1f1-ecb47fde6a7d-plugins-config-volume\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:36:22.863682 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:22.863671    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cbq4b\" (UniqueName: \"kubernetes.io/projected/7e347e28-a5be-4179-8f6e-9e8107b979a2-kube-api-access-cbq4b\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:36:22.863682 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:22.863682    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7e347e28-a5be-4179-8f6e-9e8107b979a2-plugins-config-volume\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:36:22.863682 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:22.863691    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c9ttp\" (UniqueName: \"kubernetes.io/projected/34acbdfe-3681-424d-b1f1-ecb47fde6a7d-kube-api-access-c9ttp\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:36:22.892688 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:22.892600    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:36:22.991533 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:22.991495    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:36:23.182719 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:23.182644    2579 generic.go:356] "Generic (PLEG): container finished" podID="34acbdfe-3681-424d-b1f1-ecb47fde6a7d" containerID="77ebcd2a77c28de1318d306c4c946275ede245c102b7ce5d0ebb0b9f433267c9" exitCode=0
Sep 16 02:36:23.182719 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:23.182683    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-60674e-f95445b5c-x5gpp"
Sep 16 02:36:23.182919 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:23.182775    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-60674e-f95445b5c-x5gpp" event={"ID":"34acbdfe-3681-424d-b1f1-ecb47fde6a7d","Type":"ContainerDied","Data":"77ebcd2a77c28de1318d306c4c946275ede245c102b7ce5d0ebb0b9f433267c9"}
Sep 16 02:36:23.182919 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:23.182835    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-60674e-f95445b5c-x5gpp" event={"ID":"34acbdfe-3681-424d-b1f1-ecb47fde6a7d","Type":"ContainerDied","Data":"a5e3866f8594c97e3df5b590a15184116c815ab61aadd3aac8de177425a35b3d"}
Sep 16 02:36:23.182919 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:23.182871    2579 scope.go:117] "RemoveContainer" containerID="77ebcd2a77c28de1318d306c4c946275ede245c102b7ce5d0ebb0b9f433267c9"
Sep 16 02:36:23.184291 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:23.184261    2579 generic.go:356] "Generic (PLEG): container finished" podID="7e347e28-a5be-4179-8f6e-9e8107b979a2" containerID="57f4b7db4002e499803dfe1f22543619457d17b19c6412333ac30679fa0b277e" exitCode=0
Sep 16 02:36:23.184397 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:23.184307    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-60674e-69589755b6-wpf7z" event={"ID":"7e347e28-a5be-4179-8f6e-9e8107b979a2","Type":"ContainerDied","Data":"57f4b7db4002e499803dfe1f22543619457d17b19c6412333ac30679fa0b277e"}
Sep 16 02:36:23.184397 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:23.184335    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-60674e-69589755b6-wpf7z" event={"ID":"7e347e28-a5be-4179-8f6e-9e8107b979a2","Type":"ContainerDied","Data":"794b5c550af327573728d73e34ae99f5de56f2f6dba359eb21189b05ba47800b"}
Sep 16 02:36:23.184397 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:23.184378    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-60674e-69589755b6-wpf7z"
Sep 16 02:36:23.193980 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:23.193962    2579 scope.go:117] "RemoveContainer" containerID="77ebcd2a77c28de1318d306c4c946275ede245c102b7ce5d0ebb0b9f433267c9"
Sep 16 02:36:23.194294 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:36:23.194266    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"77ebcd2a77c28de1318d306c4c946275ede245c102b7ce5d0ebb0b9f433267c9\": container with ID starting with 77ebcd2a77c28de1318d306c4c946275ede245c102b7ce5d0ebb0b9f433267c9 not found: ID does not exist" containerID="77ebcd2a77c28de1318d306c4c946275ede245c102b7ce5d0ebb0b9f433267c9"
Sep 16 02:36:23.194411 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:23.194305    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"77ebcd2a77c28de1318d306c4c946275ede245c102b7ce5d0ebb0b9f433267c9"} err="failed to get container status \"77ebcd2a77c28de1318d306c4c946275ede245c102b7ce5d0ebb0b9f433267c9\": rpc error: code = NotFound desc = could not find container \"77ebcd2a77c28de1318d306c4c946275ede245c102b7ce5d0ebb0b9f433267c9\": container with ID starting with 77ebcd2a77c28de1318d306c4c946275ede245c102b7ce5d0ebb0b9f433267c9 not found: ID does not exist"
Sep 16 02:36:23.194411 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:23.194330    2579 scope.go:117] "RemoveContainer" containerID="57f4b7db4002e499803dfe1f22543619457d17b19c6412333ac30679fa0b277e"
Sep 16 02:36:23.202982 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:23.202962    2579 scope.go:117] "RemoveContainer" containerID="57f4b7db4002e499803dfe1f22543619457d17b19c6412333ac30679fa0b277e"
Sep 16 02:36:23.203502 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:36:23.203364    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"57f4b7db4002e499803dfe1f22543619457d17b19c6412333ac30679fa0b277e\": container with ID starting with 57f4b7db4002e499803dfe1f22543619457d17b19c6412333ac30679fa0b277e not found: ID does not exist" containerID="57f4b7db4002e499803dfe1f22543619457d17b19c6412333ac30679fa0b277e"
Sep 16 02:36:23.203502 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:23.203406    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"57f4b7db4002e499803dfe1f22543619457d17b19c6412333ac30679fa0b277e"} err="failed to get container status \"57f4b7db4002e499803dfe1f22543619457d17b19c6412333ac30679fa0b277e\": rpc error: code = NotFound desc = could not find container \"57f4b7db4002e499803dfe1f22543619457d17b19c6412333ac30679fa0b277e\": container with ID starting with 57f4b7db4002e499803dfe1f22543619457d17b19c6412333ac30679fa0b277e not found: ID does not exist"
Sep 16 02:36:23.212400 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:23.212275    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-60674e-f95445b5c-x5gpp"]
Sep 16 02:36:23.214274 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:23.214250    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-60674e-f95445b5c-x5gpp"]
Sep 16 02:36:23.222802 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:23.222781    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-60674e-69589755b6-wpf7z"]
Sep 16 02:36:23.226352 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:23.226334    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-60674e-69589755b6-wpf7z"]
Sep 16 02:36:23.545782 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:23.545746    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:36:24.104435 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:24.104404    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="34acbdfe-3681-424d-b1f1-ecb47fde6a7d" path="/var/lib/kubelet/pods/34acbdfe-3681-424d-b1f1-ecb47fde6a7d/volumes"
Sep 16 02:36:24.104825 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:24.104810    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7e347e28-a5be-4179-8f6e-9e8107b979a2" path="/var/lib/kubelet/pods/7e347e28-a5be-4179-8f6e-9e8107b979a2/volumes"
Sep 16 02:36:24.453527 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:24.453495    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:36:25.122322 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:25.122288    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:36:28.263346 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:28.263313    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:36:28.812811 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:28.812782    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:36:28.838776 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:28.838748    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-0bdfbb54-f5857fc4-787jq"]
Sep 16 02:36:28.839020 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:28.838996    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-0bdfbb54-f5857fc4-787jq" podUID="6eb34ba4-487a-4c81-9e60-470c8faa2795" containerName="payload-processing" containerID="cri-o://8321e28543a23cd0866bf0f1cff47ca9c4d726e6ee913a970343d8d990371c79" gracePeriod=30
Sep 16 02:36:28.913889 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:28.913847    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:36:28.996556 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:28.996522    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0bdfbb54-q5j4n"]
Sep 16 02:36:29.000601 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:29.000569    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:36:29.009280 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:29.002823    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0bdfbb54-q5j4n"]
Sep 16 02:36:29.072882 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:29.072764    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:36:29.111462 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:29.111433    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-0bdfbb54-f5857fc4-787jq"
Sep 16 02:36:29.135598 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:29.135566    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:36:29.222306 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:29.222272    2579 generic.go:356] "Generic (PLEG): container finished" podID="6eb34ba4-487a-4c81-9e60-470c8faa2795" containerID="8321e28543a23cd0866bf0f1cff47ca9c4d726e6ee913a970343d8d990371c79" exitCode=0
Sep 16 02:36:29.222517 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:29.222319    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-0bdfbb54-f5857fc4-787jq"
Sep 16 02:36:29.222517 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:29.222355    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-0bdfbb54-f5857fc4-787jq" event={"ID":"6eb34ba4-487a-4c81-9e60-470c8faa2795","Type":"ContainerDied","Data":"8321e28543a23cd0866bf0f1cff47ca9c4d726e6ee913a970343d8d990371c79"}
Sep 16 02:36:29.222517 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:29.222437    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-0bdfbb54-f5857fc4-787jq" event={"ID":"6eb34ba4-487a-4c81-9e60-470c8faa2795","Type":"ContainerDied","Data":"216c547bce40ce60520e93a089eadc3d58c2615340bf699fa86aed58646b1bb3"}
Sep 16 02:36:29.222517 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:29.222495    2579 scope.go:117] "RemoveContainer" containerID="8321e28543a23cd0866bf0f1cff47ca9c4d726e6ee913a970343d8d990371c79"
Sep 16 02:36:29.227983 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:29.227965    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6eb34ba4-487a-4c81-9e60-470c8faa2795-plugins-config-volume\") pod \"6eb34ba4-487a-4c81-9e60-470c8faa2795\" (UID: \"6eb34ba4-487a-4c81-9e60-470c8faa2795\") "
Sep 16 02:36:29.228106 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:29.228008    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ltb59\" (UniqueName: \"kubernetes.io/projected/6eb34ba4-487a-4c81-9e60-470c8faa2795-kube-api-access-ltb59\") pod \"6eb34ba4-487a-4c81-9e60-470c8faa2795\" (UID: \"6eb34ba4-487a-4c81-9e60-470c8faa2795\") "
Sep 16 02:36:29.228322 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:29.228302    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6eb34ba4-487a-4c81-9e60-470c8faa2795-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "6eb34ba4-487a-4c81-9e60-470c8faa2795" (UID: "6eb34ba4-487a-4c81-9e60-470c8faa2795"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 02:36:29.230238 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:29.230212    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6eb34ba4-487a-4c81-9e60-470c8faa2795-kube-api-access-ltb59" (OuterVolumeSpecName: "kube-api-access-ltb59") pod "6eb34ba4-487a-4c81-9e60-470c8faa2795" (UID: "6eb34ba4-487a-4c81-9e60-470c8faa2795"). InnerVolumeSpecName "kube-api-access-ltb59". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 02:36:29.231541 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:29.231520    2579 scope.go:117] "RemoveContainer" containerID="8321e28543a23cd0866bf0f1cff47ca9c4d726e6ee913a970343d8d990371c79"
Sep 16 02:36:29.231828 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:36:29.231811    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8321e28543a23cd0866bf0f1cff47ca9c4d726e6ee913a970343d8d990371c79\": container with ID starting with 8321e28543a23cd0866bf0f1cff47ca9c4d726e6ee913a970343d8d990371c79 not found: ID does not exist" containerID="8321e28543a23cd0866bf0f1cff47ca9c4d726e6ee913a970343d8d990371c79"
Sep 16 02:36:29.231897 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:29.231836    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8321e28543a23cd0866bf0f1cff47ca9c4d726e6ee913a970343d8d990371c79"} err="failed to get container status \"8321e28543a23cd0866bf0f1cff47ca9c4d726e6ee913a970343d8d990371c79\": rpc error: code = NotFound desc = could not find container \"8321e28543a23cd0866bf0f1cff47ca9c4d726e6ee913a970343d8d990371c79\": container with ID starting with 8321e28543a23cd0866bf0f1cff47ca9c4d726e6ee913a970343d8d990371c79 not found: ID does not exist"
Sep 16 02:36:29.240657 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:29.240628    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:36:29.329747 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:29.329673    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6eb34ba4-487a-4c81-9e60-470c8faa2795-plugins-config-volume\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:36:29.329747 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:29.329703    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ltb59\" (UniqueName: \"kubernetes.io/projected/6eb34ba4-487a-4c81-9e60-470c8faa2795-kube-api-access-ltb59\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:36:29.342881 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:29.342844    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:36:29.442337 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:29.442301    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:36:29.518645 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:29.518612    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:36:29.544232 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:29.544203    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-0bdfbb54-f5857fc4-787jq"]
Sep 16 02:36:29.548661 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:29.548638    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-0bdfbb54-f5857fc4-787jq"]
Sep 16 02:36:29.576218 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:29.576189    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:36:29.877411 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:29.877375    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:36:30.103205 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:30.103173    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6eb34ba4-487a-4c81-9e60-470c8faa2795" path="/var/lib/kubelet/pods/6eb34ba4-487a-4c81-9e60-470c8faa2795/volumes"
Sep 16 02:36:30.103589 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:30.103574    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="be849b25-20ad-4ea1-8a08-c0c1d35c46bb" path="/var/lib/kubelet/pods/be849b25-20ad-4ea1-8a08-c0c1d35c46bb/volumes"
Sep 16 02:36:30.270308 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:30.270275    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:36:30.625527 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:30.625420    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:36:31.515656 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:31.515624    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:36:32.170264 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:32.170229    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:36:32.428957 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:32.428865    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:36:32.519820 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:32.519790    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:36:33.056617 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:33.056585    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:36:34.341638 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:34.341597    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:36:35.368781 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:35.368749    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:36:35.469682 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:35.469650    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:36:35.815884 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:35.815846    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:36:35.916833 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:35.916802    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:36:35.989028 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:35.988988    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:36:40.092230 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:40.092195    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:36:40.673805 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:40.673760    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:36:40.689423 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:40.689399    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-1a323c-5d57f9fdfc-fcm9x"]
Sep 16 02:36:40.689643 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:40.689619    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-1a323c-5d57f9fdfc-fcm9x" podUID="c03ab1c3-17db-4555-99aa-f3ab4a2f11f0" containerName="payload-processing" containerID="cri-o://176f9e73189f12784e44ca0ad56cb90df6c3aa84fe24f8525b9ba6438d418497" gracePeriod=30
Sep 16 02:36:40.706551 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:40.706520    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-1a323c-dc54d6f4-c5k58"]
Sep 16 02:36:40.706942 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:40.706913    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-1a323c-dc54d6f4-c5k58" podUID="6f71a2e1-266a-40ff-b337-7822a912a8b0" containerName="payload-pre-processing" containerID="cri-o://168bdcfb558c3ccad944b529e6b9145c8c41a7c7b32d1a301102b3d622364ec2" gracePeriod=30
Sep 16 02:36:40.743239 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:40.743199    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:36:40.840807 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:40.840729    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:36:40.894807 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:40.894764    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:36:41.000218 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:41.000197    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-1a323c-5d57f9fdfc-fcm9x"
Sep 16 02:36:41.003022 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:41.003002    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-1a323c-dc54d6f4-c5k58"
Sep 16 02:36:41.142958 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:41.142930    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c03ab1c3-17db-4555-99aa-f3ab4a2f11f0-plugins-config-volume\") pod \"c03ab1c3-17db-4555-99aa-f3ab4a2f11f0\" (UID: \"c03ab1c3-17db-4555-99aa-f3ab4a2f11f0\") "
Sep 16 02:36:41.143304 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:41.142976    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6f71a2e1-266a-40ff-b337-7822a912a8b0-plugins-config-volume\") pod \"6f71a2e1-266a-40ff-b337-7822a912a8b0\" (UID: \"6f71a2e1-266a-40ff-b337-7822a912a8b0\") "
Sep 16 02:36:41.143304 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:41.143028    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xxrxz\" (UniqueName: \"kubernetes.io/projected/6f71a2e1-266a-40ff-b337-7822a912a8b0-kube-api-access-xxrxz\") pod \"6f71a2e1-266a-40ff-b337-7822a912a8b0\" (UID: \"6f71a2e1-266a-40ff-b337-7822a912a8b0\") "
Sep 16 02:36:41.143304 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:41.143050    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j5v9m\" (UniqueName: \"kubernetes.io/projected/c03ab1c3-17db-4555-99aa-f3ab4a2f11f0-kube-api-access-j5v9m\") pod \"c03ab1c3-17db-4555-99aa-f3ab4a2f11f0\" (UID: \"c03ab1c3-17db-4555-99aa-f3ab4a2f11f0\") "
Sep 16 02:36:41.143405 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:41.143328    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c03ab1c3-17db-4555-99aa-f3ab4a2f11f0-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c03ab1c3-17db-4555-99aa-f3ab4a2f11f0" (UID: "c03ab1c3-17db-4555-99aa-f3ab4a2f11f0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 02:36:41.143405 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:41.143367    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6f71a2e1-266a-40ff-b337-7822a912a8b0-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "6f71a2e1-266a-40ff-b337-7822a912a8b0" (UID: "6f71a2e1-266a-40ff-b337-7822a912a8b0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 02:36:41.145559 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:41.145529    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c03ab1c3-17db-4555-99aa-f3ab4a2f11f0-kube-api-access-j5v9m" (OuterVolumeSpecName: "kube-api-access-j5v9m") pod "c03ab1c3-17db-4555-99aa-f3ab4a2f11f0" (UID: "c03ab1c3-17db-4555-99aa-f3ab4a2f11f0"). InnerVolumeSpecName "kube-api-access-j5v9m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 02:36:41.145753 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:41.145731    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6f71a2e1-266a-40ff-b337-7822a912a8b0-kube-api-access-xxrxz" (OuterVolumeSpecName: "kube-api-access-xxrxz") pod "6f71a2e1-266a-40ff-b337-7822a912a8b0" (UID: "6f71a2e1-266a-40ff-b337-7822a912a8b0"). InnerVolumeSpecName "kube-api-access-xxrxz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 02:36:41.218769 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:41.218715    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:36:41.243970 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:41.243903    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xxrxz\" (UniqueName: \"kubernetes.io/projected/6f71a2e1-266a-40ff-b337-7822a912a8b0-kube-api-access-xxrxz\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:36:41.243970 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:41.243929    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j5v9m\" (UniqueName: \"kubernetes.io/projected/c03ab1c3-17db-4555-99aa-f3ab4a2f11f0-kube-api-access-j5v9m\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:36:41.243970 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:41.243938    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c03ab1c3-17db-4555-99aa-f3ab4a2f11f0-plugins-config-volume\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:36:41.243970 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:41.243947    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6f71a2e1-266a-40ff-b337-7822a912a8b0-plugins-config-volume\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:36:41.296191 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:41.296161    2579 generic.go:356] "Generic (PLEG): container finished" podID="c03ab1c3-17db-4555-99aa-f3ab4a2f11f0" containerID="176f9e73189f12784e44ca0ad56cb90df6c3aa84fe24f8525b9ba6438d418497" exitCode=0
Sep 16 02:36:41.296361 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:41.296207    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-1a323c-5d57f9fdfc-fcm9x"
Sep 16 02:36:41.296361 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:41.296270    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-1a323c-5d57f9fdfc-fcm9x" event={"ID":"c03ab1c3-17db-4555-99aa-f3ab4a2f11f0","Type":"ContainerDied","Data":"176f9e73189f12784e44ca0ad56cb90df6c3aa84fe24f8525b9ba6438d418497"}
Sep 16 02:36:41.296361 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:41.296337    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-1a323c-5d57f9fdfc-fcm9x" event={"ID":"c03ab1c3-17db-4555-99aa-f3ab4a2f11f0","Type":"ContainerDied","Data":"82c107635a35c875ef07b7593f9a260c12391087fb53d3f3b105ca7ea4a2b383"}
Sep 16 02:36:41.296521 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:41.296371    2579 scope.go:117] "RemoveContainer" containerID="176f9e73189f12784e44ca0ad56cb90df6c3aa84fe24f8525b9ba6438d418497"
Sep 16 02:36:41.297775 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:41.297754    2579 generic.go:356] "Generic (PLEG): container finished" podID="6f71a2e1-266a-40ff-b337-7822a912a8b0" containerID="168bdcfb558c3ccad944b529e6b9145c8c41a7c7b32d1a301102b3d622364ec2" exitCode=0
Sep 16 02:36:41.297882 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:41.297815    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-1a323c-dc54d6f4-c5k58"
Sep 16 02:36:41.297882 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:41.297852    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-1a323c-dc54d6f4-c5k58" event={"ID":"6f71a2e1-266a-40ff-b337-7822a912a8b0","Type":"ContainerDied","Data":"168bdcfb558c3ccad944b529e6b9145c8c41a7c7b32d1a301102b3d622364ec2"}
Sep 16 02:36:41.298130 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:41.297897    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-1a323c-dc54d6f4-c5k58" event={"ID":"6f71a2e1-266a-40ff-b337-7822a912a8b0","Type":"ContainerDied","Data":"4353a57f59e4a60ba3c90992fdf86343e2a2fde9f0c2ec6f0d29cdd7228f6e52"}
Sep 16 02:36:41.306136 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:41.306109    2579 scope.go:117] "RemoveContainer" containerID="176f9e73189f12784e44ca0ad56cb90df6c3aa84fe24f8525b9ba6438d418497"
Sep 16 02:36:41.306432 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:36:41.306408    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"176f9e73189f12784e44ca0ad56cb90df6c3aa84fe24f8525b9ba6438d418497\": container with ID starting with 176f9e73189f12784e44ca0ad56cb90df6c3aa84fe24f8525b9ba6438d418497 not found: ID does not exist" containerID="176f9e73189f12784e44ca0ad56cb90df6c3aa84fe24f8525b9ba6438d418497"
Sep 16 02:36:41.306527 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:41.306444    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"176f9e73189f12784e44ca0ad56cb90df6c3aa84fe24f8525b9ba6438d418497"} err="failed to get container status \"176f9e73189f12784e44ca0ad56cb90df6c3aa84fe24f8525b9ba6438d418497\": rpc error: code = NotFound desc = could not find container \"176f9e73189f12784e44ca0ad56cb90df6c3aa84fe24f8525b9ba6438d418497\": container with ID starting with 176f9e73189f12784e44ca0ad56cb90df6c3aa84fe24f8525b9ba6438d418497 not found: ID does not exist"
Sep 16 02:36:41.306527 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:41.306493    2579 scope.go:117] "RemoveContainer" containerID="168bdcfb558c3ccad944b529e6b9145c8c41a7c7b32d1a301102b3d622364ec2"
Sep 16 02:36:41.315083 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:41.315063    2579 scope.go:117] "RemoveContainer" containerID="168bdcfb558c3ccad944b529e6b9145c8c41a7c7b32d1a301102b3d622364ec2"
Sep 16 02:36:41.315375 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:36:41.315351    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"168bdcfb558c3ccad944b529e6b9145c8c41a7c7b32d1a301102b3d622364ec2\": container with ID starting with 168bdcfb558c3ccad944b529e6b9145c8c41a7c7b32d1a301102b3d622364ec2 not found: ID does not exist" containerID="168bdcfb558c3ccad944b529e6b9145c8c41a7c7b32d1a301102b3d622364ec2"
Sep 16 02:36:41.315463 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:41.315379    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"168bdcfb558c3ccad944b529e6b9145c8c41a7c7b32d1a301102b3d622364ec2"} err="failed to get container status \"168bdcfb558c3ccad944b529e6b9145c8c41a7c7b32d1a301102b3d622364ec2\": rpc error: code = NotFound desc = could not find container \"168bdcfb558c3ccad944b529e6b9145c8c41a7c7b32d1a301102b3d622364ec2\": container with ID starting with 168bdcfb558c3ccad944b529e6b9145c8c41a7c7b32d1a301102b3d622364ec2 not found: ID does not exist"
Sep 16 02:36:41.323010 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:41.322988    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-1a323c-dc54d6f4-c5k58"]
Sep 16 02:36:41.325140 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:41.325119    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-1a323c-dc54d6f4-c5k58"]
Sep 16 02:36:41.332662 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:41.332640    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-1a323c-5d57f9fdfc-fcm9x"]
Sep 16 02:36:41.335771 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:41.335752    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-1a323c-5d57f9fdfc-fcm9x"]
Sep 16 02:36:41.439309 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:41.439253    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:36:42.103285 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:42.103255    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6f71a2e1-266a-40ff-b337-7822a912a8b0" path="/var/lib/kubelet/pods/6f71a2e1-266a-40ff-b337-7822a912a8b0/volumes"
Sep 16 02:36:42.103638 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:42.103624    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c03ab1c3-17db-4555-99aa-f3ab4a2f11f0" path="/var/lib/kubelet/pods/c03ab1c3-17db-4555-99aa-f3ab4a2f11f0/volumes"
Sep 16 02:36:42.117685 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:42.117658    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:36:42.829532 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:42.829463    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:36:47.113416 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:47.113374    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:36:47.307272 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:47.307223    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:36:47.372338 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:47.372245    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:36:47.912610 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:47.912552    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:36:48.205241 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:48.205193    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:36:48.305349 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:48.305310    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:36:52.894714 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:52.894625    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:36:53.722637 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:53.722599    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:36:54.199603 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:54.199559    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:36:54.291825 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:54.291790    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:36:55.409001 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:55.408969    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:36:55.635545 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:55.635503    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:36:56.565279 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:56.565241    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:36:56.672563 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:56.672520    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:36:56.726039 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:56.725999    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:36:57.009649 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:57.009612    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:36:57.108815 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:57.108776    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:36:57.159388 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:57.159354    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-24aabe-5d76d4564c-9blzj"]
Sep 16 02:36:57.159788 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:57.159774    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e347e28-a5be-4179-8f6e-9e8107b979a2" containerName="payload-pre-processing"
Sep 16 02:36:57.159842 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:57.159793    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e347e28-a5be-4179-8f6e-9e8107b979a2" containerName="payload-pre-processing"
Sep 16 02:36:57.159842 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:57.159812    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c03ab1c3-17db-4555-99aa-f3ab4a2f11f0" containerName="payload-processing"
Sep 16 02:36:57.159842 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:57.159817    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="c03ab1c3-17db-4555-99aa-f3ab4a2f11f0" containerName="payload-processing"
Sep 16 02:36:57.159842 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:57.159824    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="16bc04a7-12ff-437a-8cc9-65c213ad457e" containerName="main"
Sep 16 02:36:57.159842 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:57.159830    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="16bc04a7-12ff-437a-8cc9-65c213ad457e" containerName="main"
Sep 16 02:36:57.159842 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:57.159837    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6eb34ba4-487a-4c81-9e60-470c8faa2795" containerName="payload-processing"
Sep 16 02:36:57.159842 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:57.159842    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="6eb34ba4-487a-4c81-9e60-470c8faa2795" containerName="payload-processing"
Sep 16 02:36:57.160116 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:57.159851    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="34acbdfe-3681-424d-b1f1-ecb47fde6a7d" containerName="payload-processing"
Sep 16 02:36:57.160116 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:57.159856    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="34acbdfe-3681-424d-b1f1-ecb47fde6a7d" containerName="payload-processing"
Sep 16 02:36:57.160116 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:57.159879    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6f71a2e1-266a-40ff-b337-7822a912a8b0" containerName="payload-pre-processing"
Sep 16 02:36:57.160116 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:57.159884    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="6f71a2e1-266a-40ff-b337-7822a912a8b0" containerName="payload-pre-processing"
Sep 16 02:36:57.160116 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:57.159891    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="be849b25-20ad-4ea1-8a08-c0c1d35c46bb" containerName="revoke-keys"
Sep 16 02:36:57.160116 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:57.159895    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="be849b25-20ad-4ea1-8a08-c0c1d35c46bb" containerName="revoke-keys"
Sep 16 02:36:57.160116 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:57.159974    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="16bc04a7-12ff-437a-8cc9-65c213ad457e" containerName="main"
Sep 16 02:36:57.160116 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:57.159984    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="6f71a2e1-266a-40ff-b337-7822a912a8b0" containerName="payload-pre-processing"
Sep 16 02:36:57.160116 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:57.159990    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="34acbdfe-3681-424d-b1f1-ecb47fde6a7d" containerName="payload-processing"
Sep 16 02:36:57.160116 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:57.159995    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="7e347e28-a5be-4179-8f6e-9e8107b979a2" containerName="payload-pre-processing"
Sep 16 02:36:57.160116 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:57.160001    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="6eb34ba4-487a-4c81-9e60-470c8faa2795" containerName="payload-processing"
Sep 16 02:36:57.160116 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:57.160007    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="c03ab1c3-17db-4555-99aa-f3ab4a2f11f0" containerName="payload-processing"
Sep 16 02:36:57.160116 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:57.160012    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="be849b25-20ad-4ea1-8a08-c0c1d35c46bb" containerName="revoke-keys"
Sep 16 02:36:57.167455 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:57.167433    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-24aabe-5d76d4564c-9blzj"
Sep 16 02:36:57.169589 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:57.169565    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w2-24aabe\""
Sep 16 02:36:57.169722 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:57.169701    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w2-24aabe-dockercfg-dcwtf\""
Sep 16 02:36:57.171122 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:57.171099    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-24aabe-5d76d4564c-9blzj"]
Sep 16 02:36:57.183368 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:57.183348    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0a019f4b-414a-44fa-9535-e1846e70866c-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-24aabe-5d76d4564c-9blzj\" (UID: \"0a019f4b-414a-44fa-9535-e1846e70866c\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-24aabe-5d76d4564c-9blzj"
Sep 16 02:36:57.183487 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:57.183377    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2lpjk\" (UniqueName: \"kubernetes.io/projected/0a019f4b-414a-44fa-9535-e1846e70866c-kube-api-access-2lpjk\") pod \"payload-processing-e2e-shared-a-w2-24aabe-5d76d4564c-9blzj\" (UID: \"0a019f4b-414a-44fa-9535-e1846e70866c\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-24aabe-5d76d4564c-9blzj"
Sep 16 02:36:57.284994 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:57.284902    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0a019f4b-414a-44fa-9535-e1846e70866c-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-24aabe-5d76d4564c-9blzj\" (UID: \"0a019f4b-414a-44fa-9535-e1846e70866c\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-24aabe-5d76d4564c-9blzj"
Sep 16 02:36:57.284994 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:57.284957    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2lpjk\" (UniqueName: \"kubernetes.io/projected/0a019f4b-414a-44fa-9535-e1846e70866c-kube-api-access-2lpjk\") pod \"payload-processing-e2e-shared-a-w2-24aabe-5d76d4564c-9blzj\" (UID: \"0a019f4b-414a-44fa-9535-e1846e70866c\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-24aabe-5d76d4564c-9blzj"
Sep 16 02:36:57.285854 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:57.285830    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0a019f4b-414a-44fa-9535-e1846e70866c-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-24aabe-5d76d4564c-9blzj\" (UID: \"0a019f4b-414a-44fa-9535-e1846e70866c\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-24aabe-5d76d4564c-9blzj"
Sep 16 02:36:57.292723 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:57.292689    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2lpjk\" (UniqueName: \"kubernetes.io/projected/0a019f4b-414a-44fa-9535-e1846e70866c-kube-api-access-2lpjk\") pod \"payload-processing-e2e-shared-a-w2-24aabe-5d76d4564c-9blzj\" (UID: \"0a019f4b-414a-44fa-9535-e1846e70866c\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-24aabe-5d76d4564c-9blzj"
Sep 16 02:36:57.479130 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:57.479095    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-24aabe-5d76d4564c-9blzj"
Sep 16 02:36:57.524556 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:57.524526    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:36:57.755128 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:57.755104    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-24aabe-5d76d4564c-9blzj"]
Sep 16 02:36:57.757125 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:36:57.757093    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a019f4b_414a_44fa_9535_e1846e70866c.slice/crio-f469a6e69d22d6f098b4c820a38014a993888c3da2f2095390e8e5ef6ffa03fd WatchSource:0}: Error finding container f469a6e69d22d6f098b4c820a38014a993888c3da2f2095390e8e5ef6ffa03fd: Status 404 returned error can't find the container with id f469a6e69d22d6f098b4c820a38014a993888c3da2f2095390e8e5ef6ffa03fd
Sep 16 02:36:57.877989 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:57.877950    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4108a843-ptd2l"]
Sep 16 02:36:57.882173 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:57.882153    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4108a843-ptd2l"
Sep 16 02:36:57.884830 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:57.884614    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-jpd4f\""
Sep 16 02:36:57.887587 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:57.887556    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4108a843-ptd2l"]
Sep 16 02:36:57.992337 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:57.992297    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b24ca5c-48e2-4be6-b2ed-80db57d9ab8b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-4108a843-ptd2l\" (UID: \"4b24ca5c-48e2-4be6-b2ed-80db57d9ab8b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4108a843-ptd2l"
Sep 16 02:36:58.093210 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:58.093111    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b24ca5c-48e2-4be6-b2ed-80db57d9ab8b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-4108a843-ptd2l\" (UID: \"4b24ca5c-48e2-4be6-b2ed-80db57d9ab8b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4108a843-ptd2l"
Sep 16 02:36:58.093794 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:58.093772    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b24ca5c-48e2-4be6-b2ed-80db57d9ab8b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-4108a843-ptd2l\" (UID: \"4b24ca5c-48e2-4be6-b2ed-80db57d9ab8b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4108a843-ptd2l"
Sep 16 02:36:58.199183 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:58.199146    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4108a843-ptd2l"
Sep 16 02:36:58.390777 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:58.390693    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-24aabe-5d76d4564c-9blzj" event={"ID":"0a019f4b-414a-44fa-9535-e1846e70866c","Type":"ContainerStarted","Data":"107d9927529090744e2eda1749c1bd3806e35224e8b5e40992e39b9b5cdfb372"}
Sep 16 02:36:58.390777 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:58.390745    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-24aabe-5d76d4564c-9blzj" event={"ID":"0a019f4b-414a-44fa-9535-e1846e70866c","Type":"ContainerStarted","Data":"f469a6e69d22d6f098b4c820a38014a993888c3da2f2095390e8e5ef6ffa03fd"}
Sep 16 02:36:58.390972 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:58.390792    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-24aabe-5d76d4564c-9blzj"
Sep 16 02:36:58.408100 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:58.408054    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-24aabe-5d76d4564c-9blzj" podStartSLOduration=1.408039808 podStartE2EDuration="1.408039808s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 02:36:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 02:36:58.406518479 +0000 UTC m=+1438.790033727" watchObservedRunningTime="2026-09-16 02:36:58.408039808 +0000 UTC m=+1438.791555053"
Sep 16 02:36:58.473417 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:58.473217    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4108a843-ptd2l"]
Sep 16 02:36:58.475624 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:36:58.475595    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b24ca5c_48e2_4be6_b2ed_80db57d9ab8b.slice/crio-978c55a848a1a8bc67b5b6fe2c88fc6b0770d6e11000563a4cb4703d138dcea4 WatchSource:0}: Error finding container 978c55a848a1a8bc67b5b6fe2c88fc6b0770d6e11000563a4cb4703d138dcea4: Status 404 returned error can't find the container with id 978c55a848a1a8bc67b5b6fe2c88fc6b0770d6e11000563a4cb4703d138dcea4
Sep 16 02:36:58.529154 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:58.529129    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:36:59.398822 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:59.398724    2579 generic.go:356] "Generic (PLEG): container finished" podID="4b24ca5c-48e2-4be6-b2ed-80db57d9ab8b" containerID="019d9b3e5172bc377b7153a02ef38c13fcc39d225fe87625f46a95b9a25bfd77" exitCode=0
Sep 16 02:36:59.399234 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:59.398827    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4108a843-ptd2l" event={"ID":"4b24ca5c-48e2-4be6-b2ed-80db57d9ab8b","Type":"ContainerDied","Data":"019d9b3e5172bc377b7153a02ef38c13fcc39d225fe87625f46a95b9a25bfd77"}
Sep 16 02:36:59.399234 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:59.398880    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4108a843-ptd2l" event={"ID":"4b24ca5c-48e2-4be6-b2ed-80db57d9ab8b","Type":"ContainerStarted","Data":"978c55a848a1a8bc67b5b6fe2c88fc6b0770d6e11000563a4cb4703d138dcea4"}
Sep 16 02:36:59.431650 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:36:59.431619    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:37:00.046908 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:37:00.046866    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:37:00.542160 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:37:00.542134    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4108a843-ptd2l"
Sep 16 02:37:00.619311 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:37:00.619282    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b24ca5c-48e2-4be6-b2ed-80db57d9ab8b-maas-api-service-ca\") pod \"4b24ca5c-48e2-4be6-b2ed-80db57d9ab8b\" (UID: \"4b24ca5c-48e2-4be6-b2ed-80db57d9ab8b\") "
Sep 16 02:37:00.619693 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:37:00.619664    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4b24ca5c-48e2-4be6-b2ed-80db57d9ab8b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4b24ca5c-48e2-4be6-b2ed-80db57d9ab8b" (UID: "4b24ca5c-48e2-4be6-b2ed-80db57d9ab8b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 02:37:00.720839 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:37:00.720811    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b24ca5c-48e2-4be6-b2ed-80db57d9ab8b-maas-api-service-ca\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:37:00.926976 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:37:00.926944    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:37:01.412119 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:37:01.412095    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4108a843-ptd2l"
Sep 16 02:37:01.412119 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:37:01.412095    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4108a843-ptd2l" event={"ID":"4b24ca5c-48e2-4be6-b2ed-80db57d9ab8b","Type":"ContainerDied","Data":"978c55a848a1a8bc67b5b6fe2c88fc6b0770d6e11000563a4cb4703d138dcea4"}
Sep 16 02:37:01.412369 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:37:01.412158    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="978c55a848a1a8bc67b5b6fe2c88fc6b0770d6e11000563a4cb4703d138dcea4"
Sep 16 02:37:04.097866 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:37:04.097835    2579 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-pre-processing-6cd864bc95-mh4jh" secret="" err="secret \"payload-processing-dockercfg-zzx6j\" not found"
Sep 16 02:37:04.123870 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:37:04.123173    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:37:04.195999 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:37:04.195963    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:37:04.255229 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:37:04.255184    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:37:04.300518 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:37:04.300490    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4108a843-ptd2l"]
Sep 16 02:37:04.304367 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:37:04.304340    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4108a843-ptd2l"]
Sep 16 02:37:04.355205 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:37:04.355106    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:37:04.799434 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:37:04.799398    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:37:05.391242 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:37:05.391212    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:37:05.646992 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:37:05.646901    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:37:05.796209 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:37:05.796178    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:37:06.078146 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:37:06.078099    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:37:06.103820 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:37:06.103792    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4b24ca5c-48e2-4be6-b2ed-80db57d9ab8b" path="/var/lib/kubelet/pods/4b24ca5c-48e2-4be6-b2ed-80db57d9ab8b/volumes"
Sep 16 02:37:06.245080 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:37:06.245045    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:37:06.356508 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:37:06.356412    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:37:08.125065 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:37:08.125032    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:37:08.549359 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:37:08.549328    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:37:08.641638 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:37:08.641591    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:37:08.879340 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:37:08.879239    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:37:09.033226 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:37:09.033185    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:37:09.208148 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:37:09.208083    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:37:09.401818 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:37:09.401785    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-24aabe-5d76d4564c-9blzj"
Sep 16 02:37:09.581940 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:37:09.581790    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:37:09.673698 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:37:09.673653    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:37:10.038717 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:37:10.038684    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:37:10.103206 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:37:10.103169    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:37:10.249956 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:37:10.249924    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:37:10.593843 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:37:10.593805    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:37:10.687192 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:37:10.687142    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:37:10.998546 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:37:10.998508    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:37:12.037211 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:37:12.037174    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:37:12.710249 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:37:12.710213    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:37:13.089705 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:37:13.089615    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:37:13.190619 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:37:13.190575    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:37:13.633737 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:37:13.633688    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:37:14.459972 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:37:14.459942    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:37:14.597970 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:37:14.597933    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:37:15.027541 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:37:15.027500    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:37:15.118927 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:37:15.118884    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:37:15.125689 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:37:15.125662    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-a53a44-566f584d6-hnt5w"]
Sep 16 02:37:15.126057 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:37:15.126044    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b24ca5c-48e2-4be6-b2ed-80db57d9ab8b" containerName="revoke-keys"
Sep 16 02:37:15.126113 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:37:15.126059    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b24ca5c-48e2-4be6-b2ed-80db57d9ab8b" containerName="revoke-keys"
Sep 16 02:37:15.126149 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:37:15.126131    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="4b24ca5c-48e2-4be6-b2ed-80db57d9ab8b" containerName="revoke-keys"
Sep 16 02:37:15.130429 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:37:15.130408    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-a53a44-566f584d6-hnt5w"
Sep 16 02:37:15.132723 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:37:15.132694    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w2-a53a44\""
Sep 16 02:37:15.132880 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:37:15.132863    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w2-a53a44-dockercfg-grbm7\""
Sep 16 02:37:15.137754 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:37:15.137734    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-a53a44-566f584d6-hnt5w"]
Sep 16 02:37:15.162867 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:37:15.162841    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-a53a44-6c64b45999-td78d"]
Sep 16 02:37:15.167059 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:37:15.167040    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-a53a44-6c64b45999-td78d"
Sep 16 02:37:15.173069 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:37:15.173047    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-a53a44-6c64b45999-td78d"]
Sep 16 02:37:15.267842 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:37:15.267813    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b8e5853d-4887-4741-ae00-33e0959aafea-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-a53a44-6c64b45999-td78d\" (UID: \"b8e5853d-4887-4741-ae00-33e0959aafea\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-a53a44-6c64b45999-td78d"
Sep 16 02:37:15.268011 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:37:15.267854    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rxffg\" (UniqueName: \"kubernetes.io/projected/b2097872-a162-49ab-95f4-06bd1aa0ed40-kube-api-access-rxffg\") pod \"payload-pre-processing-e2e-shared-b-w2-a53a44-566f584d6-hnt5w\" (UID: \"b2097872-a162-49ab-95f4-06bd1aa0ed40\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-a53a44-566f584d6-hnt5w"
Sep 16 02:37:15.268011 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:37:15.267881    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4rvrp\" (UniqueName: \"kubernetes.io/projected/b8e5853d-4887-4741-ae00-33e0959aafea-kube-api-access-4rvrp\") pod \"payload-processing-e2e-shared-b-w2-a53a44-6c64b45999-td78d\" (UID: \"b8e5853d-4887-4741-ae00-33e0959aafea\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-a53a44-6c64b45999-td78d"
Sep 16 02:37:15.268011 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:37:15.267930    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b2097872-a162-49ab-95f4-06bd1aa0ed40-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-a53a44-566f584d6-hnt5w\" (UID: \"b2097872-a162-49ab-95f4-06bd1aa0ed40\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-a53a44-566f584d6-hnt5w"
Sep 16 02:37:15.369244 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:37:15.369147    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b8e5853d-4887-4741-ae00-33e0959aafea-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-a53a44-6c64b45999-td78d\" (UID: \"b8e5853d-4887-4741-ae00-33e0959aafea\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-a53a44-6c64b45999-td78d"
Sep 16 02:37:15.369244 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:37:15.369216    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rxffg\" (UniqueName: \"kubernetes.io/projected/b2097872-a162-49ab-95f4-06bd1aa0ed40-kube-api-access-rxffg\") pod \"payload-pre-processing-e2e-shared-b-w2-a53a44-566f584d6-hnt5w\" (UID: \"b2097872-a162-49ab-95f4-06bd1aa0ed40\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-a53a44-566f584d6-hnt5w"
Sep 16 02:37:15.369523 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:37:15.369262    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4rvrp\" (UniqueName: \"kubernetes.io/projected/b8e5853d-4887-4741-ae00-33e0959aafea-kube-api-access-4rvrp\") pod \"payload-processing-e2e-shared-b-w2-a53a44-6c64b45999-td78d\" (UID: \"b8e5853d-4887-4741-ae00-33e0959aafea\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-a53a44-6c64b45999-td78d"
Sep 16 02:37:15.369523 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:37:15.369289    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b2097872-a162-49ab-95f4-06bd1aa0ed40-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-a53a44-566f584d6-hnt5w\" (UID: \"b2097872-a162-49ab-95f4-06bd1aa0ed40\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-a53a44-566f584d6-hnt5w"
Sep 16 02:37:15.370134 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:37:15.370111    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b8e5853d-4887-4741-ae00-33e0959aafea-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-a53a44-6c64b45999-td78d\" (UID: \"b8e5853d-4887-4741-ae00-33e0959aafea\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-a53a44-6c64b45999-td78d"
Sep 16 02:37:15.370275 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:37:15.370212    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b2097872-a162-49ab-95f4-06bd1aa0ed40-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-a53a44-566f584d6-hnt5w\" (UID: \"b2097872-a162-49ab-95f4-06bd1aa0ed40\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-a53a44-566f584d6-hnt5w"
Sep 16 02:37:15.377095 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:37:15.377060    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4rvrp\" (UniqueName: \"kubernetes.io/projected/b8e5853d-4887-4741-ae00-33e0959aafea-kube-api-access-4rvrp\") pod \"payload-processing-e2e-shared-b-w2-a53a44-6c64b45999-td78d\" (UID: \"b8e5853d-4887-4741-ae00-33e0959aafea\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-a53a44-6c64b45999-td78d"
Sep 16 02:37:15.377190 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:37:15.377154    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rxffg\" (UniqueName: \"kubernetes.io/projected/b2097872-a162-49ab-95f4-06bd1aa0ed40-kube-api-access-rxffg\") pod \"payload-pre-processing-e2e-shared-b-w2-a53a44-566f584d6-hnt5w\" (UID: \"b2097872-a162-49ab-95f4-06bd1aa0ed40\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-a53a44-566f584d6-hnt5w"
Sep 16 02:37:15.443100 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:37:15.443071    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-a53a44-566f584d6-hnt5w"
Sep 16 02:37:15.478201 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:37:15.478173    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-a53a44-6c64b45999-td78d"
Sep 16 02:37:15.661213 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:37:15.661114    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:37:15.719113 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:37:15.719092    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-a53a44-566f584d6-hnt5w"]
Sep 16 02:37:15.721309 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:37:15.721287    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2097872_a162_49ab_95f4_06bd1aa0ed40.slice/crio-aa227b7b8972906bfffacb913e23747a96b2bc310938f253d87aa78f6eaf3903 WatchSource:0}: Error finding container aa227b7b8972906bfffacb913e23747a96b2bc310938f253d87aa78f6eaf3903: Status 404 returned error can't find the container with id aa227b7b8972906bfffacb913e23747a96b2bc310938f253d87aa78f6eaf3903
Sep 16 02:37:15.766420 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:37:15.766386    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-a53a44-6c64b45999-td78d"]
Sep 16 02:37:15.768305 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:37:15.768267    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8e5853d_4887_4741_ae00_33e0959aafea.slice/crio-0207e9c8e613ba111440f37c96bce593c96fc472ff9aeb5a083e6b1ffeee9c6f WatchSource:0}: Error finding container 0207e9c8e613ba111440f37c96bce593c96fc472ff9aeb5a083e6b1ffeee9c6f: Status 404 returned error can't find the container with id 0207e9c8e613ba111440f37c96bce593c96fc472ff9aeb5a083e6b1ffeee9c6f
Sep 16 02:37:16.499776 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:37:16.499737    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-a53a44-6c64b45999-td78d" event={"ID":"b8e5853d-4887-4741-ae00-33e0959aafea","Type":"ContainerStarted","Data":"6e2a1ce2afa3400410d4b499318a5410483d2c714b91a86eaad4067fd94ec6ad"}
Sep 16 02:37:16.500241 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:37:16.499808    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-a53a44-6c64b45999-td78d"
Sep 16 02:37:16.500241 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:37:16.499838    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-a53a44-6c64b45999-td78d" event={"ID":"b8e5853d-4887-4741-ae00-33e0959aafea","Type":"ContainerStarted","Data":"0207e9c8e613ba111440f37c96bce593c96fc472ff9aeb5a083e6b1ffeee9c6f"}
Sep 16 02:37:16.501622 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:37:16.501598    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-a53a44-566f584d6-hnt5w" event={"ID":"b2097872-a162-49ab-95f4-06bd1aa0ed40","Type":"ContainerStarted","Data":"83f341696678104e73c1015b323ebe754055ba138e035a18efa155f2f47db180"}
Sep 16 02:37:16.501744 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:37:16.501637    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-a53a44-566f584d6-hnt5w" event={"ID":"b2097872-a162-49ab-95f4-06bd1aa0ed40","Type":"ContainerStarted","Data":"aa227b7b8972906bfffacb913e23747a96b2bc310938f253d87aa78f6eaf3903"}
Sep 16 02:37:16.501744 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:37:16.501671    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-a53a44-566f584d6-hnt5w"
Sep 16 02:37:16.518648 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:37:16.518621    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:37:16.519984 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:37:16.519942    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-a53a44-6c64b45999-td78d" podStartSLOduration=1.5199223929999999 podStartE2EDuration="1.519922393s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 02:37:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 02:37:16.516152783 +0000 UTC m=+1456.899668032" watchObservedRunningTime="2026-09-16 02:37:16.519922393 +0000 UTC m=+1456.903437640"
Sep 16 02:37:16.533203 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:37:16.533153    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-a53a44-566f584d6-hnt5w" podStartSLOduration=1.533135157 podStartE2EDuration="1.533135157s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 02:37:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 02:37:16.531066351 +0000 UTC m=+1456.914581596" watchObservedRunningTime="2026-09-16 02:37:16.533135157 +0000 UTC m=+1456.916650402"
Sep 16 02:37:17.246625 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:37:17.246575    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:37:17.866333 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:37:17.866300    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:37:27.510747 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:37:27.510715    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-a53a44-566f584d6-hnt5w"
Sep 16 02:37:27.511169 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:37:27.510844    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-a53a44-6c64b45999-td78d"
Sep 16 02:37:30.965746 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:37:30.965705    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:37:32.092906 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:37:32.092868    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:37:33.276656 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:37:33.276620    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:37:34.149090 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:37:34.149058    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:37:34.844089 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:37:34.844037    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:37:35.496439 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:37:35.496398    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:37:36.194668 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:37:36.194632    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:37:37.120762 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:37:37.120727    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:37:38.006745 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:37:38.006704    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:37:40.105770 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:37:40.105731    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:37:40.672160 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:37:40.672120    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:37:40.746232 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:37:40.746198    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:37:40.811921 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:37:40.811885    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:37:40.883645 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:37:40.883610    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:37:41.000921 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:37:41.000892    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:37:41.241061 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:37:41.241030    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:37:41.722606 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:37:41.722574    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:37:41.824826 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:37:41.824796    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:37:41.926158 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:37:41.926122    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:37:42.238733 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:37:42.238693    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:37:42.868304 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:37:42.868271    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:37:43.423004 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:37:43.422968    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:37:43.524277 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:37:43.524243    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:37:49.035282 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:37:49.035247    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:37:49.090536 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:37:49.090456    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:37:49.193249 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:37:49.193212    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:37:49.307268 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:37:49.307185    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:37:49.434743 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:37:49.434698    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:37:49.885220 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:37:49.885184    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:38:00.170597 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:38:00.170566    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-c8q8z_ebf846fa-af7d-4fdd-9662-b7c533c36ddd/console-operator/2.log"
Sep 16 02:38:00.182389 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:38:00.182368    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-c8q8z_ebf846fa-af7d-4fdd-9662-b7c533c36ddd/console-operator/2.log"
Sep 16 02:38:03.965068 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:38:03.965025    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w2-24aabe/auth-test-model-24aabe-kserve-d9468cd5f-4zt8g"]
Sep 16 02:38:03.968974 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:38:03.968947    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w2-24aabe/auth-test-model-24aabe-kserve-d9468cd5f-4zt8g"
Sep 16 02:38:03.972027 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:38:03.971996    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w2-24aabe\"/\"openshift-service-ca.crt\""
Sep 16 02:38:03.972168 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:38:03.972001    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w2-24aabe\"/\"kube-root-ca.crt\""
Sep 16 02:38:03.972168 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:38:03.972054    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w2-24aabe\"/\"default-dockercfg-scwfc\""
Sep 16 02:38:03.972168 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:38:03.972001    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w2-24aabe\"/\"auth-test-model-24aabe-kserve-self-signed-certs\""
Sep 16 02:38:03.977087 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:38:03.977067    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w2-24aabe/auth-test-model-24aabe-kserve-d9468cd5f-4zt8g"]
Sep 16 02:38:03.997867 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:38:03.997826    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:38:04.125661 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:38:04.125628    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2adfafdf-5d46-43df-ba80-ad7ce0743ea1-tls-certs\") pod \"auth-test-model-24aabe-kserve-d9468cd5f-4zt8g\" (UID: \"2adfafdf-5d46-43df-ba80-ad7ce0743ea1\") " pod="ai-tenant-e2e-shared-a-w2-24aabe/auth-test-model-24aabe-kserve-d9468cd5f-4zt8g"
Sep 16 02:38:04.125831 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:38:04.125683    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2adfafdf-5d46-43df-ba80-ad7ce0743ea1-model-cache\") pod \"auth-test-model-24aabe-kserve-d9468cd5f-4zt8g\" (UID: \"2adfafdf-5d46-43df-ba80-ad7ce0743ea1\") " pod="ai-tenant-e2e-shared-a-w2-24aabe/auth-test-model-24aabe-kserve-d9468cd5f-4zt8g"
Sep 16 02:38:04.125831 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:38:04.125757    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2adfafdf-5d46-43df-ba80-ad7ce0743ea1-home\") pod \"auth-test-model-24aabe-kserve-d9468cd5f-4zt8g\" (UID: \"2adfafdf-5d46-43df-ba80-ad7ce0743ea1\") " pod="ai-tenant-e2e-shared-a-w2-24aabe/auth-test-model-24aabe-kserve-d9468cd5f-4zt8g"
Sep 16 02:38:04.125831 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:38:04.125808    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2adfafdf-5d46-43df-ba80-ad7ce0743ea1-dshm\") pod \"auth-test-model-24aabe-kserve-d9468cd5f-4zt8g\" (UID: \"2adfafdf-5d46-43df-ba80-ad7ce0743ea1\") " pod="ai-tenant-e2e-shared-a-w2-24aabe/auth-test-model-24aabe-kserve-d9468cd5f-4zt8g"
Sep 16 02:38:04.126007 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:38:04.125836    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2adfafdf-5d46-43df-ba80-ad7ce0743ea1-tmp-dir\") pod \"auth-test-model-24aabe-kserve-d9468cd5f-4zt8g\" (UID: \"2adfafdf-5d46-43df-ba80-ad7ce0743ea1\") " pod="ai-tenant-e2e-shared-a-w2-24aabe/auth-test-model-24aabe-kserve-d9468cd5f-4zt8g"
Sep 16 02:38:04.126007 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:38:04.125926    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nz6vt\" (UniqueName: \"kubernetes.io/projected/2adfafdf-5d46-43df-ba80-ad7ce0743ea1-kube-api-access-nz6vt\") pod \"auth-test-model-24aabe-kserve-d9468cd5f-4zt8g\" (UID: \"2adfafdf-5d46-43df-ba80-ad7ce0743ea1\") " pod="ai-tenant-e2e-shared-a-w2-24aabe/auth-test-model-24aabe-kserve-d9468cd5f-4zt8g"
Sep 16 02:38:04.226753 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:38:04.226677    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2adfafdf-5d46-43df-ba80-ad7ce0743ea1-tls-certs\") pod \"auth-test-model-24aabe-kserve-d9468cd5f-4zt8g\" (UID: \"2adfafdf-5d46-43df-ba80-ad7ce0743ea1\") " pod="ai-tenant-e2e-shared-a-w2-24aabe/auth-test-model-24aabe-kserve-d9468cd5f-4zt8g"
Sep 16 02:38:04.226753 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:38:04.226724    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2adfafdf-5d46-43df-ba80-ad7ce0743ea1-model-cache\") pod \"auth-test-model-24aabe-kserve-d9468cd5f-4zt8g\" (UID: \"2adfafdf-5d46-43df-ba80-ad7ce0743ea1\") " pod="ai-tenant-e2e-shared-a-w2-24aabe/auth-test-model-24aabe-kserve-d9468cd5f-4zt8g"
Sep 16 02:38:04.226957 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:38:04.226759    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2adfafdf-5d46-43df-ba80-ad7ce0743ea1-home\") pod \"auth-test-model-24aabe-kserve-d9468cd5f-4zt8g\" (UID: \"2adfafdf-5d46-43df-ba80-ad7ce0743ea1\") " pod="ai-tenant-e2e-shared-a-w2-24aabe/auth-test-model-24aabe-kserve-d9468cd5f-4zt8g"
Sep 16 02:38:04.226957 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:38:04.226779    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2adfafdf-5d46-43df-ba80-ad7ce0743ea1-dshm\") pod \"auth-test-model-24aabe-kserve-d9468cd5f-4zt8g\" (UID: \"2adfafdf-5d46-43df-ba80-ad7ce0743ea1\") " pod="ai-tenant-e2e-shared-a-w2-24aabe/auth-test-model-24aabe-kserve-d9468cd5f-4zt8g"
Sep 16 02:38:04.226957 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:38:04.226796    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2adfafdf-5d46-43df-ba80-ad7ce0743ea1-tmp-dir\") pod \"auth-test-model-24aabe-kserve-d9468cd5f-4zt8g\" (UID: \"2adfafdf-5d46-43df-ba80-ad7ce0743ea1\") " pod="ai-tenant-e2e-shared-a-w2-24aabe/auth-test-model-24aabe-kserve-d9468cd5f-4zt8g"
Sep 16 02:38:04.226957 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:38:04.226848    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nz6vt\" (UniqueName: \"kubernetes.io/projected/2adfafdf-5d46-43df-ba80-ad7ce0743ea1-kube-api-access-nz6vt\") pod \"auth-test-model-24aabe-kserve-d9468cd5f-4zt8g\" (UID: \"2adfafdf-5d46-43df-ba80-ad7ce0743ea1\") " pod="ai-tenant-e2e-shared-a-w2-24aabe/auth-test-model-24aabe-kserve-d9468cd5f-4zt8g"
Sep 16 02:38:04.227247 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:38:04.227223    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2adfafdf-5d46-43df-ba80-ad7ce0743ea1-model-cache\") pod \"auth-test-model-24aabe-kserve-d9468cd5f-4zt8g\" (UID: \"2adfafdf-5d46-43df-ba80-ad7ce0743ea1\") " pod="ai-tenant-e2e-shared-a-w2-24aabe/auth-test-model-24aabe-kserve-d9468cd5f-4zt8g"
Sep 16 02:38:04.227324 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:38:04.227281    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2adfafdf-5d46-43df-ba80-ad7ce0743ea1-home\") pod \"auth-test-model-24aabe-kserve-d9468cd5f-4zt8g\" (UID: \"2adfafdf-5d46-43df-ba80-ad7ce0743ea1\") " pod="ai-tenant-e2e-shared-a-w2-24aabe/auth-test-model-24aabe-kserve-d9468cd5f-4zt8g"
Sep 16 02:38:04.227381 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:38:04.227315    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2adfafdf-5d46-43df-ba80-ad7ce0743ea1-tmp-dir\") pod \"auth-test-model-24aabe-kserve-d9468cd5f-4zt8g\" (UID: \"2adfafdf-5d46-43df-ba80-ad7ce0743ea1\") " pod="ai-tenant-e2e-shared-a-w2-24aabe/auth-test-model-24aabe-kserve-d9468cd5f-4zt8g"
Sep 16 02:38:04.229176 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:38:04.229147    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2adfafdf-5d46-43df-ba80-ad7ce0743ea1-dshm\") pod \"auth-test-model-24aabe-kserve-d9468cd5f-4zt8g\" (UID: \"2adfafdf-5d46-43df-ba80-ad7ce0743ea1\") " pod="ai-tenant-e2e-shared-a-w2-24aabe/auth-test-model-24aabe-kserve-d9468cd5f-4zt8g"
Sep 16 02:38:04.229422 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:38:04.229406    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2adfafdf-5d46-43df-ba80-ad7ce0743ea1-tls-certs\") pod \"auth-test-model-24aabe-kserve-d9468cd5f-4zt8g\" (UID: \"2adfafdf-5d46-43df-ba80-ad7ce0743ea1\") " pod="ai-tenant-e2e-shared-a-w2-24aabe/auth-test-model-24aabe-kserve-d9468cd5f-4zt8g"
Sep 16 02:38:04.235191 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:38:04.235169    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nz6vt\" (UniqueName: \"kubernetes.io/projected/2adfafdf-5d46-43df-ba80-ad7ce0743ea1-kube-api-access-nz6vt\") pod \"auth-test-model-24aabe-kserve-d9468cd5f-4zt8g\" (UID: \"2adfafdf-5d46-43df-ba80-ad7ce0743ea1\") " pod="ai-tenant-e2e-shared-a-w2-24aabe/auth-test-model-24aabe-kserve-d9468cd5f-4zt8g"
Sep 16 02:38:04.282168 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:38:04.282140    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w2-24aabe/auth-test-model-24aabe-kserve-d9468cd5f-4zt8g"
Sep 16 02:38:04.554190 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:38:04.554165    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w2-24aabe/auth-test-model-24aabe-kserve-d9468cd5f-4zt8g"]
Sep 16 02:38:04.556301 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:38:04.556270    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2adfafdf_5d46_43df_ba80_ad7ce0743ea1.slice/crio-0578c2a188c86b52d2de7606da2328c7bf95c04f7be4afe816a42f415f5307b7 WatchSource:0}: Error finding container 0578c2a188c86b52d2de7606da2328c7bf95c04f7be4afe816a42f415f5307b7: Status 404 returned error can't find the container with id 0578c2a188c86b52d2de7606da2328c7bf95c04f7be4afe816a42f415f5307b7
Sep 16 02:38:04.768499 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:38:04.768447    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w2-24aabe/auth-test-model-24aabe-kserve-d9468cd5f-4zt8g" event={"ID":"2adfafdf-5d46-43df-ba80-ad7ce0743ea1","Type":"ContainerStarted","Data":"b7ca49232610df30e8d2cc88967fd6ba3a2b0e7ba3be03c5523631b67ebf977b"}
Sep 16 02:38:04.768499 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:38:04.768506    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w2-24aabe/auth-test-model-24aabe-kserve-d9468cd5f-4zt8g" event={"ID":"2adfafdf-5d46-43df-ba80-ad7ce0743ea1","Type":"ContainerStarted","Data":"0578c2a188c86b52d2de7606da2328c7bf95c04f7be4afe816a42f415f5307b7"}
Sep 16 02:38:04.785312 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:38:04.785264    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w2-24aabe/auth-test-model-24aabe-kserve-d9468cd5f-4zt8g" podStartSLOduration=1.785244256 podStartE2EDuration="1.785244256s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 02:38:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 02:38:04.783481579 +0000 UTC m=+1505.166996815" watchObservedRunningTime="2026-09-16 02:38:04.785244256 +0000 UTC m=+1505.168759500"
Sep 16 02:38:14.282427 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:38:14.282394    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w2-24aabe/auth-test-model-24aabe-kserve-d9468cd5f-4zt8g"
Sep 16 02:38:14.282819 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:38:14.282450    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w2-24aabe/auth-test-model-24aabe-kserve-d9468cd5f-4zt8g"
Sep 16 02:38:14.294874 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:38:14.294853    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w2-24aabe/auth-test-model-24aabe-kserve-d9468cd5f-4zt8g"
Sep 16 02:38:14.841176 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:38:14.841145    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w2-24aabe/auth-test-model-24aabe-kserve-d9468cd5f-4zt8g"
Sep 16 02:38:15.980145 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:38:15.980081    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:38:21.624513 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:38:21.624412    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:38:21.724620 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:38:21.724564    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:38:21.826790 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:38:21.826751    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:38:21.927767 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:38:21.927685    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:38:32.097575 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:38:32.097534    2579 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-pre-processing-6cd864bc95-mh4jh" secret="" err="secret \"payload-processing-dockercfg-zzx6j\" not found"
Sep 16 02:38:36.211957 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:38:36.211881    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:38:37.955100 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:38:37.955061    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:38:40.763450 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:38:40.763399    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:38:40.812601 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:38:40.812567    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:38:43.797684 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:38:43.797652    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:38:43.898403 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:38:43.898367    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:38:43.998229 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:38:43.998199    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:38:44.100453 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:38:44.100334    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:38:45.281297 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:38:45.281264    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w2-a53a44/auth-test-model-a53a44-kserve-59f8574c79-7zfnt"]
Sep 16 02:38:45.285171 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:38:45.285154    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w2-a53a44/auth-test-model-a53a44-kserve-59f8574c79-7zfnt"
Sep 16 02:38:45.288439 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:38:45.288403    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w2-a53a44\"/\"kube-root-ca.crt\""
Sep 16 02:38:45.288439 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:38:45.288430    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w2-a53a44\"/\"auth-test-model-a53a44-kserve-self-signed-certs\""
Sep 16 02:38:45.288700 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:38:45.288439    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w2-a53a44\"/\"default-dockercfg-tw8hg\""
Sep 16 02:38:45.288700 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:38:45.288430    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w2-a53a44\"/\"openshift-service-ca.crt\""
Sep 16 02:38:45.293364 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:38:45.293339    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w2-a53a44/auth-test-model-a53a44-kserve-59f8574c79-7zfnt"]
Sep 16 02:38:45.306833 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:38:45.306799    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:38:45.394643 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:38:45.394609    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zfzdj\" (UniqueName: \"kubernetes.io/projected/eb4f9053-822f-4b52-8de8-ee555b5fac9c-kube-api-access-zfzdj\") pod \"auth-test-model-a53a44-kserve-59f8574c79-7zfnt\" (UID: \"eb4f9053-822f-4b52-8de8-ee555b5fac9c\") " pod="ai-tenant-e2e-shared-b-w2-a53a44/auth-test-model-a53a44-kserve-59f8574c79-7zfnt"
Sep 16 02:38:45.394801 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:38:45.394650    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eb4f9053-822f-4b52-8de8-ee555b5fac9c-home\") pod \"auth-test-model-a53a44-kserve-59f8574c79-7zfnt\" (UID: \"eb4f9053-822f-4b52-8de8-ee555b5fac9c\") " pod="ai-tenant-e2e-shared-b-w2-a53a44/auth-test-model-a53a44-kserve-59f8574c79-7zfnt"
Sep 16 02:38:45.394801 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:38:45.394710    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eb4f9053-822f-4b52-8de8-ee555b5fac9c-tls-certs\") pod \"auth-test-model-a53a44-kserve-59f8574c79-7zfnt\" (UID: \"eb4f9053-822f-4b52-8de8-ee555b5fac9c\") " pod="ai-tenant-e2e-shared-b-w2-a53a44/auth-test-model-a53a44-kserve-59f8574c79-7zfnt"
Sep 16 02:38:45.394801 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:38:45.394739    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eb4f9053-822f-4b52-8de8-ee555b5fac9c-dshm\") pod \"auth-test-model-a53a44-kserve-59f8574c79-7zfnt\" (UID: \"eb4f9053-822f-4b52-8de8-ee555b5fac9c\") " pod="ai-tenant-e2e-shared-b-w2-a53a44/auth-test-model-a53a44-kserve-59f8574c79-7zfnt"
Sep 16 02:38:45.394936 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:38:45.394806    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eb4f9053-822f-4b52-8de8-ee555b5fac9c-tmp-dir\") pod \"auth-test-model-a53a44-kserve-59f8574c79-7zfnt\" (UID: \"eb4f9053-822f-4b52-8de8-ee555b5fac9c\") " pod="ai-tenant-e2e-shared-b-w2-a53a44/auth-test-model-a53a44-kserve-59f8574c79-7zfnt"
Sep 16 02:38:45.394936 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:38:45.394845    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eb4f9053-822f-4b52-8de8-ee555b5fac9c-model-cache\") pod \"auth-test-model-a53a44-kserve-59f8574c79-7zfnt\" (UID: \"eb4f9053-822f-4b52-8de8-ee555b5fac9c\") " pod="ai-tenant-e2e-shared-b-w2-a53a44/auth-test-model-a53a44-kserve-59f8574c79-7zfnt"
Sep 16 02:38:45.496098 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:38:45.496060    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zfzdj\" (UniqueName: \"kubernetes.io/projected/eb4f9053-822f-4b52-8de8-ee555b5fac9c-kube-api-access-zfzdj\") pod \"auth-test-model-a53a44-kserve-59f8574c79-7zfnt\" (UID: \"eb4f9053-822f-4b52-8de8-ee555b5fac9c\") " pod="ai-tenant-e2e-shared-b-w2-a53a44/auth-test-model-a53a44-kserve-59f8574c79-7zfnt"
Sep 16 02:38:45.496279 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:38:45.496110    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eb4f9053-822f-4b52-8de8-ee555b5fac9c-home\") pod \"auth-test-model-a53a44-kserve-59f8574c79-7zfnt\" (UID: \"eb4f9053-822f-4b52-8de8-ee555b5fac9c\") " pod="ai-tenant-e2e-shared-b-w2-a53a44/auth-test-model-a53a44-kserve-59f8574c79-7zfnt"
Sep 16 02:38:45.496279 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:38:45.496143    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eb4f9053-822f-4b52-8de8-ee555b5fac9c-tls-certs\") pod \"auth-test-model-a53a44-kserve-59f8574c79-7zfnt\" (UID: \"eb4f9053-822f-4b52-8de8-ee555b5fac9c\") " pod="ai-tenant-e2e-shared-b-w2-a53a44/auth-test-model-a53a44-kserve-59f8574c79-7zfnt"
Sep 16 02:38:45.496279 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:38:45.496166    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eb4f9053-822f-4b52-8de8-ee555b5fac9c-dshm\") pod \"auth-test-model-a53a44-kserve-59f8574c79-7zfnt\" (UID: \"eb4f9053-822f-4b52-8de8-ee555b5fac9c\") " pod="ai-tenant-e2e-shared-b-w2-a53a44/auth-test-model-a53a44-kserve-59f8574c79-7zfnt"
Sep 16 02:38:45.496279 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:38:45.496197    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eb4f9053-822f-4b52-8de8-ee555b5fac9c-tmp-dir\") pod \"auth-test-model-a53a44-kserve-59f8574c79-7zfnt\" (UID: \"eb4f9053-822f-4b52-8de8-ee555b5fac9c\") " pod="ai-tenant-e2e-shared-b-w2-a53a44/auth-test-model-a53a44-kserve-59f8574c79-7zfnt"
Sep 16 02:38:45.496279 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:38:45.496236    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eb4f9053-822f-4b52-8de8-ee555b5fac9c-model-cache\") pod \"auth-test-model-a53a44-kserve-59f8574c79-7zfnt\" (UID: \"eb4f9053-822f-4b52-8de8-ee555b5fac9c\") " pod="ai-tenant-e2e-shared-b-w2-a53a44/auth-test-model-a53a44-kserve-59f8574c79-7zfnt"
Sep 16 02:38:45.496669 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:38:45.496652    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eb4f9053-822f-4b52-8de8-ee555b5fac9c-home\") pod \"auth-test-model-a53a44-kserve-59f8574c79-7zfnt\" (UID: \"eb4f9053-822f-4b52-8de8-ee555b5fac9c\") " pod="ai-tenant-e2e-shared-b-w2-a53a44/auth-test-model-a53a44-kserve-59f8574c79-7zfnt"
Sep 16 02:38:45.496719 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:38:45.496699    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eb4f9053-822f-4b52-8de8-ee555b5fac9c-tmp-dir\") pod \"auth-test-model-a53a44-kserve-59f8574c79-7zfnt\" (UID: \"eb4f9053-822f-4b52-8de8-ee555b5fac9c\") " pod="ai-tenant-e2e-shared-b-w2-a53a44/auth-test-model-a53a44-kserve-59f8574c79-7zfnt"
Sep 16 02:38:45.496719 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:38:45.496709    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eb4f9053-822f-4b52-8de8-ee555b5fac9c-model-cache\") pod \"auth-test-model-a53a44-kserve-59f8574c79-7zfnt\" (UID: \"eb4f9053-822f-4b52-8de8-ee555b5fac9c\") " pod="ai-tenant-e2e-shared-b-w2-a53a44/auth-test-model-a53a44-kserve-59f8574c79-7zfnt"
Sep 16 02:38:45.498673 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:38:45.498654    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eb4f9053-822f-4b52-8de8-ee555b5fac9c-dshm\") pod \"auth-test-model-a53a44-kserve-59f8574c79-7zfnt\" (UID: \"eb4f9053-822f-4b52-8de8-ee555b5fac9c\") " pod="ai-tenant-e2e-shared-b-w2-a53a44/auth-test-model-a53a44-kserve-59f8574c79-7zfnt"
Sep 16 02:38:45.498819 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:38:45.498802    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eb4f9053-822f-4b52-8de8-ee555b5fac9c-tls-certs\") pod \"auth-test-model-a53a44-kserve-59f8574c79-7zfnt\" (UID: \"eb4f9053-822f-4b52-8de8-ee555b5fac9c\") " pod="ai-tenant-e2e-shared-b-w2-a53a44/auth-test-model-a53a44-kserve-59f8574c79-7zfnt"
Sep 16 02:38:45.503520 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:38:45.503496    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zfzdj\" (UniqueName: \"kubernetes.io/projected/eb4f9053-822f-4b52-8de8-ee555b5fac9c-kube-api-access-zfzdj\") pod \"auth-test-model-a53a44-kserve-59f8574c79-7zfnt\" (UID: \"eb4f9053-822f-4b52-8de8-ee555b5fac9c\") " pod="ai-tenant-e2e-shared-b-w2-a53a44/auth-test-model-a53a44-kserve-59f8574c79-7zfnt"
Sep 16 02:38:45.596880 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:38:45.596812    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w2-a53a44/auth-test-model-a53a44-kserve-59f8574c79-7zfnt"
Sep 16 02:38:45.867189 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:38:45.867164    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w2-a53a44/auth-test-model-a53a44-kserve-59f8574c79-7zfnt"]
Sep 16 02:38:45.869072 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:38:45.869041    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podeb4f9053_822f_4b52_8de8_ee555b5fac9c.slice/crio-f7975259e9a71fbb4efdfb1a58504171203f8aad668676288ff4a16b2b961621 WatchSource:0}: Error finding container f7975259e9a71fbb4efdfb1a58504171203f8aad668676288ff4a16b2b961621: Status 404 returned error can't find the container with id f7975259e9a71fbb4efdfb1a58504171203f8aad668676288ff4a16b2b961621
Sep 16 02:38:45.996872 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:38:45.996830    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w2-a53a44/auth-test-model-a53a44-kserve-59f8574c79-7zfnt" event={"ID":"eb4f9053-822f-4b52-8de8-ee555b5fac9c","Type":"ContainerStarted","Data":"9e621a911501b111072aeb8d83bd18294f5482ec24b268d8d7232c1291aadfcf"}
Sep 16 02:38:45.997034 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:38:45.996880    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w2-a53a44/auth-test-model-a53a44-kserve-59f8574c79-7zfnt" event={"ID":"eb4f9053-822f-4b52-8de8-ee555b5fac9c","Type":"ContainerStarted","Data":"f7975259e9a71fbb4efdfb1a58504171203f8aad668676288ff4a16b2b961621"}
Sep 16 02:38:46.012243 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:38:46.012189    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w2-a53a44/auth-test-model-a53a44-kserve-59f8574c79-7zfnt" podStartSLOduration=1.012169091 podStartE2EDuration="1.012169091s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 02:38:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 02:38:46.011397662 +0000 UTC m=+1546.394912908" watchObservedRunningTime="2026-09-16 02:38:46.012169091 +0000 UTC m=+1546.395684337"
Sep 16 02:38:54.616159 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:38:54.616128    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:38:55.463535 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:38:55.463482    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:38:55.597252 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:38:55.597168    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w2-a53a44/auth-test-model-a53a44-kserve-59f8574c79-7zfnt"
Sep 16 02:38:55.597432 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:38:55.597270    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w2-a53a44/auth-test-model-a53a44-kserve-59f8574c79-7zfnt"
Sep 16 02:38:55.610232 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:38:55.610203    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w2-a53a44/auth-test-model-a53a44-kserve-59f8574c79-7zfnt"
Sep 16 02:38:56.065799 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:38:56.065770    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w2-a53a44/auth-test-model-a53a44-kserve-59f8574c79-7zfnt"
Sep 16 02:39:01.045104 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:01.045065    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:39:01.155886 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:01.155854    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:39:01.246173 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:01.246140    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:39:01.346133 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:01.346016    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:39:02.154795 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:02.154736    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:39:02.256749 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:02.256708    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:39:02.353881 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:02.353842    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:39:04.156926 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:04.156868    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:39:10.150737 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:10.150701    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:39:10.211205 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:10.211169    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:39:10.919749 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:10.919711    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:39:16.461718 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:16.461665    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:39:16.586211 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:16.586175    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:39:16.671456 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:16.671408    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:39:16.774908 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:16.774835    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:39:17.099517 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:17.099417    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:39:17.223267 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:17.222994    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:39:17.322600 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:17.322568    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:39:18.029415 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:18.029370    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:39:18.667973 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:18.667936    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:39:19.448614 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:19.448555    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:39:20.125695 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:20.125661    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:39:20.222789 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:20.222754    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:39:20.433488 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:20.433362    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:39:21.148771 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:21.148735    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:39:21.592135 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:21.592107    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-542ed5/distinct-w3-542ed5-kserve-64fdd5d5f-jjxfp"]
Sep 16 02:39:21.603916 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:21.603891    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-542ed5/premium-w3-542ed5-kserve-56d8467ff6-mqzvc"]
Sep 16 02:39:21.613815 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:21.613784    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-542ed5/simulator-w3-542ed5-kserve-77ff54b49-nwttm"]
Sep 16 02:39:21.663218 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:21.663187    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:39:21.893237 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:21.893158    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:39:22.652391 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:22.652352    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:39:22.752555 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:22.752525    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:39:22.805163 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:22.805127    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:39:23.087594 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:23.087562    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:39:23.186095 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:23.186062    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:39:26.929082 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:26.929035    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:39:27.029991 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:27.029936    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:39:32.609110 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:32.609062    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:39:32.640883 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:32.640841    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-542ed5-84b56598c8-pcz52"]
Sep 16 02:39:32.641122 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:32.641092    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-542ed5-84b56598c8-pcz52" podUID="2169a66a-2ca0-4cb1-81f8-6dcc588e9652" containerName="payload-pre-processing" containerID="cri-o://835a1b970d3712febafe0d52175ae00b74911bd1887a25afc65733ae7fd4f200" gracePeriod=30
Sep 16 02:39:32.707641 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:32.707607    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:39:32.789968 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:32.789921    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:39:32.839776 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:32.839629    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:39:32.912105 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:32.912082    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-542ed5-84b56598c8-pcz52"
Sep 16 02:39:32.971600 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:32.971571    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2169a66a-2ca0-4cb1-81f8-6dcc588e9652-plugins-config-volume\") pod \"2169a66a-2ca0-4cb1-81f8-6dcc588e9652\" (UID: \"2169a66a-2ca0-4cb1-81f8-6dcc588e9652\") "
Sep 16 02:39:32.971762 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:32.971632    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fg62z\" (UniqueName: \"kubernetes.io/projected/2169a66a-2ca0-4cb1-81f8-6dcc588e9652-kube-api-access-fg62z\") pod \"2169a66a-2ca0-4cb1-81f8-6dcc588e9652\" (UID: \"2169a66a-2ca0-4cb1-81f8-6dcc588e9652\") "
Sep 16 02:39:32.971942 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:32.971917    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2169a66a-2ca0-4cb1-81f8-6dcc588e9652-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2169a66a-2ca0-4cb1-81f8-6dcc588e9652" (UID: "2169a66a-2ca0-4cb1-81f8-6dcc588e9652"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 02:39:32.973806 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:32.973782    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2169a66a-2ca0-4cb1-81f8-6dcc588e9652-kube-api-access-fg62z" (OuterVolumeSpecName: "kube-api-access-fg62z") pod "2169a66a-2ca0-4cb1-81f8-6dcc588e9652" (UID: "2169a66a-2ca0-4cb1-81f8-6dcc588e9652"). InnerVolumeSpecName "kube-api-access-fg62z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 02:39:33.073253 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:33.073222    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2169a66a-2ca0-4cb1-81f8-6dcc588e9652-plugins-config-volume\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:39:33.073253 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:33.073247    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fg62z\" (UniqueName: \"kubernetes.io/projected/2169a66a-2ca0-4cb1-81f8-6dcc588e9652-kube-api-access-fg62z\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:39:33.245594 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:33.245555    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:39:33.260934 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:33.260905    2579 generic.go:356] "Generic (PLEG): container finished" podID="2169a66a-2ca0-4cb1-81f8-6dcc588e9652" containerID="835a1b970d3712febafe0d52175ae00b74911bd1887a25afc65733ae7fd4f200" exitCode=0
Sep 16 02:39:33.261077 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:33.260946    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-542ed5-84b56598c8-pcz52"
Sep 16 02:39:33.261077 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:33.260993    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-542ed5-84b56598c8-pcz52" event={"ID":"2169a66a-2ca0-4cb1-81f8-6dcc588e9652","Type":"ContainerDied","Data":"835a1b970d3712febafe0d52175ae00b74911bd1887a25afc65733ae7fd4f200"}
Sep 16 02:39:33.261077 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:33.261051    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-542ed5-84b56598c8-pcz52" event={"ID":"2169a66a-2ca0-4cb1-81f8-6dcc588e9652","Type":"ContainerDied","Data":"5984ac8df234170a0b2e503de3be48ac35c93b5bd382ec314e7faed0891f816b"}
Sep 16 02:39:33.261185 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:33.261078    2579 scope.go:117] "RemoveContainer" containerID="835a1b970d3712febafe0d52175ae00b74911bd1887a25afc65733ae7fd4f200"
Sep 16 02:39:33.270904 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:33.270888    2579 scope.go:117] "RemoveContainer" containerID="835a1b970d3712febafe0d52175ae00b74911bd1887a25afc65733ae7fd4f200"
Sep 16 02:39:33.271180 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:39:33.271161    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"835a1b970d3712febafe0d52175ae00b74911bd1887a25afc65733ae7fd4f200\": container with ID starting with 835a1b970d3712febafe0d52175ae00b74911bd1887a25afc65733ae7fd4f200 not found: ID does not exist" containerID="835a1b970d3712febafe0d52175ae00b74911bd1887a25afc65733ae7fd4f200"
Sep 16 02:39:33.271252 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:33.271190    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"835a1b970d3712febafe0d52175ae00b74911bd1887a25afc65733ae7fd4f200"} err="failed to get container status \"835a1b970d3712febafe0d52175ae00b74911bd1887a25afc65733ae7fd4f200\": rpc error: code = NotFound desc = could not find container \"835a1b970d3712febafe0d52175ae00b74911bd1887a25afc65733ae7fd4f200\": container with ID starting with 835a1b970d3712febafe0d52175ae00b74911bd1887a25afc65733ae7fd4f200 not found: ID does not exist"
Sep 16 02:39:33.290816 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:33.290793    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-542ed5-84b56598c8-pcz52"]
Sep 16 02:39:33.297129 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:33.297108    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-542ed5-84b56598c8-pcz52"]
Sep 16 02:39:33.343279 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:33.343253    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:39:34.104236 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:34.104207    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2169a66a-2ca0-4cb1-81f8-6dcc588e9652" path="/var/lib/kubelet/pods/2169a66a-2ca0-4cb1-81f8-6dcc588e9652/volumes"
Sep 16 02:39:34.111072 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:34.111043    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:39:34.794523 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:34.794491    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:39:35.421260 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:35.421219    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:39:36.108735 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:36.108698    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:39:36.225410 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:36.225381    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:39:36.614125 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:36.614074    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w3-542ed5/distinct-w3-542ed5-kserve-64fdd5d5f-jjxfp" podUID="44750d09-f1af-47cb-9a01-ce4b408f092f" containerName="main" containerID="cri-o://e71e2b36976904eff4f8c247ea886a51f195b055480dde7644a5567ec4be98d8" gracePeriod=45
Sep 16 02:39:36.625942 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:36.625909    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w3-542ed5/premium-w3-542ed5-kserve-56d8467ff6-mqzvc" podUID="01ea8c54-637f-488b-b324-2eba77119902" containerName="main" containerID="cri-o://8ffc3fbf2a3f909e816bb92b5ced4352afc3c33a96fed59f56009d15a09cfba3" gracePeriod=45
Sep 16 02:39:36.640860 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:36.640803    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w3-542ed5/simulator-w3-542ed5-kserve-77ff54b49-nwttm" podUID="99191806-ff18-4c21-afa7-701c9c9a85fa" containerName="main" containerID="cri-o://91b0943e1e232cfcd0e07cb06546019798067ce42f9fb2626df3b9b1a16d90a1" gracePeriod=45
Sep 16 02:39:36.921673 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:36.921641    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-542ed5/distinct-w3-542ed5-kserve-64fdd5d5f-jjxfp"
Sep 16 02:39:36.943422 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:36.943392    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:39:36.944963 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:36.944940    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-542ed5/premium-w3-542ed5-kserve-56d8467ff6-mqzvc"
Sep 16 02:39:36.948891 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:36.948874    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-542ed5/simulator-w3-542ed5-kserve-77ff54b49-nwttm"
Sep 16 02:39:37.015685 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:37.015653    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/01ea8c54-637f-488b-b324-2eba77119902-home\") pod \"01ea8c54-637f-488b-b324-2eba77119902\" (UID: \"01ea8c54-637f-488b-b324-2eba77119902\") "
Sep 16 02:39:37.015848 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:37.015702    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/99191806-ff18-4c21-afa7-701c9c9a85fa-tmp-dir\") pod \"99191806-ff18-4c21-afa7-701c9c9a85fa\" (UID: \"99191806-ff18-4c21-afa7-701c9c9a85fa\") "
Sep 16 02:39:37.015848 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:37.015731    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b8qks\" (UniqueName: \"kubernetes.io/projected/99191806-ff18-4c21-afa7-701c9c9a85fa-kube-api-access-b8qks\") pod \"99191806-ff18-4c21-afa7-701c9c9a85fa\" (UID: \"99191806-ff18-4c21-afa7-701c9c9a85fa\") "
Sep 16 02:39:37.015848 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:37.015760    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/99191806-ff18-4c21-afa7-701c9c9a85fa-dshm\") pod \"99191806-ff18-4c21-afa7-701c9c9a85fa\" (UID: \"99191806-ff18-4c21-afa7-701c9c9a85fa\") "
Sep 16 02:39:37.015848 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:37.015811    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/99191806-ff18-4c21-afa7-701c9c9a85fa-tls-certs\") pod \"99191806-ff18-4c21-afa7-701c9c9a85fa\" (UID: \"99191806-ff18-4c21-afa7-701c9c9a85fa\") "
Sep 16 02:39:37.015848 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:37.015833    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/99191806-ff18-4c21-afa7-701c9c9a85fa-model-cache\") pod \"99191806-ff18-4c21-afa7-701c9c9a85fa\" (UID: \"99191806-ff18-4c21-afa7-701c9c9a85fa\") "
Sep 16 02:39:37.016136 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:37.015942    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/01ea8c54-637f-488b-b324-2eba77119902-home" (OuterVolumeSpecName: "home") pod "01ea8c54-637f-488b-b324-2eba77119902" (UID: "01ea8c54-637f-488b-b324-2eba77119902"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 02:39:37.016136 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:37.015983    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99191806-ff18-4c21-afa7-701c9c9a85fa-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "99191806-ff18-4c21-afa7-701c9c9a85fa" (UID: "99191806-ff18-4c21-afa7-701c9c9a85fa"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 02:39:37.016136 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:37.016067    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/44750d09-f1af-47cb-9a01-ce4b408f092f-dshm\") pod \"44750d09-f1af-47cb-9a01-ce4b408f092f\" (UID: \"44750d09-f1af-47cb-9a01-ce4b408f092f\") "
Sep 16 02:39:37.016136 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:37.016082    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99191806-ff18-4c21-afa7-701c9c9a85fa-model-cache" (OuterVolumeSpecName: "model-cache") pod "99191806-ff18-4c21-afa7-701c9c9a85fa" (UID: "99191806-ff18-4c21-afa7-701c9c9a85fa"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 02:39:37.016136 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:37.016121    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bf89z\" (UniqueName: \"kubernetes.io/projected/01ea8c54-637f-488b-b324-2eba77119902-kube-api-access-bf89z\") pod \"01ea8c54-637f-488b-b324-2eba77119902\" (UID: \"01ea8c54-637f-488b-b324-2eba77119902\") "
Sep 16 02:39:37.016403 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:37.016153    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/44750d09-f1af-47cb-9a01-ce4b408f092f-home\") pod \"44750d09-f1af-47cb-9a01-ce4b408f092f\" (UID: \"44750d09-f1af-47cb-9a01-ce4b408f092f\") "
Sep 16 02:39:37.016403 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:37.016192    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/01ea8c54-637f-488b-b324-2eba77119902-tls-certs\") pod \"01ea8c54-637f-488b-b324-2eba77119902\" (UID: \"01ea8c54-637f-488b-b324-2eba77119902\") "
Sep 16 02:39:37.016403 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:37.016235    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/44750d09-f1af-47cb-9a01-ce4b408f092f-tls-certs\") pod \"44750d09-f1af-47cb-9a01-ce4b408f092f\" (UID: \"44750d09-f1af-47cb-9a01-ce4b408f092f\") "
Sep 16 02:39:37.016403 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:37.016264    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/44750d09-f1af-47cb-9a01-ce4b408f092f-model-cache\") pod \"44750d09-f1af-47cb-9a01-ce4b408f092f\" (UID: \"44750d09-f1af-47cb-9a01-ce4b408f092f\") "
Sep 16 02:39:37.016403 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:37.016304    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/99191806-ff18-4c21-afa7-701c9c9a85fa-home\") pod \"99191806-ff18-4c21-afa7-701c9c9a85fa\" (UID: \"99191806-ff18-4c21-afa7-701c9c9a85fa\") "
Sep 16 02:39:37.016403 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:37.016344    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/44750d09-f1af-47cb-9a01-ce4b408f092f-tmp-dir\") pod \"44750d09-f1af-47cb-9a01-ce4b408f092f\" (UID: \"44750d09-f1af-47cb-9a01-ce4b408f092f\") "
Sep 16 02:39:37.016403 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:37.016390    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sswhj\" (UniqueName: \"kubernetes.io/projected/44750d09-f1af-47cb-9a01-ce4b408f092f-kube-api-access-sswhj\") pod \"44750d09-f1af-47cb-9a01-ce4b408f092f\" (UID: \"44750d09-f1af-47cb-9a01-ce4b408f092f\") "
Sep 16 02:39:37.016736 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:37.016414    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/01ea8c54-637f-488b-b324-2eba77119902-model-cache\") pod \"01ea8c54-637f-488b-b324-2eba77119902\" (UID: \"01ea8c54-637f-488b-b324-2eba77119902\") "
Sep 16 02:39:37.016736 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:37.016441    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/01ea8c54-637f-488b-b324-2eba77119902-tmp-dir\") pod \"01ea8c54-637f-488b-b324-2eba77119902\" (UID: \"01ea8c54-637f-488b-b324-2eba77119902\") "
Sep 16 02:39:37.016736 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:37.016462    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44750d09-f1af-47cb-9a01-ce4b408f092f-home" (OuterVolumeSpecName: "home") pod "44750d09-f1af-47cb-9a01-ce4b408f092f" (UID: "44750d09-f1af-47cb-9a01-ce4b408f092f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 02:39:37.016736 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:37.016501    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/01ea8c54-637f-488b-b324-2eba77119902-dshm\") pod \"01ea8c54-637f-488b-b324-2eba77119902\" (UID: \"01ea8c54-637f-488b-b324-2eba77119902\") "
Sep 16 02:39:37.016736 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:37.016662    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99191806-ff18-4c21-afa7-701c9c9a85fa-home" (OuterVolumeSpecName: "home") pod "99191806-ff18-4c21-afa7-701c9c9a85fa" (UID: "99191806-ff18-4c21-afa7-701c9c9a85fa"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 02:39:37.016950 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:37.016917    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/01ea8c54-637f-488b-b324-2eba77119902-home\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:39:37.016950 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:37.016939    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/99191806-ff18-4c21-afa7-701c9c9a85fa-tmp-dir\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:39:37.017055 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:37.016954    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/99191806-ff18-4c21-afa7-701c9c9a85fa-model-cache\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:39:37.017055 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:37.016966    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/44750d09-f1af-47cb-9a01-ce4b408f092f-home\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:39:37.017055 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:37.016979    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/99191806-ff18-4c21-afa7-701c9c9a85fa-home\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:39:37.017259 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:37.017151    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44750d09-f1af-47cb-9a01-ce4b408f092f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "44750d09-f1af-47cb-9a01-ce4b408f092f" (UID: "44750d09-f1af-47cb-9a01-ce4b408f092f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 02:39:37.017661 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:37.017371    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/01ea8c54-637f-488b-b324-2eba77119902-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "01ea8c54-637f-488b-b324-2eba77119902" (UID: "01ea8c54-637f-488b-b324-2eba77119902"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 02:39:37.017661 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:37.017560    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44750d09-f1af-47cb-9a01-ce4b408f092f-model-cache" (OuterVolumeSpecName: "model-cache") pod "44750d09-f1af-47cb-9a01-ce4b408f092f" (UID: "44750d09-f1af-47cb-9a01-ce4b408f092f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 02:39:37.017661 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:37.017586    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/01ea8c54-637f-488b-b324-2eba77119902-model-cache" (OuterVolumeSpecName: "model-cache") pod "01ea8c54-637f-488b-b324-2eba77119902" (UID: "01ea8c54-637f-488b-b324-2eba77119902"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 02:39:37.018964 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:37.018930    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44750d09-f1af-47cb-9a01-ce4b408f092f-dshm" (OuterVolumeSpecName: "dshm") pod "44750d09-f1af-47cb-9a01-ce4b408f092f" (UID: "44750d09-f1af-47cb-9a01-ce4b408f092f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 02:39:37.019340 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:37.019314    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/99191806-ff18-4c21-afa7-701c9c9a85fa-kube-api-access-b8qks" (OuterVolumeSpecName: "kube-api-access-b8qks") pod "99191806-ff18-4c21-afa7-701c9c9a85fa" (UID: "99191806-ff18-4c21-afa7-701c9c9a85fa"). InnerVolumeSpecName "kube-api-access-b8qks". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 02:39:37.019592 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:37.019454    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99191806-ff18-4c21-afa7-701c9c9a85fa-dshm" (OuterVolumeSpecName: "dshm") pod "99191806-ff18-4c21-afa7-701c9c9a85fa" (UID: "99191806-ff18-4c21-afa7-701c9c9a85fa"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 02:39:37.019592 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:37.019498    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/01ea8c54-637f-488b-b324-2eba77119902-kube-api-access-bf89z" (OuterVolumeSpecName: "kube-api-access-bf89z") pod "01ea8c54-637f-488b-b324-2eba77119902" (UID: "01ea8c54-637f-488b-b324-2eba77119902"). InnerVolumeSpecName "kube-api-access-bf89z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 02:39:37.019798 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:37.019767    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/99191806-ff18-4c21-afa7-701c9c9a85fa-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "99191806-ff18-4c21-afa7-701c9c9a85fa" (UID: "99191806-ff18-4c21-afa7-701c9c9a85fa"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 02:39:37.020367 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:37.020340    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/01ea8c54-637f-488b-b324-2eba77119902-dshm" (OuterVolumeSpecName: "dshm") pod "01ea8c54-637f-488b-b324-2eba77119902" (UID: "01ea8c54-637f-488b-b324-2eba77119902"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 02:39:37.020367 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:37.020348    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/44750d09-f1af-47cb-9a01-ce4b408f092f-kube-api-access-sswhj" (OuterVolumeSpecName: "kube-api-access-sswhj") pod "44750d09-f1af-47cb-9a01-ce4b408f092f" (UID: "44750d09-f1af-47cb-9a01-ce4b408f092f"). InnerVolumeSpecName "kube-api-access-sswhj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 02:39:37.020925 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:37.020902    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/44750d09-f1af-47cb-9a01-ce4b408f092f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "44750d09-f1af-47cb-9a01-ce4b408f092f" (UID: "44750d09-f1af-47cb-9a01-ce4b408f092f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 02:39:37.021189 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:37.021171    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/01ea8c54-637f-488b-b324-2eba77119902-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "01ea8c54-637f-488b-b324-2eba77119902" (UID: "01ea8c54-637f-488b-b324-2eba77119902"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 02:39:37.118421 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:37.118389    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/44750d09-f1af-47cb-9a01-ce4b408f092f-tmp-dir\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:39:37.118421 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:37.118419    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sswhj\" (UniqueName: \"kubernetes.io/projected/44750d09-f1af-47cb-9a01-ce4b408f092f-kube-api-access-sswhj\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:39:37.118674 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:37.118432    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/01ea8c54-637f-488b-b324-2eba77119902-model-cache\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:39:37.118674 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:37.118443    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/01ea8c54-637f-488b-b324-2eba77119902-tmp-dir\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:39:37.118674 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:37.118453    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/01ea8c54-637f-488b-b324-2eba77119902-dshm\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:39:37.118674 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:37.118463    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b8qks\" (UniqueName: \"kubernetes.io/projected/99191806-ff18-4c21-afa7-701c9c9a85fa-kube-api-access-b8qks\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:39:37.118674 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:37.118507    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/99191806-ff18-4c21-afa7-701c9c9a85fa-dshm\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:39:37.118674 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:37.118518    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/99191806-ff18-4c21-afa7-701c9c9a85fa-tls-certs\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:39:37.118674 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:37.118530    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/44750d09-f1af-47cb-9a01-ce4b408f092f-dshm\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:39:37.118674 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:37.118541    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bf89z\" (UniqueName: \"kubernetes.io/projected/01ea8c54-637f-488b-b324-2eba77119902-kube-api-access-bf89z\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:39:37.118674 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:37.118552    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/01ea8c54-637f-488b-b324-2eba77119902-tls-certs\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:39:37.118674 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:37.118564    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/44750d09-f1af-47cb-9a01-ce4b408f092f-tls-certs\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:39:37.118674 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:37.118574    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/44750d09-f1af-47cb-9a01-ce4b408f092f-model-cache\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:39:37.284743 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:37.284713    2579 generic.go:356] "Generic (PLEG): container finished" podID="01ea8c54-637f-488b-b324-2eba77119902" containerID="8ffc3fbf2a3f909e816bb92b5ced4352afc3c33a96fed59f56009d15a09cfba3" exitCode=0
Sep 16 02:39:37.284919 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:37.284752    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-542ed5/premium-w3-542ed5-kserve-56d8467ff6-mqzvc"
Sep 16 02:39:37.284919 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:37.284792    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-542ed5/premium-w3-542ed5-kserve-56d8467ff6-mqzvc" event={"ID":"01ea8c54-637f-488b-b324-2eba77119902","Type":"ContainerDied","Data":"8ffc3fbf2a3f909e816bb92b5ced4352afc3c33a96fed59f56009d15a09cfba3"}
Sep 16 02:39:37.284919 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:37.284830    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-542ed5/premium-w3-542ed5-kserve-56d8467ff6-mqzvc" event={"ID":"01ea8c54-637f-488b-b324-2eba77119902","Type":"ContainerDied","Data":"dc93abd435d484faddd3743aac0fa37119449e6b7e3e2943a9d075e2fa1b2362"}
Sep 16 02:39:37.284919 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:37.284846    2579 scope.go:117] "RemoveContainer" containerID="8ffc3fbf2a3f909e816bb92b5ced4352afc3c33a96fed59f56009d15a09cfba3"
Sep 16 02:39:37.286316 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:37.286281    2579 generic.go:356] "Generic (PLEG): container finished" podID="99191806-ff18-4c21-afa7-701c9c9a85fa" containerID="91b0943e1e232cfcd0e07cb06546019798067ce42f9fb2626df3b9b1a16d90a1" exitCode=0
Sep 16 02:39:37.286457 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:37.286328    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-542ed5/simulator-w3-542ed5-kserve-77ff54b49-nwttm"
Sep 16 02:39:37.286457 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:37.286332    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-542ed5/simulator-w3-542ed5-kserve-77ff54b49-nwttm" event={"ID":"99191806-ff18-4c21-afa7-701c9c9a85fa","Type":"ContainerDied","Data":"91b0943e1e232cfcd0e07cb06546019798067ce42f9fb2626df3b9b1a16d90a1"}
Sep 16 02:39:37.286457 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:37.286365    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-542ed5/simulator-w3-542ed5-kserve-77ff54b49-nwttm" event={"ID":"99191806-ff18-4c21-afa7-701c9c9a85fa","Type":"ContainerDied","Data":"2f9f486eadf2682f2cbabe56581e050a665ef9e5ef1eeea139ba307f09117b34"}
Sep 16 02:39:37.288214 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:37.288186    2579 generic.go:356] "Generic (PLEG): container finished" podID="44750d09-f1af-47cb-9a01-ce4b408f092f" containerID="e71e2b36976904eff4f8c247ea886a51f195b055480dde7644a5567ec4be98d8" exitCode=0
Sep 16 02:39:37.288338 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:37.288222    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-542ed5/distinct-w3-542ed5-kserve-64fdd5d5f-jjxfp"
Sep 16 02:39:37.288338 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:37.288247    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-542ed5/distinct-w3-542ed5-kserve-64fdd5d5f-jjxfp" event={"ID":"44750d09-f1af-47cb-9a01-ce4b408f092f","Type":"ContainerDied","Data":"e71e2b36976904eff4f8c247ea886a51f195b055480dde7644a5567ec4be98d8"}
Sep 16 02:39:37.288338 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:37.288272    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-542ed5/distinct-w3-542ed5-kserve-64fdd5d5f-jjxfp" event={"ID":"44750d09-f1af-47cb-9a01-ce4b408f092f","Type":"ContainerDied","Data":"8ed3e033c8fd0ca35b23f214af1e18d5f9d5a93f4a1f60e98b382fef7fc629bd"}
Sep 16 02:39:37.295216 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:37.295200    2579 scope.go:117] "RemoveContainer" containerID="8ffc3fbf2a3f909e816bb92b5ced4352afc3c33a96fed59f56009d15a09cfba3"
Sep 16 02:39:37.295616 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:39:37.295596    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8ffc3fbf2a3f909e816bb92b5ced4352afc3c33a96fed59f56009d15a09cfba3\": container with ID starting with 8ffc3fbf2a3f909e816bb92b5ced4352afc3c33a96fed59f56009d15a09cfba3 not found: ID does not exist" containerID="8ffc3fbf2a3f909e816bb92b5ced4352afc3c33a96fed59f56009d15a09cfba3"
Sep 16 02:39:37.295718 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:37.295626    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8ffc3fbf2a3f909e816bb92b5ced4352afc3c33a96fed59f56009d15a09cfba3"} err="failed to get container status \"8ffc3fbf2a3f909e816bb92b5ced4352afc3c33a96fed59f56009d15a09cfba3\": rpc error: code = NotFound desc = could not find container \"8ffc3fbf2a3f909e816bb92b5ced4352afc3c33a96fed59f56009d15a09cfba3\": container with ID starting with 8ffc3fbf2a3f909e816bb92b5ced4352afc3c33a96fed59f56009d15a09cfba3 not found: ID does not exist"
Sep 16 02:39:37.295718 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:37.295648    2579 scope.go:117] "RemoveContainer" containerID="91b0943e1e232cfcd0e07cb06546019798067ce42f9fb2626df3b9b1a16d90a1"
Sep 16 02:39:37.304618 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:37.304604    2579 scope.go:117] "RemoveContainer" containerID="91b0943e1e232cfcd0e07cb06546019798067ce42f9fb2626df3b9b1a16d90a1"
Sep 16 02:39:37.304873 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:39:37.304855    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"91b0943e1e232cfcd0e07cb06546019798067ce42f9fb2626df3b9b1a16d90a1\": container with ID starting with 91b0943e1e232cfcd0e07cb06546019798067ce42f9fb2626df3b9b1a16d90a1 not found: ID does not exist" containerID="91b0943e1e232cfcd0e07cb06546019798067ce42f9fb2626df3b9b1a16d90a1"
Sep 16 02:39:37.304925 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:37.304878    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"91b0943e1e232cfcd0e07cb06546019798067ce42f9fb2626df3b9b1a16d90a1"} err="failed to get container status \"91b0943e1e232cfcd0e07cb06546019798067ce42f9fb2626df3b9b1a16d90a1\": rpc error: code = NotFound desc = could not find container \"91b0943e1e232cfcd0e07cb06546019798067ce42f9fb2626df3b9b1a16d90a1\": container with ID starting with 91b0943e1e232cfcd0e07cb06546019798067ce42f9fb2626df3b9b1a16d90a1 not found: ID does not exist"
Sep 16 02:39:37.304925 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:37.304893    2579 scope.go:117] "RemoveContainer" containerID="e71e2b36976904eff4f8c247ea886a51f195b055480dde7644a5567ec4be98d8"
Sep 16 02:39:37.311606 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:37.311584    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-542ed5/premium-w3-542ed5-kserve-56d8467ff6-mqzvc"]
Sep 16 02:39:37.313696 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:37.313677    2579 scope.go:117] "RemoveContainer" containerID="e71e2b36976904eff4f8c247ea886a51f195b055480dde7644a5567ec4be98d8"
Sep 16 02:39:37.313997 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:39:37.313973    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e71e2b36976904eff4f8c247ea886a51f195b055480dde7644a5567ec4be98d8\": container with ID starting with e71e2b36976904eff4f8c247ea886a51f195b055480dde7644a5567ec4be98d8 not found: ID does not exist" containerID="e71e2b36976904eff4f8c247ea886a51f195b055480dde7644a5567ec4be98d8"
Sep 16 02:39:37.314081 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:37.314002    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e71e2b36976904eff4f8c247ea886a51f195b055480dde7644a5567ec4be98d8"} err="failed to get container status \"e71e2b36976904eff4f8c247ea886a51f195b055480dde7644a5567ec4be98d8\": rpc error: code = NotFound desc = could not find container \"e71e2b36976904eff4f8c247ea886a51f195b055480dde7644a5567ec4be98d8\": container with ID starting with e71e2b36976904eff4f8c247ea886a51f195b055480dde7644a5567ec4be98d8 not found: ID does not exist"
Sep 16 02:39:37.314796 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:37.314775    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w3-542ed5/premium-w3-542ed5-kserve-56d8467ff6-mqzvc"]
Sep 16 02:39:37.323393 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:37.323368    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-542ed5/simulator-w3-542ed5-kserve-77ff54b49-nwttm"]
Sep 16 02:39:37.326601 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:37.326574    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w3-542ed5/simulator-w3-542ed5-kserve-77ff54b49-nwttm"]
Sep 16 02:39:37.335461 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:37.335437    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-542ed5/distinct-w3-542ed5-kserve-64fdd5d5f-jjxfp"]
Sep 16 02:39:37.340828 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:37.340808    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w3-542ed5/distinct-w3-542ed5-kserve-64fdd5d5f-jjxfp"]
Sep 16 02:39:37.828058 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:37.828027    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:39:38.103189 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:38.103150    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="01ea8c54-637f-488b-b324-2eba77119902" path="/var/lib/kubelet/pods/01ea8c54-637f-488b-b324-2eba77119902/volumes"
Sep 16 02:39:38.103647 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:38.103626    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="44750d09-f1af-47cb-9a01-ce4b408f092f" path="/var/lib/kubelet/pods/44750d09-f1af-47cb-9a01-ce4b408f092f/volumes"
Sep 16 02:39:38.104037 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:38.104019    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="99191806-ff18-4c21-afa7-701c9c9a85fa" path="/var/lib/kubelet/pods/99191806-ff18-4c21-afa7-701c9c9a85fa/volumes"
Sep 16 02:39:38.454821 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:38.454791    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:39:38.957457 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:38.957393    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:39:39.019642 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:39.019610    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:39:39.100325 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:39.100293    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:39:39.307278 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:39.307177    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:39:40.234733 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:40.234694    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:39:41.028791 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:41.028759    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:39:41.126812 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:41.126775    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:39:41.332883 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:41.332809    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-619550/distinct-2-w0-619550-kserve-6949cd8666-knndp"]
Sep 16 02:39:41.368610 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:41.368579    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-619550/simulator-w0-619550-kserve-5958b89dff-t28jc"]
Sep 16 02:39:41.416039 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:41.416005    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:39:41.574746 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:41.574709    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:39:41.895604 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:41.895559    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-542ed5/simulator-w3-542ed5-kserve-77ff54b49-nwttm" podUID="99191806-ff18-4c21-afa7-701c9c9a85fa" containerName="main" probeResult="failure" output="Get \"https://10.132.0.52:8000/ready\": context deadline exceeded"
Sep 16 02:39:46.529743 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:46.529706    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:39:47.801186 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:47.801144    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:39:47.948534 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:47.948493    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:39:48.334356 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:48.334325    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:39:49.420058 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:49.420019    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:39:50.141967 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:50.141926    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:39:50.938097 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:50.938062    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:39:50.966965 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:50.966917    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-619550-5984cc5ccf-58g44"]
Sep 16 02:39:50.967280 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:50.967250    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w0-619550-5984cc5ccf-58g44" podUID="f84078e6-c7af-4767-8fd4-c68bb7d27bfa" containerName="payload-processing" containerID="cri-o://1e5ad5ca25ab553a3afc3e699a07eb2f3018d8759632959c358c1fa0ce951221" gracePeriod=30
Sep 16 02:39:50.974102 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:50.974069    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-619550-597b7898-4xksl"]
Sep 16 02:39:50.974408 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:50.974309    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-619550-597b7898-4xksl" podUID="452d6b9b-47b4-458f-a332-9c2ed48c3266" containerName="payload-pre-processing" containerID="cri-o://f369ea688d4aa23f2b39a11da4f87356b339216a551f9fdeeea6433f43d78d65" gracePeriod=30
Sep 16 02:39:51.006098 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:51.006072    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:39:51.067063 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:51.067034    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:39:51.169308 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:51.169276    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:39:51.232268 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:51.232171    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:39:51.260485 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:51.260450    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-619550-5984cc5ccf-58g44"
Sep 16 02:39:51.263543 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:51.263523    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-619550-597b7898-4xksl"
Sep 16 02:39:51.300053 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:51.300018    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:39:51.361400 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:51.361374    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z9s2t\" (UniqueName: \"kubernetes.io/projected/452d6b9b-47b4-458f-a332-9c2ed48c3266-kube-api-access-z9s2t\") pod \"452d6b9b-47b4-458f-a332-9c2ed48c3266\" (UID: \"452d6b9b-47b4-458f-a332-9c2ed48c3266\") "
Sep 16 02:39:51.361609 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:51.361427    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f84078e6-c7af-4767-8fd4-c68bb7d27bfa-plugins-config-volume\") pod \"f84078e6-c7af-4767-8fd4-c68bb7d27bfa\" (UID: \"f84078e6-c7af-4767-8fd4-c68bb7d27bfa\") "
Sep 16 02:39:51.361609 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:51.361496    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/452d6b9b-47b4-458f-a332-9c2ed48c3266-plugins-config-volume\") pod \"452d6b9b-47b4-458f-a332-9c2ed48c3266\" (UID: \"452d6b9b-47b4-458f-a332-9c2ed48c3266\") "
Sep 16 02:39:51.361609 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:51.361586    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t848s\" (UniqueName: \"kubernetes.io/projected/f84078e6-c7af-4767-8fd4-c68bb7d27bfa-kube-api-access-t848s\") pod \"f84078e6-c7af-4767-8fd4-c68bb7d27bfa\" (UID: \"f84078e6-c7af-4767-8fd4-c68bb7d27bfa\") "
Sep 16 02:39:51.361909 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:51.361873    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f84078e6-c7af-4767-8fd4-c68bb7d27bfa-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f84078e6-c7af-4767-8fd4-c68bb7d27bfa" (UID: "f84078e6-c7af-4767-8fd4-c68bb7d27bfa"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 02:39:51.361981 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:51.361918    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/452d6b9b-47b4-458f-a332-9c2ed48c3266-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "452d6b9b-47b4-458f-a332-9c2ed48c3266" (UID: "452d6b9b-47b4-458f-a332-9c2ed48c3266"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 02:39:51.363894 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:51.363854    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/452d6b9b-47b4-458f-a332-9c2ed48c3266-kube-api-access-z9s2t" (OuterVolumeSpecName: "kube-api-access-z9s2t") pod "452d6b9b-47b4-458f-a332-9c2ed48c3266" (UID: "452d6b9b-47b4-458f-a332-9c2ed48c3266"). InnerVolumeSpecName "kube-api-access-z9s2t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 02:39:51.363982 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:51.363923    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f84078e6-c7af-4767-8fd4-c68bb7d27bfa-kube-api-access-t848s" (OuterVolumeSpecName: "kube-api-access-t848s") pod "f84078e6-c7af-4767-8fd4-c68bb7d27bfa" (UID: "f84078e6-c7af-4767-8fd4-c68bb7d27bfa"). InnerVolumeSpecName "kube-api-access-t848s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 02:39:51.373422 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:51.373393    2579 generic.go:356] "Generic (PLEG): container finished" podID="452d6b9b-47b4-458f-a332-9c2ed48c3266" containerID="f369ea688d4aa23f2b39a11da4f87356b339216a551f9fdeeea6433f43d78d65" exitCode=0
Sep 16 02:39:51.373521 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:51.373443    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-619550-597b7898-4xksl"
Sep 16 02:39:51.373521 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:51.373504    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-619550-597b7898-4xksl" event={"ID":"452d6b9b-47b4-458f-a332-9c2ed48c3266","Type":"ContainerDied","Data":"f369ea688d4aa23f2b39a11da4f87356b339216a551f9fdeeea6433f43d78d65"}
Sep 16 02:39:51.373615 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:51.373557    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-619550-597b7898-4xksl" event={"ID":"452d6b9b-47b4-458f-a332-9c2ed48c3266","Type":"ContainerDied","Data":"90d644c6f542ba17f289ac4193db6a71f904420011676fb40adf85949c2aab82"}
Sep 16 02:39:51.373615 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:51.373581    2579 scope.go:117] "RemoveContainer" containerID="f369ea688d4aa23f2b39a11da4f87356b339216a551f9fdeeea6433f43d78d65"
Sep 16 02:39:51.375204 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:51.375181    2579 generic.go:356] "Generic (PLEG): container finished" podID="f84078e6-c7af-4767-8fd4-c68bb7d27bfa" containerID="1e5ad5ca25ab553a3afc3e699a07eb2f3018d8759632959c358c1fa0ce951221" exitCode=0
Sep 16 02:39:51.375319 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:51.375274    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-619550-5984cc5ccf-58g44" event={"ID":"f84078e6-c7af-4767-8fd4-c68bb7d27bfa","Type":"ContainerDied","Data":"1e5ad5ca25ab553a3afc3e699a07eb2f3018d8759632959c358c1fa0ce951221"}
Sep 16 02:39:51.375319 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:51.375284    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-619550-5984cc5ccf-58g44"
Sep 16 02:39:51.375319 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:51.375313    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-619550-5984cc5ccf-58g44" event={"ID":"f84078e6-c7af-4767-8fd4-c68bb7d27bfa","Type":"ContainerDied","Data":"c956be1654b3d392f6c9e5cc4e0d9373377cae334a7434476fe7c9b519e68bfc"}
Sep 16 02:39:51.385007 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:51.384987    2579 scope.go:117] "RemoveContainer" containerID="f369ea688d4aa23f2b39a11da4f87356b339216a551f9fdeeea6433f43d78d65"
Sep 16 02:39:51.385344 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:39:51.385315    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f369ea688d4aa23f2b39a11da4f87356b339216a551f9fdeeea6433f43d78d65\": container with ID starting with f369ea688d4aa23f2b39a11da4f87356b339216a551f9fdeeea6433f43d78d65 not found: ID does not exist" containerID="f369ea688d4aa23f2b39a11da4f87356b339216a551f9fdeeea6433f43d78d65"
Sep 16 02:39:51.385461 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:51.385353    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f369ea688d4aa23f2b39a11da4f87356b339216a551f9fdeeea6433f43d78d65"} err="failed to get container status \"f369ea688d4aa23f2b39a11da4f87356b339216a551f9fdeeea6433f43d78d65\": rpc error: code = NotFound desc = could not find container \"f369ea688d4aa23f2b39a11da4f87356b339216a551f9fdeeea6433f43d78d65\": container with ID starting with f369ea688d4aa23f2b39a11da4f87356b339216a551f9fdeeea6433f43d78d65 not found: ID does not exist"
Sep 16 02:39:51.385461 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:51.385377    2579 scope.go:117] "RemoveContainer" containerID="1e5ad5ca25ab553a3afc3e699a07eb2f3018d8759632959c358c1fa0ce951221"
Sep 16 02:39:51.394036 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:51.394020    2579 scope.go:117] "RemoveContainer" containerID="1e5ad5ca25ab553a3afc3e699a07eb2f3018d8759632959c358c1fa0ce951221"
Sep 16 02:39:51.394292 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:39:51.394273    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1e5ad5ca25ab553a3afc3e699a07eb2f3018d8759632959c358c1fa0ce951221\": container with ID starting with 1e5ad5ca25ab553a3afc3e699a07eb2f3018d8759632959c358c1fa0ce951221 not found: ID does not exist" containerID="1e5ad5ca25ab553a3afc3e699a07eb2f3018d8759632959c358c1fa0ce951221"
Sep 16 02:39:51.394336 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:51.394299    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1e5ad5ca25ab553a3afc3e699a07eb2f3018d8759632959c358c1fa0ce951221"} err="failed to get container status \"1e5ad5ca25ab553a3afc3e699a07eb2f3018d8759632959c358c1fa0ce951221\": rpc error: code = NotFound desc = could not find container \"1e5ad5ca25ab553a3afc3e699a07eb2f3018d8759632959c358c1fa0ce951221\": container with ID starting with 1e5ad5ca25ab553a3afc3e699a07eb2f3018d8759632959c358c1fa0ce951221 not found: ID does not exist"
Sep 16 02:39:51.394549 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:51.394531    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-619550-597b7898-4xksl"]
Sep 16 02:39:51.399101 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:51.399083    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-619550-597b7898-4xksl"]
Sep 16 02:39:51.408369 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:51.408351    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-619550-5984cc5ccf-58g44"]
Sep 16 02:39:51.411459 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:51.411434    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-619550-5984cc5ccf-58g44"]
Sep 16 02:39:51.463085 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:51.463051    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/452d6b9b-47b4-458f-a332-9c2ed48c3266-plugins-config-volume\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:39:51.463085 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:51.463078    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t848s\" (UniqueName: \"kubernetes.io/projected/f84078e6-c7af-4767-8fd4-c68bb7d27bfa-kube-api-access-t848s\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:39:51.463251 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:51.463094    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z9s2t\" (UniqueName: \"kubernetes.io/projected/452d6b9b-47b4-458f-a332-9c2ed48c3266-kube-api-access-z9s2t\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:39:51.463251 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:51.463107    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f84078e6-c7af-4767-8fd4-c68bb7d27bfa-plugins-config-volume\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:39:51.599664 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:51.599584    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:39:52.105908 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:52.105835    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="452d6b9b-47b4-458f-a332-9c2ed48c3266" path="/var/lib/kubelet/pods/452d6b9b-47b4-458f-a332-9c2ed48c3266/volumes"
Sep 16 02:39:52.106496 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:52.106453    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f84078e6-c7af-4767-8fd4-c68bb7d27bfa" path="/var/lib/kubelet/pods/f84078e6-c7af-4767-8fd4-c68bb7d27bfa/volumes"
Sep 16 02:39:52.111663 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:52.111639    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:39:52.823371 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:52.823327    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:39:53.621345 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:53.621307    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:39:56.355001 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:56.354957    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-619550/distinct-2-w0-619550-kserve-6949cd8666-knndp" podUID="72a6b825-de7c-4806-8db6-f116f9cd615b" containerName="main" containerID="cri-o://d4aaad0ca86a465ba9be107d4334c51a7278b10c12f93acc0e67c40247429b75" gracePeriod=45
Sep 16 02:39:56.388543 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:56.388506    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-619550/simulator-w0-619550-kserve-5958b89dff-t28jc" podUID="b6e49a5a-de18-4720-aaac-b1b3b1c617b7" containerName="main" containerID="cri-o://4bdcd06293f14fdfee8e2881fb8c0c4b4bdf4143016db237779f96adb5cc2a26" gracePeriod=45
Sep 16 02:39:56.627692 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:56.627662    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-619550/distinct-2-w0-619550-kserve-6949cd8666-knndp"
Sep 16 02:39:56.682092 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:56.682063    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-619550/simulator-w0-619550-kserve-5958b89dff-t28jc"
Sep 16 02:39:56.814496 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:56.814438    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b6e49a5a-de18-4720-aaac-b1b3b1c617b7-tmp-dir\") pod \"b6e49a5a-de18-4720-aaac-b1b3b1c617b7\" (UID: \"b6e49a5a-de18-4720-aaac-b1b3b1c617b7\") "
Sep 16 02:39:56.814692 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:56.814507    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b6e49a5a-de18-4720-aaac-b1b3b1c617b7-tls-certs\") pod \"b6e49a5a-de18-4720-aaac-b1b3b1c617b7\" (UID: \"b6e49a5a-de18-4720-aaac-b1b3b1c617b7\") "
Sep 16 02:39:56.814692 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:56.814530    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/72a6b825-de7c-4806-8db6-f116f9cd615b-dshm\") pod \"72a6b825-de7c-4806-8db6-f116f9cd615b\" (UID: \"72a6b825-de7c-4806-8db6-f116f9cd615b\") "
Sep 16 02:39:56.814692 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:56.814555    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b6e49a5a-de18-4720-aaac-b1b3b1c617b7-model-cache\") pod \"b6e49a5a-de18-4720-aaac-b1b3b1c617b7\" (UID: \"b6e49a5a-de18-4720-aaac-b1b3b1c617b7\") "
Sep 16 02:39:56.814692 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:56.814574    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5csn9\" (UniqueName: \"kubernetes.io/projected/72a6b825-de7c-4806-8db6-f116f9cd615b-kube-api-access-5csn9\") pod \"72a6b825-de7c-4806-8db6-f116f9cd615b\" (UID: \"72a6b825-de7c-4806-8db6-f116f9cd615b\") "
Sep 16 02:39:56.814692 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:56.814592    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/72a6b825-de7c-4806-8db6-f116f9cd615b-model-cache\") pod \"72a6b825-de7c-4806-8db6-f116f9cd615b\" (UID: \"72a6b825-de7c-4806-8db6-f116f9cd615b\") "
Sep 16 02:39:56.814692 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:56.814651    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b6e49a5a-de18-4720-aaac-b1b3b1c617b7-home\") pod \"b6e49a5a-de18-4720-aaac-b1b3b1c617b7\" (UID: \"b6e49a5a-de18-4720-aaac-b1b3b1c617b7\") "
Sep 16 02:39:56.815000 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:56.814707    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b6e49a5a-de18-4720-aaac-b1b3b1c617b7-dshm\") pod \"b6e49a5a-de18-4720-aaac-b1b3b1c617b7\" (UID: \"b6e49a5a-de18-4720-aaac-b1b3b1c617b7\") "
Sep 16 02:39:56.815000 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:56.814731    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4mggc\" (UniqueName: \"kubernetes.io/projected/b6e49a5a-de18-4720-aaac-b1b3b1c617b7-kube-api-access-4mggc\") pod \"b6e49a5a-de18-4720-aaac-b1b3b1c617b7\" (UID: \"b6e49a5a-de18-4720-aaac-b1b3b1c617b7\") "
Sep 16 02:39:56.815000 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:56.814737    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6e49a5a-de18-4720-aaac-b1b3b1c617b7-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b6e49a5a-de18-4720-aaac-b1b3b1c617b7" (UID: "b6e49a5a-de18-4720-aaac-b1b3b1c617b7"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 02:39:56.815000 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:56.814760    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/72a6b825-de7c-4806-8db6-f116f9cd615b-tls-certs\") pod \"72a6b825-de7c-4806-8db6-f116f9cd615b\" (UID: \"72a6b825-de7c-4806-8db6-f116f9cd615b\") "
Sep 16 02:39:56.815000 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:56.814790    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/72a6b825-de7c-4806-8db6-f116f9cd615b-home\") pod \"72a6b825-de7c-4806-8db6-f116f9cd615b\" (UID: \"72a6b825-de7c-4806-8db6-f116f9cd615b\") "
Sep 16 02:39:56.815000 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:56.814815    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/72a6b825-de7c-4806-8db6-f116f9cd615b-tmp-dir\") pod \"72a6b825-de7c-4806-8db6-f116f9cd615b\" (UID: \"72a6b825-de7c-4806-8db6-f116f9cd615b\") "
Sep 16 02:39:56.815000 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:56.814877    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6e49a5a-de18-4720-aaac-b1b3b1c617b7-model-cache" (OuterVolumeSpecName: "model-cache") pod "b6e49a5a-de18-4720-aaac-b1b3b1c617b7" (UID: "b6e49a5a-de18-4720-aaac-b1b3b1c617b7"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 02:39:56.815000 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:56.814876    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/72a6b825-de7c-4806-8db6-f116f9cd615b-model-cache" (OuterVolumeSpecName: "model-cache") pod "72a6b825-de7c-4806-8db6-f116f9cd615b" (UID: "72a6b825-de7c-4806-8db6-f116f9cd615b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 02:39:56.815419 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:56.815009    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6e49a5a-de18-4720-aaac-b1b3b1c617b7-home" (OuterVolumeSpecName: "home") pod "b6e49a5a-de18-4720-aaac-b1b3b1c617b7" (UID: "b6e49a5a-de18-4720-aaac-b1b3b1c617b7"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 02:39:56.815419 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:56.815212    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/72a6b825-de7c-4806-8db6-f116f9cd615b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "72a6b825-de7c-4806-8db6-f116f9cd615b" (UID: "72a6b825-de7c-4806-8db6-f116f9cd615b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 02:39:56.815419 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:56.815325    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b6e49a5a-de18-4720-aaac-b1b3b1c617b7-tmp-dir\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:39:56.815419 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:56.815345    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b6e49a5a-de18-4720-aaac-b1b3b1c617b7-model-cache\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:39:56.815419 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:56.815359    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/72a6b825-de7c-4806-8db6-f116f9cd615b-model-cache\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:39:56.815419 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:56.815372    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b6e49a5a-de18-4720-aaac-b1b3b1c617b7-home\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:39:56.815918 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:56.815562    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/72a6b825-de7c-4806-8db6-f116f9cd615b-home" (OuterVolumeSpecName: "home") pod "72a6b825-de7c-4806-8db6-f116f9cd615b" (UID: "72a6b825-de7c-4806-8db6-f116f9cd615b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 02:39:56.817211 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:56.817172    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/72a6b825-de7c-4806-8db6-f116f9cd615b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "72a6b825-de7c-4806-8db6-f116f9cd615b" (UID: "72a6b825-de7c-4806-8db6-f116f9cd615b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 02:39:56.817211 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:56.817188    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/72a6b825-de7c-4806-8db6-f116f9cd615b-kube-api-access-5csn9" (OuterVolumeSpecName: "kube-api-access-5csn9") pod "72a6b825-de7c-4806-8db6-f116f9cd615b" (UID: "72a6b825-de7c-4806-8db6-f116f9cd615b"). InnerVolumeSpecName "kube-api-access-5csn9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 02:39:56.817489 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:56.817432    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6e49a5a-de18-4720-aaac-b1b3b1c617b7-dshm" (OuterVolumeSpecName: "dshm") pod "b6e49a5a-de18-4720-aaac-b1b3b1c617b7" (UID: "b6e49a5a-de18-4720-aaac-b1b3b1c617b7"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 02:39:56.817489 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:56.817434    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/72a6b825-de7c-4806-8db6-f116f9cd615b-dshm" (OuterVolumeSpecName: "dshm") pod "72a6b825-de7c-4806-8db6-f116f9cd615b" (UID: "72a6b825-de7c-4806-8db6-f116f9cd615b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 02:39:56.817634 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:56.817616    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b6e49a5a-de18-4720-aaac-b1b3b1c617b7-kube-api-access-4mggc" (OuterVolumeSpecName: "kube-api-access-4mggc") pod "b6e49a5a-de18-4720-aaac-b1b3b1c617b7" (UID: "b6e49a5a-de18-4720-aaac-b1b3b1c617b7"). InnerVolumeSpecName "kube-api-access-4mggc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 02:39:56.817715 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:56.817698    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b6e49a5a-de18-4720-aaac-b1b3b1c617b7-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b6e49a5a-de18-4720-aaac-b1b3b1c617b7" (UID: "b6e49a5a-de18-4720-aaac-b1b3b1c617b7"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 02:39:56.916626 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:56.916555    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b6e49a5a-de18-4720-aaac-b1b3b1c617b7-tls-certs\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:39:56.916626 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:56.916577    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/72a6b825-de7c-4806-8db6-f116f9cd615b-dshm\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:39:56.916626 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:56.916586    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5csn9\" (UniqueName: \"kubernetes.io/projected/72a6b825-de7c-4806-8db6-f116f9cd615b-kube-api-access-5csn9\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:39:56.916626 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:56.916594    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b6e49a5a-de18-4720-aaac-b1b3b1c617b7-dshm\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:39:56.916626 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:56.916603    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4mggc\" (UniqueName: \"kubernetes.io/projected/b6e49a5a-de18-4720-aaac-b1b3b1c617b7-kube-api-access-4mggc\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:39:56.916626 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:56.916611    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/72a6b825-de7c-4806-8db6-f116f9cd615b-tls-certs\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:39:56.916626 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:56.916619    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/72a6b825-de7c-4806-8db6-f116f9cd615b-home\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:39:56.916626 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:56.916626    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/72a6b825-de7c-4806-8db6-f116f9cd615b-tmp-dir\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:39:57.409031 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:57.408998    2579 generic.go:356] "Generic (PLEG): container finished" podID="b6e49a5a-de18-4720-aaac-b1b3b1c617b7" containerID="4bdcd06293f14fdfee8e2881fb8c0c4b4bdf4143016db237779f96adb5cc2a26" exitCode=0
Sep 16 02:39:57.409498 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:57.409045    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-619550/simulator-w0-619550-kserve-5958b89dff-t28jc"
Sep 16 02:39:57.409498 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:57.409088    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-619550/simulator-w0-619550-kserve-5958b89dff-t28jc" event={"ID":"b6e49a5a-de18-4720-aaac-b1b3b1c617b7","Type":"ContainerDied","Data":"4bdcd06293f14fdfee8e2881fb8c0c4b4bdf4143016db237779f96adb5cc2a26"}
Sep 16 02:39:57.409498 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:57.409128    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-619550/simulator-w0-619550-kserve-5958b89dff-t28jc" event={"ID":"b6e49a5a-de18-4720-aaac-b1b3b1c617b7","Type":"ContainerDied","Data":"31aa4d2c5db9b93de40e53455bd96863a724c5ba7b2d7fe9b5ab89ba78f067c7"}
Sep 16 02:39:57.409498 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:57.409146    2579 scope.go:117] "RemoveContainer" containerID="4bdcd06293f14fdfee8e2881fb8c0c4b4bdf4143016db237779f96adb5cc2a26"
Sep 16 02:39:57.410607 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:57.410412    2579 generic.go:356] "Generic (PLEG): container finished" podID="72a6b825-de7c-4806-8db6-f116f9cd615b" containerID="d4aaad0ca86a465ba9be107d4334c51a7278b10c12f93acc0e67c40247429b75" exitCode=0
Sep 16 02:39:57.410607 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:57.410442    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-619550/distinct-2-w0-619550-kserve-6949cd8666-knndp" event={"ID":"72a6b825-de7c-4806-8db6-f116f9cd615b","Type":"ContainerDied","Data":"d4aaad0ca86a465ba9be107d4334c51a7278b10c12f93acc0e67c40247429b75"}
Sep 16 02:39:57.410607 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:57.410452    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-619550/distinct-2-w0-619550-kserve-6949cd8666-knndp"
Sep 16 02:39:57.410607 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:57.410487    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-619550/distinct-2-w0-619550-kserve-6949cd8666-knndp" event={"ID":"72a6b825-de7c-4806-8db6-f116f9cd615b","Type":"ContainerDied","Data":"5a5b891957a7cc788945e7586188be3b54a2f4585cfca7e0404e44e76b2ba69a"}
Sep 16 02:39:57.425587 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:57.421278    2579 scope.go:117] "RemoveContainer" containerID="4bdcd06293f14fdfee8e2881fb8c0c4b4bdf4143016db237779f96adb5cc2a26"
Sep 16 02:39:57.425587 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:39:57.423460    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4bdcd06293f14fdfee8e2881fb8c0c4b4bdf4143016db237779f96adb5cc2a26\": container with ID starting with 4bdcd06293f14fdfee8e2881fb8c0c4b4bdf4143016db237779f96adb5cc2a26 not found: ID does not exist" containerID="4bdcd06293f14fdfee8e2881fb8c0c4b4bdf4143016db237779f96adb5cc2a26"
Sep 16 02:39:57.425587 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:57.423521    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4bdcd06293f14fdfee8e2881fb8c0c4b4bdf4143016db237779f96adb5cc2a26"} err="failed to get container status \"4bdcd06293f14fdfee8e2881fb8c0c4b4bdf4143016db237779f96adb5cc2a26\": rpc error: code = NotFound desc = could not find container \"4bdcd06293f14fdfee8e2881fb8c0c4b4bdf4143016db237779f96adb5cc2a26\": container with ID starting with 4bdcd06293f14fdfee8e2881fb8c0c4b4bdf4143016db237779f96adb5cc2a26 not found: ID does not exist"
Sep 16 02:39:57.425587 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:57.423556    2579 scope.go:117] "RemoveContainer" containerID="d4aaad0ca86a465ba9be107d4334c51a7278b10c12f93acc0e67c40247429b75"
Sep 16 02:39:57.435183 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:57.435157    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-619550/simulator-w0-619550-kserve-5958b89dff-t28jc"]
Sep 16 02:39:57.437505 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:57.437484    2579 scope.go:117] "RemoveContainer" containerID="d4aaad0ca86a465ba9be107d4334c51a7278b10c12f93acc0e67c40247429b75"
Sep 16 02:39:57.437760 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:39:57.437742    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d4aaad0ca86a465ba9be107d4334c51a7278b10c12f93acc0e67c40247429b75\": container with ID starting with d4aaad0ca86a465ba9be107d4334c51a7278b10c12f93acc0e67c40247429b75 not found: ID does not exist" containerID="d4aaad0ca86a465ba9be107d4334c51a7278b10c12f93acc0e67c40247429b75"
Sep 16 02:39:57.437839 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:57.437773    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d4aaad0ca86a465ba9be107d4334c51a7278b10c12f93acc0e67c40247429b75"} err="failed to get container status \"d4aaad0ca86a465ba9be107d4334c51a7278b10c12f93acc0e67c40247429b75\": rpc error: code = NotFound desc = could not find container \"d4aaad0ca86a465ba9be107d4334c51a7278b10c12f93acc0e67c40247429b75\": container with ID starting with d4aaad0ca86a465ba9be107d4334c51a7278b10c12f93acc0e67c40247429b75 not found: ID does not exist"
Sep 16 02:39:57.439024 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:57.439002    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-619550/simulator-w0-619550-kserve-5958b89dff-t28jc"]
Sep 16 02:39:57.450722 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:57.450702    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-619550/distinct-2-w0-619550-kserve-6949cd8666-knndp"]
Sep 16 02:39:57.454548 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:57.454525    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:39:57.455237 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:57.455219    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-619550/distinct-2-w0-619550-kserve-6949cd8666-knndp"]
Sep 16 02:39:57.517224 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:57.517197    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:39:57.768899 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:57.768865    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:39:58.102699 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:58.102626    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="72a6b825-de7c-4806-8db6-f116f9cd615b" path="/var/lib/kubelet/pods/72a6b825-de7c-4806-8db6-f116f9cd615b/volumes"
Sep 16 02:39:58.103033 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:58.103019    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b6e49a5a-de18-4720-aaac-b1b3b1c617b7" path="/var/lib/kubelet/pods/b6e49a5a-de18-4720-aaac-b1b3b1c617b7/volumes"
Sep 16 02:39:58.374171 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:58.374094    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:39:58.474242 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:58.474211    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:39:58.909154 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:39:58.909123    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:40:01.710903 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:40:01.710863    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:40:01.779438 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:40:01.779405    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:40:01.838544 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:40:01.838506    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:40:02.021144 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:40:02.021053    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:40:02.094787 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:40:02.094738    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:40:02.096990 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:40:02.096968    2579 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-pre-processing-6cd864bc95-mh4jh" secret="" err="secret \"payload-processing-dockercfg-zzx6j\" not found"
Sep 16 02:40:02.325796 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:40:02.325706    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:40:02.425812 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:40:02.425780    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:40:03.066216 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:40:03.066183    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:40:04.655703 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:40:04.655665    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:40:07.956651 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:40:07.956613    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:40:08.053636 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:40:08.053602    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:40:08.268933 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:40:08.268842    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:40:08.687543 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:40:08.687505    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:40:08.780136 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:40:08.780097    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:40:08.924479 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:40:08.924448    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:40:14.623675 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:40:14.623643    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:40:14.733682 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:40:14.733640    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:40:14.824416 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:40:14.824382    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:40:15.018004 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:40:15.017970    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:40:18.516599 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:40:18.516558    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:40:18.566825 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:40:18.566790    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:40:18.768893 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:40:18.768800    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:40:21.025917 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:40:21.025885    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:40:21.084100 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:40:21.084045    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:40:22.056817 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:40:22.056786    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:40:22.399334 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:40:22.399241    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:40:22.606611 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:40:22.606573    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:40:24.277172 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:40:24.277140    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:40:25.112011 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:40:25.111965    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:40:25.254628 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:40:25.254588    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:40:25.353065 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:40:25.353035    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:40:26.416057 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:40:26.416014    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:40:26.515951 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:40:26.515921    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:40:26.571032 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:40:26.571004    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:40:26.876338 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:40:26.876309    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:40:26.970200 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:40:26.970161    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:40:29.870184 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:40:29.870148    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:40:38.280870 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:40:38.280835    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:40:49.414462 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:40:49.414418    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:40:50.032920 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:40:50.032883    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:40:50.438360 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:40:50.438329    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:40:50.852853 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:40:50.852772    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:40:50.953645 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:40:50.953611    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:40:51.180235 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:40:51.180159    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:40:51.471791 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:40:51.471759    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:40:54.492675 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:40:54.492641    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:40:54.628234 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:40:54.628191    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:40:54.733156 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:40:54.733109    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:40:55.435774 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:40:55.435744    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:40:56.434019 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:40:56.433983    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:40:56.533533 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:40:56.533498    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:40:56.884196 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:40:56.884165    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:40:56.982355 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:40:56.982320    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:40:56.983408 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:40:56.983388    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w1-8e10f1-84cb959658-664wr"]
Sep 16 02:40:56.983787 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:40:56.983774    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="01ea8c54-637f-488b-b324-2eba77119902" containerName="main"
Sep 16 02:40:56.983829 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:40:56.983788    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="01ea8c54-637f-488b-b324-2eba77119902" containerName="main"
Sep 16 02:40:56.983829 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:40:56.983803    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b6e49a5a-de18-4720-aaac-b1b3b1c617b7" containerName="main"
Sep 16 02:40:56.983829 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:40:56.983808    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="b6e49a5a-de18-4720-aaac-b1b3b1c617b7" containerName="main"
Sep 16 02:40:56.983829 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:40:56.983824    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2169a66a-2ca0-4cb1-81f8-6dcc588e9652" containerName="payload-pre-processing"
Sep 16 02:40:56.983955 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:40:56.983830    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="2169a66a-2ca0-4cb1-81f8-6dcc588e9652" containerName="payload-pre-processing"
Sep 16 02:40:56.983955 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:40:56.983837    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44750d09-f1af-47cb-9a01-ce4b408f092f" containerName="main"
Sep 16 02:40:56.983955 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:40:56.983842    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="44750d09-f1af-47cb-9a01-ce4b408f092f" containerName="main"
Sep 16 02:40:56.983955 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:40:56.983849    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="452d6b9b-47b4-458f-a332-9c2ed48c3266" containerName="payload-pre-processing"
Sep 16 02:40:56.983955 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:40:56.983854    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="452d6b9b-47b4-458f-a332-9c2ed48c3266" containerName="payload-pre-processing"
Sep 16 02:40:56.983955 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:40:56.983863    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="72a6b825-de7c-4806-8db6-f116f9cd615b" containerName="main"
Sep 16 02:40:56.983955 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:40:56.983867    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="72a6b825-de7c-4806-8db6-f116f9cd615b" containerName="main"
Sep 16 02:40:56.983955 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:40:56.983875    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f84078e6-c7af-4767-8fd4-c68bb7d27bfa" containerName="payload-processing"
Sep 16 02:40:56.983955 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:40:56.983881    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="f84078e6-c7af-4767-8fd4-c68bb7d27bfa" containerName="payload-processing"
Sep 16 02:40:56.983955 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:40:56.983886    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="99191806-ff18-4c21-afa7-701c9c9a85fa" containerName="main"
Sep 16 02:40:56.983955 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:40:56.983891    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="99191806-ff18-4c21-afa7-701c9c9a85fa" containerName="main"
Sep 16 02:40:56.984322 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:40:56.983965    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="99191806-ff18-4c21-afa7-701c9c9a85fa" containerName="main"
Sep 16 02:40:56.984322 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:40:56.983973    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="f84078e6-c7af-4767-8fd4-c68bb7d27bfa" containerName="payload-processing"
Sep 16 02:40:56.984322 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:40:56.983978    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="b6e49a5a-de18-4720-aaac-b1b3b1c617b7" containerName="main"
Sep 16 02:40:56.984322 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:40:56.983985    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="452d6b9b-47b4-458f-a332-9c2ed48c3266" containerName="payload-pre-processing"
Sep 16 02:40:56.984322 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:40:56.983990    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="2169a66a-2ca0-4cb1-81f8-6dcc588e9652" containerName="payload-pre-processing"
Sep 16 02:40:56.984322 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:40:56.983996    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="01ea8c54-637f-488b-b324-2eba77119902" containerName="main"
Sep 16 02:40:56.984322 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:40:56.984003    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="44750d09-f1af-47cb-9a01-ce4b408f092f" containerName="main"
Sep 16 02:40:56.984322 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:40:56.984008    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="72a6b825-de7c-4806-8db6-f116f9cd615b" containerName="main"
Sep 16 02:40:56.987209 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:40:56.987190    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-8e10f1-84cb959658-664wr"
Sep 16 02:40:56.989638 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:40:56.989615    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w1-8e10f1-dockercfg-l6xdw\""
Sep 16 02:40:56.989748 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:40:56.989668    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w1-8e10f1\""
Sep 16 02:40:56.994782 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:40:56.994759    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w1-8e10f1-84cb959658-664wr"]
Sep 16 02:40:57.018195 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:40:57.018166    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w1-8e10f1-77cb4456c5-nscms"]
Sep 16 02:40:57.021652 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:40:57.021634    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-8e10f1-77cb4456c5-nscms"
Sep 16 02:40:57.031241 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:40:57.031216    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w1-8e10f1-77cb4456c5-nscms"]
Sep 16 02:40:57.047254 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:40:57.047229    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:40:57.070098 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:40:57.070070    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j2kn2\" (UniqueName: \"kubernetes.io/projected/b6610657-a9dc-45d4-ba19-1a71096a5121-kube-api-access-j2kn2\") pod \"payload-pre-processing-e2e-shared-b-w1-8e10f1-84cb959658-664wr\" (UID: \"b6610657-a9dc-45d4-ba19-1a71096a5121\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-8e10f1-84cb959658-664wr"
Sep 16 02:40:57.070205 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:40:57.070139    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dde66f21-667a-4e8e-916f-04a8de158844-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w1-8e10f1-77cb4456c5-nscms\" (UID: \"dde66f21-667a-4e8e-916f-04a8de158844\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w1-8e10f1-77cb4456c5-nscms"
Sep 16 02:40:57.070205 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:40:57.070176    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r7nq8\" (UniqueName: \"kubernetes.io/projected/dde66f21-667a-4e8e-916f-04a8de158844-kube-api-access-r7nq8\") pod \"payload-processing-e2e-shared-b-w1-8e10f1-77cb4456c5-nscms\" (UID: \"dde66f21-667a-4e8e-916f-04a8de158844\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w1-8e10f1-77cb4456c5-nscms"
Sep 16 02:40:57.070301 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:40:57.070207    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b6610657-a9dc-45d4-ba19-1a71096a5121-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w1-8e10f1-84cb959658-664wr\" (UID: \"b6610657-a9dc-45d4-ba19-1a71096a5121\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-8e10f1-84cb959658-664wr"
Sep 16 02:40:57.170922 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:40:57.170842    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dde66f21-667a-4e8e-916f-04a8de158844-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w1-8e10f1-77cb4456c5-nscms\" (UID: \"dde66f21-667a-4e8e-916f-04a8de158844\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w1-8e10f1-77cb4456c5-nscms"
Sep 16 02:40:57.170922 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:40:57.170886    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r7nq8\" (UniqueName: \"kubernetes.io/projected/dde66f21-667a-4e8e-916f-04a8de158844-kube-api-access-r7nq8\") pod \"payload-processing-e2e-shared-b-w1-8e10f1-77cb4456c5-nscms\" (UID: \"dde66f21-667a-4e8e-916f-04a8de158844\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w1-8e10f1-77cb4456c5-nscms"
Sep 16 02:40:57.170922 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:40:57.170910    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b6610657-a9dc-45d4-ba19-1a71096a5121-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w1-8e10f1-84cb959658-664wr\" (UID: \"b6610657-a9dc-45d4-ba19-1a71096a5121\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-8e10f1-84cb959658-664wr"
Sep 16 02:40:57.171142 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:40:57.170946    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j2kn2\" (UniqueName: \"kubernetes.io/projected/b6610657-a9dc-45d4-ba19-1a71096a5121-kube-api-access-j2kn2\") pod \"payload-pre-processing-e2e-shared-b-w1-8e10f1-84cb959658-664wr\" (UID: \"b6610657-a9dc-45d4-ba19-1a71096a5121\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-8e10f1-84cb959658-664wr"
Sep 16 02:40:57.171808 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:40:57.171785    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b6610657-a9dc-45d4-ba19-1a71096a5121-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w1-8e10f1-84cb959658-664wr\" (UID: \"b6610657-a9dc-45d4-ba19-1a71096a5121\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-8e10f1-84cb959658-664wr"
Sep 16 02:40:57.171860 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:40:57.171785    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dde66f21-667a-4e8e-916f-04a8de158844-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w1-8e10f1-77cb4456c5-nscms\" (UID: \"dde66f21-667a-4e8e-916f-04a8de158844\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w1-8e10f1-77cb4456c5-nscms"
Sep 16 02:40:57.178608 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:40:57.178582    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j2kn2\" (UniqueName: \"kubernetes.io/projected/b6610657-a9dc-45d4-ba19-1a71096a5121-kube-api-access-j2kn2\") pod \"payload-pre-processing-e2e-shared-b-w1-8e10f1-84cb959658-664wr\" (UID: \"b6610657-a9dc-45d4-ba19-1a71096a5121\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-8e10f1-84cb959658-664wr"
Sep 16 02:40:57.178730 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:40:57.178582    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r7nq8\" (UniqueName: \"kubernetes.io/projected/dde66f21-667a-4e8e-916f-04a8de158844-kube-api-access-r7nq8\") pod \"payload-processing-e2e-shared-b-w1-8e10f1-77cb4456c5-nscms\" (UID: \"dde66f21-667a-4e8e-916f-04a8de158844\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w1-8e10f1-77cb4456c5-nscms"
Sep 16 02:40:57.298523 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:40:57.298488    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-8e10f1-84cb959658-664wr"
Sep 16 02:40:57.332619 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:40:57.332586    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-8e10f1-77cb4456c5-nscms"
Sep 16 02:40:57.333961 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:40:57.333931    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:40:57.571827 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:40:57.571804    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w1-8e10f1-84cb959658-664wr"]
Sep 16 02:40:57.574531 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:40:57.574501    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6610657_a9dc_45d4_ba19_1a71096a5121.slice/crio-32058704802a32ffc6dcba0b4321802b5f7f8088b7b59aa79ddad35335250cef WatchSource:0}: Error finding container 32058704802a32ffc6dcba0b4321802b5f7f8088b7b59aa79ddad35335250cef: Status 404 returned error can't find the container with id 32058704802a32ffc6dcba0b4321802b5f7f8088b7b59aa79ddad35335250cef
Sep 16 02:40:57.617610 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:40:57.617580    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w1-8e10f1-77cb4456c5-nscms"]
Sep 16 02:40:57.618070 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:40:57.618051    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddde66f21_667a_4e8e_916f_04a8de158844.slice/crio-40d2eea8cdd1617890f1d5fa9327c7cf50655c13555222b32a24ff6793cdc256 WatchSource:0}: Error finding container 40d2eea8cdd1617890f1d5fa9327c7cf50655c13555222b32a24ff6793cdc256: Status 404 returned error can't find the container with id 40d2eea8cdd1617890f1d5fa9327c7cf50655c13555222b32a24ff6793cdc256
Sep 16 02:40:57.724925 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:40:57.724889    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-8e10f1-84cb959658-664wr" event={"ID":"b6610657-a9dc-45d4-ba19-1a71096a5121","Type":"ContainerStarted","Data":"152d9ce369af04ab5ca001f759b8045875151089967facf29c51dfa9094ecb49"}
Sep 16 02:40:57.725117 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:40:57.724954    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-8e10f1-84cb959658-664wr"
Sep 16 02:40:57.725117 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:40:57.724980    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-8e10f1-84cb959658-664wr" event={"ID":"b6610657-a9dc-45d4-ba19-1a71096a5121","Type":"ContainerStarted","Data":"32058704802a32ffc6dcba0b4321802b5f7f8088b7b59aa79ddad35335250cef"}
Sep 16 02:40:57.726839 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:40:57.726805    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-8e10f1-77cb4456c5-nscms" event={"ID":"dde66f21-667a-4e8e-916f-04a8de158844","Type":"ContainerStarted","Data":"15aa2f030a99e6ae41ef0501175612b4eb5213f59b0ef6eb89a37a2a8317f1d0"}
Sep 16 02:40:57.726970 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:40:57.726862    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-8e10f1-77cb4456c5-nscms" event={"ID":"dde66f21-667a-4e8e-916f-04a8de158844","Type":"ContainerStarted","Data":"40d2eea8cdd1617890f1d5fa9327c7cf50655c13555222b32a24ff6793cdc256"}
Sep 16 02:40:57.726970 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:40:57.726892    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-8e10f1-77cb4456c5-nscms"
Sep 16 02:40:57.740981 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:40:57.740908    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-8e10f1-84cb959658-664wr" podStartSLOduration=1.740886378 podStartE2EDuration="1.740886378s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 02:40:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 02:40:57.740442278 +0000 UTC m=+1678.123957524" watchObservedRunningTime="2026-09-16 02:40:57.740886378 +0000 UTC m=+1678.124401630"
Sep 16 02:40:57.755109 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:40:57.755054    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-8e10f1-77cb4456c5-nscms" podStartSLOduration=1.755039724 podStartE2EDuration="1.755039724s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 02:40:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 02:40:57.754832926 +0000 UTC m=+1678.138348173" watchObservedRunningTime="2026-09-16 02:40:57.755039724 +0000 UTC m=+1678.138555007"
Sep 16 02:40:58.052746 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:40:58.052711    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:40:58.732955 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:40:58.732912    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:41:08.734982 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:08.734953    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-8e10f1-84cb959658-664wr"
Sep 16 02:41:08.735437 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:08.735021    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-8e10f1-77cb4456c5-nscms"
Sep 16 02:41:15.183395 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:15.183355    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w2-24aabe/auth-test-model-24aabe-kserve-d9468cd5f-4zt8g"]
Sep 16 02:41:15.206510 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:15.206442    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:41:16.097159 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:16.097129    2579 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-pre-processing-6cd864bc95-mh4jh" secret="" err="secret \"payload-processing-dockercfg-zzx6j\" not found"
Sep 16 02:41:16.104200 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:16.104165    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w2-a53a44/auth-test-model-a53a44-kserve-59f8574c79-7zfnt"]
Sep 16 02:41:16.112156 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:16.112122    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:41:18.220990 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:18.220950    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:41:18.917767 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:18.917722    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:41:19.712717 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:19.712677    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:41:19.815268 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:19.815240    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:41:20.466758 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:20.466719    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:41:22.173939 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:22.173866    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w2-24aabe/sub-test-model-24aabe-kserve-6ff87cb44f-qbc9q"]
Sep 16 02:41:22.178035 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:22.178013    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w2-24aabe/sub-test-model-24aabe-kserve-6ff87cb44f-qbc9q"
Sep 16 02:41:22.180289 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:22.180266    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w2-24aabe\"/\"sub-test-model-24aabe-kserve-self-signed-certs\""
Sep 16 02:41:22.186811 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:22.186785    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w2-24aabe/sub-test-model-24aabe-kserve-6ff87cb44f-qbc9q"]
Sep 16 02:41:22.209325 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:22.209284    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:41:22.209548 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:22.209530    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a56c54a4-83b5-44c8-8cb7-613642bebe2c-home\") pod \"sub-test-model-24aabe-kserve-6ff87cb44f-qbc9q\" (UID: \"a56c54a4-83b5-44c8-8cb7-613642bebe2c\") " pod="ai-tenant-e2e-shared-a-w2-24aabe/sub-test-model-24aabe-kserve-6ff87cb44f-qbc9q"
Sep 16 02:41:22.209620 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:22.209602    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a56c54a4-83b5-44c8-8cb7-613642bebe2c-tmp-dir\") pod \"sub-test-model-24aabe-kserve-6ff87cb44f-qbc9q\" (UID: \"a56c54a4-83b5-44c8-8cb7-613642bebe2c\") " pod="ai-tenant-e2e-shared-a-w2-24aabe/sub-test-model-24aabe-kserve-6ff87cb44f-qbc9q"
Sep 16 02:41:22.209678 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:22.209664    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a56c54a4-83b5-44c8-8cb7-613642bebe2c-tls-certs\") pod \"sub-test-model-24aabe-kserve-6ff87cb44f-qbc9q\" (UID: \"a56c54a4-83b5-44c8-8cb7-613642bebe2c\") " pod="ai-tenant-e2e-shared-a-w2-24aabe/sub-test-model-24aabe-kserve-6ff87cb44f-qbc9q"
Sep 16 02:41:22.209723 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:22.209687    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a56c54a4-83b5-44c8-8cb7-613642bebe2c-dshm\") pod \"sub-test-model-24aabe-kserve-6ff87cb44f-qbc9q\" (UID: \"a56c54a4-83b5-44c8-8cb7-613642bebe2c\") " pod="ai-tenant-e2e-shared-a-w2-24aabe/sub-test-model-24aabe-kserve-6ff87cb44f-qbc9q"
Sep 16 02:41:22.209759 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:22.209733    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a56c54a4-83b5-44c8-8cb7-613642bebe2c-model-cache\") pod \"sub-test-model-24aabe-kserve-6ff87cb44f-qbc9q\" (UID: \"a56c54a4-83b5-44c8-8cb7-613642bebe2c\") " pod="ai-tenant-e2e-shared-a-w2-24aabe/sub-test-model-24aabe-kserve-6ff87cb44f-qbc9q"
Sep 16 02:41:22.209759 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:22.209750    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z7n54\" (UniqueName: \"kubernetes.io/projected/a56c54a4-83b5-44c8-8cb7-613642bebe2c-kube-api-access-z7n54\") pod \"sub-test-model-24aabe-kserve-6ff87cb44f-qbc9q\" (UID: \"a56c54a4-83b5-44c8-8cb7-613642bebe2c\") " pod="ai-tenant-e2e-shared-a-w2-24aabe/sub-test-model-24aabe-kserve-6ff87cb44f-qbc9q"
Sep 16 02:41:22.310136 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:22.310110    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a56c54a4-83b5-44c8-8cb7-613642bebe2c-tmp-dir\") pod \"sub-test-model-24aabe-kserve-6ff87cb44f-qbc9q\" (UID: \"a56c54a4-83b5-44c8-8cb7-613642bebe2c\") " pod="ai-tenant-e2e-shared-a-w2-24aabe/sub-test-model-24aabe-kserve-6ff87cb44f-qbc9q"
Sep 16 02:41:22.310245 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:22.310154    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a56c54a4-83b5-44c8-8cb7-613642bebe2c-tls-certs\") pod \"sub-test-model-24aabe-kserve-6ff87cb44f-qbc9q\" (UID: \"a56c54a4-83b5-44c8-8cb7-613642bebe2c\") " pod="ai-tenant-e2e-shared-a-w2-24aabe/sub-test-model-24aabe-kserve-6ff87cb44f-qbc9q"
Sep 16 02:41:22.310245 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:22.310170    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a56c54a4-83b5-44c8-8cb7-613642bebe2c-dshm\") pod \"sub-test-model-24aabe-kserve-6ff87cb44f-qbc9q\" (UID: \"a56c54a4-83b5-44c8-8cb7-613642bebe2c\") " pod="ai-tenant-e2e-shared-a-w2-24aabe/sub-test-model-24aabe-kserve-6ff87cb44f-qbc9q"
Sep 16 02:41:22.310245 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:22.310189    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a56c54a4-83b5-44c8-8cb7-613642bebe2c-model-cache\") pod \"sub-test-model-24aabe-kserve-6ff87cb44f-qbc9q\" (UID: \"a56c54a4-83b5-44c8-8cb7-613642bebe2c\") " pod="ai-tenant-e2e-shared-a-w2-24aabe/sub-test-model-24aabe-kserve-6ff87cb44f-qbc9q"
Sep 16 02:41:22.310245 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:22.310207    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z7n54\" (UniqueName: \"kubernetes.io/projected/a56c54a4-83b5-44c8-8cb7-613642bebe2c-kube-api-access-z7n54\") pod \"sub-test-model-24aabe-kserve-6ff87cb44f-qbc9q\" (UID: \"a56c54a4-83b5-44c8-8cb7-613642bebe2c\") " pod="ai-tenant-e2e-shared-a-w2-24aabe/sub-test-model-24aabe-kserve-6ff87cb44f-qbc9q"
Sep 16 02:41:22.310245 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:22.310232    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a56c54a4-83b5-44c8-8cb7-613642bebe2c-home\") pod \"sub-test-model-24aabe-kserve-6ff87cb44f-qbc9q\" (UID: \"a56c54a4-83b5-44c8-8cb7-613642bebe2c\") " pod="ai-tenant-e2e-shared-a-w2-24aabe/sub-test-model-24aabe-kserve-6ff87cb44f-qbc9q"
Sep 16 02:41:22.310599 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:22.310575    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a56c54a4-83b5-44c8-8cb7-613642bebe2c-tmp-dir\") pod \"sub-test-model-24aabe-kserve-6ff87cb44f-qbc9q\" (UID: \"a56c54a4-83b5-44c8-8cb7-613642bebe2c\") " pod="ai-tenant-e2e-shared-a-w2-24aabe/sub-test-model-24aabe-kserve-6ff87cb44f-qbc9q"
Sep 16 02:41:22.310599 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:22.310595    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a56c54a4-83b5-44c8-8cb7-613642bebe2c-model-cache\") pod \"sub-test-model-24aabe-kserve-6ff87cb44f-qbc9q\" (UID: \"a56c54a4-83b5-44c8-8cb7-613642bebe2c\") " pod="ai-tenant-e2e-shared-a-w2-24aabe/sub-test-model-24aabe-kserve-6ff87cb44f-qbc9q"
Sep 16 02:41:22.310696 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:22.310667    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a56c54a4-83b5-44c8-8cb7-613642bebe2c-home\") pod \"sub-test-model-24aabe-kserve-6ff87cb44f-qbc9q\" (UID: \"a56c54a4-83b5-44c8-8cb7-613642bebe2c\") " pod="ai-tenant-e2e-shared-a-w2-24aabe/sub-test-model-24aabe-kserve-6ff87cb44f-qbc9q"
Sep 16 02:41:22.312551 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:22.312524    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a56c54a4-83b5-44c8-8cb7-613642bebe2c-dshm\") pod \"sub-test-model-24aabe-kserve-6ff87cb44f-qbc9q\" (UID: \"a56c54a4-83b5-44c8-8cb7-613642bebe2c\") " pod="ai-tenant-e2e-shared-a-w2-24aabe/sub-test-model-24aabe-kserve-6ff87cb44f-qbc9q"
Sep 16 02:41:22.312834 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:22.312818    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a56c54a4-83b5-44c8-8cb7-613642bebe2c-tls-certs\") pod \"sub-test-model-24aabe-kserve-6ff87cb44f-qbc9q\" (UID: \"a56c54a4-83b5-44c8-8cb7-613642bebe2c\") " pod="ai-tenant-e2e-shared-a-w2-24aabe/sub-test-model-24aabe-kserve-6ff87cb44f-qbc9q"
Sep 16 02:41:22.317696 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:22.317675    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z7n54\" (UniqueName: \"kubernetes.io/projected/a56c54a4-83b5-44c8-8cb7-613642bebe2c-kube-api-access-z7n54\") pod \"sub-test-model-24aabe-kserve-6ff87cb44f-qbc9q\" (UID: \"a56c54a4-83b5-44c8-8cb7-613642bebe2c\") " pod="ai-tenant-e2e-shared-a-w2-24aabe/sub-test-model-24aabe-kserve-6ff87cb44f-qbc9q"
Sep 16 02:41:22.488890 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:22.488865    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w2-24aabe/sub-test-model-24aabe-kserve-6ff87cb44f-qbc9q"
Sep 16 02:41:22.761969 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:22.761879    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w2-24aabe/sub-test-model-24aabe-kserve-6ff87cb44f-qbc9q"]
Sep 16 02:41:22.765216 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:41:22.765182    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda56c54a4_83b5_44c8_8cb7_613642bebe2c.slice/crio-79dab5149dc7bc47c7fe5c6a6bd550623a2dc9345f5bc8dda99e822041c42533 WatchSource:0}: Error finding container 79dab5149dc7bc47c7fe5c6a6bd550623a2dc9345f5bc8dda99e822041c42533: Status 404 returned error can't find the container with id 79dab5149dc7bc47c7fe5c6a6bd550623a2dc9345f5bc8dda99e822041c42533
Sep 16 02:41:22.861221 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:22.861184    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w2-24aabe/sub-test-model-24aabe-kserve-6ff87cb44f-qbc9q" event={"ID":"a56c54a4-83b5-44c8-8cb7-613642bebe2c","Type":"ContainerStarted","Data":"832a2a18798abd35463abf39c1c73967af71ac19ade22786ed225ef1ca5270f0"}
Sep 16 02:41:22.861378 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:22.861231    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w2-24aabe/sub-test-model-24aabe-kserve-6ff87cb44f-qbc9q" event={"ID":"a56c54a4-83b5-44c8-8cb7-613642bebe2c","Type":"ContainerStarted","Data":"79dab5149dc7bc47c7fe5c6a6bd550623a2dc9345f5bc8dda99e822041c42533"}
Sep 16 02:41:22.880193 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:22.880134    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w2-24aabe/sub-test-model-24aabe-kserve-6ff87cb44f-qbc9q" podStartSLOduration=0.880098617 podStartE2EDuration="880.098617ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 02:41:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 02:41:22.878801503 +0000 UTC m=+1703.262316749" watchObservedRunningTime="2026-09-16 02:41:22.880098617 +0000 UTC m=+1703.263613864"
Sep 16 02:41:29.445954 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:29.445887    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:41:29.796763 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:29.796684    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:41:30.191648 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:30.191615    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:41:30.205422 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:30.205395    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w2-24aabe/auth-test-model-24aabe-kserve-d9468cd5f-4zt8g" podUID="2adfafdf-5d46-43df-ba80-ad7ce0743ea1" containerName="main" containerID="cri-o://b7ca49232610df30e8d2cc88967fd6ba3a2b0e7ba3be03c5523631b67ebf977b" gracePeriod=45
Sep 16 02:41:30.457893 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:30.457868    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w2-24aabe/auth-test-model-24aabe-kserve-d9468cd5f-4zt8g"
Sep 16 02:41:30.485985 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:30.485952    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2adfafdf-5d46-43df-ba80-ad7ce0743ea1-home\") pod \"2adfafdf-5d46-43df-ba80-ad7ce0743ea1\" (UID: \"2adfafdf-5d46-43df-ba80-ad7ce0743ea1\") "
Sep 16 02:41:30.486136 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:30.485991    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2adfafdf-5d46-43df-ba80-ad7ce0743ea1-tls-certs\") pod \"2adfafdf-5d46-43df-ba80-ad7ce0743ea1\" (UID: \"2adfafdf-5d46-43df-ba80-ad7ce0743ea1\") "
Sep 16 02:41:30.486136 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:30.486024    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2adfafdf-5d46-43df-ba80-ad7ce0743ea1-dshm\") pod \"2adfafdf-5d46-43df-ba80-ad7ce0743ea1\" (UID: \"2adfafdf-5d46-43df-ba80-ad7ce0743ea1\") "
Sep 16 02:41:30.486136 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:30.486046    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2adfafdf-5d46-43df-ba80-ad7ce0743ea1-tmp-dir\") pod \"2adfafdf-5d46-43df-ba80-ad7ce0743ea1\" (UID: \"2adfafdf-5d46-43df-ba80-ad7ce0743ea1\") "
Sep 16 02:41:30.486306 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:30.486149    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nz6vt\" (UniqueName: \"kubernetes.io/projected/2adfafdf-5d46-43df-ba80-ad7ce0743ea1-kube-api-access-nz6vt\") pod \"2adfafdf-5d46-43df-ba80-ad7ce0743ea1\" (UID: \"2adfafdf-5d46-43df-ba80-ad7ce0743ea1\") "
Sep 16 02:41:30.486306 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:30.486183    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2adfafdf-5d46-43df-ba80-ad7ce0743ea1-model-cache\") pod \"2adfafdf-5d46-43df-ba80-ad7ce0743ea1\" (UID: \"2adfafdf-5d46-43df-ba80-ad7ce0743ea1\") "
Sep 16 02:41:30.486306 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:30.486226    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2adfafdf-5d46-43df-ba80-ad7ce0743ea1-home" (OuterVolumeSpecName: "home") pod "2adfafdf-5d46-43df-ba80-ad7ce0743ea1" (UID: "2adfafdf-5d46-43df-ba80-ad7ce0743ea1"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 02:41:30.486445 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:30.486409    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2adfafdf-5d46-43df-ba80-ad7ce0743ea1-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "2adfafdf-5d46-43df-ba80-ad7ce0743ea1" (UID: "2adfafdf-5d46-43df-ba80-ad7ce0743ea1"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 02:41:30.486548 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:30.486530    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2adfafdf-5d46-43df-ba80-ad7ce0743ea1-home\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:41:30.486608 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:30.486554    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2adfafdf-5d46-43df-ba80-ad7ce0743ea1-tmp-dir\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:41:30.486608 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:30.486551    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2adfafdf-5d46-43df-ba80-ad7ce0743ea1-model-cache" (OuterVolumeSpecName: "model-cache") pod "2adfafdf-5d46-43df-ba80-ad7ce0743ea1" (UID: "2adfafdf-5d46-43df-ba80-ad7ce0743ea1"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 02:41:30.488433 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:30.488411    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2adfafdf-5d46-43df-ba80-ad7ce0743ea1-dshm" (OuterVolumeSpecName: "dshm") pod "2adfafdf-5d46-43df-ba80-ad7ce0743ea1" (UID: "2adfafdf-5d46-43df-ba80-ad7ce0743ea1"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 02:41:30.488557 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:30.488432    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2adfafdf-5d46-43df-ba80-ad7ce0743ea1-kube-api-access-nz6vt" (OuterVolumeSpecName: "kube-api-access-nz6vt") pod "2adfafdf-5d46-43df-ba80-ad7ce0743ea1" (UID: "2adfafdf-5d46-43df-ba80-ad7ce0743ea1"). InnerVolumeSpecName "kube-api-access-nz6vt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 02:41:30.488663 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:30.488643    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2adfafdf-5d46-43df-ba80-ad7ce0743ea1-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "2adfafdf-5d46-43df-ba80-ad7ce0743ea1" (UID: "2adfafdf-5d46-43df-ba80-ad7ce0743ea1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 02:41:30.587879 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:30.587853    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2adfafdf-5d46-43df-ba80-ad7ce0743ea1-tls-certs\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:41:30.587879 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:30.587876    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2adfafdf-5d46-43df-ba80-ad7ce0743ea1-dshm\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:41:30.587879 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:30.587885    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nz6vt\" (UniqueName: \"kubernetes.io/projected/2adfafdf-5d46-43df-ba80-ad7ce0743ea1-kube-api-access-nz6vt\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:41:30.588088 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:30.587895    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2adfafdf-5d46-43df-ba80-ad7ce0743ea1-model-cache\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:41:30.906644 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:30.906558    2579 generic.go:356] "Generic (PLEG): container finished" podID="2adfafdf-5d46-43df-ba80-ad7ce0743ea1" containerID="b7ca49232610df30e8d2cc88967fd6ba3a2b0e7ba3be03c5523631b67ebf977b" exitCode=0
Sep 16 02:41:30.906644 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:30.906599    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w2-24aabe/auth-test-model-24aabe-kserve-d9468cd5f-4zt8g"
Sep 16 02:41:30.906827 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:30.906649    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w2-24aabe/auth-test-model-24aabe-kserve-d9468cd5f-4zt8g" event={"ID":"2adfafdf-5d46-43df-ba80-ad7ce0743ea1","Type":"ContainerDied","Data":"b7ca49232610df30e8d2cc88967fd6ba3a2b0e7ba3be03c5523631b67ebf977b"}
Sep 16 02:41:30.906827 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:30.906689    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w2-24aabe/auth-test-model-24aabe-kserve-d9468cd5f-4zt8g" event={"ID":"2adfafdf-5d46-43df-ba80-ad7ce0743ea1","Type":"ContainerDied","Data":"0578c2a188c86b52d2de7606da2328c7bf95c04f7be4afe816a42f415f5307b7"}
Sep 16 02:41:30.906827 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:30.906709    2579 scope.go:117] "RemoveContainer" containerID="b7ca49232610df30e8d2cc88967fd6ba3a2b0e7ba3be03c5523631b67ebf977b"
Sep 16 02:41:30.923027 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:30.923008    2579 scope.go:117] "RemoveContainer" containerID="b7ca49232610df30e8d2cc88967fd6ba3a2b0e7ba3be03c5523631b67ebf977b"
Sep 16 02:41:30.923342 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:41:30.923321    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b7ca49232610df30e8d2cc88967fd6ba3a2b0e7ba3be03c5523631b67ebf977b\": container with ID starting with b7ca49232610df30e8d2cc88967fd6ba3a2b0e7ba3be03c5523631b67ebf977b not found: ID does not exist" containerID="b7ca49232610df30e8d2cc88967fd6ba3a2b0e7ba3be03c5523631b67ebf977b"
Sep 16 02:41:30.923392 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:30.923349    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b7ca49232610df30e8d2cc88967fd6ba3a2b0e7ba3be03c5523631b67ebf977b"} err="failed to get container status \"b7ca49232610df30e8d2cc88967fd6ba3a2b0e7ba3be03c5523631b67ebf977b\": rpc error: code = NotFound desc = could not find container \"b7ca49232610df30e8d2cc88967fd6ba3a2b0e7ba3be03c5523631b67ebf977b\": container with ID starting with b7ca49232610df30e8d2cc88967fd6ba3a2b0e7ba3be03c5523631b67ebf977b not found: ID does not exist"
Sep 16 02:41:30.931020 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:30.930998    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w2-24aabe/auth-test-model-24aabe-kserve-d9468cd5f-4zt8g"]
Sep 16 02:41:30.934707 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:30.934687    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w2-24aabe/auth-test-model-24aabe-kserve-d9468cd5f-4zt8g"]
Sep 16 02:41:31.124418 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:31.124382    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w2-a53a44/auth-test-model-a53a44-kserve-59f8574c79-7zfnt" podUID="eb4f9053-822f-4b52-8de8-ee555b5fac9c" containerName="main" containerID="cri-o://9e621a911501b111072aeb8d83bd18294f5482ec24b268d8d7232c1291aadfcf" gracePeriod=45
Sep 16 02:41:31.371145 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:31.371118    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w2-a53a44/auth-test-model-a53a44-kserve-59f8574c79-7zfnt"
Sep 16 02:41:31.495888 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:31.495811    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eb4f9053-822f-4b52-8de8-ee555b5fac9c-tls-certs\") pod \"eb4f9053-822f-4b52-8de8-ee555b5fac9c\" (UID: \"eb4f9053-822f-4b52-8de8-ee555b5fac9c\") "
Sep 16 02:41:31.495888 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:31.495860    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eb4f9053-822f-4b52-8de8-ee555b5fac9c-home\") pod \"eb4f9053-822f-4b52-8de8-ee555b5fac9c\" (UID: \"eb4f9053-822f-4b52-8de8-ee555b5fac9c\") "
Sep 16 02:41:31.496364 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:31.495907    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eb4f9053-822f-4b52-8de8-ee555b5fac9c-tmp-dir\") pod \"eb4f9053-822f-4b52-8de8-ee555b5fac9c\" (UID: \"eb4f9053-822f-4b52-8de8-ee555b5fac9c\") "
Sep 16 02:41:31.496364 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:31.495954    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eb4f9053-822f-4b52-8de8-ee555b5fac9c-model-cache\") pod \"eb4f9053-822f-4b52-8de8-ee555b5fac9c\" (UID: \"eb4f9053-822f-4b52-8de8-ee555b5fac9c\") "
Sep 16 02:41:31.496364 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:31.495983    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eb4f9053-822f-4b52-8de8-ee555b5fac9c-dshm\") pod \"eb4f9053-822f-4b52-8de8-ee555b5fac9c\" (UID: \"eb4f9053-822f-4b52-8de8-ee555b5fac9c\") "
Sep 16 02:41:31.496364 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:31.496035    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zfzdj\" (UniqueName: \"kubernetes.io/projected/eb4f9053-822f-4b52-8de8-ee555b5fac9c-kube-api-access-zfzdj\") pod \"eb4f9053-822f-4b52-8de8-ee555b5fac9c\" (UID: \"eb4f9053-822f-4b52-8de8-ee555b5fac9c\") "
Sep 16 02:41:31.496364 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:31.496190    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb4f9053-822f-4b52-8de8-ee555b5fac9c-home" (OuterVolumeSpecName: "home") pod "eb4f9053-822f-4b52-8de8-ee555b5fac9c" (UID: "eb4f9053-822f-4b52-8de8-ee555b5fac9c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 02:41:31.496364 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:31.496198    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb4f9053-822f-4b52-8de8-ee555b5fac9c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "eb4f9053-822f-4b52-8de8-ee555b5fac9c" (UID: "eb4f9053-822f-4b52-8de8-ee555b5fac9c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 02:41:31.496364 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:31.496210    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb4f9053-822f-4b52-8de8-ee555b5fac9c-model-cache" (OuterVolumeSpecName: "model-cache") pod "eb4f9053-822f-4b52-8de8-ee555b5fac9c" (UID: "eb4f9053-822f-4b52-8de8-ee555b5fac9c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 02:41:31.496364 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:31.496290    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eb4f9053-822f-4b52-8de8-ee555b5fac9c-home\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:41:31.496364 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:31.496302    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eb4f9053-822f-4b52-8de8-ee555b5fac9c-tmp-dir\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:41:31.496364 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:31.496316    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eb4f9053-822f-4b52-8de8-ee555b5fac9c-model-cache\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:41:31.498279 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:31.498256    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eb4f9053-822f-4b52-8de8-ee555b5fac9c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "eb4f9053-822f-4b52-8de8-ee555b5fac9c" (UID: "eb4f9053-822f-4b52-8de8-ee555b5fac9c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 02:41:31.498375 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:31.498304    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb4f9053-822f-4b52-8de8-ee555b5fac9c-dshm" (OuterVolumeSpecName: "dshm") pod "eb4f9053-822f-4b52-8de8-ee555b5fac9c" (UID: "eb4f9053-822f-4b52-8de8-ee555b5fac9c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 02:41:31.498433 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:31.498419    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb4f9053-822f-4b52-8de8-ee555b5fac9c-kube-api-access-zfzdj" (OuterVolumeSpecName: "kube-api-access-zfzdj") pod "eb4f9053-822f-4b52-8de8-ee555b5fac9c" (UID: "eb4f9053-822f-4b52-8de8-ee555b5fac9c"). InnerVolumeSpecName "kube-api-access-zfzdj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 02:41:31.597651 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:31.597618    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zfzdj\" (UniqueName: \"kubernetes.io/projected/eb4f9053-822f-4b52-8de8-ee555b5fac9c-kube-api-access-zfzdj\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:41:31.597651 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:31.597646    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eb4f9053-822f-4b52-8de8-ee555b5fac9c-tls-certs\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:41:31.597651 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:31.597657    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eb4f9053-822f-4b52-8de8-ee555b5fac9c-dshm\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:41:31.913612 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:31.913528    2579 generic.go:356] "Generic (PLEG): container finished" podID="eb4f9053-822f-4b52-8de8-ee555b5fac9c" containerID="9e621a911501b111072aeb8d83bd18294f5482ec24b268d8d7232c1291aadfcf" exitCode=0
Sep 16 02:41:31.913612 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:31.913577    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w2-a53a44/auth-test-model-a53a44-kserve-59f8574c79-7zfnt"
Sep 16 02:41:31.913612 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:31.913594    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w2-a53a44/auth-test-model-a53a44-kserve-59f8574c79-7zfnt" event={"ID":"eb4f9053-822f-4b52-8de8-ee555b5fac9c","Type":"ContainerDied","Data":"9e621a911501b111072aeb8d83bd18294f5482ec24b268d8d7232c1291aadfcf"}
Sep 16 02:41:31.913906 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:31.913630    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w2-a53a44/auth-test-model-a53a44-kserve-59f8574c79-7zfnt" event={"ID":"eb4f9053-822f-4b52-8de8-ee555b5fac9c","Type":"ContainerDied","Data":"f7975259e9a71fbb4efdfb1a58504171203f8aad668676288ff4a16b2b961621"}
Sep 16 02:41:31.913906 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:31.913654    2579 scope.go:117] "RemoveContainer" containerID="9e621a911501b111072aeb8d83bd18294f5482ec24b268d8d7232c1291aadfcf"
Sep 16 02:41:31.922805 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:31.922787    2579 scope.go:117] "RemoveContainer" containerID="9e621a911501b111072aeb8d83bd18294f5482ec24b268d8d7232c1291aadfcf"
Sep 16 02:41:31.923037 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:41:31.923017    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9e621a911501b111072aeb8d83bd18294f5482ec24b268d8d7232c1291aadfcf\": container with ID starting with 9e621a911501b111072aeb8d83bd18294f5482ec24b268d8d7232c1291aadfcf not found: ID does not exist" containerID="9e621a911501b111072aeb8d83bd18294f5482ec24b268d8d7232c1291aadfcf"
Sep 16 02:41:31.923120 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:31.923043    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9e621a911501b111072aeb8d83bd18294f5482ec24b268d8d7232c1291aadfcf"} err="failed to get container status \"9e621a911501b111072aeb8d83bd18294f5482ec24b268d8d7232c1291aadfcf\": rpc error: code = NotFound desc = could not find container \"9e621a911501b111072aeb8d83bd18294f5482ec24b268d8d7232c1291aadfcf\": container with ID starting with 9e621a911501b111072aeb8d83bd18294f5482ec24b268d8d7232c1291aadfcf not found: ID does not exist"
Sep 16 02:41:31.934235 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:31.934206    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w2-a53a44/auth-test-model-a53a44-kserve-59f8574c79-7zfnt"]
Sep 16 02:41:31.936093 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:31.936066    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w2-a53a44/auth-test-model-a53a44-kserve-59f8574c79-7zfnt"]
Sep 16 02:41:32.040967 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:32.040932    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:41:32.102691 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:32.102666    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2adfafdf-5d46-43df-ba80-ad7ce0743ea1" path="/var/lib/kubelet/pods/2adfafdf-5d46-43df-ba80-ad7ce0743ea1/volumes"
Sep 16 02:41:32.103099 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:32.103086    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eb4f9053-822f-4b52-8de8-ee555b5fac9c" path="/var/lib/kubelet/pods/eb4f9053-822f-4b52-8de8-ee555b5fac9c/volumes"
Sep 16 02:41:32.489719 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:32.489691    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w2-24aabe/sub-test-model-24aabe-kserve-6ff87cb44f-qbc9q"
Sep 16 02:41:32.489908 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:32.489737    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w2-24aabe/sub-test-model-24aabe-kserve-6ff87cb44f-qbc9q"
Sep 16 02:41:32.502350 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:32.502325    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w2-24aabe/sub-test-model-24aabe-kserve-6ff87cb44f-qbc9q"
Sep 16 02:41:32.675728 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:32.675692    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:41:32.769894 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:32.769788    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:41:32.931937 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:32.931905    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w2-24aabe/sub-test-model-24aabe-kserve-6ff87cb44f-qbc9q"
Sep 16 02:41:33.014061 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:33.014017    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:41:33.660729 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:33.660694    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:41:34.550090 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:34.550056    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:41:35.237362 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:35.237331    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:41:35.296220 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:35.296182    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:41:35.396847 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:35.396807    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:41:35.702725 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:35.702691    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:41:35.787847 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:35.787812    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:41:35.788805 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:35.788786    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-77f43ab3-547f5fcc66-xq82k"]
Sep 16 02:41:35.789220 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:35.789207    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eb4f9053-822f-4b52-8de8-ee555b5fac9c" containerName="main"
Sep 16 02:41:35.789263 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:35.789223    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="eb4f9053-822f-4b52-8de8-ee555b5fac9c" containerName="main"
Sep 16 02:41:35.789263 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:35.789235    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2adfafdf-5d46-43df-ba80-ad7ce0743ea1" containerName="main"
Sep 16 02:41:35.789263 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:35.789239    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="2adfafdf-5d46-43df-ba80-ad7ce0743ea1" containerName="main"
Sep 16 02:41:35.789356 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:35.789329    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="2adfafdf-5d46-43df-ba80-ad7ce0743ea1" containerName="main"
Sep 16 02:41:35.789356 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:35.789347    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="eb4f9053-822f-4b52-8de8-ee555b5fac9c" containerName="main"
Sep 16 02:41:35.792356 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:35.792335    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-77f43ab3-547f5fcc66-xq82k"
Sep 16 02:41:35.794528 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:35.794459    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-77f43ab3\""
Sep 16 02:41:35.803197 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:35.803168    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-77f43ab3-547f5fcc66-xq82k"]
Sep 16 02:41:35.838656 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:35.838629    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-77f43ab3-6dcc766745-mphkm"]
Sep 16 02:41:35.838961 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:35.838939    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f7f95091-2ef8-45b6-96dd-25fa307ac07e-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-77f43ab3-547f5fcc66-xq82k\" (UID: \"f7f95091-2ef8-45b6-96dd-25fa307ac07e\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-77f43ab3-547f5fcc66-xq82k"
Sep 16 02:41:35.839034 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:35.839019    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k8dt5\" (UniqueName: \"kubernetes.io/projected/f7f95091-2ef8-45b6-96dd-25fa307ac07e-kube-api-access-k8dt5\") pod \"payload-pre-processing-e2e-mt-77f43ab3-547f5fcc66-xq82k\" (UID: \"f7f95091-2ef8-45b6-96dd-25fa307ac07e\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-77f43ab3-547f5fcc66-xq82k"
Sep 16 02:41:35.842102 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:35.842084    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-77f43ab3-6dcc766745-mphkm"
Sep 16 02:41:35.844353 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:35.844331    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-77f43ab3-dockercfg-mmjq4\""
Sep 16 02:41:35.851986 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:35.851964    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-77f43ab3-6dcc766745-mphkm"]
Sep 16 02:41:35.940429 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:35.940393    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1cbe69b2-8c94-4465-888e-280cdf86caf2-plugins-config-volume\") pod \"payload-processing-e2e-mt-77f43ab3-6dcc766745-mphkm\" (UID: \"1cbe69b2-8c94-4465-888e-280cdf86caf2\") " pod="openshift-ingress/payload-processing-e2e-mt-77f43ab3-6dcc766745-mphkm"
Sep 16 02:41:35.940600 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:35.940442    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wq4xb\" (UniqueName: \"kubernetes.io/projected/1cbe69b2-8c94-4465-888e-280cdf86caf2-kube-api-access-wq4xb\") pod \"payload-processing-e2e-mt-77f43ab3-6dcc766745-mphkm\" (UID: \"1cbe69b2-8c94-4465-888e-280cdf86caf2\") " pod="openshift-ingress/payload-processing-e2e-mt-77f43ab3-6dcc766745-mphkm"
Sep 16 02:41:35.940600 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:35.940519    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k8dt5\" (UniqueName: \"kubernetes.io/projected/f7f95091-2ef8-45b6-96dd-25fa307ac07e-kube-api-access-k8dt5\") pod \"payload-pre-processing-e2e-mt-77f43ab3-547f5fcc66-xq82k\" (UID: \"f7f95091-2ef8-45b6-96dd-25fa307ac07e\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-77f43ab3-547f5fcc66-xq82k"
Sep 16 02:41:35.940600 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:35.940589    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f7f95091-2ef8-45b6-96dd-25fa307ac07e-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-77f43ab3-547f5fcc66-xq82k\" (UID: \"f7f95091-2ef8-45b6-96dd-25fa307ac07e\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-77f43ab3-547f5fcc66-xq82k"
Sep 16 02:41:35.941292 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:35.941270    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f7f95091-2ef8-45b6-96dd-25fa307ac07e-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-77f43ab3-547f5fcc66-xq82k\" (UID: \"f7f95091-2ef8-45b6-96dd-25fa307ac07e\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-77f43ab3-547f5fcc66-xq82k"
Sep 16 02:41:35.951022 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:35.951000    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k8dt5\" (UniqueName: \"kubernetes.io/projected/f7f95091-2ef8-45b6-96dd-25fa307ac07e-kube-api-access-k8dt5\") pod \"payload-pre-processing-e2e-mt-77f43ab3-547f5fcc66-xq82k\" (UID: \"f7f95091-2ef8-45b6-96dd-25fa307ac07e\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-77f43ab3-547f5fcc66-xq82k"
Sep 16 02:41:36.041369 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:36.041282    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wq4xb\" (UniqueName: \"kubernetes.io/projected/1cbe69b2-8c94-4465-888e-280cdf86caf2-kube-api-access-wq4xb\") pod \"payload-processing-e2e-mt-77f43ab3-6dcc766745-mphkm\" (UID: \"1cbe69b2-8c94-4465-888e-280cdf86caf2\") " pod="openshift-ingress/payload-processing-e2e-mt-77f43ab3-6dcc766745-mphkm"
Sep 16 02:41:36.041592 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:36.041390    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1cbe69b2-8c94-4465-888e-280cdf86caf2-plugins-config-volume\") pod \"payload-processing-e2e-mt-77f43ab3-6dcc766745-mphkm\" (UID: \"1cbe69b2-8c94-4465-888e-280cdf86caf2\") " pod="openshift-ingress/payload-processing-e2e-mt-77f43ab3-6dcc766745-mphkm"
Sep 16 02:41:36.042056 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:36.042037    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1cbe69b2-8c94-4465-888e-280cdf86caf2-plugins-config-volume\") pod \"payload-processing-e2e-mt-77f43ab3-6dcc766745-mphkm\" (UID: \"1cbe69b2-8c94-4465-888e-280cdf86caf2\") " pod="openshift-ingress/payload-processing-e2e-mt-77f43ab3-6dcc766745-mphkm"
Sep 16 02:41:36.048807 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:36.048784    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wq4xb\" (UniqueName: \"kubernetes.io/projected/1cbe69b2-8c94-4465-888e-280cdf86caf2-kube-api-access-wq4xb\") pod \"payload-processing-e2e-mt-77f43ab3-6dcc766745-mphkm\" (UID: \"1cbe69b2-8c94-4465-888e-280cdf86caf2\") " pod="openshift-ingress/payload-processing-e2e-mt-77f43ab3-6dcc766745-mphkm"
Sep 16 02:41:36.104011 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:36.103985    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-77f43ab3-547f5fcc66-xq82k"
Sep 16 02:41:36.153071 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:36.153042    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-77f43ab3-6dcc766745-mphkm"
Sep 16 02:41:36.379150 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:36.379120    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-77f43ab3-547f5fcc66-xq82k"]
Sep 16 02:41:36.381981 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:41:36.381956    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7f95091_2ef8_45b6_96dd_25fa307ac07e.slice/crio-f8687fce86c26e9a91cba46b299b29a3029e0fddf16129c8150006367d9817c1 WatchSource:0}: Error finding container f8687fce86c26e9a91cba46b299b29a3029e0fddf16129c8150006367d9817c1: Status 404 returned error can't find the container with id f8687fce86c26e9a91cba46b299b29a3029e0fddf16129c8150006367d9817c1
Sep 16 02:41:36.441201 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:36.441179    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-77f43ab3-6dcc766745-mphkm"]
Sep 16 02:41:36.443013 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:41:36.442984    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1cbe69b2_8c94_4465_888e_280cdf86caf2.slice/crio-4bb829fcd638de0e766be5ff3ce994c136f812ee6fe38c0c63c3c45cca04d3cc WatchSource:0}: Error finding container 4bb829fcd638de0e766be5ff3ce994c136f812ee6fe38c0c63c3c45cca04d3cc: Status 404 returned error can't find the container with id 4bb829fcd638de0e766be5ff3ce994c136f812ee6fe38c0c63c3c45cca04d3cc
Sep 16 02:41:36.943607 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:36.943574    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-77f43ab3-6dcc766745-mphkm" event={"ID":"1cbe69b2-8c94-4465-888e-280cdf86caf2","Type":"ContainerStarted","Data":"016380ef62205699b7f5194024b5f0402a33b075aafbc7f2152a7ae2b73c2e37"}
Sep 16 02:41:36.943830 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:36.943623    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-77f43ab3-6dcc766745-mphkm" event={"ID":"1cbe69b2-8c94-4465-888e-280cdf86caf2","Type":"ContainerStarted","Data":"4bb829fcd638de0e766be5ff3ce994c136f812ee6fe38c0c63c3c45cca04d3cc"}
Sep 16 02:41:36.943830 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:36.943669    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-77f43ab3-6dcc766745-mphkm"
Sep 16 02:41:36.945302 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:36.945278    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-77f43ab3-547f5fcc66-xq82k" event={"ID":"f7f95091-2ef8-45b6-96dd-25fa307ac07e","Type":"ContainerStarted","Data":"888ab86525b3df534f70435fcc8e3fe2dbde1cd4947f649f7b07bec795a082ac"}
Sep 16 02:41:36.945399 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:36.945322    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-77f43ab3-547f5fcc66-xq82k" event={"ID":"f7f95091-2ef8-45b6-96dd-25fa307ac07e","Type":"ContainerStarted","Data":"f8687fce86c26e9a91cba46b299b29a3029e0fddf16129c8150006367d9817c1"}
Sep 16 02:41:36.945399 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:36.945344    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-77f43ab3-547f5fcc66-xq82k"
Sep 16 02:41:36.960730 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:36.960679    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-77f43ab3-6dcc766745-mphkm" podStartSLOduration=1.9606598179999999 podStartE2EDuration="1.960659818s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 02:41:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 02:41:36.958447753 +0000 UTC m=+1717.341963001" watchObservedRunningTime="2026-09-16 02:41:36.960659818 +0000 UTC m=+1717.344175067"
Sep 16 02:41:36.976195 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:36.976155    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-77f43ab3-547f5fcc66-xq82k" podStartSLOduration=1.9761372179999999 podStartE2EDuration="1.976137218s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 02:41:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 02:41:36.974965806 +0000 UTC m=+1717.358481052" watchObservedRunningTime="2026-09-16 02:41:36.976137218 +0000 UTC m=+1717.359652488"
Sep 16 02:41:47.952615 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:47.952582    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-77f43ab3-6dcc766745-mphkm"
Sep 16 02:41:47.953215 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:47.952666    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-77f43ab3-547f5fcc66-xq82k"
Sep 16 02:41:57.927292 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:57.927258    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:41:58.188259 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:58.188207    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:41:58.814402 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:58.814361    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:41:59.533516 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:41:59.533463    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:42:00.230765 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:42:00.230727    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:42:00.853355 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:42:00.853317    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:42:05.377782 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:42:05.377752    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w2-a53a44/sub-test-model-a53a44-kserve-bccfdbcbc-hlxr6"]
Sep 16 02:42:05.381391 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:42:05.381368    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w2-a53a44/sub-test-model-a53a44-kserve-bccfdbcbc-hlxr6"
Sep 16 02:42:05.384920 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:42:05.384896    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w2-a53a44\"/\"default-dockercfg-tw8hg\""
Sep 16 02:42:05.385933 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:42:05.385909    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w2-a53a44\"/\"openshift-service-ca.crt\""
Sep 16 02:42:05.386071 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:42:05.385970    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w2-a53a44\"/\"kube-root-ca.crt\""
Sep 16 02:42:05.386195 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:42:05.386180    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w2-a53a44\"/\"sub-test-model-a53a44-kserve-self-signed-certs\""
Sep 16 02:42:05.391432 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:42:05.391408    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w2-a53a44/sub-test-model-a53a44-kserve-bccfdbcbc-hlxr6"]
Sep 16 02:42:05.394683 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:42:05.394660    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:42:05.416385 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:42:05.416362    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5d607480-7df3-4f34-a7a2-d27f28f110c6-home\") pod \"sub-test-model-a53a44-kserve-bccfdbcbc-hlxr6\" (UID: \"5d607480-7df3-4f34-a7a2-d27f28f110c6\") " pod="ai-tenant-e2e-shared-b-w2-a53a44/sub-test-model-a53a44-kserve-bccfdbcbc-hlxr6"
Sep 16 02:42:05.416520 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:42:05.416391    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5d607480-7df3-4f34-a7a2-d27f28f110c6-model-cache\") pod \"sub-test-model-a53a44-kserve-bccfdbcbc-hlxr6\" (UID: \"5d607480-7df3-4f34-a7a2-d27f28f110c6\") " pod="ai-tenant-e2e-shared-b-w2-a53a44/sub-test-model-a53a44-kserve-bccfdbcbc-hlxr6"
Sep 16 02:42:05.416520 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:42:05.416416    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5d607480-7df3-4f34-a7a2-d27f28f110c6-dshm\") pod \"sub-test-model-a53a44-kserve-bccfdbcbc-hlxr6\" (UID: \"5d607480-7df3-4f34-a7a2-d27f28f110c6\") " pod="ai-tenant-e2e-shared-b-w2-a53a44/sub-test-model-a53a44-kserve-bccfdbcbc-hlxr6"
Sep 16 02:42:05.416611 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:42:05.416527    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5d607480-7df3-4f34-a7a2-d27f28f110c6-tmp-dir\") pod \"sub-test-model-a53a44-kserve-bccfdbcbc-hlxr6\" (UID: \"5d607480-7df3-4f34-a7a2-d27f28f110c6\") " pod="ai-tenant-e2e-shared-b-w2-a53a44/sub-test-model-a53a44-kserve-bccfdbcbc-hlxr6"
Sep 16 02:42:05.416611 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:42:05.416586    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5d607480-7df3-4f34-a7a2-d27f28f110c6-tls-certs\") pod \"sub-test-model-a53a44-kserve-bccfdbcbc-hlxr6\" (UID: \"5d607480-7df3-4f34-a7a2-d27f28f110c6\") " pod="ai-tenant-e2e-shared-b-w2-a53a44/sub-test-model-a53a44-kserve-bccfdbcbc-hlxr6"
Sep 16 02:42:05.416677 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:42:05.416636    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x6wl9\" (UniqueName: \"kubernetes.io/projected/5d607480-7df3-4f34-a7a2-d27f28f110c6-kube-api-access-x6wl9\") pod \"sub-test-model-a53a44-kserve-bccfdbcbc-hlxr6\" (UID: \"5d607480-7df3-4f34-a7a2-d27f28f110c6\") " pod="ai-tenant-e2e-shared-b-w2-a53a44/sub-test-model-a53a44-kserve-bccfdbcbc-hlxr6"
Sep 16 02:42:05.517768 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:42:05.517738    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5d607480-7df3-4f34-a7a2-d27f28f110c6-tls-certs\") pod \"sub-test-model-a53a44-kserve-bccfdbcbc-hlxr6\" (UID: \"5d607480-7df3-4f34-a7a2-d27f28f110c6\") " pod="ai-tenant-e2e-shared-b-w2-a53a44/sub-test-model-a53a44-kserve-bccfdbcbc-hlxr6"
Sep 16 02:42:05.517941 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:42:05.517788    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x6wl9\" (UniqueName: \"kubernetes.io/projected/5d607480-7df3-4f34-a7a2-d27f28f110c6-kube-api-access-x6wl9\") pod \"sub-test-model-a53a44-kserve-bccfdbcbc-hlxr6\" (UID: \"5d607480-7df3-4f34-a7a2-d27f28f110c6\") " pod="ai-tenant-e2e-shared-b-w2-a53a44/sub-test-model-a53a44-kserve-bccfdbcbc-hlxr6"
Sep 16 02:42:05.517941 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:42:05.517818    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5d607480-7df3-4f34-a7a2-d27f28f110c6-home\") pod \"sub-test-model-a53a44-kserve-bccfdbcbc-hlxr6\" (UID: \"5d607480-7df3-4f34-a7a2-d27f28f110c6\") " pod="ai-tenant-e2e-shared-b-w2-a53a44/sub-test-model-a53a44-kserve-bccfdbcbc-hlxr6"
Sep 16 02:42:05.517941 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:42:05.517845    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5d607480-7df3-4f34-a7a2-d27f28f110c6-model-cache\") pod \"sub-test-model-a53a44-kserve-bccfdbcbc-hlxr6\" (UID: \"5d607480-7df3-4f34-a7a2-d27f28f110c6\") " pod="ai-tenant-e2e-shared-b-w2-a53a44/sub-test-model-a53a44-kserve-bccfdbcbc-hlxr6"
Sep 16 02:42:05.517941 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:42:05.517871    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5d607480-7df3-4f34-a7a2-d27f28f110c6-dshm\") pod \"sub-test-model-a53a44-kserve-bccfdbcbc-hlxr6\" (UID: \"5d607480-7df3-4f34-a7a2-d27f28f110c6\") " pod="ai-tenant-e2e-shared-b-w2-a53a44/sub-test-model-a53a44-kserve-bccfdbcbc-hlxr6"
Sep 16 02:42:05.517941 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:42:05.517905    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5d607480-7df3-4f34-a7a2-d27f28f110c6-tmp-dir\") pod \"sub-test-model-a53a44-kserve-bccfdbcbc-hlxr6\" (UID: \"5d607480-7df3-4f34-a7a2-d27f28f110c6\") " pod="ai-tenant-e2e-shared-b-w2-a53a44/sub-test-model-a53a44-kserve-bccfdbcbc-hlxr6"
Sep 16 02:42:05.518321 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:42:05.518298    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5d607480-7df3-4f34-a7a2-d27f28f110c6-home\") pod \"sub-test-model-a53a44-kserve-bccfdbcbc-hlxr6\" (UID: \"5d607480-7df3-4f34-a7a2-d27f28f110c6\") " pod="ai-tenant-e2e-shared-b-w2-a53a44/sub-test-model-a53a44-kserve-bccfdbcbc-hlxr6"
Sep 16 02:42:05.518388 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:42:05.518320    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5d607480-7df3-4f34-a7a2-d27f28f110c6-tmp-dir\") pod \"sub-test-model-a53a44-kserve-bccfdbcbc-hlxr6\" (UID: \"5d607480-7df3-4f34-a7a2-d27f28f110c6\") " pod="ai-tenant-e2e-shared-b-w2-a53a44/sub-test-model-a53a44-kserve-bccfdbcbc-hlxr6"
Sep 16 02:42:05.518388 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:42:05.518368    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5d607480-7df3-4f34-a7a2-d27f28f110c6-model-cache\") pod \"sub-test-model-a53a44-kserve-bccfdbcbc-hlxr6\" (UID: \"5d607480-7df3-4f34-a7a2-d27f28f110c6\") " pod="ai-tenant-e2e-shared-b-w2-a53a44/sub-test-model-a53a44-kserve-bccfdbcbc-hlxr6"
Sep 16 02:42:05.520438 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:42:05.520415    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5d607480-7df3-4f34-a7a2-d27f28f110c6-dshm\") pod \"sub-test-model-a53a44-kserve-bccfdbcbc-hlxr6\" (UID: \"5d607480-7df3-4f34-a7a2-d27f28f110c6\") " pod="ai-tenant-e2e-shared-b-w2-a53a44/sub-test-model-a53a44-kserve-bccfdbcbc-hlxr6"
Sep 16 02:42:05.520559 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:42:05.520419    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5d607480-7df3-4f34-a7a2-d27f28f110c6-tls-certs\") pod \"sub-test-model-a53a44-kserve-bccfdbcbc-hlxr6\" (UID: \"5d607480-7df3-4f34-a7a2-d27f28f110c6\") " pod="ai-tenant-e2e-shared-b-w2-a53a44/sub-test-model-a53a44-kserve-bccfdbcbc-hlxr6"
Sep 16 02:42:05.526216 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:42:05.526195    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x6wl9\" (UniqueName: \"kubernetes.io/projected/5d607480-7df3-4f34-a7a2-d27f28f110c6-kube-api-access-x6wl9\") pod \"sub-test-model-a53a44-kserve-bccfdbcbc-hlxr6\" (UID: \"5d607480-7df3-4f34-a7a2-d27f28f110c6\") " pod="ai-tenant-e2e-shared-b-w2-a53a44/sub-test-model-a53a44-kserve-bccfdbcbc-hlxr6"
Sep 16 02:42:05.692166 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:42:05.692135    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w2-a53a44/sub-test-model-a53a44-kserve-bccfdbcbc-hlxr6"
Sep 16 02:42:05.966863 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:42:05.966835    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w2-a53a44/sub-test-model-a53a44-kserve-bccfdbcbc-hlxr6"]
Sep 16 02:42:05.968670 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:42:05.968637    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5d607480_7df3_4f34_a7a2_d27f28f110c6.slice/crio-e5e5a389861a238d6bb42267338e48cec8bb5361627d337c3f6896d4238ef70e WatchSource:0}: Error finding container e5e5a389861a238d6bb42267338e48cec8bb5361627d337c3f6896d4238ef70e: Status 404 returned error can't find the container with id e5e5a389861a238d6bb42267338e48cec8bb5361627d337c3f6896d4238ef70e
Sep 16 02:42:06.103422 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:42:06.103383    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w2-a53a44/sub-test-model-a53a44-kserve-bccfdbcbc-hlxr6" event={"ID":"5d607480-7df3-4f34-a7a2-d27f28f110c6","Type":"ContainerStarted","Data":"dde1e8cfbf81516e6a707b0cd8739bd7f7bb244afd7d80756d0c52dcd84e4ffe"}
Sep 16 02:42:06.103583 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:42:06.103436    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w2-a53a44/sub-test-model-a53a44-kserve-bccfdbcbc-hlxr6" event={"ID":"5d607480-7df3-4f34-a7a2-d27f28f110c6","Type":"ContainerStarted","Data":"e5e5a389861a238d6bb42267338e48cec8bb5361627d337c3f6896d4238ef70e"}
Sep 16 02:42:06.118692 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:42:06.118650    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w2-a53a44/sub-test-model-a53a44-kserve-bccfdbcbc-hlxr6" podStartSLOduration=1.118635489 podStartE2EDuration="1.118635489s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 02:42:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 02:42:06.11774583 +0000 UTC m=+1746.501261100" watchObservedRunningTime="2026-09-16 02:42:06.118635489 +0000 UTC m=+1746.502150731"
Sep 16 02:42:15.692356 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:42:15.692325    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w2-a53a44/sub-test-model-a53a44-kserve-bccfdbcbc-hlxr6"
Sep 16 02:42:15.692783 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:42:15.692375    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w2-a53a44/sub-test-model-a53a44-kserve-bccfdbcbc-hlxr6"
Sep 16 02:42:15.705550 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:42:15.705528    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w2-a53a44/sub-test-model-a53a44-kserve-bccfdbcbc-hlxr6"
Sep 16 02:42:16.075065 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:42:16.074983    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-77f43ab3/e2e-lifecycle-model-77f43ab3-kserve-76cd4bf5f9-87x9b"]
Sep 16 02:42:16.079230 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:42:16.079209    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-77f43ab3/e2e-lifecycle-model-77f43ab3-kserve-76cd4bf5f9-87x9b"
Sep 16 02:42:16.081526 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:42:16.081500    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-77f43ab3\"/\"kube-root-ca.crt\""
Sep 16 02:42:16.081647 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:42:16.081511    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-77f43ab3\"/\"openshift-service-ca.crt\""
Sep 16 02:42:16.082354 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:42:16.082329    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-77f43ab3\"/\"e2e-lifecycle-model-77f43ab3-kserve-self-signed-certs\""
Sep 16 02:42:16.082494 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:42:16.082331    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-77f43ab3\"/\"default-dockercfg-whfnv\""
Sep 16 02:42:16.088559 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:42:16.088162    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-77f43ab3/e2e-lifecycle-model-77f43ab3-kserve-76cd4bf5f9-87x9b"]
Sep 16 02:42:16.108257 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:42:16.108228    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:42:16.160880 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:42:16.160850    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w2-a53a44/sub-test-model-a53a44-kserve-bccfdbcbc-hlxr6"
Sep 16 02:42:16.223086 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:42:16.223056    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b8ae3d33-d1dd-4d95-85b5-d7deed379ddd-model-cache\") pod \"e2e-lifecycle-model-77f43ab3-kserve-76cd4bf5f9-87x9b\" (UID: \"b8ae3d33-d1dd-4d95-85b5-d7deed379ddd\") " pod="ai-tenant-e2e-mt-77f43ab3/e2e-lifecycle-model-77f43ab3-kserve-76cd4bf5f9-87x9b"
Sep 16 02:42:16.223086 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:42:16.223088    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b8ae3d33-d1dd-4d95-85b5-d7deed379ddd-tmp-dir\") pod \"e2e-lifecycle-model-77f43ab3-kserve-76cd4bf5f9-87x9b\" (UID: \"b8ae3d33-d1dd-4d95-85b5-d7deed379ddd\") " pod="ai-tenant-e2e-mt-77f43ab3/e2e-lifecycle-model-77f43ab3-kserve-76cd4bf5f9-87x9b"
Sep 16 02:42:16.223300 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:42:16.223126    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b8ae3d33-d1dd-4d95-85b5-d7deed379ddd-home\") pod \"e2e-lifecycle-model-77f43ab3-kserve-76cd4bf5f9-87x9b\" (UID: \"b8ae3d33-d1dd-4d95-85b5-d7deed379ddd\") " pod="ai-tenant-e2e-mt-77f43ab3/e2e-lifecycle-model-77f43ab3-kserve-76cd4bf5f9-87x9b"
Sep 16 02:42:16.223300 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:42:16.223209    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v7hj6\" (UniqueName: \"kubernetes.io/projected/b8ae3d33-d1dd-4d95-85b5-d7deed379ddd-kube-api-access-v7hj6\") pod \"e2e-lifecycle-model-77f43ab3-kserve-76cd4bf5f9-87x9b\" (UID: \"b8ae3d33-d1dd-4d95-85b5-d7deed379ddd\") " pod="ai-tenant-e2e-mt-77f43ab3/e2e-lifecycle-model-77f43ab3-kserve-76cd4bf5f9-87x9b"
Sep 16 02:42:16.223300 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:42:16.223228    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b8ae3d33-d1dd-4d95-85b5-d7deed379ddd-tls-certs\") pod \"e2e-lifecycle-model-77f43ab3-kserve-76cd4bf5f9-87x9b\" (UID: \"b8ae3d33-d1dd-4d95-85b5-d7deed379ddd\") " pod="ai-tenant-e2e-mt-77f43ab3/e2e-lifecycle-model-77f43ab3-kserve-76cd4bf5f9-87x9b"
Sep 16 02:42:16.223484 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:42:16.223300    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b8ae3d33-d1dd-4d95-85b5-d7deed379ddd-dshm\") pod \"e2e-lifecycle-model-77f43ab3-kserve-76cd4bf5f9-87x9b\" (UID: \"b8ae3d33-d1dd-4d95-85b5-d7deed379ddd\") " pod="ai-tenant-e2e-mt-77f43ab3/e2e-lifecycle-model-77f43ab3-kserve-76cd4bf5f9-87x9b"
Sep 16 02:42:16.324763 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:42:16.324729    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b8ae3d33-d1dd-4d95-85b5-d7deed379ddd-home\") pod \"e2e-lifecycle-model-77f43ab3-kserve-76cd4bf5f9-87x9b\" (UID: \"b8ae3d33-d1dd-4d95-85b5-d7deed379ddd\") " pod="ai-tenant-e2e-mt-77f43ab3/e2e-lifecycle-model-77f43ab3-kserve-76cd4bf5f9-87x9b"
Sep 16 02:42:16.324933 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:42:16.324795    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v7hj6\" (UniqueName: \"kubernetes.io/projected/b8ae3d33-d1dd-4d95-85b5-d7deed379ddd-kube-api-access-v7hj6\") pod \"e2e-lifecycle-model-77f43ab3-kserve-76cd4bf5f9-87x9b\" (UID: \"b8ae3d33-d1dd-4d95-85b5-d7deed379ddd\") " pod="ai-tenant-e2e-mt-77f43ab3/e2e-lifecycle-model-77f43ab3-kserve-76cd4bf5f9-87x9b"
Sep 16 02:42:16.324933 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:42:16.324816    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b8ae3d33-d1dd-4d95-85b5-d7deed379ddd-tls-certs\") pod \"e2e-lifecycle-model-77f43ab3-kserve-76cd4bf5f9-87x9b\" (UID: \"b8ae3d33-d1dd-4d95-85b5-d7deed379ddd\") " pod="ai-tenant-e2e-mt-77f43ab3/e2e-lifecycle-model-77f43ab3-kserve-76cd4bf5f9-87x9b"
Sep 16 02:42:16.324933 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:42:16.324835    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b8ae3d33-d1dd-4d95-85b5-d7deed379ddd-dshm\") pod \"e2e-lifecycle-model-77f43ab3-kserve-76cd4bf5f9-87x9b\" (UID: \"b8ae3d33-d1dd-4d95-85b5-d7deed379ddd\") " pod="ai-tenant-e2e-mt-77f43ab3/e2e-lifecycle-model-77f43ab3-kserve-76cd4bf5f9-87x9b"
Sep 16 02:42:16.324933 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:42:16.324884    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b8ae3d33-d1dd-4d95-85b5-d7deed379ddd-model-cache\") pod \"e2e-lifecycle-model-77f43ab3-kserve-76cd4bf5f9-87x9b\" (UID: \"b8ae3d33-d1dd-4d95-85b5-d7deed379ddd\") " pod="ai-tenant-e2e-mt-77f43ab3/e2e-lifecycle-model-77f43ab3-kserve-76cd4bf5f9-87x9b"
Sep 16 02:42:16.324933 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:42:16.324906    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b8ae3d33-d1dd-4d95-85b5-d7deed379ddd-tmp-dir\") pod \"e2e-lifecycle-model-77f43ab3-kserve-76cd4bf5f9-87x9b\" (UID: \"b8ae3d33-d1dd-4d95-85b5-d7deed379ddd\") " pod="ai-tenant-e2e-mt-77f43ab3/e2e-lifecycle-model-77f43ab3-kserve-76cd4bf5f9-87x9b"
Sep 16 02:42:16.325241 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:42:16.325162    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b8ae3d33-d1dd-4d95-85b5-d7deed379ddd-home\") pod \"e2e-lifecycle-model-77f43ab3-kserve-76cd4bf5f9-87x9b\" (UID: \"b8ae3d33-d1dd-4d95-85b5-d7deed379ddd\") " pod="ai-tenant-e2e-mt-77f43ab3/e2e-lifecycle-model-77f43ab3-kserve-76cd4bf5f9-87x9b"
Sep 16 02:42:16.325241 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:42:16.325207    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b8ae3d33-d1dd-4d95-85b5-d7deed379ddd-model-cache\") pod \"e2e-lifecycle-model-77f43ab3-kserve-76cd4bf5f9-87x9b\" (UID: \"b8ae3d33-d1dd-4d95-85b5-d7deed379ddd\") " pod="ai-tenant-e2e-mt-77f43ab3/e2e-lifecycle-model-77f43ab3-kserve-76cd4bf5f9-87x9b"
Sep 16 02:42:16.325328 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:42:16.325289    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b8ae3d33-d1dd-4d95-85b5-d7deed379ddd-tmp-dir\") pod \"e2e-lifecycle-model-77f43ab3-kserve-76cd4bf5f9-87x9b\" (UID: \"b8ae3d33-d1dd-4d95-85b5-d7deed379ddd\") " pod="ai-tenant-e2e-mt-77f43ab3/e2e-lifecycle-model-77f43ab3-kserve-76cd4bf5f9-87x9b"
Sep 16 02:42:16.327185 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:42:16.327169    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b8ae3d33-d1dd-4d95-85b5-d7deed379ddd-dshm\") pod \"e2e-lifecycle-model-77f43ab3-kserve-76cd4bf5f9-87x9b\" (UID: \"b8ae3d33-d1dd-4d95-85b5-d7deed379ddd\") " pod="ai-tenant-e2e-mt-77f43ab3/e2e-lifecycle-model-77f43ab3-kserve-76cd4bf5f9-87x9b"
Sep 16 02:42:16.327449 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:42:16.327430    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b8ae3d33-d1dd-4d95-85b5-d7deed379ddd-tls-certs\") pod \"e2e-lifecycle-model-77f43ab3-kserve-76cd4bf5f9-87x9b\" (UID: \"b8ae3d33-d1dd-4d95-85b5-d7deed379ddd\") " pod="ai-tenant-e2e-mt-77f43ab3/e2e-lifecycle-model-77f43ab3-kserve-76cd4bf5f9-87x9b"
Sep 16 02:42:16.332982 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:42:16.332959    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v7hj6\" (UniqueName: \"kubernetes.io/projected/b8ae3d33-d1dd-4d95-85b5-d7deed379ddd-kube-api-access-v7hj6\") pod \"e2e-lifecycle-model-77f43ab3-kserve-76cd4bf5f9-87x9b\" (UID: \"b8ae3d33-d1dd-4d95-85b5-d7deed379ddd\") " pod="ai-tenant-e2e-mt-77f43ab3/e2e-lifecycle-model-77f43ab3-kserve-76cd4bf5f9-87x9b"
Sep 16 02:42:16.392017 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:42:16.391991    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-77f43ab3/e2e-lifecycle-model-77f43ab3-kserve-76cd4bf5f9-87x9b"
Sep 16 02:42:16.665739 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:42:16.665711    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-77f43ab3/e2e-lifecycle-model-77f43ab3-kserve-76cd4bf5f9-87x9b"]
Sep 16 02:42:16.667654 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:42:16.667622    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb8ae3d33_d1dd_4d95_85b5_d7deed379ddd.slice/crio-cb452ea799fb923dc596e2d421f161735dccecb85e894ba96e09b0844479bd71 WatchSource:0}: Error finding container cb452ea799fb923dc596e2d421f161735dccecb85e894ba96e09b0844479bd71: Status 404 returned error can't find the container with id cb452ea799fb923dc596e2d421f161735dccecb85e894ba96e09b0844479bd71
Sep 16 02:42:17.156178 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:42:17.156137    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-77f43ab3/e2e-lifecycle-model-77f43ab3-kserve-76cd4bf5f9-87x9b" event={"ID":"b8ae3d33-d1dd-4d95-85b5-d7deed379ddd","Type":"ContainerStarted","Data":"38cc065db9af9c135318ccfabe0c2449d1ec21700efaff4fbf7e68ae1829a8f9"}
Sep 16 02:42:17.156692 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:42:17.156188    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-77f43ab3/e2e-lifecycle-model-77f43ab3-kserve-76cd4bf5f9-87x9b" event={"ID":"b8ae3d33-d1dd-4d95-85b5-d7deed379ddd","Type":"ContainerStarted","Data":"cb452ea799fb923dc596e2d421f161735dccecb85e894ba96e09b0844479bd71"}
Sep 16 02:42:17.172060 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:42:17.171995    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-77f43ab3/e2e-lifecycle-model-77f43ab3-kserve-76cd4bf5f9-87x9b" podStartSLOduration=1.17197663 podStartE2EDuration="1.17197663s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 02:42:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 02:42:17.171780328 +0000 UTC m=+1757.555295573" watchObservedRunningTime="2026-09-16 02:42:17.17197663 +0000 UTC m=+1757.555491875"
Sep 16 02:42:18.545023 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:42:18.544976    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:42:26.393092 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:42:26.393051    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-77f43ab3/e2e-lifecycle-model-77f43ab3-kserve-76cd4bf5f9-87x9b"
Sep 16 02:42:26.393628 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:42:26.393107    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-77f43ab3/e2e-lifecycle-model-77f43ab3-kserve-76cd4bf5f9-87x9b"
Sep 16 02:42:26.405698 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:42:26.405674    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-77f43ab3/e2e-lifecycle-model-77f43ab3-kserve-76cd4bf5f9-87x9b"
Sep 16 02:42:27.218428 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:42:27.218398    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-77f43ab3/e2e-lifecycle-model-77f43ab3-kserve-76cd4bf5f9-87x9b"
Sep 16 02:42:32.096446 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:42:32.096406    2579 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-pre-processing-6cd864bc95-mh4jh" secret="" err="secret \"payload-processing-dockercfg-zzx6j\" not found"
Sep 16 02:42:36.956791 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:42:36.956719    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:42:58.701795 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:42:58.701750    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:42:59.533235 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:42:59.533184    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:43:00.204295 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:00.204260    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-c8q8z_ebf846fa-af7d-4fdd-9662-b7c533c36ddd/console-operator/2.log"
Sep 16 02:43:00.216834 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:00.216806    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-c8q8z_ebf846fa-af7d-4fdd-9662-b7c533c36ddd/console-operator/2.log"
Sep 16 02:43:00.264935 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:00.264903    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w2-24aabe/sub-test-model-24aabe-kserve-6ff87cb44f-qbc9q"]
Sep 16 02:43:00.281500 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:00.281441    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:43:00.311528 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:00.311503    2579 scope.go:117] "RemoveContainer" containerID="019d9b3e5172bc377b7153a02ef38c13fcc39d225fe87625f46a95b9a25bfd77"
Sep 16 02:43:00.320539 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:00.320519    2579 scope.go:117] "RemoveContainer" containerID="dd7267484e97e8deb82d4f5e2a83998ce34af57a9a408c30d8283536e938bfc6"
Sep 16 02:43:00.379147 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:00.379103    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:43:01.074194 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:01.074155    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:43:01.372935 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:01.372858    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w2-a53a44/sub-test-model-a53a44-kserve-bccfdbcbc-hlxr6"]
Sep 16 02:43:01.393372 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:01.393339    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:43:02.024493 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:02.024445    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:43:02.640908 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:02.640866    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:43:07.080548 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:07.080511    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w2-24aabe/rate-test-model-24aabe-kserve-9dc995687-7w6gs"]
Sep 16 02:43:07.084533 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:07.084504    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w2-24aabe/rate-test-model-24aabe-kserve-9dc995687-7w6gs"
Sep 16 02:43:07.086790 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:07.086767    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w2-24aabe\"/\"rate-test-model-24aabe-kserve-self-signed-certs\""
Sep 16 02:43:07.093297 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:07.093199    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w2-24aabe/rate-test-model-24aabe-kserve-9dc995687-7w6gs"]
Sep 16 02:43:07.103613 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:07.103573    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:43:07.208460 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:07.208419    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3336085d-16ba-4eae-9d4e-a4a3e949a615-dshm\") pod \"rate-test-model-24aabe-kserve-9dc995687-7w6gs\" (UID: \"3336085d-16ba-4eae-9d4e-a4a3e949a615\") " pod="ai-tenant-e2e-shared-a-w2-24aabe/rate-test-model-24aabe-kserve-9dc995687-7w6gs"
Sep 16 02:43:07.208647 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:07.208538    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-59mhg\" (UniqueName: \"kubernetes.io/projected/3336085d-16ba-4eae-9d4e-a4a3e949a615-kube-api-access-59mhg\") pod \"rate-test-model-24aabe-kserve-9dc995687-7w6gs\" (UID: \"3336085d-16ba-4eae-9d4e-a4a3e949a615\") " pod="ai-tenant-e2e-shared-a-w2-24aabe/rate-test-model-24aabe-kserve-9dc995687-7w6gs"
Sep 16 02:43:07.208647 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:07.208570    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3336085d-16ba-4eae-9d4e-a4a3e949a615-home\") pod \"rate-test-model-24aabe-kserve-9dc995687-7w6gs\" (UID: \"3336085d-16ba-4eae-9d4e-a4a3e949a615\") " pod="ai-tenant-e2e-shared-a-w2-24aabe/rate-test-model-24aabe-kserve-9dc995687-7w6gs"
Sep 16 02:43:07.208765 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:07.208735    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3336085d-16ba-4eae-9d4e-a4a3e949a615-tls-certs\") pod \"rate-test-model-24aabe-kserve-9dc995687-7w6gs\" (UID: \"3336085d-16ba-4eae-9d4e-a4a3e949a615\") " pod="ai-tenant-e2e-shared-a-w2-24aabe/rate-test-model-24aabe-kserve-9dc995687-7w6gs"
Sep 16 02:43:07.208825 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:07.208778    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3336085d-16ba-4eae-9d4e-a4a3e949a615-tmp-dir\") pod \"rate-test-model-24aabe-kserve-9dc995687-7w6gs\" (UID: \"3336085d-16ba-4eae-9d4e-a4a3e949a615\") " pod="ai-tenant-e2e-shared-a-w2-24aabe/rate-test-model-24aabe-kserve-9dc995687-7w6gs"
Sep 16 02:43:07.208869 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:07.208857    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3336085d-16ba-4eae-9d4e-a4a3e949a615-model-cache\") pod \"rate-test-model-24aabe-kserve-9dc995687-7w6gs\" (UID: \"3336085d-16ba-4eae-9d4e-a4a3e949a615\") " pod="ai-tenant-e2e-shared-a-w2-24aabe/rate-test-model-24aabe-kserve-9dc995687-7w6gs"
Sep 16 02:43:07.309798 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:07.309770    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3336085d-16ba-4eae-9d4e-a4a3e949a615-dshm\") pod \"rate-test-model-24aabe-kserve-9dc995687-7w6gs\" (UID: \"3336085d-16ba-4eae-9d4e-a4a3e949a615\") " pod="ai-tenant-e2e-shared-a-w2-24aabe/rate-test-model-24aabe-kserve-9dc995687-7w6gs"
Sep 16 02:43:07.309955 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:07.309808    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-59mhg\" (UniqueName: \"kubernetes.io/projected/3336085d-16ba-4eae-9d4e-a4a3e949a615-kube-api-access-59mhg\") pod \"rate-test-model-24aabe-kserve-9dc995687-7w6gs\" (UID: \"3336085d-16ba-4eae-9d4e-a4a3e949a615\") " pod="ai-tenant-e2e-shared-a-w2-24aabe/rate-test-model-24aabe-kserve-9dc995687-7w6gs"
Sep 16 02:43:07.309955 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:07.309824    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3336085d-16ba-4eae-9d4e-a4a3e949a615-home\") pod \"rate-test-model-24aabe-kserve-9dc995687-7w6gs\" (UID: \"3336085d-16ba-4eae-9d4e-a4a3e949a615\") " pod="ai-tenant-e2e-shared-a-w2-24aabe/rate-test-model-24aabe-kserve-9dc995687-7w6gs"
Sep 16 02:43:07.309955 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:07.309896    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3336085d-16ba-4eae-9d4e-a4a3e949a615-tls-certs\") pod \"rate-test-model-24aabe-kserve-9dc995687-7w6gs\" (UID: \"3336085d-16ba-4eae-9d4e-a4a3e949a615\") " pod="ai-tenant-e2e-shared-a-w2-24aabe/rate-test-model-24aabe-kserve-9dc995687-7w6gs"
Sep 16 02:43:07.309955 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:07.309920    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3336085d-16ba-4eae-9d4e-a4a3e949a615-tmp-dir\") pod \"rate-test-model-24aabe-kserve-9dc995687-7w6gs\" (UID: \"3336085d-16ba-4eae-9d4e-a4a3e949a615\") " pod="ai-tenant-e2e-shared-a-w2-24aabe/rate-test-model-24aabe-kserve-9dc995687-7w6gs"
Sep 16 02:43:07.310160 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:07.309973    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3336085d-16ba-4eae-9d4e-a4a3e949a615-model-cache\") pod \"rate-test-model-24aabe-kserve-9dc995687-7w6gs\" (UID: \"3336085d-16ba-4eae-9d4e-a4a3e949a615\") " pod="ai-tenant-e2e-shared-a-w2-24aabe/rate-test-model-24aabe-kserve-9dc995687-7w6gs"
Sep 16 02:43:07.310298 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:07.310273    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3336085d-16ba-4eae-9d4e-a4a3e949a615-home\") pod \"rate-test-model-24aabe-kserve-9dc995687-7w6gs\" (UID: \"3336085d-16ba-4eae-9d4e-a4a3e949a615\") " pod="ai-tenant-e2e-shared-a-w2-24aabe/rate-test-model-24aabe-kserve-9dc995687-7w6gs"
Sep 16 02:43:07.310298 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:07.310295    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3336085d-16ba-4eae-9d4e-a4a3e949a615-model-cache\") pod \"rate-test-model-24aabe-kserve-9dc995687-7w6gs\" (UID: \"3336085d-16ba-4eae-9d4e-a4a3e949a615\") " pod="ai-tenant-e2e-shared-a-w2-24aabe/rate-test-model-24aabe-kserve-9dc995687-7w6gs"
Sep 16 02:43:07.310458 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:07.310352    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3336085d-16ba-4eae-9d4e-a4a3e949a615-tmp-dir\") pod \"rate-test-model-24aabe-kserve-9dc995687-7w6gs\" (UID: \"3336085d-16ba-4eae-9d4e-a4a3e949a615\") " pod="ai-tenant-e2e-shared-a-w2-24aabe/rate-test-model-24aabe-kserve-9dc995687-7w6gs"
Sep 16 02:43:07.312370 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:07.312342    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3336085d-16ba-4eae-9d4e-a4a3e949a615-dshm\") pod \"rate-test-model-24aabe-kserve-9dc995687-7w6gs\" (UID: \"3336085d-16ba-4eae-9d4e-a4a3e949a615\") " pod="ai-tenant-e2e-shared-a-w2-24aabe/rate-test-model-24aabe-kserve-9dc995687-7w6gs"
Sep 16 02:43:07.312589 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:07.312574    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3336085d-16ba-4eae-9d4e-a4a3e949a615-tls-certs\") pod \"rate-test-model-24aabe-kserve-9dc995687-7w6gs\" (UID: \"3336085d-16ba-4eae-9d4e-a4a3e949a615\") " pod="ai-tenant-e2e-shared-a-w2-24aabe/rate-test-model-24aabe-kserve-9dc995687-7w6gs"
Sep 16 02:43:07.317243 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:07.317224    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-59mhg\" (UniqueName: \"kubernetes.io/projected/3336085d-16ba-4eae-9d4e-a4a3e949a615-kube-api-access-59mhg\") pod \"rate-test-model-24aabe-kserve-9dc995687-7w6gs\" (UID: \"3336085d-16ba-4eae-9d4e-a4a3e949a615\") " pod="ai-tenant-e2e-shared-a-w2-24aabe/rate-test-model-24aabe-kserve-9dc995687-7w6gs"
Sep 16 02:43:07.396458 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:07.396391    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w2-24aabe/rate-test-model-24aabe-kserve-9dc995687-7w6gs"
Sep 16 02:43:07.670430 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:07.670403    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w2-24aabe/rate-test-model-24aabe-kserve-9dc995687-7w6gs"]
Sep 16 02:43:07.671786 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:43:07.671758    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3336085d_16ba_4eae_9d4e_a4a3e949a615.slice/crio-84a7f61c35ed9c2f44cf6645cbb0039a83f833c9f86ea5f3afd81327689f7149 WatchSource:0}: Error finding container 84a7f61c35ed9c2f44cf6645cbb0039a83f833c9f86ea5f3afd81327689f7149: Status 404 returned error can't find the container with id 84a7f61c35ed9c2f44cf6645cbb0039a83f833c9f86ea5f3afd81327689f7149
Sep 16 02:43:08.422003 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:08.421970    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w2-24aabe/rate-test-model-24aabe-kserve-9dc995687-7w6gs" event={"ID":"3336085d-16ba-4eae-9d4e-a4a3e949a615","Type":"ContainerStarted","Data":"2c19994ba4881b26d5a77904d5cbe8286162e83f23f24e7757547ddbf98b620d"}
Sep 16 02:43:08.422398 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:08.422016    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w2-24aabe/rate-test-model-24aabe-kserve-9dc995687-7w6gs" event={"ID":"3336085d-16ba-4eae-9d4e-a4a3e949a615","Type":"ContainerStarted","Data":"84a7f61c35ed9c2f44cf6645cbb0039a83f833c9f86ea5f3afd81327689f7149"}
Sep 16 02:43:08.441379 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:08.441325    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w2-24aabe/rate-test-model-24aabe-kserve-9dc995687-7w6gs" podStartSLOduration=1.4413100189999999 podStartE2EDuration="1.441310019s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 02:43:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 02:43:08.439697021 +0000 UTC m=+1808.823212264" watchObservedRunningTime="2026-09-16 02:43:08.441310019 +0000 UTC m=+1808.824825264"
Sep 16 02:43:13.846813 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:13.846771    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:43:13.864433 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:13.864402    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-77f43ab3-6dcc766745-mphkm"]
Sep 16 02:43:13.864712 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:13.864685    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-77f43ab3-6dcc766745-mphkm" podUID="1cbe69b2-8c94-4465-888e-280cdf86caf2" containerName="payload-processing" containerID="cri-o://016380ef62205699b7f5194024b5f0402a33b075aafbc7f2152a7ae2b73c2e37" gracePeriod=30
Sep 16 02:43:13.875357 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:13.875331    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-77f43ab3-547f5fcc66-xq82k"]
Sep 16 02:43:13.876243 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:13.875573    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-77f43ab3-547f5fcc66-xq82k" podUID="f7f95091-2ef8-45b6-96dd-25fa307ac07e" containerName="payload-pre-processing" containerID="cri-o://888ab86525b3df534f70435fcc8e3fe2dbde1cd4947f649f7b07bec795a082ac" gracePeriod=30
Sep 16 02:43:13.945760 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:13.945727    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:43:14.047447 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:14.046838    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:43:14.145210 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:14.145181    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:43:14.161110 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:14.161079    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-77f43ab3-6dcc766745-mphkm"
Sep 16 02:43:14.164690 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:14.164656    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-77f43ab3-547f5fcc66-xq82k"
Sep 16 02:43:14.170451 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:14.170422    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wq4xb\" (UniqueName: \"kubernetes.io/projected/1cbe69b2-8c94-4465-888e-280cdf86caf2-kube-api-access-wq4xb\") pod \"1cbe69b2-8c94-4465-888e-280cdf86caf2\" (UID: \"1cbe69b2-8c94-4465-888e-280cdf86caf2\") "
Sep 16 02:43:14.170599 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:14.170500    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f7f95091-2ef8-45b6-96dd-25fa307ac07e-plugins-config-volume\") pod \"f7f95091-2ef8-45b6-96dd-25fa307ac07e\" (UID: \"f7f95091-2ef8-45b6-96dd-25fa307ac07e\") "
Sep 16 02:43:14.170599 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:14.170567    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1cbe69b2-8c94-4465-888e-280cdf86caf2-plugins-config-volume\") pod \"1cbe69b2-8c94-4465-888e-280cdf86caf2\" (UID: \"1cbe69b2-8c94-4465-888e-280cdf86caf2\") "
Sep 16 02:43:14.170710 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:14.170602    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k8dt5\" (UniqueName: \"kubernetes.io/projected/f7f95091-2ef8-45b6-96dd-25fa307ac07e-kube-api-access-k8dt5\") pod \"f7f95091-2ef8-45b6-96dd-25fa307ac07e\" (UID: \"f7f95091-2ef8-45b6-96dd-25fa307ac07e\") "
Sep 16 02:43:14.170891 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:14.170858    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f7f95091-2ef8-45b6-96dd-25fa307ac07e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f7f95091-2ef8-45b6-96dd-25fa307ac07e" (UID: "f7f95091-2ef8-45b6-96dd-25fa307ac07e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 02:43:14.170980 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:14.170889    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1cbe69b2-8c94-4465-888e-280cdf86caf2-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "1cbe69b2-8c94-4465-888e-280cdf86caf2" (UID: "1cbe69b2-8c94-4465-888e-280cdf86caf2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 02:43:14.171038 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:14.171003    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1cbe69b2-8c94-4465-888e-280cdf86caf2-plugins-config-volume\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:43:14.171038 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:14.171022    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f7f95091-2ef8-45b6-96dd-25fa307ac07e-plugins-config-volume\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:43:14.172839 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:14.172812    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1cbe69b2-8c94-4465-888e-280cdf86caf2-kube-api-access-wq4xb" (OuterVolumeSpecName: "kube-api-access-wq4xb") pod "1cbe69b2-8c94-4465-888e-280cdf86caf2" (UID: "1cbe69b2-8c94-4465-888e-280cdf86caf2"). InnerVolumeSpecName "kube-api-access-wq4xb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 02:43:14.172929 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:14.172902    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7f95091-2ef8-45b6-96dd-25fa307ac07e-kube-api-access-k8dt5" (OuterVolumeSpecName: "kube-api-access-k8dt5") pod "f7f95091-2ef8-45b6-96dd-25fa307ac07e" (UID: "f7f95091-2ef8-45b6-96dd-25fa307ac07e"). InnerVolumeSpecName "kube-api-access-k8dt5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 02:43:14.271741 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:14.271709    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k8dt5\" (UniqueName: \"kubernetes.io/projected/f7f95091-2ef8-45b6-96dd-25fa307ac07e-kube-api-access-k8dt5\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:43:14.271741 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:14.271739    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wq4xb\" (UniqueName: \"kubernetes.io/projected/1cbe69b2-8c94-4465-888e-280cdf86caf2-kube-api-access-wq4xb\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:43:14.453247 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:14.453216    2579 generic.go:356] "Generic (PLEG): container finished" podID="1cbe69b2-8c94-4465-888e-280cdf86caf2" containerID="016380ef62205699b7f5194024b5f0402a33b075aafbc7f2152a7ae2b73c2e37" exitCode=0
Sep 16 02:43:14.453430 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:14.453251    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-77f43ab3-6dcc766745-mphkm" event={"ID":"1cbe69b2-8c94-4465-888e-280cdf86caf2","Type":"ContainerDied","Data":"016380ef62205699b7f5194024b5f0402a33b075aafbc7f2152a7ae2b73c2e37"}
Sep 16 02:43:14.453430 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:14.453280    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-77f43ab3-6dcc766745-mphkm"
Sep 16 02:43:14.453430 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:14.453319    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-77f43ab3-6dcc766745-mphkm" event={"ID":"1cbe69b2-8c94-4465-888e-280cdf86caf2","Type":"ContainerDied","Data":"4bb829fcd638de0e766be5ff3ce994c136f812ee6fe38c0c63c3c45cca04d3cc"}
Sep 16 02:43:14.453430 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:14.453350    2579 scope.go:117] "RemoveContainer" containerID="016380ef62205699b7f5194024b5f0402a33b075aafbc7f2152a7ae2b73c2e37"
Sep 16 02:43:14.454763 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:14.454744    2579 generic.go:356] "Generic (PLEG): container finished" podID="f7f95091-2ef8-45b6-96dd-25fa307ac07e" containerID="888ab86525b3df534f70435fcc8e3fe2dbde1cd4947f649f7b07bec795a082ac" exitCode=0
Sep 16 02:43:14.454880 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:14.454803    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-77f43ab3-547f5fcc66-xq82k"
Sep 16 02:43:14.454880 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:14.454814    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-77f43ab3-547f5fcc66-xq82k" event={"ID":"f7f95091-2ef8-45b6-96dd-25fa307ac07e","Type":"ContainerDied","Data":"888ab86525b3df534f70435fcc8e3fe2dbde1cd4947f649f7b07bec795a082ac"}
Sep 16 02:43:14.454880 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:14.454855    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-77f43ab3-547f5fcc66-xq82k" event={"ID":"f7f95091-2ef8-45b6-96dd-25fa307ac07e","Type":"ContainerDied","Data":"f8687fce86c26e9a91cba46b299b29a3029e0fddf16129c8150006367d9817c1"}
Sep 16 02:43:14.463968 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:14.463949    2579 scope.go:117] "RemoveContainer" containerID="016380ef62205699b7f5194024b5f0402a33b075aafbc7f2152a7ae2b73c2e37"
Sep 16 02:43:14.464333 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:43:14.464305    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"016380ef62205699b7f5194024b5f0402a33b075aafbc7f2152a7ae2b73c2e37\": container with ID starting with 016380ef62205699b7f5194024b5f0402a33b075aafbc7f2152a7ae2b73c2e37 not found: ID does not exist" containerID="016380ef62205699b7f5194024b5f0402a33b075aafbc7f2152a7ae2b73c2e37"
Sep 16 02:43:14.464427 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:14.464343    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"016380ef62205699b7f5194024b5f0402a33b075aafbc7f2152a7ae2b73c2e37"} err="failed to get container status \"016380ef62205699b7f5194024b5f0402a33b075aafbc7f2152a7ae2b73c2e37\": rpc error: code = NotFound desc = could not find container \"016380ef62205699b7f5194024b5f0402a33b075aafbc7f2152a7ae2b73c2e37\": container with ID starting with 016380ef62205699b7f5194024b5f0402a33b075aafbc7f2152a7ae2b73c2e37 not found: ID does not exist"
Sep 16 02:43:14.464427 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:14.464363    2579 scope.go:117] "RemoveContainer" containerID="888ab86525b3df534f70435fcc8e3fe2dbde1cd4947f649f7b07bec795a082ac"
Sep 16 02:43:14.474048 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:14.474027    2579 scope.go:117] "RemoveContainer" containerID="888ab86525b3df534f70435fcc8e3fe2dbde1cd4947f649f7b07bec795a082ac"
Sep 16 02:43:14.474642 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:43:14.474358    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"888ab86525b3df534f70435fcc8e3fe2dbde1cd4947f649f7b07bec795a082ac\": container with ID starting with 888ab86525b3df534f70435fcc8e3fe2dbde1cd4947f649f7b07bec795a082ac not found: ID does not exist" containerID="888ab86525b3df534f70435fcc8e3fe2dbde1cd4947f649f7b07bec795a082ac"
Sep 16 02:43:14.474642 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:14.474393    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"888ab86525b3df534f70435fcc8e3fe2dbde1cd4947f649f7b07bec795a082ac"} err="failed to get container status \"888ab86525b3df534f70435fcc8e3fe2dbde1cd4947f649f7b07bec795a082ac\": rpc error: code = NotFound desc = could not find container \"888ab86525b3df534f70435fcc8e3fe2dbde1cd4947f649f7b07bec795a082ac\": container with ID starting with 888ab86525b3df534f70435fcc8e3fe2dbde1cd4947f649f7b07bec795a082ac not found: ID does not exist"
Sep 16 02:43:14.494362 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:14.494338    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-77f43ab3-547f5fcc66-xq82k"]
Sep 16 02:43:14.497636 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:14.497613    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-77f43ab3-547f5fcc66-xq82k"]
Sep 16 02:43:14.501757 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:14.501735    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-77f43ab3-6dcc766745-mphkm"]
Sep 16 02:43:14.507699 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:14.507676    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-77f43ab3-6dcc766745-mphkm"]
Sep 16 02:43:14.508025 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:43:14.508001    2579 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7f95091_2ef8_45b6_96dd_25fa307ac07e.slice/crio-f8687fce86c26e9a91cba46b299b29a3029e0fddf16129c8150006367d9817c1\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1cbe69b2_8c94_4465_888e_280cdf86caf2.slice/crio-4bb829fcd638de0e766be5ff3ce994c136f812ee6fe38c0c63c3c45cca04d3cc\": RecentStats: unable to find data in memory cache]"
Sep 16 02:43:14.518770 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:14.518744    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:43:15.173988 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:15.173955    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:43:15.285882 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:15.285837    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w2-24aabe/sub-test-model-24aabe-kserve-6ff87cb44f-qbc9q" podUID="a56c54a4-83b5-44c8-8cb7-613642bebe2c" containerName="main" containerID="cri-o://832a2a18798abd35463abf39c1c73967af71ac19ade22786ed225ef1ca5270f0" gracePeriod=45
Sep 16 02:43:15.466105 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:15.466070    2579 generic.go:356] "Generic (PLEG): container finished" podID="a56c54a4-83b5-44c8-8cb7-613642bebe2c" containerID="832a2a18798abd35463abf39c1c73967af71ac19ade22786ed225ef1ca5270f0" exitCode=0
Sep 16 02:43:15.466276 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:15.466127    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w2-24aabe/sub-test-model-24aabe-kserve-6ff87cb44f-qbc9q" event={"ID":"a56c54a4-83b5-44c8-8cb7-613642bebe2c","Type":"ContainerDied","Data":"832a2a18798abd35463abf39c1c73967af71ac19ade22786ed225ef1ca5270f0"}
Sep 16 02:43:15.543831 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:15.543806    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w2-24aabe/sub-test-model-24aabe-kserve-6ff87cb44f-qbc9q"
Sep 16 02:43:15.582780 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:15.582738    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a56c54a4-83b5-44c8-8cb7-613642bebe2c-tmp-dir\") pod \"a56c54a4-83b5-44c8-8cb7-613642bebe2c\" (UID: \"a56c54a4-83b5-44c8-8cb7-613642bebe2c\") "
Sep 16 02:43:15.582956 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:15.582826    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a56c54a4-83b5-44c8-8cb7-613642bebe2c-dshm\") pod \"a56c54a4-83b5-44c8-8cb7-613642bebe2c\" (UID: \"a56c54a4-83b5-44c8-8cb7-613642bebe2c\") "
Sep 16 02:43:15.582956 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:15.582871    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a56c54a4-83b5-44c8-8cb7-613642bebe2c-model-cache\") pod \"a56c54a4-83b5-44c8-8cb7-613642bebe2c\" (UID: \"a56c54a4-83b5-44c8-8cb7-613642bebe2c\") "
Sep 16 02:43:15.582956 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:15.582896    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z7n54\" (UniqueName: \"kubernetes.io/projected/a56c54a4-83b5-44c8-8cb7-613642bebe2c-kube-api-access-z7n54\") pod \"a56c54a4-83b5-44c8-8cb7-613642bebe2c\" (UID: \"a56c54a4-83b5-44c8-8cb7-613642bebe2c\") "
Sep 16 02:43:15.583138 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:15.582984    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a56c54a4-83b5-44c8-8cb7-613642bebe2c-tls-certs\") pod \"a56c54a4-83b5-44c8-8cb7-613642bebe2c\" (UID: \"a56c54a4-83b5-44c8-8cb7-613642bebe2c\") "
Sep 16 02:43:15.583138 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:15.582992    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a56c54a4-83b5-44c8-8cb7-613642bebe2c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a56c54a4-83b5-44c8-8cb7-613642bebe2c" (UID: "a56c54a4-83b5-44c8-8cb7-613642bebe2c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 02:43:15.583138 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:15.583024    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a56c54a4-83b5-44c8-8cb7-613642bebe2c-home\") pod \"a56c54a4-83b5-44c8-8cb7-613642bebe2c\" (UID: \"a56c54a4-83b5-44c8-8cb7-613642bebe2c\") "
Sep 16 02:43:15.583138 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:15.583077    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a56c54a4-83b5-44c8-8cb7-613642bebe2c-model-cache" (OuterVolumeSpecName: "model-cache") pod "a56c54a4-83b5-44c8-8cb7-613642bebe2c" (UID: "a56c54a4-83b5-44c8-8cb7-613642bebe2c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 02:43:15.583637 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:15.583238    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a56c54a4-83b5-44c8-8cb7-613642bebe2c-home" (OuterVolumeSpecName: "home") pod "a56c54a4-83b5-44c8-8cb7-613642bebe2c" (UID: "a56c54a4-83b5-44c8-8cb7-613642bebe2c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 02:43:15.583637 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:15.583371    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a56c54a4-83b5-44c8-8cb7-613642bebe2c-home\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:43:15.583637 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:15.583389    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a56c54a4-83b5-44c8-8cb7-613642bebe2c-tmp-dir\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:43:15.583637 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:15.583403    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a56c54a4-83b5-44c8-8cb7-613642bebe2c-model-cache\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:43:15.585226 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:15.585198    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a56c54a4-83b5-44c8-8cb7-613642bebe2c-dshm" (OuterVolumeSpecName: "dshm") pod "a56c54a4-83b5-44c8-8cb7-613642bebe2c" (UID: "a56c54a4-83b5-44c8-8cb7-613642bebe2c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 02:43:15.585367 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:15.585344    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a56c54a4-83b5-44c8-8cb7-613642bebe2c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a56c54a4-83b5-44c8-8cb7-613642bebe2c" (UID: "a56c54a4-83b5-44c8-8cb7-613642bebe2c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 02:43:15.585617 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:15.585591    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a56c54a4-83b5-44c8-8cb7-613642bebe2c-kube-api-access-z7n54" (OuterVolumeSpecName: "kube-api-access-z7n54") pod "a56c54a4-83b5-44c8-8cb7-613642bebe2c" (UID: "a56c54a4-83b5-44c8-8cb7-613642bebe2c"). InnerVolumeSpecName "kube-api-access-z7n54". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 02:43:15.684680 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:15.684649    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a56c54a4-83b5-44c8-8cb7-613642bebe2c-tls-certs\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:43:15.684680 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:15.684673    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a56c54a4-83b5-44c8-8cb7-613642bebe2c-dshm\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:43:15.684680 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:15.684682    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z7n54\" (UniqueName: \"kubernetes.io/projected/a56c54a4-83b5-44c8-8cb7-613642bebe2c-kube-api-access-z7n54\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:43:15.923862 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:15.923759    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:43:16.103447 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:16.103401    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1cbe69b2-8c94-4465-888e-280cdf86caf2" path="/var/lib/kubelet/pods/1cbe69b2-8c94-4465-888e-280cdf86caf2/volumes"
Sep 16 02:43:16.103846 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:16.103828    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f7f95091-2ef8-45b6-96dd-25fa307ac07e" path="/var/lib/kubelet/pods/f7f95091-2ef8-45b6-96dd-25fa307ac07e/volumes"
Sep 16 02:43:16.393513 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:16.393460    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w2-a53a44/sub-test-model-a53a44-kserve-bccfdbcbc-hlxr6" podUID="5d607480-7df3-4f34-a7a2-d27f28f110c6" containerName="main" containerID="cri-o://dde1e8cfbf81516e6a707b0cd8739bd7f7bb244afd7d80756d0c52dcd84e4ffe" gracePeriod=45
Sep 16 02:43:16.473516 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:16.473487    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w2-24aabe/sub-test-model-24aabe-kserve-6ff87cb44f-qbc9q"
Sep 16 02:43:16.473516 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:16.473502    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w2-24aabe/sub-test-model-24aabe-kserve-6ff87cb44f-qbc9q" event={"ID":"a56c54a4-83b5-44c8-8cb7-613642bebe2c","Type":"ContainerDied","Data":"79dab5149dc7bc47c7fe5c6a6bd550623a2dc9345f5bc8dda99e822041c42533"}
Sep 16 02:43:16.473747 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:16.473557    2579 scope.go:117] "RemoveContainer" containerID="832a2a18798abd35463abf39c1c73967af71ac19ade22786ed225ef1ca5270f0"
Sep 16 02:43:16.488483 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:16.488447    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w2-24aabe/sub-test-model-24aabe-kserve-6ff87cb44f-qbc9q"]
Sep 16 02:43:16.490826 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:16.490806    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w2-24aabe/sub-test-model-24aabe-kserve-6ff87cb44f-qbc9q"]
Sep 16 02:43:16.637491 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:16.637444    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:43:16.648515 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:16.648442    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w2-a53a44/sub-test-model-a53a44-kserve-bccfdbcbc-hlxr6"
Sep 16 02:43:16.692644 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:16.692614    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5d607480-7df3-4f34-a7a2-d27f28f110c6-dshm\") pod \"5d607480-7df3-4f34-a7a2-d27f28f110c6\" (UID: \"5d607480-7df3-4f34-a7a2-d27f28f110c6\") "
Sep 16 02:43:16.692789 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:16.692661    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5d607480-7df3-4f34-a7a2-d27f28f110c6-home\") pod \"5d607480-7df3-4f34-a7a2-d27f28f110c6\" (UID: \"5d607480-7df3-4f34-a7a2-d27f28f110c6\") "
Sep 16 02:43:16.692789 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:16.692693    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5d607480-7df3-4f34-a7a2-d27f28f110c6-tmp-dir\") pod \"5d607480-7df3-4f34-a7a2-d27f28f110c6\" (UID: \"5d607480-7df3-4f34-a7a2-d27f28f110c6\") "
Sep 16 02:43:16.692789 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:16.692712    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5d607480-7df3-4f34-a7a2-d27f28f110c6-tls-certs\") pod \"5d607480-7df3-4f34-a7a2-d27f28f110c6\" (UID: \"5d607480-7df3-4f34-a7a2-d27f28f110c6\") "
Sep 16 02:43:16.692789 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:16.692772    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5d607480-7df3-4f34-a7a2-d27f28f110c6-model-cache\") pod \"5d607480-7df3-4f34-a7a2-d27f28f110c6\" (UID: \"5d607480-7df3-4f34-a7a2-d27f28f110c6\") "
Sep 16 02:43:16.692993 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:16.692820    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x6wl9\" (UniqueName: \"kubernetes.io/projected/5d607480-7df3-4f34-a7a2-d27f28f110c6-kube-api-access-x6wl9\") pod \"5d607480-7df3-4f34-a7a2-d27f28f110c6\" (UID: \"5d607480-7df3-4f34-a7a2-d27f28f110c6\") "
Sep 16 02:43:16.692993 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:16.692951    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d607480-7df3-4f34-a7a2-d27f28f110c6-home" (OuterVolumeSpecName: "home") pod "5d607480-7df3-4f34-a7a2-d27f28f110c6" (UID: "5d607480-7df3-4f34-a7a2-d27f28f110c6"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 02:43:16.693099 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:16.692994    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d607480-7df3-4f34-a7a2-d27f28f110c6-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "5d607480-7df3-4f34-a7a2-d27f28f110c6" (UID: "5d607480-7df3-4f34-a7a2-d27f28f110c6"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 02:43:16.693099 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:16.693044    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d607480-7df3-4f34-a7a2-d27f28f110c6-model-cache" (OuterVolumeSpecName: "model-cache") pod "5d607480-7df3-4f34-a7a2-d27f28f110c6" (UID: "5d607480-7df3-4f34-a7a2-d27f28f110c6"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 02:43:16.693209 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:16.693142    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5d607480-7df3-4f34-a7a2-d27f28f110c6-model-cache\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:43:16.693209 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:16.693160    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5d607480-7df3-4f34-a7a2-d27f28f110c6-home\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:43:16.693209 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:16.693171    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5d607480-7df3-4f34-a7a2-d27f28f110c6-tmp-dir\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:43:16.695060 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:16.695040    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d607480-7df3-4f34-a7a2-d27f28f110c6-dshm" (OuterVolumeSpecName: "dshm") pod "5d607480-7df3-4f34-a7a2-d27f28f110c6" (UID: "5d607480-7df3-4f34-a7a2-d27f28f110c6"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 02:43:16.695142 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:16.695053    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5d607480-7df3-4f34-a7a2-d27f28f110c6-kube-api-access-x6wl9" (OuterVolumeSpecName: "kube-api-access-x6wl9") pod "5d607480-7df3-4f34-a7a2-d27f28f110c6" (UID: "5d607480-7df3-4f34-a7a2-d27f28f110c6"). InnerVolumeSpecName "kube-api-access-x6wl9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 02:43:16.695142 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:16.695111    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5d607480-7df3-4f34-a7a2-d27f28f110c6-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "5d607480-7df3-4f34-a7a2-d27f28f110c6" (UID: "5d607480-7df3-4f34-a7a2-d27f28f110c6"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 02:43:16.794644 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:16.794607    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5d607480-7df3-4f34-a7a2-d27f28f110c6-tls-certs\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:43:16.794644 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:16.794640    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x6wl9\" (UniqueName: \"kubernetes.io/projected/5d607480-7df3-4f34-a7a2-d27f28f110c6-kube-api-access-x6wl9\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:43:16.794644 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:16.794650    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5d607480-7df3-4f34-a7a2-d27f28f110c6-dshm\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:43:17.247881 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:17.247845    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:43:17.397350 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:17.397317    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w2-24aabe/rate-test-model-24aabe-kserve-9dc995687-7w6gs"
Sep 16 02:43:17.397350 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:17.397360    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w2-24aabe/rate-test-model-24aabe-kserve-9dc995687-7w6gs"
Sep 16 02:43:17.409897 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:17.409874    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w2-24aabe/rate-test-model-24aabe-kserve-9dc995687-7w6gs"
Sep 16 02:43:17.481695 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:17.481670    2579 generic.go:356] "Generic (PLEG): container finished" podID="5d607480-7df3-4f34-a7a2-d27f28f110c6" containerID="dde1e8cfbf81516e6a707b0cd8739bd7f7bb244afd7d80756d0c52dcd84e4ffe" exitCode=0
Sep 16 02:43:17.481864 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:17.481708    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w2-a53a44/sub-test-model-a53a44-kserve-bccfdbcbc-hlxr6"
Sep 16 02:43:17.481864 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:17.481761    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w2-a53a44/sub-test-model-a53a44-kserve-bccfdbcbc-hlxr6" event={"ID":"5d607480-7df3-4f34-a7a2-d27f28f110c6","Type":"ContainerDied","Data":"dde1e8cfbf81516e6a707b0cd8739bd7f7bb244afd7d80756d0c52dcd84e4ffe"}
Sep 16 02:43:17.481864 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:17.481795    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w2-a53a44/sub-test-model-a53a44-kserve-bccfdbcbc-hlxr6" event={"ID":"5d607480-7df3-4f34-a7a2-d27f28f110c6","Type":"ContainerDied","Data":"e5e5a389861a238d6bb42267338e48cec8bb5361627d337c3f6896d4238ef70e"}
Sep 16 02:43:17.481864 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:17.481813    2579 scope.go:117] "RemoveContainer" containerID="dde1e8cfbf81516e6a707b0cd8739bd7f7bb244afd7d80756d0c52dcd84e4ffe"
Sep 16 02:43:17.491118 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:17.491099    2579 scope.go:117] "RemoveContainer" containerID="dde1e8cfbf81516e6a707b0cd8739bd7f7bb244afd7d80756d0c52dcd84e4ffe"
Sep 16 02:43:17.491359 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:43:17.491340    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dde1e8cfbf81516e6a707b0cd8739bd7f7bb244afd7d80756d0c52dcd84e4ffe\": container with ID starting with dde1e8cfbf81516e6a707b0cd8739bd7f7bb244afd7d80756d0c52dcd84e4ffe not found: ID does not exist" containerID="dde1e8cfbf81516e6a707b0cd8739bd7f7bb244afd7d80756d0c52dcd84e4ffe"
Sep 16 02:43:17.491412 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:17.491369    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dde1e8cfbf81516e6a707b0cd8739bd7f7bb244afd7d80756d0c52dcd84e4ffe"} err="failed to get container status \"dde1e8cfbf81516e6a707b0cd8739bd7f7bb244afd7d80756d0c52dcd84e4ffe\": rpc error: code = NotFound desc = could not find container \"dde1e8cfbf81516e6a707b0cd8739bd7f7bb244afd7d80756d0c52dcd84e4ffe\": container with ID starting with dde1e8cfbf81516e6a707b0cd8739bd7f7bb244afd7d80756d0c52dcd84e4ffe not found: ID does not exist"
Sep 16 02:43:17.494271 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:17.494250    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w2-24aabe/rate-test-model-24aabe-kserve-9dc995687-7w6gs"
Sep 16 02:43:17.502802 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:17.502749    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w2-a53a44/sub-test-model-a53a44-kserve-bccfdbcbc-hlxr6"]
Sep 16 02:43:17.504666 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:17.504649    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w2-a53a44/sub-test-model-a53a44-kserve-bccfdbcbc-hlxr6"]
Sep 16 02:43:18.102441 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:18.102407    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5d607480-7df3-4f34-a7a2-d27f28f110c6" path="/var/lib/kubelet/pods/5d607480-7df3-4f34-a7a2-d27f28f110c6/volumes"
Sep 16 02:43:18.102834 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:18.102820    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a56c54a4-83b5-44c8-8cb7-613642bebe2c" path="/var/lib/kubelet/pods/a56c54a4-83b5-44c8-8cb7-613642bebe2c/volumes"
Sep 16 02:43:22.230027 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:22.229996    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-77f43ab3/e2e-lifecycle-model-77f43ab3-kserve-76cd4bf5f9-87x9b"]
Sep 16 02:43:22.269242 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:22.269211    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:43:30.877969 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:30.877933    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w2-a53a44/rate-test-model-a53a44-kserve-f9d5fb459-7vsp9"]
Sep 16 02:43:30.878612 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:30.878591    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7f95091-2ef8-45b6-96dd-25fa307ac07e" containerName="payload-pre-processing"
Sep 16 02:43:30.878688 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:30.878615    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7f95091-2ef8-45b6-96dd-25fa307ac07e" containerName="payload-pre-processing"
Sep 16 02:43:30.878688 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:30.878628    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1cbe69b2-8c94-4465-888e-280cdf86caf2" containerName="payload-processing"
Sep 16 02:43:30.878688 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:30.878639    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="1cbe69b2-8c94-4465-888e-280cdf86caf2" containerName="payload-processing"
Sep 16 02:43:30.878688 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:30.878653    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a56c54a4-83b5-44c8-8cb7-613642bebe2c" containerName="main"
Sep 16 02:43:30.878688 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:30.878661    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="a56c54a4-83b5-44c8-8cb7-613642bebe2c" containerName="main"
Sep 16 02:43:30.878688 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:30.878672    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d607480-7df3-4f34-a7a2-d27f28f110c6" containerName="main"
Sep 16 02:43:30.878688 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:30.878681    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d607480-7df3-4f34-a7a2-d27f28f110c6" containerName="main"
Sep 16 02:43:30.879055 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:30.878793    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="f7f95091-2ef8-45b6-96dd-25fa307ac07e" containerName="payload-pre-processing"
Sep 16 02:43:30.879055 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:30.878806    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="5d607480-7df3-4f34-a7a2-d27f28f110c6" containerName="main"
Sep 16 02:43:30.879055 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:30.878818    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="a56c54a4-83b5-44c8-8cb7-613642bebe2c" containerName="main"
Sep 16 02:43:30.879055 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:30.878829    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="1cbe69b2-8c94-4465-888e-280cdf86caf2" containerName="payload-processing"
Sep 16 02:43:30.885343 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:30.885308    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w2-a53a44/rate-test-model-a53a44-kserve-f9d5fb459-7vsp9"
Sep 16 02:43:30.890850 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:30.890824    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w2-a53a44\"/\"default-dockercfg-tw8hg\""
Sep 16 02:43:30.891238 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:30.890944    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w2-a53a44\"/\"kube-root-ca.crt\""
Sep 16 02:43:30.891782 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:30.891251    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w2-a53a44\"/\"rate-test-model-a53a44-kserve-self-signed-certs\""
Sep 16 02:43:30.891782 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:30.891064    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w2-a53a44\"/\"openshift-service-ca.crt\""
Sep 16 02:43:30.892904 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:30.892883    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w2-a53a44/rate-test-model-a53a44-kserve-f9d5fb459-7vsp9"]
Sep 16 02:43:30.922166 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:30.922142    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5d02335d-cd4c-4f59-a223-85f952504e4a-model-cache\") pod \"rate-test-model-a53a44-kserve-f9d5fb459-7vsp9\" (UID: \"5d02335d-cd4c-4f59-a223-85f952504e4a\") " pod="ai-tenant-e2e-shared-b-w2-a53a44/rate-test-model-a53a44-kserve-f9d5fb459-7vsp9"
Sep 16 02:43:30.922302 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:30.922175    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mkxph\" (UniqueName: \"kubernetes.io/projected/5d02335d-cd4c-4f59-a223-85f952504e4a-kube-api-access-mkxph\") pod \"rate-test-model-a53a44-kserve-f9d5fb459-7vsp9\" (UID: \"5d02335d-cd4c-4f59-a223-85f952504e4a\") " pod="ai-tenant-e2e-shared-b-w2-a53a44/rate-test-model-a53a44-kserve-f9d5fb459-7vsp9"
Sep 16 02:43:30.922302 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:30.922215    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5d02335d-cd4c-4f59-a223-85f952504e4a-tmp-dir\") pod \"rate-test-model-a53a44-kserve-f9d5fb459-7vsp9\" (UID: \"5d02335d-cd4c-4f59-a223-85f952504e4a\") " pod="ai-tenant-e2e-shared-b-w2-a53a44/rate-test-model-a53a44-kserve-f9d5fb459-7vsp9"
Sep 16 02:43:30.922302 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:30.922273    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5d02335d-cd4c-4f59-a223-85f952504e4a-dshm\") pod \"rate-test-model-a53a44-kserve-f9d5fb459-7vsp9\" (UID: \"5d02335d-cd4c-4f59-a223-85f952504e4a\") " pod="ai-tenant-e2e-shared-b-w2-a53a44/rate-test-model-a53a44-kserve-f9d5fb459-7vsp9"
Sep 16 02:43:30.922302 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:30.922303    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5d02335d-cd4c-4f59-a223-85f952504e4a-tls-certs\") pod \"rate-test-model-a53a44-kserve-f9d5fb459-7vsp9\" (UID: \"5d02335d-cd4c-4f59-a223-85f952504e4a\") " pod="ai-tenant-e2e-shared-b-w2-a53a44/rate-test-model-a53a44-kserve-f9d5fb459-7vsp9"
Sep 16 02:43:30.922492 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:30.922349    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5d02335d-cd4c-4f59-a223-85f952504e4a-home\") pod \"rate-test-model-a53a44-kserve-f9d5fb459-7vsp9\" (UID: \"5d02335d-cd4c-4f59-a223-85f952504e4a\") " pod="ai-tenant-e2e-shared-b-w2-a53a44/rate-test-model-a53a44-kserve-f9d5fb459-7vsp9"
Sep 16 02:43:30.930852 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:30.930826    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:43:31.023227 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:31.023198    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5d02335d-cd4c-4f59-a223-85f952504e4a-tmp-dir\") pod \"rate-test-model-a53a44-kserve-f9d5fb459-7vsp9\" (UID: \"5d02335d-cd4c-4f59-a223-85f952504e4a\") " pod="ai-tenant-e2e-shared-b-w2-a53a44/rate-test-model-a53a44-kserve-f9d5fb459-7vsp9"
Sep 16 02:43:31.023384 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:31.023231    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5d02335d-cd4c-4f59-a223-85f952504e4a-dshm\") pod \"rate-test-model-a53a44-kserve-f9d5fb459-7vsp9\" (UID: \"5d02335d-cd4c-4f59-a223-85f952504e4a\") " pod="ai-tenant-e2e-shared-b-w2-a53a44/rate-test-model-a53a44-kserve-f9d5fb459-7vsp9"
Sep 16 02:43:31.023384 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:31.023248    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5d02335d-cd4c-4f59-a223-85f952504e4a-tls-certs\") pod \"rate-test-model-a53a44-kserve-f9d5fb459-7vsp9\" (UID: \"5d02335d-cd4c-4f59-a223-85f952504e4a\") " pod="ai-tenant-e2e-shared-b-w2-a53a44/rate-test-model-a53a44-kserve-f9d5fb459-7vsp9"
Sep 16 02:43:31.023384 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:31.023298    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5d02335d-cd4c-4f59-a223-85f952504e4a-home\") pod \"rate-test-model-a53a44-kserve-f9d5fb459-7vsp9\" (UID: \"5d02335d-cd4c-4f59-a223-85f952504e4a\") " pod="ai-tenant-e2e-shared-b-w2-a53a44/rate-test-model-a53a44-kserve-f9d5fb459-7vsp9"
Sep 16 02:43:31.023384 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:31.023342    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5d02335d-cd4c-4f59-a223-85f952504e4a-model-cache\") pod \"rate-test-model-a53a44-kserve-f9d5fb459-7vsp9\" (UID: \"5d02335d-cd4c-4f59-a223-85f952504e4a\") " pod="ai-tenant-e2e-shared-b-w2-a53a44/rate-test-model-a53a44-kserve-f9d5fb459-7vsp9"
Sep 16 02:43:31.023384 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:31.023369    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mkxph\" (UniqueName: \"kubernetes.io/projected/5d02335d-cd4c-4f59-a223-85f952504e4a-kube-api-access-mkxph\") pod \"rate-test-model-a53a44-kserve-f9d5fb459-7vsp9\" (UID: \"5d02335d-cd4c-4f59-a223-85f952504e4a\") " pod="ai-tenant-e2e-shared-b-w2-a53a44/rate-test-model-a53a44-kserve-f9d5fb459-7vsp9"
Sep 16 02:43:31.023673 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:31.023650    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5d02335d-cd4c-4f59-a223-85f952504e4a-tmp-dir\") pod \"rate-test-model-a53a44-kserve-f9d5fb459-7vsp9\" (UID: \"5d02335d-cd4c-4f59-a223-85f952504e4a\") " pod="ai-tenant-e2e-shared-b-w2-a53a44/rate-test-model-a53a44-kserve-f9d5fb459-7vsp9"
Sep 16 02:43:31.023727 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:31.023710    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5d02335d-cd4c-4f59-a223-85f952504e4a-home\") pod \"rate-test-model-a53a44-kserve-f9d5fb459-7vsp9\" (UID: \"5d02335d-cd4c-4f59-a223-85f952504e4a\") " pod="ai-tenant-e2e-shared-b-w2-a53a44/rate-test-model-a53a44-kserve-f9d5fb459-7vsp9"
Sep 16 02:43:31.023807 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:31.023783    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5d02335d-cd4c-4f59-a223-85f952504e4a-model-cache\") pod \"rate-test-model-a53a44-kserve-f9d5fb459-7vsp9\" (UID: \"5d02335d-cd4c-4f59-a223-85f952504e4a\") " pod="ai-tenant-e2e-shared-b-w2-a53a44/rate-test-model-a53a44-kserve-f9d5fb459-7vsp9"
Sep 16 02:43:31.025836 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:31.025799    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5d02335d-cd4c-4f59-a223-85f952504e4a-dshm\") pod \"rate-test-model-a53a44-kserve-f9d5fb459-7vsp9\" (UID: \"5d02335d-cd4c-4f59-a223-85f952504e4a\") " pod="ai-tenant-e2e-shared-b-w2-a53a44/rate-test-model-a53a44-kserve-f9d5fb459-7vsp9"
Sep 16 02:43:31.025985 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:31.025969    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5d02335d-cd4c-4f59-a223-85f952504e4a-tls-certs\") pod \"rate-test-model-a53a44-kserve-f9d5fb459-7vsp9\" (UID: \"5d02335d-cd4c-4f59-a223-85f952504e4a\") " pod="ai-tenant-e2e-shared-b-w2-a53a44/rate-test-model-a53a44-kserve-f9d5fb459-7vsp9"
Sep 16 02:43:31.034017 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:31.033994    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mkxph\" (UniqueName: \"kubernetes.io/projected/5d02335d-cd4c-4f59-a223-85f952504e4a-kube-api-access-mkxph\") pod \"rate-test-model-a53a44-kserve-f9d5fb459-7vsp9\" (UID: \"5d02335d-cd4c-4f59-a223-85f952504e4a\") " pod="ai-tenant-e2e-shared-b-w2-a53a44/rate-test-model-a53a44-kserve-f9d5fb459-7vsp9"
Sep 16 02:43:31.200887 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:31.200861    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w2-a53a44/rate-test-model-a53a44-kserve-f9d5fb459-7vsp9"
Sep 16 02:43:31.473391 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:31.473363    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w2-a53a44/rate-test-model-a53a44-kserve-f9d5fb459-7vsp9"]
Sep 16 02:43:31.475235 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:43:31.475213    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5d02335d_cd4c_4f59_a223_85f952504e4a.slice/crio-b335b713075ea0bbe3259c115ae40b58853c9d9bfd184980a25f090a499e3056 WatchSource:0}: Error finding container b335b713075ea0bbe3259c115ae40b58853c9d9bfd184980a25f090a499e3056: Status 404 returned error can't find the container with id b335b713075ea0bbe3259c115ae40b58853c9d9bfd184980a25f090a499e3056
Sep 16 02:43:31.553893 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:31.553852    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w2-a53a44/rate-test-model-a53a44-kserve-f9d5fb459-7vsp9" event={"ID":"5d02335d-cd4c-4f59-a223-85f952504e4a","Type":"ContainerStarted","Data":"b645f6bbf26de67465a17001724e57acd7625a9d4e3b4498a4d9fdc73f6c86a0"}
Sep 16 02:43:31.554035 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:31.553904    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w2-a53a44/rate-test-model-a53a44-kserve-f9d5fb459-7vsp9" event={"ID":"5d02335d-cd4c-4f59-a223-85f952504e4a","Type":"ContainerStarted","Data":"b335b713075ea0bbe3259c115ae40b58853c9d9bfd184980a25f090a499e3056"}
Sep 16 02:43:31.572037 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:31.571983    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w2-a53a44/rate-test-model-a53a44-kserve-f9d5fb459-7vsp9" podStartSLOduration=1.571967098 podStartE2EDuration="1.571967098s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 02:43:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 02:43:31.569924513 +0000 UTC m=+1831.953439758" watchObservedRunningTime="2026-09-16 02:43:31.571967098 +0000 UTC m=+1831.955482343"
Sep 16 02:43:37.250109 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:37.250066    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-77f43ab3/e2e-lifecycle-model-77f43ab3-kserve-76cd4bf5f9-87x9b" podUID="b8ae3d33-d1dd-4d95-85b5-d7deed379ddd" containerName="main" containerID="cri-o://38cc065db9af9c135318ccfabe0c2449d1ec21700efaff4fbf7e68ae1829a8f9" gracePeriod=45
Sep 16 02:43:37.517039 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:37.517015    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-77f43ab3/e2e-lifecycle-model-77f43ab3-kserve-76cd4bf5f9-87x9b"
Sep 16 02:43:37.585256 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:37.585225    2579 generic.go:356] "Generic (PLEG): container finished" podID="b8ae3d33-d1dd-4d95-85b5-d7deed379ddd" containerID="38cc065db9af9c135318ccfabe0c2449d1ec21700efaff4fbf7e68ae1829a8f9" exitCode=0
Sep 16 02:43:37.585409 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:37.585262    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-77f43ab3/e2e-lifecycle-model-77f43ab3-kserve-76cd4bf5f9-87x9b"
Sep 16 02:43:37.585409 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:37.585330    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-77f43ab3/e2e-lifecycle-model-77f43ab3-kserve-76cd4bf5f9-87x9b" event={"ID":"b8ae3d33-d1dd-4d95-85b5-d7deed379ddd","Type":"ContainerDied","Data":"38cc065db9af9c135318ccfabe0c2449d1ec21700efaff4fbf7e68ae1829a8f9"}
Sep 16 02:43:37.585409 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:37.585375    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-77f43ab3/e2e-lifecycle-model-77f43ab3-kserve-76cd4bf5f9-87x9b" event={"ID":"b8ae3d33-d1dd-4d95-85b5-d7deed379ddd","Type":"ContainerDied","Data":"cb452ea799fb923dc596e2d421f161735dccecb85e894ba96e09b0844479bd71"}
Sep 16 02:43:37.585409 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:37.585396    2579 scope.go:117] "RemoveContainer" containerID="38cc065db9af9c135318ccfabe0c2449d1ec21700efaff4fbf7e68ae1829a8f9"
Sep 16 02:43:37.587793 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:37.587774    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b8ae3d33-d1dd-4d95-85b5-d7deed379ddd-dshm\") pod \"b8ae3d33-d1dd-4d95-85b5-d7deed379ddd\" (UID: \"b8ae3d33-d1dd-4d95-85b5-d7deed379ddd\") "
Sep 16 02:43:37.587929 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:37.587851    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v7hj6\" (UniqueName: \"kubernetes.io/projected/b8ae3d33-d1dd-4d95-85b5-d7deed379ddd-kube-api-access-v7hj6\") pod \"b8ae3d33-d1dd-4d95-85b5-d7deed379ddd\" (UID: \"b8ae3d33-d1dd-4d95-85b5-d7deed379ddd\") "
Sep 16 02:43:37.587929 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:37.587876    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b8ae3d33-d1dd-4d95-85b5-d7deed379ddd-model-cache\") pod \"b8ae3d33-d1dd-4d95-85b5-d7deed379ddd\" (UID: \"b8ae3d33-d1dd-4d95-85b5-d7deed379ddd\") "
Sep 16 02:43:37.587929 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:37.587909    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b8ae3d33-d1dd-4d95-85b5-d7deed379ddd-tmp-dir\") pod \"b8ae3d33-d1dd-4d95-85b5-d7deed379ddd\" (UID: \"b8ae3d33-d1dd-4d95-85b5-d7deed379ddd\") "
Sep 16 02:43:37.588114 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:37.587992    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b8ae3d33-d1dd-4d95-85b5-d7deed379ddd-home\") pod \"b8ae3d33-d1dd-4d95-85b5-d7deed379ddd\" (UID: \"b8ae3d33-d1dd-4d95-85b5-d7deed379ddd\") "
Sep 16 02:43:37.588114 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:37.588017    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b8ae3d33-d1dd-4d95-85b5-d7deed379ddd-tls-certs\") pod \"b8ae3d33-d1dd-4d95-85b5-d7deed379ddd\" (UID: \"b8ae3d33-d1dd-4d95-85b5-d7deed379ddd\") "
Sep 16 02:43:37.588501 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:37.588218    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b8ae3d33-d1dd-4d95-85b5-d7deed379ddd-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b8ae3d33-d1dd-4d95-85b5-d7deed379ddd" (UID: "b8ae3d33-d1dd-4d95-85b5-d7deed379ddd"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 02:43:37.588501 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:37.588231    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b8ae3d33-d1dd-4d95-85b5-d7deed379ddd-model-cache" (OuterVolumeSpecName: "model-cache") pod "b8ae3d33-d1dd-4d95-85b5-d7deed379ddd" (UID: "b8ae3d33-d1dd-4d95-85b5-d7deed379ddd"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 02:43:37.588501 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:37.588354    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b8ae3d33-d1dd-4d95-85b5-d7deed379ddd-model-cache\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:43:37.588501 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:37.588414    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b8ae3d33-d1dd-4d95-85b5-d7deed379ddd-tmp-dir\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:43:37.588501 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:37.588425    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b8ae3d33-d1dd-4d95-85b5-d7deed379ddd-home" (OuterVolumeSpecName: "home") pod "b8ae3d33-d1dd-4d95-85b5-d7deed379ddd" (UID: "b8ae3d33-d1dd-4d95-85b5-d7deed379ddd"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 02:43:37.590445 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:37.590411    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b8ae3d33-d1dd-4d95-85b5-d7deed379ddd-dshm" (OuterVolumeSpecName: "dshm") pod "b8ae3d33-d1dd-4d95-85b5-d7deed379ddd" (UID: "b8ae3d33-d1dd-4d95-85b5-d7deed379ddd"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 02:43:37.590590 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:37.590545    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b8ae3d33-d1dd-4d95-85b5-d7deed379ddd-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b8ae3d33-d1dd-4d95-85b5-d7deed379ddd" (UID: "b8ae3d33-d1dd-4d95-85b5-d7deed379ddd"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 02:43:37.590686 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:37.590665    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b8ae3d33-d1dd-4d95-85b5-d7deed379ddd-kube-api-access-v7hj6" (OuterVolumeSpecName: "kube-api-access-v7hj6") pod "b8ae3d33-d1dd-4d95-85b5-d7deed379ddd" (UID: "b8ae3d33-d1dd-4d95-85b5-d7deed379ddd"). InnerVolumeSpecName "kube-api-access-v7hj6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 02:43:37.608507 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:37.608463    2579 scope.go:117] "RemoveContainer" containerID="38cc065db9af9c135318ccfabe0c2449d1ec21700efaff4fbf7e68ae1829a8f9"
Sep 16 02:43:37.608802 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:43:37.608782    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"38cc065db9af9c135318ccfabe0c2449d1ec21700efaff4fbf7e68ae1829a8f9\": container with ID starting with 38cc065db9af9c135318ccfabe0c2449d1ec21700efaff4fbf7e68ae1829a8f9 not found: ID does not exist" containerID="38cc065db9af9c135318ccfabe0c2449d1ec21700efaff4fbf7e68ae1829a8f9"
Sep 16 02:43:37.608860 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:37.608811    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"38cc065db9af9c135318ccfabe0c2449d1ec21700efaff4fbf7e68ae1829a8f9"} err="failed to get container status \"38cc065db9af9c135318ccfabe0c2449d1ec21700efaff4fbf7e68ae1829a8f9\": rpc error: code = NotFound desc = could not find container \"38cc065db9af9c135318ccfabe0c2449d1ec21700efaff4fbf7e68ae1829a8f9\": container with ID starting with 38cc065db9af9c135318ccfabe0c2449d1ec21700efaff4fbf7e68ae1829a8f9 not found: ID does not exist"
Sep 16 02:43:37.689082 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:37.689058    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b8ae3d33-d1dd-4d95-85b5-d7deed379ddd-home\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:43:37.689082 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:37.689079    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b8ae3d33-d1dd-4d95-85b5-d7deed379ddd-tls-certs\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:43:37.689082 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:37.689087    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b8ae3d33-d1dd-4d95-85b5-d7deed379ddd-dshm\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:43:37.689277 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:37.689095    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v7hj6\" (UniqueName: \"kubernetes.io/projected/b8ae3d33-d1dd-4d95-85b5-d7deed379ddd-kube-api-access-v7hj6\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:43:37.905929 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:37.905897    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-77f43ab3/e2e-lifecycle-model-77f43ab3-kserve-76cd4bf5f9-87x9b"]
Sep 16 02:43:37.909488 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:37.909445    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-77f43ab3/e2e-lifecycle-model-77f43ab3-kserve-76cd4bf5f9-87x9b"]
Sep 16 02:43:38.102559 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:38.102530    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b8ae3d33-d1dd-4d95-85b5-d7deed379ddd" path="/var/lib/kubelet/pods/b8ae3d33-d1dd-4d95-85b5-d7deed379ddd/volumes"
Sep 16 02:43:39.097105 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:39.097073    2579 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-pre-processing-6cd864bc95-mh4jh" secret="" err="secret \"payload-processing-dockercfg-zzx6j\" not found"
Sep 16 02:43:41.201286 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:41.201257    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w2-a53a44/rate-test-model-a53a44-kserve-f9d5fb459-7vsp9"
Sep 16 02:43:41.201684 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:41.201307    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w2-a53a44/rate-test-model-a53a44-kserve-f9d5fb459-7vsp9"
Sep 16 02:43:41.213951 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:41.213927    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w2-a53a44/rate-test-model-a53a44-kserve-f9d5fb459-7vsp9"
Sep 16 02:43:41.616376 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:41.616297    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w2-a53a44/rate-test-model-a53a44-kserve-f9d5fb459-7vsp9"
Sep 16 02:43:58.701780 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:43:58.701746    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:44:00.104052 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:00.104021    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:44:04.838299 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:04.838259    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:44:10.775658 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:10.775621    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:44:10.880077 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:10.880043    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:44:11.035977 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:11.035875    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:44:11.271240 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:11.271211    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w2-24aabe/rate-test-model-24aabe-kserve-9dc995687-7w6gs"]
Sep 16 02:44:11.289154 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:11.289053    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:44:12.248889 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:12.248848    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:44:12.349048 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:12.349001    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:44:12.393319 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:12.393286    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w2-a53a44/rate-test-model-a53a44-kserve-f9d5fb459-7vsp9"]
Sep 16 02:44:12.412134 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:12.412103    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:44:12.841910 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:12.841860    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:44:13.037575 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:13.037537    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:44:13.112545 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:13.112433    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:44:13.179158 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:13.179108    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:44:13.727905 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:13.727863    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:44:14.477742 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:14.477711    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:44:16.373973 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:16.373939    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:44:16.521202 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:16.521164    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:44:16.626929 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:16.626835    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:44:17.217547 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:17.217513    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:44:17.970315 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:17.970278    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:44:18.629980 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:18.629951    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:44:18.680298 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:18.680262    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:44:18.999969 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:18.999935    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:44:19.116771 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:19.116731    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:44:19.172296 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:19.172260    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-fd614e-768875496b-mxqjp"]
Sep 16 02:44:19.172699 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:19.172683    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b8ae3d33-d1dd-4d95-85b5-d7deed379ddd" containerName="main"
Sep 16 02:44:19.172757 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:19.172702    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="b8ae3d33-d1dd-4d95-85b5-d7deed379ddd" containerName="main"
Sep 16 02:44:19.172799 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:19.172788    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="b8ae3d33-d1dd-4d95-85b5-d7deed379ddd" containerName="main"
Sep 16 02:44:19.177829 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:19.177794    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-fd614e-768875496b-mxqjp"
Sep 16 02:44:19.181293 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:19.181269    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-fd614e-dockercfg-968lr\""
Sep 16 02:44:19.181792 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:19.181771    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-fd614e\""
Sep 16 02:44:19.181955 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:19.181930    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:44:19.186104 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:19.186084    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-fd614e-768875496b-mxqjp"]
Sep 16 02:44:19.194549 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:19.194526    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-fd614e-66fccddb69-57t8f"]
Sep 16 02:44:19.197827 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:19.197801    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-fd614e-66fccddb69-57t8f"
Sep 16 02:44:19.209201 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:19.209177    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-fd614e-66fccddb69-57t8f"]
Sep 16 02:44:19.261694 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:19.261627    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9fefbf80-ce02-4fcf-ac3d-3af84b5b32ef-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-fd614e-768875496b-mxqjp\" (UID: \"9fefbf80-ce02-4fcf-ac3d-3af84b5b32ef\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-fd614e-768875496b-mxqjp"
Sep 16 02:44:19.261694 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:19.261666    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c1ed1477-844b-4b24-8a3d-910f1eb44b2b-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-fd614e-66fccddb69-57t8f\" (UID: \"c1ed1477-844b-4b24-8a3d-910f1eb44b2b\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-fd614e-66fccddb69-57t8f"
Sep 16 02:44:19.261873 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:19.261794    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fqns4\" (UniqueName: \"kubernetes.io/projected/c1ed1477-844b-4b24-8a3d-910f1eb44b2b-kube-api-access-fqns4\") pod \"payload-processing-e2e-ipp-a-fd614e-66fccddb69-57t8f\" (UID: \"c1ed1477-844b-4b24-8a3d-910f1eb44b2b\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-fd614e-66fccddb69-57t8f"
Sep 16 02:44:19.261873 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:19.261852    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-csh8t\" (UniqueName: \"kubernetes.io/projected/9fefbf80-ce02-4fcf-ac3d-3af84b5b32ef-kube-api-access-csh8t\") pod \"payload-pre-processing-e2e-ipp-a-fd614e-768875496b-mxqjp\" (UID: \"9fefbf80-ce02-4fcf-ac3d-3af84b5b32ef\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-fd614e-768875496b-mxqjp"
Sep 16 02:44:19.363427 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:19.363375    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9fefbf80-ce02-4fcf-ac3d-3af84b5b32ef-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-fd614e-768875496b-mxqjp\" (UID: \"9fefbf80-ce02-4fcf-ac3d-3af84b5b32ef\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-fd614e-768875496b-mxqjp"
Sep 16 02:44:19.363648 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:19.363436    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c1ed1477-844b-4b24-8a3d-910f1eb44b2b-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-fd614e-66fccddb69-57t8f\" (UID: \"c1ed1477-844b-4b24-8a3d-910f1eb44b2b\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-fd614e-66fccddb69-57t8f"
Sep 16 02:44:19.363648 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:19.363571    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fqns4\" (UniqueName: \"kubernetes.io/projected/c1ed1477-844b-4b24-8a3d-910f1eb44b2b-kube-api-access-fqns4\") pod \"payload-processing-e2e-ipp-a-fd614e-66fccddb69-57t8f\" (UID: \"c1ed1477-844b-4b24-8a3d-910f1eb44b2b\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-fd614e-66fccddb69-57t8f"
Sep 16 02:44:19.363648 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:19.363609    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-csh8t\" (UniqueName: \"kubernetes.io/projected/9fefbf80-ce02-4fcf-ac3d-3af84b5b32ef-kube-api-access-csh8t\") pod \"payload-pre-processing-e2e-ipp-a-fd614e-768875496b-mxqjp\" (UID: \"9fefbf80-ce02-4fcf-ac3d-3af84b5b32ef\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-fd614e-768875496b-mxqjp"
Sep 16 02:44:19.364194 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:19.364163    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9fefbf80-ce02-4fcf-ac3d-3af84b5b32ef-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-fd614e-768875496b-mxqjp\" (UID: \"9fefbf80-ce02-4fcf-ac3d-3af84b5b32ef\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-fd614e-768875496b-mxqjp"
Sep 16 02:44:19.364334 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:19.364225    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c1ed1477-844b-4b24-8a3d-910f1eb44b2b-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-fd614e-66fccddb69-57t8f\" (UID: \"c1ed1477-844b-4b24-8a3d-910f1eb44b2b\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-fd614e-66fccddb69-57t8f"
Sep 16 02:44:19.371481 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:19.371442    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fqns4\" (UniqueName: \"kubernetes.io/projected/c1ed1477-844b-4b24-8a3d-910f1eb44b2b-kube-api-access-fqns4\") pod \"payload-processing-e2e-ipp-a-fd614e-66fccddb69-57t8f\" (UID: \"c1ed1477-844b-4b24-8a3d-910f1eb44b2b\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-fd614e-66fccddb69-57t8f"
Sep 16 02:44:19.371588 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:19.371445    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-csh8t\" (UniqueName: \"kubernetes.io/projected/9fefbf80-ce02-4fcf-ac3d-3af84b5b32ef-kube-api-access-csh8t\") pod \"payload-pre-processing-e2e-ipp-a-fd614e-768875496b-mxqjp\" (UID: \"9fefbf80-ce02-4fcf-ac3d-3af84b5b32ef\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-fd614e-768875496b-mxqjp"
Sep 16 02:44:19.490545 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:19.490513    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-fd614e-768875496b-mxqjp"
Sep 16 02:44:19.510065 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:19.510027    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-fd614e-66fccddb69-57t8f"
Sep 16 02:44:19.765609 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:19.765583    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-fd614e-768875496b-mxqjp"]
Sep 16 02:44:19.767299 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:44:19.767258    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9fefbf80_ce02_4fcf_ac3d_3af84b5b32ef.slice/crio-99719d214f6f937ee74f1960b634fec72a7fd0a88dfcd4694c3851c5832dcc98 WatchSource:0}: Error finding container 99719d214f6f937ee74f1960b634fec72a7fd0a88dfcd4694c3851c5832dcc98: Status 404 returned error can't find the container with id 99719d214f6f937ee74f1960b634fec72a7fd0a88dfcd4694c3851c5832dcc98
Sep 16 02:44:19.792089 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:19.792066    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-fd614e-66fccddb69-57t8f"]
Sep 16 02:44:19.795782 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:44:19.795712    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1ed1477_844b_4b24_8a3d_910f1eb44b2b.slice/crio-4754f3980e187cef53562100699cc88f71530313211962bad5932251dbf9a36f WatchSource:0}: Error finding container 4754f3980e187cef53562100699cc88f71530313211962bad5932251dbf9a36f: Status 404 returned error can't find the container with id 4754f3980e187cef53562100699cc88f71530313211962bad5932251dbf9a36f
Sep 16 02:44:19.803143 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:19.803111    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-fd614e-66fccddb69-57t8f" event={"ID":"c1ed1477-844b-4b24-8a3d-910f1eb44b2b","Type":"ContainerStarted","Data":"4754f3980e187cef53562100699cc88f71530313211962bad5932251dbf9a36f"}
Sep 16 02:44:19.804698 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:19.804674    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-fd614e-768875496b-mxqjp" event={"ID":"9fefbf80-ce02-4fcf-ac3d-3af84b5b32ef","Type":"ContainerStarted","Data":"99719d214f6f937ee74f1960b634fec72a7fd0a88dfcd4694c3851c5832dcc98"}
Sep 16 02:44:20.376483 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:20.376427    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:44:20.476816 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:20.476778    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:44:20.810798 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:20.810768    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-fd614e-768875496b-mxqjp" event={"ID":"9fefbf80-ce02-4fcf-ac3d-3af84b5b32ef","Type":"ContainerStarted","Data":"6d186c96140dfeb2e93764a14fec104a39dced8874e78a8c4fb6dd49c64702a1"}
Sep 16 02:44:20.811006 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:20.810866    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-fd614e-768875496b-mxqjp"
Sep 16 02:44:20.812519 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:20.812490    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-fd614e-66fccddb69-57t8f" event={"ID":"c1ed1477-844b-4b24-8a3d-910f1eb44b2b","Type":"ContainerStarted","Data":"420762a049b8fafe6d3ed41bc5009875a3c5a9613b0f8dbca84926e252299725"}
Sep 16 02:44:20.812616 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:20.812541    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-fd614e-66fccddb69-57t8f"
Sep 16 02:44:20.827567 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:20.827522    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-fd614e-768875496b-mxqjp" podStartSLOduration=1.827506558 podStartE2EDuration="1.827506558s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 02:44:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 02:44:20.825536368 +0000 UTC m=+1881.209051615" watchObservedRunningTime="2026-09-16 02:44:20.827506558 +0000 UTC m=+1881.211021805"
Sep 16 02:44:20.841938 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:20.841897    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-fd614e-66fccddb69-57t8f" podStartSLOduration=1.8418807400000001 podStartE2EDuration="1.84188074s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 02:44:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 02:44:20.841445848 +0000 UTC m=+1881.224961094" watchObservedRunningTime="2026-09-16 02:44:20.84188074 +0000 UTC m=+1881.225396006"
Sep 16 02:44:21.060452 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:21.060417    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:44:26.291444 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:26.291357    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w2-24aabe/rate-test-model-24aabe-kserve-9dc995687-7w6gs" podUID="3336085d-16ba-4eae-9d4e-a4a3e949a615" containerName="main" containerID="cri-o://2c19994ba4881b26d5a77904d5cbe8286162e83f23f24e7757547ddbf98b620d" gracePeriod=45
Sep 16 02:44:26.554076 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:26.554017    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w2-24aabe/rate-test-model-24aabe-kserve-9dc995687-7w6gs"
Sep 16 02:44:26.629791 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:26.629752    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-59mhg\" (UniqueName: \"kubernetes.io/projected/3336085d-16ba-4eae-9d4e-a4a3e949a615-kube-api-access-59mhg\") pod \"3336085d-16ba-4eae-9d4e-a4a3e949a615\" (UID: \"3336085d-16ba-4eae-9d4e-a4a3e949a615\") "
Sep 16 02:44:26.629951 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:26.629808    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3336085d-16ba-4eae-9d4e-a4a3e949a615-tmp-dir\") pod \"3336085d-16ba-4eae-9d4e-a4a3e949a615\" (UID: \"3336085d-16ba-4eae-9d4e-a4a3e949a615\") "
Sep 16 02:44:26.629951 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:26.629866    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3336085d-16ba-4eae-9d4e-a4a3e949a615-dshm\") pod \"3336085d-16ba-4eae-9d4e-a4a3e949a615\" (UID: \"3336085d-16ba-4eae-9d4e-a4a3e949a615\") "
Sep 16 02:44:26.629951 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:26.629924    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3336085d-16ba-4eae-9d4e-a4a3e949a615-tls-certs\") pod \"3336085d-16ba-4eae-9d4e-a4a3e949a615\" (UID: \"3336085d-16ba-4eae-9d4e-a4a3e949a615\") "
Sep 16 02:44:26.630098 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:26.629976    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3336085d-16ba-4eae-9d4e-a4a3e949a615-home\") pod \"3336085d-16ba-4eae-9d4e-a4a3e949a615\" (UID: \"3336085d-16ba-4eae-9d4e-a4a3e949a615\") "
Sep 16 02:44:26.630098 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:26.630029    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3336085d-16ba-4eae-9d4e-a4a3e949a615-model-cache\") pod \"3336085d-16ba-4eae-9d4e-a4a3e949a615\" (UID: \"3336085d-16ba-4eae-9d4e-a4a3e949a615\") "
Sep 16 02:44:26.630206 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:26.630112    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3336085d-16ba-4eae-9d4e-a4a3e949a615-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "3336085d-16ba-4eae-9d4e-a4a3e949a615" (UID: "3336085d-16ba-4eae-9d4e-a4a3e949a615"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 02:44:26.630401 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:26.630378    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3336085d-16ba-4eae-9d4e-a4a3e949a615-home" (OuterVolumeSpecName: "home") pod "3336085d-16ba-4eae-9d4e-a4a3e949a615" (UID: "3336085d-16ba-4eae-9d4e-a4a3e949a615"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 02:44:26.630401 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:26.630391    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3336085d-16ba-4eae-9d4e-a4a3e949a615-model-cache" (OuterVolumeSpecName: "model-cache") pod "3336085d-16ba-4eae-9d4e-a4a3e949a615" (UID: "3336085d-16ba-4eae-9d4e-a4a3e949a615"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 02:44:26.630654 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:26.630385    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3336085d-16ba-4eae-9d4e-a4a3e949a615-tmp-dir\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:44:26.632247 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:26.632224    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3336085d-16ba-4eae-9d4e-a4a3e949a615-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "3336085d-16ba-4eae-9d4e-a4a3e949a615" (UID: "3336085d-16ba-4eae-9d4e-a4a3e949a615"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 02:44:26.632494 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:26.632457    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3336085d-16ba-4eae-9d4e-a4a3e949a615-dshm" (OuterVolumeSpecName: "dshm") pod "3336085d-16ba-4eae-9d4e-a4a3e949a615" (UID: "3336085d-16ba-4eae-9d4e-a4a3e949a615"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 02:44:26.632625 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:26.632598    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3336085d-16ba-4eae-9d4e-a4a3e949a615-kube-api-access-59mhg" (OuterVolumeSpecName: "kube-api-access-59mhg") pod "3336085d-16ba-4eae-9d4e-a4a3e949a615" (UID: "3336085d-16ba-4eae-9d4e-a4a3e949a615"). InnerVolumeSpecName "kube-api-access-59mhg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 02:44:26.701450 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:26.701412    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:44:26.731607 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:26.731582    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3336085d-16ba-4eae-9d4e-a4a3e949a615-home\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:44:26.731607 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:26.731605    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3336085d-16ba-4eae-9d4e-a4a3e949a615-model-cache\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:44:26.731730 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:26.731616    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-59mhg\" (UniqueName: \"kubernetes.io/projected/3336085d-16ba-4eae-9d4e-a4a3e949a615-kube-api-access-59mhg\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:44:26.731730 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:26.731625    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3336085d-16ba-4eae-9d4e-a4a3e949a615-dshm\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:44:26.731730 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:26.731634    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3336085d-16ba-4eae-9d4e-a4a3e949a615-tls-certs\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:44:26.807681 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:26.807612    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:44:26.847618 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:26.847590    2579 generic.go:356] "Generic (PLEG): container finished" podID="3336085d-16ba-4eae-9d4e-a4a3e949a615" containerID="2c19994ba4881b26d5a77904d5cbe8286162e83f23f24e7757547ddbf98b620d" exitCode=0
Sep 16 02:44:26.847766 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:26.847631    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w2-24aabe/rate-test-model-24aabe-kserve-9dc995687-7w6gs"
Sep 16 02:44:26.847766 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:26.847680    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w2-24aabe/rate-test-model-24aabe-kserve-9dc995687-7w6gs" event={"ID":"3336085d-16ba-4eae-9d4e-a4a3e949a615","Type":"ContainerDied","Data":"2c19994ba4881b26d5a77904d5cbe8286162e83f23f24e7757547ddbf98b620d"}
Sep 16 02:44:26.847766 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:26.847721    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w2-24aabe/rate-test-model-24aabe-kserve-9dc995687-7w6gs" event={"ID":"3336085d-16ba-4eae-9d4e-a4a3e949a615","Type":"ContainerDied","Data":"84a7f61c35ed9c2f44cf6645cbb0039a83f833c9f86ea5f3afd81327689f7149"}
Sep 16 02:44:26.847766 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:26.847739    2579 scope.go:117] "RemoveContainer" containerID="2c19994ba4881b26d5a77904d5cbe8286162e83f23f24e7757547ddbf98b620d"
Sep 16 02:44:26.857074 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:26.857059    2579 scope.go:117] "RemoveContainer" containerID="2c19994ba4881b26d5a77904d5cbe8286162e83f23f24e7757547ddbf98b620d"
Sep 16 02:44:26.857336 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:44:26.857318    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2c19994ba4881b26d5a77904d5cbe8286162e83f23f24e7757547ddbf98b620d\": container with ID starting with 2c19994ba4881b26d5a77904d5cbe8286162e83f23f24e7757547ddbf98b620d not found: ID does not exist" containerID="2c19994ba4881b26d5a77904d5cbe8286162e83f23f24e7757547ddbf98b620d"
Sep 16 02:44:26.857383 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:26.857344    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c19994ba4881b26d5a77904d5cbe8286162e83f23f24e7757547ddbf98b620d"} err="failed to get container status \"2c19994ba4881b26d5a77904d5cbe8286162e83f23f24e7757547ddbf98b620d\": rpc error: code = NotFound desc = could not find container \"2c19994ba4881b26d5a77904d5cbe8286162e83f23f24e7757547ddbf98b620d\": container with ID starting with 2c19994ba4881b26d5a77904d5cbe8286162e83f23f24e7757547ddbf98b620d not found: ID does not exist"
Sep 16 02:44:26.868122 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:26.868099    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w2-24aabe/rate-test-model-24aabe-kserve-9dc995687-7w6gs"]
Sep 16 02:44:26.871909 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:26.871889    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w2-24aabe/rate-test-model-24aabe-kserve-9dc995687-7w6gs"]
Sep 16 02:44:26.875605 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:26.875569    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:44:27.413198 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:27.413164    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w2-a53a44/rate-test-model-a53a44-kserve-f9d5fb459-7vsp9" podUID="5d02335d-cd4c-4f59-a223-85f952504e4a" containerName="main" containerID="cri-o://b645f6bbf26de67465a17001724e57acd7625a9d4e3b4498a4d9fdc73f6c86a0" gracePeriod=45
Sep 16 02:44:27.671163 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:27.671102    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w2-a53a44/rate-test-model-a53a44-kserve-f9d5fb459-7vsp9"
Sep 16 02:44:27.742406 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:27.742375    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5d02335d-cd4c-4f59-a223-85f952504e4a-tmp-dir\") pod \"5d02335d-cd4c-4f59-a223-85f952504e4a\" (UID: \"5d02335d-cd4c-4f59-a223-85f952504e4a\") "
Sep 16 02:44:27.742581 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:27.742453    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5d02335d-cd4c-4f59-a223-85f952504e4a-tls-certs\") pod \"5d02335d-cd4c-4f59-a223-85f952504e4a\" (UID: \"5d02335d-cd4c-4f59-a223-85f952504e4a\") "
Sep 16 02:44:27.742581 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:27.742513    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5d02335d-cd4c-4f59-a223-85f952504e4a-model-cache\") pod \"5d02335d-cd4c-4f59-a223-85f952504e4a\" (UID: \"5d02335d-cd4c-4f59-a223-85f952504e4a\") "
Sep 16 02:44:27.742581 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:27.742542    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5d02335d-cd4c-4f59-a223-85f952504e4a-dshm\") pod \"5d02335d-cd4c-4f59-a223-85f952504e4a\" (UID: \"5d02335d-cd4c-4f59-a223-85f952504e4a\") "
Sep 16 02:44:27.742581 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:27.742561    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mkxph\" (UniqueName: \"kubernetes.io/projected/5d02335d-cd4c-4f59-a223-85f952504e4a-kube-api-access-mkxph\") pod \"5d02335d-cd4c-4f59-a223-85f952504e4a\" (UID: \"5d02335d-cd4c-4f59-a223-85f952504e4a\") "
Sep 16 02:44:27.742767 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:27.742596    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5d02335d-cd4c-4f59-a223-85f952504e4a-home\") pod \"5d02335d-cd4c-4f59-a223-85f952504e4a\" (UID: \"5d02335d-cd4c-4f59-a223-85f952504e4a\") "
Sep 16 02:44:27.742767 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:27.742662    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d02335d-cd4c-4f59-a223-85f952504e4a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "5d02335d-cd4c-4f59-a223-85f952504e4a" (UID: "5d02335d-cd4c-4f59-a223-85f952504e4a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 02:44:27.742874 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:27.742832    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d02335d-cd4c-4f59-a223-85f952504e4a-model-cache" (OuterVolumeSpecName: "model-cache") pod "5d02335d-cd4c-4f59-a223-85f952504e4a" (UID: "5d02335d-cd4c-4f59-a223-85f952504e4a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 02:44:27.742934 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:27.742876    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d02335d-cd4c-4f59-a223-85f952504e4a-home" (OuterVolumeSpecName: "home") pod "5d02335d-cd4c-4f59-a223-85f952504e4a" (UID: "5d02335d-cd4c-4f59-a223-85f952504e4a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 02:44:27.742934 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:27.742920    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5d02335d-cd4c-4f59-a223-85f952504e4a-tmp-dir\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:44:27.744797 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:27.744772    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d02335d-cd4c-4f59-a223-85f952504e4a-dshm" (OuterVolumeSpecName: "dshm") pod "5d02335d-cd4c-4f59-a223-85f952504e4a" (UID: "5d02335d-cd4c-4f59-a223-85f952504e4a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 02:44:27.744911 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:27.744797    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5d02335d-cd4c-4f59-a223-85f952504e4a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "5d02335d-cd4c-4f59-a223-85f952504e4a" (UID: "5d02335d-cd4c-4f59-a223-85f952504e4a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 02:44:27.744911 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:27.744853    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5d02335d-cd4c-4f59-a223-85f952504e4a-kube-api-access-mkxph" (OuterVolumeSpecName: "kube-api-access-mkxph") pod "5d02335d-cd4c-4f59-a223-85f952504e4a" (UID: "5d02335d-cd4c-4f59-a223-85f952504e4a"). InnerVolumeSpecName "kube-api-access-mkxph". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 02:44:27.844214 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:27.844193    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5d02335d-cd4c-4f59-a223-85f952504e4a-tls-certs\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:44:27.844214 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:27.844213    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5d02335d-cd4c-4f59-a223-85f952504e4a-model-cache\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:44:27.844336 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:27.844221    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5d02335d-cd4c-4f59-a223-85f952504e4a-dshm\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:44:27.844336 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:27.844229    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mkxph\" (UniqueName: \"kubernetes.io/projected/5d02335d-cd4c-4f59-a223-85f952504e4a-kube-api-access-mkxph\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:44:27.844336 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:27.844237    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5d02335d-cd4c-4f59-a223-85f952504e4a-home\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:44:27.853788 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:27.853766    2579 generic.go:356] "Generic (PLEG): container finished" podID="5d02335d-cd4c-4f59-a223-85f952504e4a" containerID="b645f6bbf26de67465a17001724e57acd7625a9d4e3b4498a4d9fdc73f6c86a0" exitCode=0
Sep 16 02:44:27.853909 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:27.853810    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w2-a53a44/rate-test-model-a53a44-kserve-f9d5fb459-7vsp9"
Sep 16 02:44:27.853909 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:27.853861    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w2-a53a44/rate-test-model-a53a44-kserve-f9d5fb459-7vsp9" event={"ID":"5d02335d-cd4c-4f59-a223-85f952504e4a","Type":"ContainerDied","Data":"b645f6bbf26de67465a17001724e57acd7625a9d4e3b4498a4d9fdc73f6c86a0"}
Sep 16 02:44:27.853909 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:27.853901    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w2-a53a44/rate-test-model-a53a44-kserve-f9d5fb459-7vsp9" event={"ID":"5d02335d-cd4c-4f59-a223-85f952504e4a","Type":"ContainerDied","Data":"b335b713075ea0bbe3259c115ae40b58853c9d9bfd184980a25f090a499e3056"}
Sep 16 02:44:27.854020 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:27.853922    2579 scope.go:117] "RemoveContainer" containerID="b645f6bbf26de67465a17001724e57acd7625a9d4e3b4498a4d9fdc73f6c86a0"
Sep 16 02:44:27.863025 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:27.863009    2579 scope.go:117] "RemoveContainer" containerID="b645f6bbf26de67465a17001724e57acd7625a9d4e3b4498a4d9fdc73f6c86a0"
Sep 16 02:44:27.863306 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:44:27.863288    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b645f6bbf26de67465a17001724e57acd7625a9d4e3b4498a4d9fdc73f6c86a0\": container with ID starting with b645f6bbf26de67465a17001724e57acd7625a9d4e3b4498a4d9fdc73f6c86a0 not found: ID does not exist" containerID="b645f6bbf26de67465a17001724e57acd7625a9d4e3b4498a4d9fdc73f6c86a0"
Sep 16 02:44:27.863370 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:27.863316    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b645f6bbf26de67465a17001724e57acd7625a9d4e3b4498a4d9fdc73f6c86a0"} err="failed to get container status \"b645f6bbf26de67465a17001724e57acd7625a9d4e3b4498a4d9fdc73f6c86a0\": rpc error: code = NotFound desc = could not find container \"b645f6bbf26de67465a17001724e57acd7625a9d4e3b4498a4d9fdc73f6c86a0\": container with ID starting with b645f6bbf26de67465a17001724e57acd7625a9d4e3b4498a4d9fdc73f6c86a0 not found: ID does not exist"
Sep 16 02:44:27.874376 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:27.874354    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w2-a53a44/rate-test-model-a53a44-kserve-f9d5fb459-7vsp9"]
Sep 16 02:44:27.876150 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:27.876132    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w2-a53a44/rate-test-model-a53a44-kserve-f9d5fb459-7vsp9"]
Sep 16 02:44:28.103008 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:28.102974    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3336085d-16ba-4eae-9d4e-a4a3e949a615" path="/var/lib/kubelet/pods/3336085d-16ba-4eae-9d4e-a4a3e949a615/volumes"
Sep 16 02:44:28.103375 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:28.103361    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5d02335d-cd4c-4f59-a223-85f952504e4a" path="/var/lib/kubelet/pods/5d02335d-cd4c-4f59-a223-85f952504e4a/volumes"
Sep 16 02:44:31.819191 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:31.819157    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-fd614e-66fccddb69-57t8f"
Sep 16 02:44:31.819741 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:31.819236    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-fd614e-768875496b-mxqjp"
Sep 16 02:44:37.395551 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:37.395511    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:44:37.621097 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:37.621064    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:44:41.487204 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:41.487168    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:44:41.587739 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:41.587702    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:44:42.120645 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:42.120608    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:44:42.220807 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:42.220774    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:44:42.508073 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:42.508036    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:44:42.962566 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:42.962516    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:44:43.065525 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:43.065490    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:44:43.746619 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:43.746587    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:44:44.420641 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:44.420596    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:44:45.123907 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:45.123872    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:44:45.757430 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:45.757393    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:44:45.852919 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:45.852880    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:44:45.953132 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:45.953098    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:44:46.494301 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:46.494269    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:44:46.846054 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:46.845941    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:44:47.518732 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:47.518697    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:44:48.228300 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:48.228268    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:44:48.842197 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:48.842166    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:44:48.895237 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:48.895203    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:44:48.994886 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:48.994853    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:44:49.274714 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:49.274676    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:44:49.391764 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:49.391734    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:44:49.407753 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:49.407725    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-d3d202-558b6d7798-4mpfq"]
Sep 16 02:44:49.408146 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:49.408134    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3336085d-16ba-4eae-9d4e-a4a3e949a615" containerName="main"
Sep 16 02:44:49.408201 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:49.408147    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="3336085d-16ba-4eae-9d4e-a4a3e949a615" containerName="main"
Sep 16 02:44:49.408201 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:49.408166    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d02335d-cd4c-4f59-a223-85f952504e4a" containerName="main"
Sep 16 02:44:49.408201 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:49.408171    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d02335d-cd4c-4f59-a223-85f952504e4a" containerName="main"
Sep 16 02:44:49.408326 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:49.408232    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="3336085d-16ba-4eae-9d4e-a4a3e949a615" containerName="main"
Sep 16 02:44:49.408326 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:49.408240    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="5d02335d-cd4c-4f59-a223-85f952504e4a" containerName="main"
Sep 16 02:44:49.411480 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:49.411454    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d3d202-558b6d7798-4mpfq"
Sep 16 02:44:49.414496 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:49.414454    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-d3d202-dockercfg-sdkh6\""
Sep 16 02:44:49.414595 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:49.414463    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-d3d202\""
Sep 16 02:44:49.433673 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:49.433651    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zn7hm\" (UniqueName: \"kubernetes.io/projected/c05721b2-9ebf-4250-9d7c-c0e464194f9c-kube-api-access-zn7hm\") pod \"payload-pre-processing-e2e-ipp-b-d3d202-558b6d7798-4mpfq\" (UID: \"c05721b2-9ebf-4250-9d7c-c0e464194f9c\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d3d202-558b6d7798-4mpfq"
Sep 16 02:44:49.433832 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:49.433806    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c05721b2-9ebf-4250-9d7c-c0e464194f9c-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-d3d202-558b6d7798-4mpfq\" (UID: \"c05721b2-9ebf-4250-9d7c-c0e464194f9c\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d3d202-558b6d7798-4mpfq"
Sep 16 02:44:49.434785 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:49.434767    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-d3d202-558b6d7798-4mpfq"]
Sep 16 02:44:49.451688 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:49.451665    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-d3d202-75996dd58f-kxn4n"]
Sep 16 02:44:49.455342 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:49.455326    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-d3d202-75996dd58f-kxn4n"
Sep 16 02:44:49.472606 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:49.472585    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-d3d202-75996dd58f-kxn4n"]
Sep 16 02:44:49.534379 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:49.534305    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c97c7775-4a18-4a8d-b068-8a625fc6a591-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-d3d202-75996dd58f-kxn4n\" (UID: \"c97c7775-4a18-4a8d-b068-8a625fc6a591\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-d3d202-75996dd58f-kxn4n"
Sep 16 02:44:49.534379 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:49.534358    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c05721b2-9ebf-4250-9d7c-c0e464194f9c-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-d3d202-558b6d7798-4mpfq\" (UID: \"c05721b2-9ebf-4250-9d7c-c0e464194f9c\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d3d202-558b6d7798-4mpfq"
Sep 16 02:44:49.534579 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:49.534431    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zn7hm\" (UniqueName: \"kubernetes.io/projected/c05721b2-9ebf-4250-9d7c-c0e464194f9c-kube-api-access-zn7hm\") pod \"payload-pre-processing-e2e-ipp-b-d3d202-558b6d7798-4mpfq\" (UID: \"c05721b2-9ebf-4250-9d7c-c0e464194f9c\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d3d202-558b6d7798-4mpfq"
Sep 16 02:44:49.534579 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:49.534462    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zxkcz\" (UniqueName: \"kubernetes.io/projected/c97c7775-4a18-4a8d-b068-8a625fc6a591-kube-api-access-zxkcz\") pod \"payload-processing-e2e-ipp-b-d3d202-75996dd58f-kxn4n\" (UID: \"c97c7775-4a18-4a8d-b068-8a625fc6a591\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-d3d202-75996dd58f-kxn4n"
Sep 16 02:44:49.535052 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:49.535036    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c05721b2-9ebf-4250-9d7c-c0e464194f9c-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-d3d202-558b6d7798-4mpfq\" (UID: \"c05721b2-9ebf-4250-9d7c-c0e464194f9c\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d3d202-558b6d7798-4mpfq"
Sep 16 02:44:49.542643 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:49.542622    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zn7hm\" (UniqueName: \"kubernetes.io/projected/c05721b2-9ebf-4250-9d7c-c0e464194f9c-kube-api-access-zn7hm\") pod \"payload-pre-processing-e2e-ipp-b-d3d202-558b6d7798-4mpfq\" (UID: \"c05721b2-9ebf-4250-9d7c-c0e464194f9c\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d3d202-558b6d7798-4mpfq"
Sep 16 02:44:49.635795 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:49.635760    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zxkcz\" (UniqueName: \"kubernetes.io/projected/c97c7775-4a18-4a8d-b068-8a625fc6a591-kube-api-access-zxkcz\") pod \"payload-processing-e2e-ipp-b-d3d202-75996dd58f-kxn4n\" (UID: \"c97c7775-4a18-4a8d-b068-8a625fc6a591\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-d3d202-75996dd58f-kxn4n"
Sep 16 02:44:49.635965 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:49.635850    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c97c7775-4a18-4a8d-b068-8a625fc6a591-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-d3d202-75996dd58f-kxn4n\" (UID: \"c97c7775-4a18-4a8d-b068-8a625fc6a591\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-d3d202-75996dd58f-kxn4n"
Sep 16 02:44:49.636562 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:49.636545    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c97c7775-4a18-4a8d-b068-8a625fc6a591-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-d3d202-75996dd58f-kxn4n\" (UID: \"c97c7775-4a18-4a8d-b068-8a625fc6a591\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-d3d202-75996dd58f-kxn4n"
Sep 16 02:44:49.643621 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:49.643599    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zxkcz\" (UniqueName: \"kubernetes.io/projected/c97c7775-4a18-4a8d-b068-8a625fc6a591-kube-api-access-zxkcz\") pod \"payload-processing-e2e-ipp-b-d3d202-75996dd58f-kxn4n\" (UID: \"c97c7775-4a18-4a8d-b068-8a625fc6a591\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-d3d202-75996dd58f-kxn4n"
Sep 16 02:44:49.722517 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:49.722455    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d3d202-558b6d7798-4mpfq"
Sep 16 02:44:49.723671 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:49.723644    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:44:49.766014 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:49.765966    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-d3d202-75996dd58f-kxn4n"
Sep 16 02:44:49.997095 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:49.997066    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-d3d202-558b6d7798-4mpfq"]
Sep 16 02:44:50.001119 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:44:50.000706    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc05721b2_9ebf_4250_9d7c_c0e464194f9c.slice/crio-a00736f82066260c5165123c05a3b5e818f383567062c073bdfc017905892255 WatchSource:0}: Error finding container a00736f82066260c5165123c05a3b5e818f383567062c073bdfc017905892255: Status 404 returned error can't find the container with id a00736f82066260c5165123c05a3b5e818f383567062c073bdfc017905892255
Sep 16 02:44:50.045131 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:50.045109    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-d3d202-75996dd58f-kxn4n"]
Sep 16 02:44:50.050635 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:44:50.050606    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc97c7775_4a18_4a8d_b068_8a625fc6a591.slice/crio-eca5f7e4588a177170f6529a585ed4c5ac3b7819130d73cec32653ed1e30bae9 WatchSource:0}: Error finding container eca5f7e4588a177170f6529a585ed4c5ac3b7819130d73cec32653ed1e30bae9: Status 404 returned error can't find the container with id eca5f7e4588a177170f6529a585ed4c5ac3b7819130d73cec32653ed1e30bae9
Sep 16 02:44:50.550272 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:50.550236    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:44:50.838608 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:50.838519    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:44:50.939115 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:50.939079    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:44:50.976028 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:50.975994    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-d3d202-75996dd58f-kxn4n" event={"ID":"c97c7775-4a18-4a8d-b068-8a625fc6a591","Type":"ContainerStarted","Data":"1f2d62609f23e54b16ec971a7b531b1ff182dd85c90c755f4e1030a5bf43fb1f"}
Sep 16 02:44:50.976563 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:50.976059    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-d3d202-75996dd58f-kxn4n"
Sep 16 02:44:50.976563 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:50.976081    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-d3d202-75996dd58f-kxn4n" event={"ID":"c97c7775-4a18-4a8d-b068-8a625fc6a591","Type":"ContainerStarted","Data":"eca5f7e4588a177170f6529a585ed4c5ac3b7819130d73cec32653ed1e30bae9"}
Sep 16 02:44:50.977781 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:50.977751    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d3d202-558b6d7798-4mpfq" event={"ID":"c05721b2-9ebf-4250-9d7c-c0e464194f9c","Type":"ContainerStarted","Data":"a50de980cf97f663ca651f3565d7edaaf2b2ed13cc5da71a966d4bf445b0afbf"}
Sep 16 02:44:50.977923 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:50.977803    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d3d202-558b6d7798-4mpfq" event={"ID":"c05721b2-9ebf-4250-9d7c-c0e464194f9c","Type":"ContainerStarted","Data":"a00736f82066260c5165123c05a3b5e818f383567062c073bdfc017905892255"}
Sep 16 02:44:50.977923 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:50.977842    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d3d202-558b6d7798-4mpfq"
Sep 16 02:44:50.992549 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:50.992504    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-d3d202-75996dd58f-kxn4n" podStartSLOduration=1.9924908879999998 podStartE2EDuration="1.992490888s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 02:44:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 02:44:50.990911653 +0000 UTC m=+1911.374426901" watchObservedRunningTime="2026-09-16 02:44:50.992490888 +0000 UTC m=+1911.376006135"
Sep 16 02:44:51.006993 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:51.006945    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d3d202-558b6d7798-4mpfq" podStartSLOduration=2.006930973 podStartE2EDuration="2.006930973s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 02:44:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 02:44:51.005715242 +0000 UTC m=+1911.389230489" watchObservedRunningTime="2026-09-16 02:44:51.006930973 +0000 UTC m=+1911.390446218"
Sep 16 02:44:51.326708 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:51.326672    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:44:57.153179 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:57.153145    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:44:57.254706 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:57.254670    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:44:58.577874 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:44:58.577838    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:45:00.137358 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:00.137322    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-d3d202-29825445-k57v7"]
Sep 16 02:45:00.142739 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:00.142718    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-fd614e-29825445-qbrbw"]
Sep 16 02:45:00.142866 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:00.142841    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-d3d202-29825445-k57v7"
Sep 16 02:45:00.144858 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:00.144835    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-jpd4f\""
Sep 16 02:45:00.146143 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:00.146122    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-fd614e-29825445-qbrbw"
Sep 16 02:45:00.149244 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:00.149225    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-d3d202-29825445-k57v7"]
Sep 16 02:45:00.152389 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:00.151991    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-fd614e-29825445-qbrbw"]
Sep 16 02:45:00.154876 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:00.154853    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-bd0fb8-29825445-hm57f"]
Sep 16 02:45:00.157890 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:00.157869    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-d3d202-29825445-k57v7"
Sep 16 02:45:00.158669 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:00.158651    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-bd0fb8-29825445-hm57f"
Sep 16 02:45:00.164496 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:00.164462    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-8e10f1-29825445-82xtd"]
Sep 16 02:45:00.169001 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:00.168975    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-fd614e-29825445-qbrbw"
Sep 16 02:45:00.169110 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:00.169042    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-bd0fb8-29825445-hm57f"]
Sep 16 02:45:00.169174 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:00.169141    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-8e10f1-29825445-82xtd"
Sep 16 02:45:00.179551 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:00.179515    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-8e10f1-29825445-82xtd"]
Sep 16 02:45:00.209086 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:00.209054    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-bd0fb8-29825445-hm57f"
Sep 16 02:45:00.218303 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:00.218278    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-8e10f1-29825445-82xtd"
Sep 16 02:45:00.450628 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:00.450604    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-d3d202-29825445-k57v7"]
Sep 16 02:45:00.452110 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:45:00.452081    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39eea28c_9ffa_4790_810b_1474c29bc639.slice/crio-0b79123239e45f209308f9a876ddb683be34922fb096271605251bd0645fc3b4 WatchSource:0}: Error finding container 0b79123239e45f209308f9a876ddb683be34922fb096271605251bd0645fc3b4: Status 404 returned error can't find the container with id 0b79123239e45f209308f9a876ddb683be34922fb096271605251bd0645fc3b4
Sep 16 02:45:00.468128 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:00.468104    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-fd614e-29825445-qbrbw"]
Sep 16 02:45:00.470153 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:45:00.470127    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93344fea_959d_47ba_8575_b3b8b1fec1be.slice/crio-7b7785d794d529f3a32baa1a98a161fff4e535508c3d6fa41b804b8c75cf12d8 WatchSource:0}: Error finding container 7b7785d794d529f3a32baa1a98a161fff4e535508c3d6fa41b804b8c75cf12d8: Status 404 returned error can't find the container with id 7b7785d794d529f3a32baa1a98a161fff4e535508c3d6fa41b804b8c75cf12d8
Sep 16 02:45:00.506213 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:00.506192    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-bd0fb8-29825445-hm57f"]
Sep 16 02:45:00.507626 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:45:00.507605    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcfbb7371_86fe_44df_bcb1_de3cf034b44e.slice/crio-4d79a00c764d3bf496284d264d62e543fef1788052369909b283b198888901da WatchSource:0}: Error finding container 4d79a00c764d3bf496284d264d62e543fef1788052369909b283b198888901da: Status 404 returned error can't find the container with id 4d79a00c764d3bf496284d264d62e543fef1788052369909b283b198888901da
Sep 16 02:45:00.528361 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:00.528338    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-8e10f1-29825445-82xtd"]
Sep 16 02:45:00.536492 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:45:00.536448    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48ad0e64_5185_4129_90bc_49d15e2d9534.slice/crio-f18a9c29f5b2c23dacc09d9947957b29e818cacda125c98037c58654a425de75 WatchSource:0}: Error finding container f18a9c29f5b2c23dacc09d9947957b29e818cacda125c98037c58654a425de75: Status 404 returned error can't find the container with id f18a9c29f5b2c23dacc09d9947957b29e818cacda125c98037c58654a425de75
Sep 16 02:45:01.031959 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:01.031868    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-fd614e-29825445-qbrbw" event={"ID":"93344fea-959d-47ba-8575-b3b8b1fec1be","Type":"ContainerStarted","Data":"cf4292e5cf32ed491f0b05d9283e3a27d791a40dee750b1f29a42732ed98b664"}
Sep 16 02:45:01.031959 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:01.031915    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-fd614e-29825445-qbrbw" event={"ID":"93344fea-959d-47ba-8575-b3b8b1fec1be","Type":"ContainerStarted","Data":"7b7785d794d529f3a32baa1a98a161fff4e535508c3d6fa41b804b8c75cf12d8"}
Sep 16 02:45:01.033599 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:01.033573    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-d3d202-29825445-k57v7" event={"ID":"39eea28c-9ffa-4790-810b-1474c29bc639","Type":"ContainerStarted","Data":"9fa31aa345bc7cef5511bf9df1a05e7189e43a0e55ebbc1b7d4dc09d67b487a0"}
Sep 16 02:45:01.033735 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:01.033610    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-d3d202-29825445-k57v7" event={"ID":"39eea28c-9ffa-4790-810b-1474c29bc639","Type":"ContainerStarted","Data":"0b79123239e45f209308f9a876ddb683be34922fb096271605251bd0645fc3b4"}
Sep 16 02:45:01.035220 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:01.035197    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-bd0fb8-29825445-hm57f" event={"ID":"cfbb7371-86fe-44df-bcb1-de3cf034b44e","Type":"ContainerStarted","Data":"9c714fd65bc6ea2178df0dcda8caeb9805750aaeaba4e441b372a08fc34899aa"}
Sep 16 02:45:01.035351 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:01.035238    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-bd0fb8-29825445-hm57f" event={"ID":"cfbb7371-86fe-44df-bcb1-de3cf034b44e","Type":"ContainerStarted","Data":"4d79a00c764d3bf496284d264d62e543fef1788052369909b283b198888901da"}
Sep 16 02:45:01.036641 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:01.036621    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-8e10f1-29825445-82xtd" event={"ID":"48ad0e64-5185-4129-90bc-49d15e2d9534","Type":"ContainerStarted","Data":"e6071188f1162f62c4e6ea7869326b9a1c77b7dd53902df4b26a870447bf3ef2"}
Sep 16 02:45:01.036742 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:01.036652    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-8e10f1-29825445-82xtd" event={"ID":"48ad0e64-5185-4129-90bc-49d15e2d9534","Type":"ContainerStarted","Data":"f18a9c29f5b2c23dacc09d9947957b29e818cacda125c98037c58654a425de75"}
Sep 16 02:45:01.048297 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:01.048252    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-fd614e-29825445-qbrbw" podStartSLOduration=1.048238633 podStartE2EDuration="1.048238633s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 02:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 02:45:01.04616779 +0000 UTC m=+1921.429683039" watchObservedRunningTime="2026-09-16 02:45:01.048238633 +0000 UTC m=+1921.431753878"
Sep 16 02:45:01.060693 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:01.060643    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-bd0fb8-29825445-hm57f" podStartSLOduration=1.060630438 podStartE2EDuration="1.060630438s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 02:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 02:45:01.058385606 +0000 UTC m=+1921.441900853" watchObservedRunningTime="2026-09-16 02:45:01.060630438 +0000 UTC m=+1921.444145683"
Sep 16 02:45:01.072810 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:01.072765    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-8e10f1-29825445-82xtd" podStartSLOduration=1.072752041 podStartE2EDuration="1.072752041s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 02:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 02:45:01.070011323 +0000 UTC m=+1921.453526572" watchObservedRunningTime="2026-09-16 02:45:01.072752041 +0000 UTC m=+1921.456267287"
Sep 16 02:45:01.083523 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:01.083460    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-d3d202-29825445-k57v7" podStartSLOduration=1.08345065 podStartE2EDuration="1.08345065s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 02:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 02:45:01.081796227 +0000 UTC m=+1921.465311472" watchObservedRunningTime="2026-09-16 02:45:01.08345065 +0000 UTC m=+1921.466965892"
Sep 16 02:45:01.985847 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:01.985806    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d3d202-558b6d7798-4mpfq"
Sep 16 02:45:01.986249 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:01.985894    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-d3d202-75996dd58f-kxn4n"
Sep 16 02:45:02.045545 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:02.045483    2579 generic.go:356] "Generic (PLEG): container finished" podID="93344fea-959d-47ba-8575-b3b8b1fec1be" containerID="cf4292e5cf32ed491f0b05d9283e3a27d791a40dee750b1f29a42732ed98b664" exitCode=0
Sep 16 02:45:02.045745 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:02.045536    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-fd614e-29825445-qbrbw" event={"ID":"93344fea-959d-47ba-8575-b3b8b1fec1be","Type":"ContainerDied","Data":"cf4292e5cf32ed491f0b05d9283e3a27d791a40dee750b1f29a42732ed98b664"}
Sep 16 02:45:02.047580 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:02.047553    2579 generic.go:356] "Generic (PLEG): container finished" podID="39eea28c-9ffa-4790-810b-1474c29bc639" containerID="9fa31aa345bc7cef5511bf9df1a05e7189e43a0e55ebbc1b7d4dc09d67b487a0" exitCode=0
Sep 16 02:45:02.047725 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:02.047658    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-d3d202-29825445-k57v7" event={"ID":"39eea28c-9ffa-4790-810b-1474c29bc639","Type":"ContainerDied","Data":"9fa31aa345bc7cef5511bf9df1a05e7189e43a0e55ebbc1b7d4dc09d67b487a0"}
Sep 16 02:45:02.049394 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:02.049374    2579 generic.go:356] "Generic (PLEG): container finished" podID="cfbb7371-86fe-44df-bcb1-de3cf034b44e" containerID="9c714fd65bc6ea2178df0dcda8caeb9805750aaeaba4e441b372a08fc34899aa" exitCode=0
Sep 16 02:45:02.049537 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:02.049458    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-bd0fb8-29825445-hm57f" event={"ID":"cfbb7371-86fe-44df-bcb1-de3cf034b44e","Type":"ContainerDied","Data":"9c714fd65bc6ea2178df0dcda8caeb9805750aaeaba4e441b372a08fc34899aa"}
Sep 16 02:45:02.051307 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:02.051281    2579 generic.go:356] "Generic (PLEG): container finished" podID="48ad0e64-5185-4129-90bc-49d15e2d9534" containerID="e6071188f1162f62c4e6ea7869326b9a1c77b7dd53902df4b26a870447bf3ef2" exitCode=0
Sep 16 02:45:02.051422 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:02.051337    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-8e10f1-29825445-82xtd" event={"ID":"48ad0e64-5185-4129-90bc-49d15e2d9534","Type":"ContainerDied","Data":"e6071188f1162f62c4e6ea7869326b9a1c77b7dd53902df4b26a870447bf3ef2"}
Sep 16 02:45:03.212720 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:03.212694    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-fd614e-29825445-qbrbw"
Sep 16 02:45:03.267046 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:03.267016    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-d3d202-29825445-k57v7"
Sep 16 02:45:03.303970 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:03.303945    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-bd0fb8-29825445-hm57f"
Sep 16 02:45:03.307079 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:03.307059    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-8e10f1-29825445-82xtd"
Sep 16 02:45:04.064916 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:04.064890    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-8e10f1-29825445-82xtd"
Sep 16 02:45:04.065095 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:04.064967    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-8e10f1-29825445-82xtd" event={"ID":"48ad0e64-5185-4129-90bc-49d15e2d9534","Type":"ContainerDied","Data":"f18a9c29f5b2c23dacc09d9947957b29e818cacda125c98037c58654a425de75"}
Sep 16 02:45:04.065095 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:04.065081    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f18a9c29f5b2c23dacc09d9947957b29e818cacda125c98037c58654a425de75"
Sep 16 02:45:04.066679 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:04.066657    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-fd614e-29825445-qbrbw"
Sep 16 02:45:04.066679 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:04.066668    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-fd614e-29825445-qbrbw" event={"ID":"93344fea-959d-47ba-8575-b3b8b1fec1be","Type":"ContainerDied","Data":"7b7785d794d529f3a32baa1a98a161fff4e535508c3d6fa41b804b8c75cf12d8"}
Sep 16 02:45:04.066861 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:04.066710    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7b7785d794d529f3a32baa1a98a161fff4e535508c3d6fa41b804b8c75cf12d8"
Sep 16 02:45:04.068263 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:04.068247    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-d3d202-29825445-k57v7"
Sep 16 02:45:04.068432 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:04.068259    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-d3d202-29825445-k57v7" event={"ID":"39eea28c-9ffa-4790-810b-1474c29bc639","Type":"ContainerDied","Data":"0b79123239e45f209308f9a876ddb683be34922fb096271605251bd0645fc3b4"}
Sep 16 02:45:04.068432 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:04.068405    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0b79123239e45f209308f9a876ddb683be34922fb096271605251bd0645fc3b4"
Sep 16 02:45:04.069835 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:04.069799    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-bd0fb8-29825445-hm57f" event={"ID":"cfbb7371-86fe-44df-bcb1-de3cf034b44e","Type":"ContainerDied","Data":"4d79a00c764d3bf496284d264d62e543fef1788052369909b283b198888901da"}
Sep 16 02:45:04.069835 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:04.069821    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-bd0fb8-29825445-hm57f"
Sep 16 02:45:04.070020 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:04.069851    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4d79a00c764d3bf496284d264d62e543fef1788052369909b283b198888901da"
Sep 16 02:45:04.609587 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:04.609559    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:45:04.706743 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:04.706702    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:45:04.952433 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:04.952400    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:45:05.245280 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:05.245192    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:45:05.444534 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:05.444491    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:45:07.096244 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:07.096217    2579 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-pre-processing-6cd864bc95-mh4jh" secret="" err="secret \"payload-processing-dockercfg-zzx6j\" not found"
Sep 16 02:45:08.117869 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:08.117834    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:45:08.841064 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:08.841029    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:45:08.933143 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:08.933113    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:45:09.631819 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:09.631788    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:45:10.541503 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:10.541416    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:45:11.211751 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:11.211715    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:45:11.926106 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:11.926062    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:45:12.701949 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:12.701920    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:45:12.806612 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:12.806578    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:45:12.907368 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:12.907339    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:45:13.210569 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:13.210526    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:45:13.311016 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:13.310978    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:45:13.393796 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:13.393766    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-7448c4-ffbd4958b-rhshb"]
Sep 16 02:45:13.394211 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:13.394197    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39eea28c-9ffa-4790-810b-1474c29bc639" containerName="cleanup"
Sep 16 02:45:13.394260 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:13.394214    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="39eea28c-9ffa-4790-810b-1474c29bc639" containerName="cleanup"
Sep 16 02:45:13.394260 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:13.394230    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="48ad0e64-5185-4129-90bc-49d15e2d9534" containerName="cleanup"
Sep 16 02:45:13.394260 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:13.394236    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="48ad0e64-5185-4129-90bc-49d15e2d9534" containerName="cleanup"
Sep 16 02:45:13.394260 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:13.394245    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cfbb7371-86fe-44df-bcb1-de3cf034b44e" containerName="cleanup"
Sep 16 02:45:13.394260 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:13.394251    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="cfbb7371-86fe-44df-bcb1-de3cf034b44e" containerName="cleanup"
Sep 16 02:45:13.394417 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:13.394266    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="93344fea-959d-47ba-8575-b3b8b1fec1be" containerName="cleanup"
Sep 16 02:45:13.394417 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:13.394271    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="93344fea-959d-47ba-8575-b3b8b1fec1be" containerName="cleanup"
Sep 16 02:45:13.394417 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:13.394343    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="39eea28c-9ffa-4790-810b-1474c29bc639" containerName="cleanup"
Sep 16 02:45:13.394417 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:13.394353    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="93344fea-959d-47ba-8575-b3b8b1fec1be" containerName="cleanup"
Sep 16 02:45:13.394417 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:13.394359    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="48ad0e64-5185-4129-90bc-49d15e2d9534" containerName="cleanup"
Sep 16 02:45:13.394417 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:13.394366    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="cfbb7371-86fe-44df-bcb1-de3cf034b44e" containerName="cleanup"
Sep 16 02:45:13.397537 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:13.397522    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-7448c4-ffbd4958b-rhshb"
Sep 16 02:45:13.399621 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:13.399606    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-7448c4\""
Sep 16 02:45:13.399713 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:13.399611    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-7448c4-dockercfg-dmbdc\""
Sep 16 02:45:13.404387 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:13.404366    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-7448c4-ffbd4958b-rhshb"]
Sep 16 02:45:13.450782 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:13.450759    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f7ntr\" (UniqueName: \"kubernetes.io/projected/2386c3bc-a0ad-4f43-a1cf-293dfba60ab6-kube-api-access-f7ntr\") pod \"payload-pre-processing-e2e-api-a-7448c4-ffbd4958b-rhshb\" (UID: \"2386c3bc-a0ad-4f43-a1cf-293dfba60ab6\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-7448c4-ffbd4958b-rhshb"
Sep 16 02:45:13.450916 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:13.450799    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2386c3bc-a0ad-4f43-a1cf-293dfba60ab6-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-7448c4-ffbd4958b-rhshb\" (UID: \"2386c3bc-a0ad-4f43-a1cf-293dfba60ab6\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-7448c4-ffbd4958b-rhshb"
Sep 16 02:45:13.465843 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:13.465770    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-7448c4-7fc96fbc87-gfx84"]
Sep 16 02:45:13.469607 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:13.469588    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-7448c4-7fc96fbc87-gfx84"
Sep 16 02:45:13.483133 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:13.483100    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-7448c4-7fc96fbc87-gfx84"]
Sep 16 02:45:13.552348 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:13.552311    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f7ntr\" (UniqueName: \"kubernetes.io/projected/2386c3bc-a0ad-4f43-a1cf-293dfba60ab6-kube-api-access-f7ntr\") pod \"payload-pre-processing-e2e-api-a-7448c4-ffbd4958b-rhshb\" (UID: \"2386c3bc-a0ad-4f43-a1cf-293dfba60ab6\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-7448c4-ffbd4958b-rhshb"
Sep 16 02:45:13.552557 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:13.552378    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2386c3bc-a0ad-4f43-a1cf-293dfba60ab6-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-7448c4-ffbd4958b-rhshb\" (UID: \"2386c3bc-a0ad-4f43-a1cf-293dfba60ab6\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-7448c4-ffbd4958b-rhshb"
Sep 16 02:45:13.552557 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:13.552424    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gm5gl\" (UniqueName: \"kubernetes.io/projected/36dde4cf-837f-4cb6-b7ca-837c8e41d07e-kube-api-access-gm5gl\") pod \"payload-processing-e2e-api-a-7448c4-7fc96fbc87-gfx84\" (UID: \"36dde4cf-837f-4cb6-b7ca-837c8e41d07e\") " pod="openshift-ingress/payload-processing-e2e-api-a-7448c4-7fc96fbc87-gfx84"
Sep 16 02:45:13.552653 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:13.552590    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/36dde4cf-837f-4cb6-b7ca-837c8e41d07e-plugins-config-volume\") pod \"payload-processing-e2e-api-a-7448c4-7fc96fbc87-gfx84\" (UID: \"36dde4cf-837f-4cb6-b7ca-837c8e41d07e\") " pod="openshift-ingress/payload-processing-e2e-api-a-7448c4-7fc96fbc87-gfx84"
Sep 16 02:45:13.553090 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:13.553069    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2386c3bc-a0ad-4f43-a1cf-293dfba60ab6-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-7448c4-ffbd4958b-rhshb\" (UID: \"2386c3bc-a0ad-4f43-a1cf-293dfba60ab6\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-7448c4-ffbd4958b-rhshb"
Sep 16 02:45:13.561265 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:13.561243    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f7ntr\" (UniqueName: \"kubernetes.io/projected/2386c3bc-a0ad-4f43-a1cf-293dfba60ab6-kube-api-access-f7ntr\") pod \"payload-pre-processing-e2e-api-a-7448c4-ffbd4958b-rhshb\" (UID: \"2386c3bc-a0ad-4f43-a1cf-293dfba60ab6\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-7448c4-ffbd4958b-rhshb"
Sep 16 02:45:13.630370 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:13.630341    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:45:13.653597 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:13.653559    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gm5gl\" (UniqueName: \"kubernetes.io/projected/36dde4cf-837f-4cb6-b7ca-837c8e41d07e-kube-api-access-gm5gl\") pod \"payload-processing-e2e-api-a-7448c4-7fc96fbc87-gfx84\" (UID: \"36dde4cf-837f-4cb6-b7ca-837c8e41d07e\") " pod="openshift-ingress/payload-processing-e2e-api-a-7448c4-7fc96fbc87-gfx84"
Sep 16 02:45:13.653772 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:13.653650    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/36dde4cf-837f-4cb6-b7ca-837c8e41d07e-plugins-config-volume\") pod \"payload-processing-e2e-api-a-7448c4-7fc96fbc87-gfx84\" (UID: \"36dde4cf-837f-4cb6-b7ca-837c8e41d07e\") " pod="openshift-ingress/payload-processing-e2e-api-a-7448c4-7fc96fbc87-gfx84"
Sep 16 02:45:13.654375 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:13.654336    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/36dde4cf-837f-4cb6-b7ca-837c8e41d07e-plugins-config-volume\") pod \"payload-processing-e2e-api-a-7448c4-7fc96fbc87-gfx84\" (UID: \"36dde4cf-837f-4cb6-b7ca-837c8e41d07e\") " pod="openshift-ingress/payload-processing-e2e-api-a-7448c4-7fc96fbc87-gfx84"
Sep 16 02:45:13.661037 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:13.661010    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gm5gl\" (UniqueName: \"kubernetes.io/projected/36dde4cf-837f-4cb6-b7ca-837c8e41d07e-kube-api-access-gm5gl\") pod \"payload-processing-e2e-api-a-7448c4-7fc96fbc87-gfx84\" (UID: \"36dde4cf-837f-4cb6-b7ca-837c8e41d07e\") " pod="openshift-ingress/payload-processing-e2e-api-a-7448c4-7fc96fbc87-gfx84"
Sep 16 02:45:13.709578 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:13.709548    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-7448c4-ffbd4958b-rhshb"
Sep 16 02:45:13.780847 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:13.780818    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-7448c4-7fc96fbc87-gfx84"
Sep 16 02:45:13.982110 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:13.982085    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-7448c4-ffbd4958b-rhshb"]
Sep 16 02:45:13.984007 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:45:13.983978    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2386c3bc_a0ad_4f43_a1cf_293dfba60ab6.slice/crio-f43661f930f074dc0e6ee14324d614254f296137f251c21ffe86e7535e4061d1 WatchSource:0}: Error finding container f43661f930f074dc0e6ee14324d614254f296137f251c21ffe86e7535e4061d1: Status 404 returned error can't find the container with id f43661f930f074dc0e6ee14324d614254f296137f251c21ffe86e7535e4061d1
Sep 16 02:45:14.057055 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:14.057033    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-7448c4-7fc96fbc87-gfx84"]
Sep 16 02:45:14.059362 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:45:14.059336    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36dde4cf_837f_4cb6_b7ca_837c8e41d07e.slice/crio-f69190d0100ac6683f3e02e6f80cbb4c9321dd24ec0fece9b83484f6dc8fe694 WatchSource:0}: Error finding container f69190d0100ac6683f3e02e6f80cbb4c9321dd24ec0fece9b83484f6dc8fe694: Status 404 returned error can't find the container with id f69190d0100ac6683f3e02e6f80cbb4c9321dd24ec0fece9b83484f6dc8fe694
Sep 16 02:45:14.126320 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:14.126289    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-7448c4-ffbd4958b-rhshb" event={"ID":"2386c3bc-a0ad-4f43-a1cf-293dfba60ab6","Type":"ContainerStarted","Data":"e6259bb8d6d0dc82c0cbd9e7d4650d893d7aea9dec4cba4f292af5ab7fa15d47"}
Sep 16 02:45:14.126497 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:14.126453    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-7448c4-ffbd4958b-rhshb"
Sep 16 02:45:14.126589 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:14.126514    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-7448c4-ffbd4958b-rhshb" event={"ID":"2386c3bc-a0ad-4f43-a1cf-293dfba60ab6","Type":"ContainerStarted","Data":"f43661f930f074dc0e6ee14324d614254f296137f251c21ffe86e7535e4061d1"}
Sep 16 02:45:14.128036 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:14.128015    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-7448c4-7fc96fbc87-gfx84" event={"ID":"36dde4cf-837f-4cb6-b7ca-837c8e41d07e","Type":"ContainerStarted","Data":"f69190d0100ac6683f3e02e6f80cbb4c9321dd24ec0fece9b83484f6dc8fe694"}
Sep 16 02:45:14.147488 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:14.147415    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-7448c4-ffbd4958b-rhshb" podStartSLOduration=1.147395753 podStartE2EDuration="1.147395753s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 02:45:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 02:45:14.14397863 +0000 UTC m=+1934.527493877" watchObservedRunningTime="2026-09-16 02:45:14.147395753 +0000 UTC m=+1934.530910999"
Sep 16 02:45:14.439521 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:14.439488    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:45:15.135752 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:15.135710    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-7448c4-7fc96fbc87-gfx84" event={"ID":"36dde4cf-837f-4cb6-b7ca-837c8e41d07e","Type":"ContainerStarted","Data":"0de0cbe6ec83937b6ad52a1782c31446cad703fda17c14908316e7b380aec6f5"}
Sep 16 02:45:15.136124 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:15.135933    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-7448c4-7fc96fbc87-gfx84"
Sep 16 02:45:15.151842 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:15.151789    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-7448c4-7fc96fbc87-gfx84" podStartSLOduration=2.151770613 podStartE2EDuration="2.151770613s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 02:45:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 02:45:15.150720092 +0000 UTC m=+1935.534235337" watchObservedRunningTime="2026-09-16 02:45:15.151770613 +0000 UTC m=+1935.535285859"
Sep 16 02:45:15.245191 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:15.245150    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:45:15.798499 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:15.798447    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:45:15.900211 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:15.900167    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:45:16.506290 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:16.506250    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:45:17.334542 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:17.334511    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:45:17.959155 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:17.959115    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:45:21.242234 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:21.242207    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-d3d202-75996dd58f-kxn4n_c97c7775-4a18-4a8d-b068-8a625fc6a591/payload-processing/0.log"
Sep 16 02:45:25.138648 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:25.138619    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-7448c4-ffbd4958b-rhshb"
Sep 16 02:45:26.142645 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:26.142616    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-7448c4-7fc96fbc87-gfx84"
Sep 16 02:45:34.599771 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:34.599730    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:45:35.224977 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:35.224929    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:45:35.982803 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:35.982765    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:45:36.611665 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:36.611627    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:45:37.435975 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:37.435930    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:45:38.052230 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:38.052191    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:45:38.826118 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:38.826084    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:45:38.926242 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:38.926166    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:45:39.366359 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:39.366320    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:45:39.643671 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:39.643572    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:45:39.748605 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:39.748570    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:45:42.803864 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:42.803833    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:45:43.166377 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:43.166284    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:45:43.299302 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:43.299270    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:45:43.739960 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:43.739919    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:45:44.374543 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:44.374511    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:45:45.244784 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:45.244740    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:45:45.886831 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:45.886789    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:45:46.632095 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:46.632059    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:45:47.271136 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:47.271089    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:45:47.372751 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:47.372717    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:45:47.502646 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:47.502613    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:45:47.965795 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:47.965758    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:45:47.971016 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:47.970992    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-e21dae-5bbc9776cc-sl9ks"]
Sep 16 02:45:47.976219 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:47.976200    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-e21dae-5bbc9776cc-sl9ks"
Sep 16 02:45:47.978681 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:47.978663    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-e21dae-dockercfg-7xcpw\""
Sep 16 02:45:47.978802 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:47.978679    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-e21dae\""
Sep 16 02:45:47.984407 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:47.984384    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-e21dae-5bbc9776cc-sl9ks"]
Sep 16 02:45:48.019888 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:48.019838    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:45:48.053270 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:48.053240    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f3694bbf-a521-4c14-a894-28075b871d83-plugins-config-volume\") pod \"payload-processing-e2e-api-b-e21dae-5bbc9776cc-sl9ks\" (UID: \"f3694bbf-a521-4c14-a894-28075b871d83\") " pod="openshift-ingress/payload-processing-e2e-api-b-e21dae-5bbc9776cc-sl9ks"
Sep 16 02:45:48.053416 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:48.053310    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sx6hc\" (UniqueName: \"kubernetes.io/projected/f3694bbf-a521-4c14-a894-28075b871d83-kube-api-access-sx6hc\") pod \"payload-processing-e2e-api-b-e21dae-5bbc9776cc-sl9ks\" (UID: \"f3694bbf-a521-4c14-a894-28075b871d83\") " pod="openshift-ingress/payload-processing-e2e-api-b-e21dae-5bbc9776cc-sl9ks"
Sep 16 02:45:48.154103 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:48.154072    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f3694bbf-a521-4c14-a894-28075b871d83-plugins-config-volume\") pod \"payload-processing-e2e-api-b-e21dae-5bbc9776cc-sl9ks\" (UID: \"f3694bbf-a521-4c14-a894-28075b871d83\") " pod="openshift-ingress/payload-processing-e2e-api-b-e21dae-5bbc9776cc-sl9ks"
Sep 16 02:45:48.154270 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:48.154163    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sx6hc\" (UniqueName: \"kubernetes.io/projected/f3694bbf-a521-4c14-a894-28075b871d83-kube-api-access-sx6hc\") pod \"payload-processing-e2e-api-b-e21dae-5bbc9776cc-sl9ks\" (UID: \"f3694bbf-a521-4c14-a894-28075b871d83\") " pod="openshift-ingress/payload-processing-e2e-api-b-e21dae-5bbc9776cc-sl9ks"
Sep 16 02:45:48.155004 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:48.154979    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f3694bbf-a521-4c14-a894-28075b871d83-plugins-config-volume\") pod \"payload-processing-e2e-api-b-e21dae-5bbc9776cc-sl9ks\" (UID: \"f3694bbf-a521-4c14-a894-28075b871d83\") " pod="openshift-ingress/payload-processing-e2e-api-b-e21dae-5bbc9776cc-sl9ks"
Sep 16 02:45:48.162453 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:48.162419    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sx6hc\" (UniqueName: \"kubernetes.io/projected/f3694bbf-a521-4c14-a894-28075b871d83-kube-api-access-sx6hc\") pod \"payload-processing-e2e-api-b-e21dae-5bbc9776cc-sl9ks\" (UID: \"f3694bbf-a521-4c14-a894-28075b871d83\") " pod="openshift-ingress/payload-processing-e2e-api-b-e21dae-5bbc9776cc-sl9ks"
Sep 16 02:45:48.288909 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:48.288835    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-e21dae-5bbc9776cc-sl9ks"
Sep 16 02:45:48.566518 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:48.566491    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-e21dae-5bbc9776cc-sl9ks"]
Sep 16 02:45:48.568437 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:45:48.568415    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3694bbf_a521_4c14_a894_28075b871d83.slice/crio-8180bfe15a1e5c79da8bd96e8757bb6931315fc17ac11750d3cc0d123d8aeea6 WatchSource:0}: Error finding container 8180bfe15a1e5c79da8bd96e8757bb6931315fc17ac11750d3cc0d123d8aeea6: Status 404 returned error can't find the container with id 8180bfe15a1e5c79da8bd96e8757bb6931315fc17ac11750d3cc0d123d8aeea6
Sep 16 02:45:49.121816 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:49.121779    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:45:49.322683 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:49.322649    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-e21dae-5bbc9776cc-sl9ks" event={"ID":"f3694bbf-a521-4c14-a894-28075b871d83","Type":"ContainerStarted","Data":"352cf5c2933a2a90981146e4920a06ebe80eb7c9bcc0bf42cf329809fcad4f86"}
Sep 16 02:45:49.323053 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:49.322708    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-e21dae-5bbc9776cc-sl9ks" event={"ID":"f3694bbf-a521-4c14-a894-28075b871d83","Type":"ContainerStarted","Data":"8180bfe15a1e5c79da8bd96e8757bb6931315fc17ac11750d3cc0d123d8aeea6"}
Sep 16 02:45:49.323053 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:49.322855    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-e21dae-5bbc9776cc-sl9ks"
Sep 16 02:45:49.340819 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:49.340772    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-e21dae-5bbc9776cc-sl9ks" podStartSLOduration=2.340758147 podStartE2EDuration="2.340758147s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 02:45:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 02:45:49.338068344 +0000 UTC m=+1969.721583589" watchObservedRunningTime="2026-09-16 02:45:49.340758147 +0000 UTC m=+1969.724273392"
Sep 16 02:45:49.836021 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:49.835977    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:45:54.674564 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:54.674480    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-ipp-a-fd614e/ipp-route-fd614e-kserve-894d7456c-28xz5"]
Sep 16 02:45:54.678624 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:54.678601    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-fd614e/ipp-route-fd614e-kserve-894d7456c-28xz5"
Sep 16 02:45:54.682210 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:54.682189    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-fd614e\"/\"kube-root-ca.crt\""
Sep 16 02:45:54.682346 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:54.682302    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-fd614e\"/\"default-dockercfg-h5p7p\""
Sep 16 02:45:54.682709 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:54.682267    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-fd614e\"/\"ipp-route-fd614e-kserve-self-signed-certs\""
Sep 16 02:45:54.682709 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:54.682184    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-fd614e\"/\"openshift-service-ca.crt\""
Sep 16 02:45:54.685193 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:54.685172    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-fd614e/ipp-route-fd614e-kserve-894d7456c-28xz5"]
Sep 16 02:45:54.707966 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:54.707928    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/35e2f4db-dc20-4c83-9e37-058576e79472-model-cache\") pod \"ipp-route-fd614e-kserve-894d7456c-28xz5\" (UID: \"35e2f4db-dc20-4c83-9e37-058576e79472\") " pod="ai-tenant-e2e-ipp-a-fd614e/ipp-route-fd614e-kserve-894d7456c-28xz5"
Sep 16 02:45:54.707966 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:54.707967    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/35e2f4db-dc20-4c83-9e37-058576e79472-tls-certs\") pod \"ipp-route-fd614e-kserve-894d7456c-28xz5\" (UID: \"35e2f4db-dc20-4c83-9e37-058576e79472\") " pod="ai-tenant-e2e-ipp-a-fd614e/ipp-route-fd614e-kserve-894d7456c-28xz5"
Sep 16 02:45:54.708135 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:54.707994    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/35e2f4db-dc20-4c83-9e37-058576e79472-tmp-dir\") pod \"ipp-route-fd614e-kserve-894d7456c-28xz5\" (UID: \"35e2f4db-dc20-4c83-9e37-058576e79472\") " pod="ai-tenant-e2e-ipp-a-fd614e/ipp-route-fd614e-kserve-894d7456c-28xz5"
Sep 16 02:45:54.708135 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:54.708009    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/35e2f4db-dc20-4c83-9e37-058576e79472-dshm\") pod \"ipp-route-fd614e-kserve-894d7456c-28xz5\" (UID: \"35e2f4db-dc20-4c83-9e37-058576e79472\") " pod="ai-tenant-e2e-ipp-a-fd614e/ipp-route-fd614e-kserve-894d7456c-28xz5"
Sep 16 02:45:54.708135 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:54.708062    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/35e2f4db-dc20-4c83-9e37-058576e79472-home\") pod \"ipp-route-fd614e-kserve-894d7456c-28xz5\" (UID: \"35e2f4db-dc20-4c83-9e37-058576e79472\") " pod="ai-tenant-e2e-ipp-a-fd614e/ipp-route-fd614e-kserve-894d7456c-28xz5"
Sep 16 02:45:54.708240 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:54.708143    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s69bw\" (UniqueName: \"kubernetes.io/projected/35e2f4db-dc20-4c83-9e37-058576e79472-kube-api-access-s69bw\") pod \"ipp-route-fd614e-kserve-894d7456c-28xz5\" (UID: \"35e2f4db-dc20-4c83-9e37-058576e79472\") " pod="ai-tenant-e2e-ipp-a-fd614e/ipp-route-fd614e-kserve-894d7456c-28xz5"
Sep 16 02:45:54.721970 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:54.721939    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:45:54.809309 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:54.809275    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s69bw\" (UniqueName: \"kubernetes.io/projected/35e2f4db-dc20-4c83-9e37-058576e79472-kube-api-access-s69bw\") pod \"ipp-route-fd614e-kserve-894d7456c-28xz5\" (UID: \"35e2f4db-dc20-4c83-9e37-058576e79472\") " pod="ai-tenant-e2e-ipp-a-fd614e/ipp-route-fd614e-kserve-894d7456c-28xz5"
Sep 16 02:45:54.809525 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:54.809356    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/35e2f4db-dc20-4c83-9e37-058576e79472-model-cache\") pod \"ipp-route-fd614e-kserve-894d7456c-28xz5\" (UID: \"35e2f4db-dc20-4c83-9e37-058576e79472\") " pod="ai-tenant-e2e-ipp-a-fd614e/ipp-route-fd614e-kserve-894d7456c-28xz5"
Sep 16 02:45:54.809525 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:54.809384    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/35e2f4db-dc20-4c83-9e37-058576e79472-tls-certs\") pod \"ipp-route-fd614e-kserve-894d7456c-28xz5\" (UID: \"35e2f4db-dc20-4c83-9e37-058576e79472\") " pod="ai-tenant-e2e-ipp-a-fd614e/ipp-route-fd614e-kserve-894d7456c-28xz5"
Sep 16 02:45:54.809525 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:54.809413    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/35e2f4db-dc20-4c83-9e37-058576e79472-tmp-dir\") pod \"ipp-route-fd614e-kserve-894d7456c-28xz5\" (UID: \"35e2f4db-dc20-4c83-9e37-058576e79472\") " pod="ai-tenant-e2e-ipp-a-fd614e/ipp-route-fd614e-kserve-894d7456c-28xz5"
Sep 16 02:45:54.809525 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:54.809432    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/35e2f4db-dc20-4c83-9e37-058576e79472-dshm\") pod \"ipp-route-fd614e-kserve-894d7456c-28xz5\" (UID: \"35e2f4db-dc20-4c83-9e37-058576e79472\") " pod="ai-tenant-e2e-ipp-a-fd614e/ipp-route-fd614e-kserve-894d7456c-28xz5"
Sep 16 02:45:54.809525 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:54.809460    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/35e2f4db-dc20-4c83-9e37-058576e79472-home\") pod \"ipp-route-fd614e-kserve-894d7456c-28xz5\" (UID: \"35e2f4db-dc20-4c83-9e37-058576e79472\") " pod="ai-tenant-e2e-ipp-a-fd614e/ipp-route-fd614e-kserve-894d7456c-28xz5"
Sep 16 02:45:54.809909 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:54.809887    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/35e2f4db-dc20-4c83-9e37-058576e79472-model-cache\") pod \"ipp-route-fd614e-kserve-894d7456c-28xz5\" (UID: \"35e2f4db-dc20-4c83-9e37-058576e79472\") " pod="ai-tenant-e2e-ipp-a-fd614e/ipp-route-fd614e-kserve-894d7456c-28xz5"
Sep 16 02:45:54.810000 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:54.809930    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/35e2f4db-dc20-4c83-9e37-058576e79472-tmp-dir\") pod \"ipp-route-fd614e-kserve-894d7456c-28xz5\" (UID: \"35e2f4db-dc20-4c83-9e37-058576e79472\") " pod="ai-tenant-e2e-ipp-a-fd614e/ipp-route-fd614e-kserve-894d7456c-28xz5"
Sep 16 02:45:54.810000 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:54.809954    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/35e2f4db-dc20-4c83-9e37-058576e79472-home\") pod \"ipp-route-fd614e-kserve-894d7456c-28xz5\" (UID: \"35e2f4db-dc20-4c83-9e37-058576e79472\") " pod="ai-tenant-e2e-ipp-a-fd614e/ipp-route-fd614e-kserve-894d7456c-28xz5"
Sep 16 02:45:54.811922 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:54.811898    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/35e2f4db-dc20-4c83-9e37-058576e79472-dshm\") pod \"ipp-route-fd614e-kserve-894d7456c-28xz5\" (UID: \"35e2f4db-dc20-4c83-9e37-058576e79472\") " pod="ai-tenant-e2e-ipp-a-fd614e/ipp-route-fd614e-kserve-894d7456c-28xz5"
Sep 16 02:45:54.812185 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:54.812158    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/35e2f4db-dc20-4c83-9e37-058576e79472-tls-certs\") pod \"ipp-route-fd614e-kserve-894d7456c-28xz5\" (UID: \"35e2f4db-dc20-4c83-9e37-058576e79472\") " pod="ai-tenant-e2e-ipp-a-fd614e/ipp-route-fd614e-kserve-894d7456c-28xz5"
Sep 16 02:45:54.817193 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:54.817168    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s69bw\" (UniqueName: \"kubernetes.io/projected/35e2f4db-dc20-4c83-9e37-058576e79472-kube-api-access-s69bw\") pod \"ipp-route-fd614e-kserve-894d7456c-28xz5\" (UID: \"35e2f4db-dc20-4c83-9e37-058576e79472\") " pod="ai-tenant-e2e-ipp-a-fd614e/ipp-route-fd614e-kserve-894d7456c-28xz5"
Sep 16 02:45:54.990581 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:54.990545    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-fd614e/ipp-route-fd614e-kserve-894d7456c-28xz5"
Sep 16 02:45:55.263548 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:55.263518    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-fd614e/ipp-route-fd614e-kserve-894d7456c-28xz5"]
Sep 16 02:45:55.264921 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:45:55.264893    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod35e2f4db_dc20_4c83_9e37_058576e79472.slice/crio-28314c84d14962c8776419fa7d42f9b4de7caa9f25228ca34003278e9ceff002 WatchSource:0}: Error finding container 28314c84d14962c8776419fa7d42f9b4de7caa9f25228ca34003278e9ceff002: Status 404 returned error can't find the container with id 28314c84d14962c8776419fa7d42f9b4de7caa9f25228ca34003278e9ceff002
Sep 16 02:45:55.356886 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:55.356853    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-fd614e/ipp-route-fd614e-kserve-894d7456c-28xz5" event={"ID":"35e2f4db-dc20-4c83-9e37-058576e79472","Type":"ContainerStarted","Data":"d28154072c7d83a8c3f65db292d5c81ad0949305f218d4ef8aec256954c7361f"}
Sep 16 02:45:55.357024 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:55.356899    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-fd614e/ipp-route-fd614e-kserve-894d7456c-28xz5" event={"ID":"35e2f4db-dc20-4c83-9e37-058576e79472","Type":"ContainerStarted","Data":"28314c84d14962c8776419fa7d42f9b4de7caa9f25228ca34003278e9ceff002"}
Sep 16 02:45:55.374242 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:45:55.374183    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-ipp-a-fd614e/ipp-route-fd614e-kserve-894d7456c-28xz5" podStartSLOduration=1.374160307 podStartE2EDuration="1.374160307s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 02:45:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 02:45:55.37311135 +0000 UTC m=+1975.756626600" watchObservedRunningTime="2026-09-16 02:45:55.374160307 +0000 UTC m=+1975.757675554"
Sep 16 02:46:00.329742 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:00.329716    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-e21dae-5bbc9776cc-sl9ks"
Sep 16 02:46:04.991218 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:04.991185    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-ipp-a-fd614e/ipp-route-fd614e-kserve-894d7456c-28xz5"
Sep 16 02:46:04.991651 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:04.991231    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-ipp-a-fd614e/ipp-route-fd614e-kserve-894d7456c-28xz5"
Sep 16 02:46:05.004403 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:05.004378    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-ipp-a-fd614e/ipp-route-fd614e-kserve-894d7456c-28xz5"
Sep 16 02:46:05.425081 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:05.424997    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-ipp-a-fd614e/ipp-route-fd614e-kserve-894d7456c-28xz5"
Sep 16 02:46:10.107618 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:10.107585    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:46:10.843317 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:10.843281    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:46:11.453748 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:11.453706    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:46:12.471209 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:12.471175    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:46:13.124778 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:13.124743    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:46:13.224582 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:13.224551    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:46:13.915868 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:13.915828    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:46:14.626130 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:14.626098    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:46:15.260068 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:15.260026    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:46:16.341021 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:16.340979    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:46:17.223853 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:17.223818    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:46:18.017639 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:18.017606    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:46:18.665599 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:18.665558    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:46:19.325901 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:19.325868    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:46:19.992821 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:19.992773    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:46:20.622695 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:20.622657    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:46:21.342124 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:21.342088    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:46:21.948831 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:21.948793    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:46:29.097194 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:29.097161    2579 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-pre-processing-6cd864bc95-mh4jh" secret="" err="secret \"payload-processing-dockercfg-zzx6j\" not found"
Sep 16 02:46:30.210936 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:30.210902    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-a-fd614e-66fccddb69-57t8f_c1ed1477-844b-4b24-8a3d-910f1eb44b2b/payload-processing/0.log"
Sep 16 02:46:30.552936 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:30.552835    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-d3d202-75996dd58f-kxn4n_c97c7775-4a18-4a8d-b068-8a625fc6a591/payload-processing/0.log"
Sep 16 02:46:30.668905 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:30.668876    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-d3d202-75996dd58f-kxn4n_c97c7775-4a18-4a8d-b068-8a625fc6a591/payload-processing/0.log"
Sep 16 02:46:30.998560 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:30.998530    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:46:31.266426 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:31.266341    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:46:31.603106 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:31.603015    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:46:31.672948 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:31.672917    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-7448c4-7fc96fbc87-gfx84"]
Sep 16 02:46:31.673155 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:31.673132    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-7448c4-7fc96fbc87-gfx84" podUID="36dde4cf-837f-4cb6-b7ca-837c8e41d07e" containerName="payload-processing" containerID="cri-o://0de0cbe6ec83937b6ad52a1782c31446cad703fda17c14908316e7b380aec6f5" gracePeriod=30
Sep 16 02:46:31.676917 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:31.676869    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:46:31.684521 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:31.683218    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-7448c4-ffbd4958b-rhshb"]
Sep 16 02:46:31.684521 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:31.683387    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-7448c4-ffbd4958b-rhshb" podUID="2386c3bc-a0ad-4f43-a1cf-293dfba60ab6" containerName="payload-pre-processing" containerID="cri-o://e6259bb8d6d0dc82c0cbd9e7d4650d893d7aea9dec4cba4f292af5ab7fa15d47" gracePeriod=30
Sep 16 02:46:31.807400 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:31.807356    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:46:31.919581 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:31.919544    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:46:31.977565 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:31.977535    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-7448c4-7fc96fbc87-gfx84"
Sep 16 02:46:31.978950 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:31.978927    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:46:31.980836 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:31.980818    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-7448c4-ffbd4958b-rhshb"
Sep 16 02:46:32.064592 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:32.064552    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2386c3bc-a0ad-4f43-a1cf-293dfba60ab6-plugins-config-volume\") pod \"2386c3bc-a0ad-4f43-a1cf-293dfba60ab6\" (UID: \"2386c3bc-a0ad-4f43-a1cf-293dfba60ab6\") "
Sep 16 02:46:32.064773 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:32.064659    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f7ntr\" (UniqueName: \"kubernetes.io/projected/2386c3bc-a0ad-4f43-a1cf-293dfba60ab6-kube-api-access-f7ntr\") pod \"2386c3bc-a0ad-4f43-a1cf-293dfba60ab6\" (UID: \"2386c3bc-a0ad-4f43-a1cf-293dfba60ab6\") "
Sep 16 02:46:32.064773 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:32.064706    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/36dde4cf-837f-4cb6-b7ca-837c8e41d07e-plugins-config-volume\") pod \"36dde4cf-837f-4cb6-b7ca-837c8e41d07e\" (UID: \"36dde4cf-837f-4cb6-b7ca-837c8e41d07e\") "
Sep 16 02:46:32.064897 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:32.064772    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gm5gl\" (UniqueName: \"kubernetes.io/projected/36dde4cf-837f-4cb6-b7ca-837c8e41d07e-kube-api-access-gm5gl\") pod \"36dde4cf-837f-4cb6-b7ca-837c8e41d07e\" (UID: \"36dde4cf-837f-4cb6-b7ca-837c8e41d07e\") "
Sep 16 02:46:32.064962 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:32.064920    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2386c3bc-a0ad-4f43-a1cf-293dfba60ab6-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2386c3bc-a0ad-4f43-a1cf-293dfba60ab6" (UID: "2386c3bc-a0ad-4f43-a1cf-293dfba60ab6"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 02:46:32.065197 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:32.065170    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2386c3bc-a0ad-4f43-a1cf-293dfba60ab6-plugins-config-volume\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:46:32.065197 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:32.065185    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/36dde4cf-837f-4cb6-b7ca-837c8e41d07e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "36dde4cf-837f-4cb6-b7ca-837c8e41d07e" (UID: "36dde4cf-837f-4cb6-b7ca-837c8e41d07e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 02:46:32.067028 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:32.067001    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2386c3bc-a0ad-4f43-a1cf-293dfba60ab6-kube-api-access-f7ntr" (OuterVolumeSpecName: "kube-api-access-f7ntr") pod "2386c3bc-a0ad-4f43-a1cf-293dfba60ab6" (UID: "2386c3bc-a0ad-4f43-a1cf-293dfba60ab6"). InnerVolumeSpecName "kube-api-access-f7ntr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 02:46:32.067127 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:32.067064    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/36dde4cf-837f-4cb6-b7ca-837c8e41d07e-kube-api-access-gm5gl" (OuterVolumeSpecName: "kube-api-access-gm5gl") pod "36dde4cf-837f-4cb6-b7ca-837c8e41d07e" (UID: "36dde4cf-837f-4cb6-b7ca-837c8e41d07e"). InnerVolumeSpecName "kube-api-access-gm5gl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 02:46:32.165898 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:32.165798    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gm5gl\" (UniqueName: \"kubernetes.io/projected/36dde4cf-837f-4cb6-b7ca-837c8e41d07e-kube-api-access-gm5gl\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:46:32.165898 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:32.165832    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f7ntr\" (UniqueName: \"kubernetes.io/projected/2386c3bc-a0ad-4f43-a1cf-293dfba60ab6-kube-api-access-f7ntr\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:46:32.165898 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:32.165851    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/36dde4cf-837f-4cb6-b7ca-837c8e41d07e-plugins-config-volume\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:46:32.300976 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:32.300942    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:46:32.427930 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:32.427843    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:46:32.568824 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:32.568785    2579 generic.go:356] "Generic (PLEG): container finished" podID="36dde4cf-837f-4cb6-b7ca-837c8e41d07e" containerID="0de0cbe6ec83937b6ad52a1782c31446cad703fda17c14908316e7b380aec6f5" exitCode=0
Sep 16 02:46:32.569017 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:32.568831    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-7448c4-7fc96fbc87-gfx84"
Sep 16 02:46:32.569017 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:32.568899    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-7448c4-7fc96fbc87-gfx84" event={"ID":"36dde4cf-837f-4cb6-b7ca-837c8e41d07e","Type":"ContainerDied","Data":"0de0cbe6ec83937b6ad52a1782c31446cad703fda17c14908316e7b380aec6f5"}
Sep 16 02:46:32.569017 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:32.568963    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-7448c4-7fc96fbc87-gfx84" event={"ID":"36dde4cf-837f-4cb6-b7ca-837c8e41d07e","Type":"ContainerDied","Data":"f69190d0100ac6683f3e02e6f80cbb4c9321dd24ec0fece9b83484f6dc8fe694"}
Sep 16 02:46:32.569017 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:32.568992    2579 scope.go:117] "RemoveContainer" containerID="0de0cbe6ec83937b6ad52a1782c31446cad703fda17c14908316e7b380aec6f5"
Sep 16 02:46:32.570515 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:32.570462    2579 generic.go:356] "Generic (PLEG): container finished" podID="2386c3bc-a0ad-4f43-a1cf-293dfba60ab6" containerID="e6259bb8d6d0dc82c0cbd9e7d4650d893d7aea9dec4cba4f292af5ab7fa15d47" exitCode=0
Sep 16 02:46:32.570627 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:32.570537    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-7448c4-ffbd4958b-rhshb"
Sep 16 02:46:32.570627 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:32.570556    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-7448c4-ffbd4958b-rhshb" event={"ID":"2386c3bc-a0ad-4f43-a1cf-293dfba60ab6","Type":"ContainerDied","Data":"e6259bb8d6d0dc82c0cbd9e7d4650d893d7aea9dec4cba4f292af5ab7fa15d47"}
Sep 16 02:46:32.570627 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:32.570594    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-7448c4-ffbd4958b-rhshb" event={"ID":"2386c3bc-a0ad-4f43-a1cf-293dfba60ab6","Type":"ContainerDied","Data":"f43661f930f074dc0e6ee14324d614254f296137f251c21ffe86e7535e4061d1"}
Sep 16 02:46:32.580724 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:32.580704    2579 scope.go:117] "RemoveContainer" containerID="0de0cbe6ec83937b6ad52a1782c31446cad703fda17c14908316e7b380aec6f5"
Sep 16 02:46:32.581180 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:46:32.581151    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0de0cbe6ec83937b6ad52a1782c31446cad703fda17c14908316e7b380aec6f5\": container with ID starting with 0de0cbe6ec83937b6ad52a1782c31446cad703fda17c14908316e7b380aec6f5 not found: ID does not exist" containerID="0de0cbe6ec83937b6ad52a1782c31446cad703fda17c14908316e7b380aec6f5"
Sep 16 02:46:32.581259 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:32.581182    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0de0cbe6ec83937b6ad52a1782c31446cad703fda17c14908316e7b380aec6f5"} err="failed to get container status \"0de0cbe6ec83937b6ad52a1782c31446cad703fda17c14908316e7b380aec6f5\": rpc error: code = NotFound desc = could not find container \"0de0cbe6ec83937b6ad52a1782c31446cad703fda17c14908316e7b380aec6f5\": container with ID starting with 0de0cbe6ec83937b6ad52a1782c31446cad703fda17c14908316e7b380aec6f5 not found: ID does not exist"
Sep 16 02:46:32.581259 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:32.581206    2579 scope.go:117] "RemoveContainer" containerID="e6259bb8d6d0dc82c0cbd9e7d4650d893d7aea9dec4cba4f292af5ab7fa15d47"
Sep 16 02:46:32.587842 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:32.587816    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-7448c4-ffbd4958b-rhshb"]
Sep 16 02:46:32.591021 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:32.591002    2579 scope.go:117] "RemoveContainer" containerID="e6259bb8d6d0dc82c0cbd9e7d4650d893d7aea9dec4cba4f292af5ab7fa15d47"
Sep 16 02:46:32.591305 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:46:32.591289    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e6259bb8d6d0dc82c0cbd9e7d4650d893d7aea9dec4cba4f292af5ab7fa15d47\": container with ID starting with e6259bb8d6d0dc82c0cbd9e7d4650d893d7aea9dec4cba4f292af5ab7fa15d47 not found: ID does not exist" containerID="e6259bb8d6d0dc82c0cbd9e7d4650d893d7aea9dec4cba4f292af5ab7fa15d47"
Sep 16 02:46:32.591370 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:32.591314    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e6259bb8d6d0dc82c0cbd9e7d4650d893d7aea9dec4cba4f292af5ab7fa15d47"} err="failed to get container status \"e6259bb8d6d0dc82c0cbd9e7d4650d893d7aea9dec4cba4f292af5ab7fa15d47\": rpc error: code = NotFound desc = could not find container \"e6259bb8d6d0dc82c0cbd9e7d4650d893d7aea9dec4cba4f292af5ab7fa15d47\": container with ID starting with e6259bb8d6d0dc82c0cbd9e7d4650d893d7aea9dec4cba4f292af5ab7fa15d47 not found: ID does not exist"
Sep 16 02:46:32.591638 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:32.591621    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-7448c4-ffbd4958b-rhshb"]
Sep 16 02:46:32.599592 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:32.599569    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-7448c4-7fc96fbc87-gfx84"]
Sep 16 02:46:32.602732 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:32.602712    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-7448c4-7fc96fbc87-gfx84"]
Sep 16 02:46:33.066573 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:33.066483    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:46:33.997309 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:33.997270    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:46:34.103560 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:34.103533    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2386c3bc-a0ad-4f43-a1cf-293dfba60ab6" path="/var/lib/kubelet/pods/2386c3bc-a0ad-4f43-a1cf-293dfba60ab6/volumes"
Sep 16 02:46:34.103934 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:34.103920    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="36dde4cf-837f-4cb6-b7ca-837c8e41d07e" path="/var/lib/kubelet/pods/36dde4cf-837f-4cb6-b7ca-837c8e41d07e/volumes"
Sep 16 02:46:34.356105 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:34.356012    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:46:34.450574 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:34.450543    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:46:34.735036 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:34.735003    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:46:35.657759 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:35.657725    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:46:36.334623 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:36.334590    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:46:37.022636 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:37.022601    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:46:37.794487 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:37.794434    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:46:37.897413 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:37.897383    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:46:38.041911 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:38.041882    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:46:38.186259 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:38.186218    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:46:38.285742 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:38.285700    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:46:38.293061 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:38.293030    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-5b136e-569bfb6474-s9pt6"]
Sep 16 02:46:38.293506 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:38.293463    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36dde4cf-837f-4cb6-b7ca-837c8e41d07e" containerName="payload-processing"
Sep 16 02:46:38.293506 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:38.293505    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="36dde4cf-837f-4cb6-b7ca-837c8e41d07e" containerName="payload-processing"
Sep 16 02:46:38.293695 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:38.293532    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2386c3bc-a0ad-4f43-a1cf-293dfba60ab6" containerName="payload-pre-processing"
Sep 16 02:46:38.293695 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:38.293538    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="2386c3bc-a0ad-4f43-a1cf-293dfba60ab6" containerName="payload-pre-processing"
Sep 16 02:46:38.293695 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:38.293606    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="36dde4cf-837f-4cb6-b7ca-837c8e41d07e" containerName="payload-processing"
Sep 16 02:46:38.293695 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:38.293614    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="2386c3bc-a0ad-4f43-a1cf-293dfba60ab6" containerName="payload-pre-processing"
Sep 16 02:46:38.296767 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:38.296750    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-5b136e-569bfb6474-s9pt6"
Sep 16 02:46:38.298893 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:38.298875    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-5b136e\""
Sep 16 02:46:38.298999 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:38.298944    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-5b136e-dockercfg-nrbw6\""
Sep 16 02:46:38.306308 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:38.306283    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-5b136e-569bfb6474-s9pt6"]
Sep 16 02:46:38.324979 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:38.324948    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xb4rk\" (UniqueName: \"kubernetes.io/projected/8c059a3e-25cd-4c4d-9822-3ee167badfc1-kube-api-access-xb4rk\") pod \"payload-pre-processing-e2e-ipp-cleanup-5b136e-569bfb6474-s9pt6\" (UID: \"8c059a3e-25cd-4c4d-9822-3ee167badfc1\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-5b136e-569bfb6474-s9pt6"
Sep 16 02:46:38.325099 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:38.325037    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8c059a3e-25cd-4c4d-9822-3ee167badfc1-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-5b136e-569bfb6474-s9pt6\" (UID: \"8c059a3e-25cd-4c4d-9822-3ee167badfc1\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-5b136e-569bfb6474-s9pt6"
Sep 16 02:46:38.326883 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:38.326864    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-5b136e-5f4c46c655-w29bt"]
Sep 16 02:46:38.330331 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:38.330315    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-5b136e-5f4c46c655-w29bt"
Sep 16 02:46:38.338565 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:38.338542    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-5b136e-5f4c46c655-w29bt"]
Sep 16 02:46:38.426524 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:38.426483    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c929e56e-b574-408c-8794-61850900ff63-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-5b136e-5f4c46c655-w29bt\" (UID: \"c929e56e-b574-408c-8794-61850900ff63\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-5b136e-5f4c46c655-w29bt"
Sep 16 02:46:38.426672 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:38.426536    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mm7kw\" (UniqueName: \"kubernetes.io/projected/c929e56e-b574-408c-8794-61850900ff63-kube-api-access-mm7kw\") pod \"payload-processing-e2e-ipp-cleanup-5b136e-5f4c46c655-w29bt\" (UID: \"c929e56e-b574-408c-8794-61850900ff63\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-5b136e-5f4c46c655-w29bt"
Sep 16 02:46:38.426672 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:38.426575    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xb4rk\" (UniqueName: \"kubernetes.io/projected/8c059a3e-25cd-4c4d-9822-3ee167badfc1-kube-api-access-xb4rk\") pod \"payload-pre-processing-e2e-ipp-cleanup-5b136e-569bfb6474-s9pt6\" (UID: \"8c059a3e-25cd-4c4d-9822-3ee167badfc1\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-5b136e-569bfb6474-s9pt6"
Sep 16 02:46:38.426672 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:38.426637    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8c059a3e-25cd-4c4d-9822-3ee167badfc1-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-5b136e-569bfb6474-s9pt6\" (UID: \"8c059a3e-25cd-4c4d-9822-3ee167badfc1\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-5b136e-569bfb6474-s9pt6"
Sep 16 02:46:38.427353 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:38.427332    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8c059a3e-25cd-4c4d-9822-3ee167badfc1-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-5b136e-569bfb6474-s9pt6\" (UID: \"8c059a3e-25cd-4c4d-9822-3ee167badfc1\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-5b136e-569bfb6474-s9pt6"
Sep 16 02:46:38.433919 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:38.433892    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xb4rk\" (UniqueName: \"kubernetes.io/projected/8c059a3e-25cd-4c4d-9822-3ee167badfc1-kube-api-access-xb4rk\") pod \"payload-pre-processing-e2e-ipp-cleanup-5b136e-569bfb6474-s9pt6\" (UID: \"8c059a3e-25cd-4c4d-9822-3ee167badfc1\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-5b136e-569bfb6474-s9pt6"
Sep 16 02:46:38.527771 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:38.527689    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c929e56e-b574-408c-8794-61850900ff63-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-5b136e-5f4c46c655-w29bt\" (UID: \"c929e56e-b574-408c-8794-61850900ff63\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-5b136e-5f4c46c655-w29bt"
Sep 16 02:46:38.527771 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:38.527744    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mm7kw\" (UniqueName: \"kubernetes.io/projected/c929e56e-b574-408c-8794-61850900ff63-kube-api-access-mm7kw\") pod \"payload-processing-e2e-ipp-cleanup-5b136e-5f4c46c655-w29bt\" (UID: \"c929e56e-b574-408c-8794-61850900ff63\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-5b136e-5f4c46c655-w29bt"
Sep 16 02:46:38.528359 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:38.528341    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c929e56e-b574-408c-8794-61850900ff63-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-5b136e-5f4c46c655-w29bt\" (UID: \"c929e56e-b574-408c-8794-61850900ff63\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-5b136e-5f4c46c655-w29bt"
Sep 16 02:46:38.536133 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:38.536107    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mm7kw\" (UniqueName: \"kubernetes.io/projected/c929e56e-b574-408c-8794-61850900ff63-kube-api-access-mm7kw\") pod \"payload-processing-e2e-ipp-cleanup-5b136e-5f4c46c655-w29bt\" (UID: \"c929e56e-b574-408c-8794-61850900ff63\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-5b136e-5f4c46c655-w29bt"
Sep 16 02:46:38.540684 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:38.540658    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:46:38.607687 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:38.607662    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-5b136e-569bfb6474-s9pt6"
Sep 16 02:46:38.642010 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:38.641980    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-5b136e-5f4c46c655-w29bt"
Sep 16 02:46:38.886815 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:38.886787    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-5b136e-569bfb6474-s9pt6"]
Sep 16 02:46:38.889042 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:46:38.889010    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c059a3e_25cd_4c4d_9822_3ee167badfc1.slice/crio-8cb653cfa85906e5895b90b402cb1af3dc822e5bbf2c3534a47e9548ac977667 WatchSource:0}: Error finding container 8cb653cfa85906e5895b90b402cb1af3dc822e5bbf2c3534a47e9548ac977667: Status 404 returned error can't find the container with id 8cb653cfa85906e5895b90b402cb1af3dc822e5bbf2c3534a47e9548ac977667
Sep 16 02:46:38.926451 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:38.926401    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-5b136e-5f4c46c655-w29bt"]
Sep 16 02:46:39.192398 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:39.192362    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:46:39.292492 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:39.292443    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:46:39.405457 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:39.405425    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:46:39.505647 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:39.505565    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:46:39.614458 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:39.614421    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-5b136e-569bfb6474-s9pt6" event={"ID":"8c059a3e-25cd-4c4d-9822-3ee167badfc1","Type":"ContainerStarted","Data":"f7387ae13d2f094945c6a3ff52717fa3d1e72ad20c25b750b008f7b7de1393bc"}
Sep 16 02:46:39.614695 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:39.614496    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-5b136e-569bfb6474-s9pt6" event={"ID":"8c059a3e-25cd-4c4d-9822-3ee167badfc1","Type":"ContainerStarted","Data":"8cb653cfa85906e5895b90b402cb1af3dc822e5bbf2c3534a47e9548ac977667"}
Sep 16 02:46:39.614695 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:39.614529    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-5b136e-569bfb6474-s9pt6"
Sep 16 02:46:39.616165 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:39.616138    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-5b136e-5f4c46c655-w29bt" event={"ID":"c929e56e-b574-408c-8794-61850900ff63","Type":"ContainerStarted","Data":"6dfc4dda31e0a441dcaa672d8b22ed22c6b9d7062728bc701d21ebe6cf92da43"}
Sep 16 02:46:39.616262 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:39.616189    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-5b136e-5f4c46c655-w29bt" event={"ID":"c929e56e-b574-408c-8794-61850900ff63","Type":"ContainerStarted","Data":"823ef249efdbd8a2c49f2260cc8d5ef880a6e9e3ae827fb11e7b50399b049f3a"}
Sep 16 02:46:39.616262 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:39.616209    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-5b136e-5f4c46c655-w29bt"
Sep 16 02:46:39.632387 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:39.632343    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-5b136e-569bfb6474-s9pt6" podStartSLOduration=1.6323302910000002 podStartE2EDuration="1.632330291s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 02:46:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 02:46:39.630194103 +0000 UTC m=+2020.013709349" watchObservedRunningTime="2026-09-16 02:46:39.632330291 +0000 UTC m=+2020.015845535"
Sep 16 02:46:39.647315 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:39.647262    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-5b136e-5f4c46c655-w29bt" podStartSLOduration=1.647244317 podStartE2EDuration="1.647244317s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 02:46:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 02:46:39.645794137 +0000 UTC m=+2020.029309385" watchObservedRunningTime="2026-09-16 02:46:39.647244317 +0000 UTC m=+2020.030759566"
Sep 16 02:46:39.788118 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:39.788029    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-e21dae-kfnfc"]
Sep 16 02:46:39.792348 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:39.792329    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-e21dae-kfnfc"
Sep 16 02:46:39.794784 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:39.794737    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-jpd4f\""
Sep 16 02:46:39.798371 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:39.798347    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-e21dae-kfnfc"]
Sep 16 02:46:39.841819 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:39.841784    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f66d4e4-5692-4ea2-807b-9a984bafa229-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-e21dae-kfnfc\" (UID: \"1f66d4e4-5692-4ea2-807b-9a984bafa229\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-e21dae-kfnfc"
Sep 16 02:46:39.942672 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:39.942631    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f66d4e4-5692-4ea2-807b-9a984bafa229-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-e21dae-kfnfc\" (UID: \"1f66d4e4-5692-4ea2-807b-9a984bafa229\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-e21dae-kfnfc"
Sep 16 02:46:39.943360 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:39.943327    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f66d4e4-5692-4ea2-807b-9a984bafa229-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-e21dae-kfnfc\" (UID: \"1f66d4e4-5692-4ea2-807b-9a984bafa229\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-e21dae-kfnfc"
Sep 16 02:46:40.106277 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:40.106200    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-e21dae-kfnfc"
Sep 16 02:46:40.170594 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:40.169519    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:46:40.334182 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:40.334149    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:46:40.388540 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:40.388517    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-e21dae-kfnfc"]
Sep 16 02:46:40.390568 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:46:40.390536    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f66d4e4_5692_4ea2_807b_9a984bafa229.slice/crio-4c2882d90c6dc711c0eb4adfc57912ab4ebee461f0300065e479db030f2c9650 WatchSource:0}: Error finding container 4c2882d90c6dc711c0eb4adfc57912ab4ebee461f0300065e479db030f2c9650: Status 404 returned error can't find the container with id 4c2882d90c6dc711c0eb4adfc57912ab4ebee461f0300065e479db030f2c9650
Sep 16 02:46:40.623723 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:40.623685    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-e21dae-kfnfc" event={"ID":"1f66d4e4-5692-4ea2-807b-9a984bafa229","Type":"ContainerStarted","Data":"4c2882d90c6dc711c0eb4adfc57912ab4ebee461f0300065e479db030f2c9650"}
Sep 16 02:46:41.144777 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:41.144733    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:46:41.630822 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:41.630786    2579 generic.go:356] "Generic (PLEG): container finished" podID="1f66d4e4-5692-4ea2-807b-9a984bafa229" containerID="52156b8ad30b96431d4551cfa1aa4900d1918d93c64f3b7c5ca280df60db7d37" exitCode=0
Sep 16 02:46:41.631188 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:41.630872    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-e21dae-kfnfc" event={"ID":"1f66d4e4-5692-4ea2-807b-9a984bafa229","Type":"ContainerDied","Data":"52156b8ad30b96431d4551cfa1aa4900d1918d93c64f3b7c5ca280df60db7d37"}
Sep 16 02:46:41.729414 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:41.729373    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:46:42.367408 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:42.367374    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:46:42.770962 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:42.770930    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-e21dae-kfnfc"
Sep 16 02:46:42.871651 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:42.871624    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f66d4e4-5692-4ea2-807b-9a984bafa229-maas-api-service-ca\") pod \"1f66d4e4-5692-4ea2-807b-9a984bafa229\" (UID: \"1f66d4e4-5692-4ea2-807b-9a984bafa229\") "
Sep 16 02:46:42.872014 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:42.871989    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f66d4e4-5692-4ea2-807b-9a984bafa229-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "1f66d4e4-5692-4ea2-807b-9a984bafa229" (UID: "1f66d4e4-5692-4ea2-807b-9a984bafa229"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 02:46:42.972913 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:42.972875    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f66d4e4-5692-4ea2-807b-9a984bafa229-maas-api-service-ca\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:46:43.125966 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:43.125923    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:46:43.643177 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:43.643147    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-e21dae-kfnfc"
Sep 16 02:46:43.643360 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:43.643178    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-e21dae-kfnfc" event={"ID":"1f66d4e4-5692-4ea2-807b-9a984bafa229","Type":"ContainerDied","Data":"4c2882d90c6dc711c0eb4adfc57912ab4ebee461f0300065e479db030f2c9650"}
Sep 16 02:46:43.643360 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:43.643231    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4c2882d90c6dc711c0eb4adfc57912ab4ebee461f0300065e479db030f2c9650"
Sep 16 02:46:43.831069 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:43.831035    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:46:44.452604 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:44.452563    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:46:45.114426 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:45.114386    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:46:45.842116 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:45.842081    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:46:50.626086 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:50.626062    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-5b136e-569bfb6474-s9pt6"
Sep 16 02:46:50.626486 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:50.626133    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-5b136e-5f4c46c655-w29bt"
Sep 16 02:46:55.050080 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:55.050049    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:46:55.151833 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:55.151798    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:46:55.222777 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:55.222739    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:46:55.248312 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:55.248272    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-e21dae-5bbc9776cc-sl9ks"]
Sep 16 02:46:55.248561 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:55.248528    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-e21dae-5bbc9776cc-sl9ks" podUID="f3694bbf-a521-4c14-a894-28075b871d83" containerName="payload-processing" containerID="cri-o://352cf5c2933a2a90981146e4920a06ebe80eb7c9bcc0bf42cf329809fcad4f86" gracePeriod=30
Sep 16 02:46:55.288877 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:55.288836    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:46:55.389822 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:55.389782    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:46:55.399264 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:55.399216    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-e21dae-kfnfc"]
Sep 16 02:46:55.402855 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:55.402829    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-e21dae-kfnfc"]
Sep 16 02:46:55.444185 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:55.444157    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:46:55.515100 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:55.515071    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-e21dae-5bbc9776cc-sl9ks"
Sep 16 02:46:55.592996 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:55.592968    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sx6hc\" (UniqueName: \"kubernetes.io/projected/f3694bbf-a521-4c14-a894-28075b871d83-kube-api-access-sx6hc\") pod \"f3694bbf-a521-4c14-a894-28075b871d83\" (UID: \"f3694bbf-a521-4c14-a894-28075b871d83\") "
Sep 16 02:46:55.593172 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:55.593018    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f3694bbf-a521-4c14-a894-28075b871d83-plugins-config-volume\") pod \"f3694bbf-a521-4c14-a894-28075b871d83\" (UID: \"f3694bbf-a521-4c14-a894-28075b871d83\") "
Sep 16 02:46:55.593448 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:55.593423    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f3694bbf-a521-4c14-a894-28075b871d83-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f3694bbf-a521-4c14-a894-28075b871d83" (UID: "f3694bbf-a521-4c14-a894-28075b871d83"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 02:46:55.595343 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:55.595317    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f3694bbf-a521-4c14-a894-28075b871d83-kube-api-access-sx6hc" (OuterVolumeSpecName: "kube-api-access-sx6hc") pod "f3694bbf-a521-4c14-a894-28075b871d83" (UID: "f3694bbf-a521-4c14-a894-28075b871d83"). InnerVolumeSpecName "kube-api-access-sx6hc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 02:46:55.694440 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:55.694412    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sx6hc\" (UniqueName: \"kubernetes.io/projected/f3694bbf-a521-4c14-a894-28075b871d83-kube-api-access-sx6hc\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:46:55.694440 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:55.694439    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f3694bbf-a521-4c14-a894-28075b871d83-plugins-config-volume\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:46:55.711161 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:55.711128    2579 generic.go:356] "Generic (PLEG): container finished" podID="f3694bbf-a521-4c14-a894-28075b871d83" containerID="352cf5c2933a2a90981146e4920a06ebe80eb7c9bcc0bf42cf329809fcad4f86" exitCode=0
Sep 16 02:46:55.711314 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:55.711177    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-e21dae-5bbc9776cc-sl9ks"
Sep 16 02:46:55.711314 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:55.711203    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-e21dae-5bbc9776cc-sl9ks" event={"ID":"f3694bbf-a521-4c14-a894-28075b871d83","Type":"ContainerDied","Data":"352cf5c2933a2a90981146e4920a06ebe80eb7c9bcc0bf42cf329809fcad4f86"}
Sep 16 02:46:55.711314 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:55.711255    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-e21dae-5bbc9776cc-sl9ks" event={"ID":"f3694bbf-a521-4c14-a894-28075b871d83","Type":"ContainerDied","Data":"8180bfe15a1e5c79da8bd96e8757bb6931315fc17ac11750d3cc0d123d8aeea6"}
Sep 16 02:46:55.711314 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:55.711277    2579 scope.go:117] "RemoveContainer" containerID="352cf5c2933a2a90981146e4920a06ebe80eb7c9bcc0bf42cf329809fcad4f86"
Sep 16 02:46:55.723576 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:55.723522    2579 scope.go:117] "RemoveContainer" containerID="352cf5c2933a2a90981146e4920a06ebe80eb7c9bcc0bf42cf329809fcad4f86"
Sep 16 02:46:55.723786 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:46:55.723768    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"352cf5c2933a2a90981146e4920a06ebe80eb7c9bcc0bf42cf329809fcad4f86\": container with ID starting with 352cf5c2933a2a90981146e4920a06ebe80eb7c9bcc0bf42cf329809fcad4f86 not found: ID does not exist" containerID="352cf5c2933a2a90981146e4920a06ebe80eb7c9bcc0bf42cf329809fcad4f86"
Sep 16 02:46:55.723845 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:55.723794    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"352cf5c2933a2a90981146e4920a06ebe80eb7c9bcc0bf42cf329809fcad4f86"} err="failed to get container status \"352cf5c2933a2a90981146e4920a06ebe80eb7c9bcc0bf42cf329809fcad4f86\": rpc error: code = NotFound desc = could not find container \"352cf5c2933a2a90981146e4920a06ebe80eb7c9bcc0bf42cf329809fcad4f86\": container with ID starting with 352cf5c2933a2a90981146e4920a06ebe80eb7c9bcc0bf42cf329809fcad4f86 not found: ID does not exist"
Sep 16 02:46:55.734362 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:55.734340    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-e21dae-5bbc9776cc-sl9ks"]
Sep 16 02:46:55.738244 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:55.738225    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-e21dae-5bbc9776cc-sl9ks"]
Sep 16 02:46:55.826693 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:55.826662    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:46:55.927293 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:55.927258    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:46:56.103763 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:56.103681    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1f66d4e4-5692-4ea2-807b-9a984bafa229" path="/var/lib/kubelet/pods/1f66d4e4-5692-4ea2-807b-9a984bafa229/volumes"
Sep 16 02:46:56.104230 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:56.104211    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f3694bbf-a521-4c14-a894-28075b871d83" path="/var/lib/kubelet/pods/f3694bbf-a521-4c14-a894-28075b871d83/volumes"
Sep 16 02:46:56.715528 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:56.715458    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:46:57.337853 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:57.337812    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:46:57.937927 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:57.937886    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:46:58.721494 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:58.721439    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:46:59.330725 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:59.330691    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:46:59.935279 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:46:59.935241    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:47:01.435060 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:01.435025    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:47:01.981486 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:01.981446    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:47:02.085036 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:02.084998    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:47:02.182396 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:02.182358    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:47:02.284750 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:02.284658    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:47:02.644534 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:02.644446    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-bd0fb8-gc2hj"]
Sep 16 02:47:02.645024 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:02.645005    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f66d4e4-5692-4ea2-807b-9a984bafa229" containerName="revoke-keys"
Sep 16 02:47:02.645104 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:02.645029    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f66d4e4-5692-4ea2-807b-9a984bafa229" containerName="revoke-keys"
Sep 16 02:47:02.645104 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:02.645054    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f3694bbf-a521-4c14-a894-28075b871d83" containerName="payload-processing"
Sep 16 02:47:02.645104 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:02.645059    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="f3694bbf-a521-4c14-a894-28075b871d83" containerName="payload-processing"
Sep 16 02:47:02.645255 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:02.645135    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="f3694bbf-a521-4c14-a894-28075b871d83" containerName="payload-processing"
Sep 16 02:47:02.645255 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:02.645143    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f66d4e4-5692-4ea2-807b-9a984bafa229" containerName="revoke-keys"
Sep 16 02:47:02.649975 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:02.649957    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-bd0fb8-gc2hj"
Sep 16 02:47:02.652690 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:02.652666    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-jpd4f\""
Sep 16 02:47:02.654879 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:02.654852    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-bd0fb8-gc2hj"]
Sep 16 02:47:02.708435 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:02.708397    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:47:02.761010 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:02.760977    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4a035cdb-0306-4400-b8fe-5f29fdd40f5c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w1-bd0fb8-gc2hj\" (UID: \"4a035cdb-0306-4400-b8fe-5f29fdd40f5c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-bd0fb8-gc2hj"
Sep 16 02:47:02.861607 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:02.861570    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4a035cdb-0306-4400-b8fe-5f29fdd40f5c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w1-bd0fb8-gc2hj\" (UID: \"4a035cdb-0306-4400-b8fe-5f29fdd40f5c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-bd0fb8-gc2hj"
Sep 16 02:47:02.862191 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:02.862173    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4a035cdb-0306-4400-b8fe-5f29fdd40f5c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w1-bd0fb8-gc2hj\" (UID: \"4a035cdb-0306-4400-b8fe-5f29fdd40f5c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-bd0fb8-gc2hj"
Sep 16 02:47:02.966081 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:02.966052    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-bd0fb8-gc2hj"
Sep 16 02:47:03.237361 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:03.237314    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-bd0fb8-gc2hj"]
Sep 16 02:47:03.239964 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:47:03.239927    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a035cdb_0306_4400_b8fe_5f29fdd40f5c.slice/crio-4e86b209595d7a1280257a19510416f1cc1c8d47900946e4c7b4a0f8b766b9d0 WatchSource:0}: Error finding container 4e86b209595d7a1280257a19510416f1cc1c8d47900946e4c7b4a0f8b766b9d0: Status 404 returned error can't find the container with id 4e86b209595d7a1280257a19510416f1cc1c8d47900946e4c7b4a0f8b766b9d0
Sep 16 02:47:03.617702 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:03.617674    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:47:03.760129 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:03.760093    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-bd0fb8-gc2hj" event={"ID":"4a035cdb-0306-4400-b8fe-5f29fdd40f5c","Type":"ContainerStarted","Data":"8f9a7d22d51294604cf5f2c1ac6357285db4a5beb528430c641b7553459a4ff9"}
Sep 16 02:47:03.760568 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:03.760148    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-bd0fb8-gc2hj" event={"ID":"4a035cdb-0306-4400-b8fe-5f29fdd40f5c","Type":"ContainerStarted","Data":"4e86b209595d7a1280257a19510416f1cc1c8d47900946e4c7b4a0f8b766b9d0"}
Sep 16 02:47:03.775874 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:03.775832    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-bd0fb8-gc2hj" podStartSLOduration=1.775816866 podStartE2EDuration="1.775816866s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 02:47:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 02:47:03.77314116 +0000 UTC m=+2044.156656408" watchObservedRunningTime="2026-09-16 02:47:03.775816866 +0000 UTC m=+2044.159332111"
Sep 16 02:47:04.244490 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:04.244384    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:47:04.539668 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:04.539565    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:47:04.769150 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:04.769114    2579 generic.go:356] "Generic (PLEG): container finished" podID="4a035cdb-0306-4400-b8fe-5f29fdd40f5c" containerID="8f9a7d22d51294604cf5f2c1ac6357285db4a5beb528430c641b7553459a4ff9" exitCode=0
Sep 16 02:47:04.769539 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:04.769158    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-bd0fb8-gc2hj" event={"ID":"4a035cdb-0306-4400-b8fe-5f29fdd40f5c","Type":"ContainerDied","Data":"8f9a7d22d51294604cf5f2c1ac6357285db4a5beb528430c641b7553459a4ff9"}
Sep 16 02:47:04.858381 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:04.858285    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:47:05.006109 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:05.006077    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:47:05.690788 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:05.690757    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:47:05.919890 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:05.919857    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-bd0fb8-gc2hj"
Sep 16 02:47:06.093574 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:06.093488    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4a035cdb-0306-4400-b8fe-5f29fdd40f5c-maas-api-service-ca\") pod \"4a035cdb-0306-4400-b8fe-5f29fdd40f5c\" (UID: \"4a035cdb-0306-4400-b8fe-5f29fdd40f5c\") "
Sep 16 02:47:06.093874 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:06.093851    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4a035cdb-0306-4400-b8fe-5f29fdd40f5c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4a035cdb-0306-4400-b8fe-5f29fdd40f5c" (UID: "4a035cdb-0306-4400-b8fe-5f29fdd40f5c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 02:47:06.194555 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:06.194519    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4a035cdb-0306-4400-b8fe-5f29fdd40f5c-maas-api-service-ca\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:47:06.449183 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:06.449146    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:47:06.784701 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:06.784620    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-bd0fb8-gc2hj"
Sep 16 02:47:06.784701 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:06.784639    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-bd0fb8-gc2hj" event={"ID":"4a035cdb-0306-4400-b8fe-5f29fdd40f5c","Type":"ContainerDied","Data":"4e86b209595d7a1280257a19510416f1cc1c8d47900946e4c7b4a0f8b766b9d0"}
Sep 16 02:47:06.784885 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:06.784721    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4e86b209595d7a1280257a19510416f1cc1c8d47900946e4c7b4a0f8b766b9d0"
Sep 16 02:47:07.332496 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:07.332440    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:47:08.112730 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:08.112699    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:47:08.718763 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:08.718732    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:47:09.470711 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:09.470675    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:47:10.134081 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:10.134040    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:47:10.776728 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:10.776689    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:47:12.784784 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:12.784750    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:47:13.327100 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:13.327065    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:47:13.332144 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:13.332116    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-bd0fb8-29825445-hm57f"]
Sep 16 02:47:13.335132 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:13.335109    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-bd0fb8-29825445-hm57f"]
Sep 16 02:47:13.400039 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:13.400006    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:47:13.456351 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:13.456311    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:47:13.505516 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:13.505460    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-bd0fb8-gc2hj"]
Sep 16 02:47:13.509160 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:13.509131    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-bd0fb8-gc2hj"]
Sep 16 02:47:13.516896 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:13.516858    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:47:13.863064 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:13.863025    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:47:13.963496 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:13.963429    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:47:14.103034 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:14.103000    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4a035cdb-0306-4400-b8fe-5f29fdd40f5c" path="/var/lib/kubelet/pods/4a035cdb-0306-4400-b8fe-5f29fdd40f5c/volumes"
Sep 16 02:47:14.103311 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:14.103296    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cfbb7371-86fe-44df-bcb1-de3cf034b44e" path="/var/lib/kubelet/pods/cfbb7371-86fe-44df-bcb1-de3cf034b44e/volumes"
Sep 16 02:47:14.602095 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:14.602059    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:47:15.337780 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:15.337745    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:47:16.017338 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:16.017302    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:47:16.730830 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:16.730789    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:47:17.349929 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:17.349881    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:47:17.887027 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:17.886996    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:47:17.909446 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:17.901969    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-5b136e-5f4c46c655-w29bt"]
Sep 16 02:47:17.909446 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:17.902281    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-5b136e-5f4c46c655-w29bt" podUID="c929e56e-b574-408c-8794-61850900ff63" containerName="payload-processing" containerID="cri-o://6dfc4dda31e0a441dcaa672d8b22ed22c6b9d7062728bc701d21ebe6cf92da43" gracePeriod=30
Sep 16 02:47:17.915190 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:17.915153    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-5b136e-569bfb6474-s9pt6"]
Sep 16 02:47:17.915827 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:17.915741    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-5b136e-569bfb6474-s9pt6" podUID="8c059a3e-25cd-4c4d-9822-3ee167badfc1" containerName="payload-pre-processing" containerID="cri-o://f7387ae13d2f094945c6a3ff52717fa3d1e72ad20c25b750b008f7b7de1393bc" gracePeriod=30
Sep 16 02:47:17.942325 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:17.942299    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:47:18.042943 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:18.042897    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:47:18.185378 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:18.185350    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-5b136e-5f4c46c655-w29bt"
Sep 16 02:47:18.207052 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:18.207032    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mm7kw\" (UniqueName: \"kubernetes.io/projected/c929e56e-b574-408c-8794-61850900ff63-kube-api-access-mm7kw\") pod \"c929e56e-b574-408c-8794-61850900ff63\" (UID: \"c929e56e-b574-408c-8794-61850900ff63\") "
Sep 16 02:47:18.207147 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:18.207130    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c929e56e-b574-408c-8794-61850900ff63-plugins-config-volume\") pod \"c929e56e-b574-408c-8794-61850900ff63\" (UID: \"c929e56e-b574-408c-8794-61850900ff63\") "
Sep 16 02:47:18.207523 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:18.207497    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c929e56e-b574-408c-8794-61850900ff63-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c929e56e-b574-408c-8794-61850900ff63" (UID: "c929e56e-b574-408c-8794-61850900ff63"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 02:47:18.209288 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:18.209266    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c929e56e-b574-408c-8794-61850900ff63-kube-api-access-mm7kw" (OuterVolumeSpecName: "kube-api-access-mm7kw") pod "c929e56e-b574-408c-8794-61850900ff63" (UID: "c929e56e-b574-408c-8794-61850900ff63"). InnerVolumeSpecName "kube-api-access-mm7kw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 02:47:18.212434 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:18.212415    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-5b136e-569bfb6474-s9pt6"
Sep 16 02:47:18.254390 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:18.254292    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:47:18.308611 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:18.308571    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xb4rk\" (UniqueName: \"kubernetes.io/projected/8c059a3e-25cd-4c4d-9822-3ee167badfc1-kube-api-access-xb4rk\") pod \"8c059a3e-25cd-4c4d-9822-3ee167badfc1\" (UID: \"8c059a3e-25cd-4c4d-9822-3ee167badfc1\") "
Sep 16 02:47:18.308787 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:18.308678    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8c059a3e-25cd-4c4d-9822-3ee167badfc1-plugins-config-volume\") pod \"8c059a3e-25cd-4c4d-9822-3ee167badfc1\" (UID: \"8c059a3e-25cd-4c4d-9822-3ee167badfc1\") "
Sep 16 02:47:18.308973 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:18.308956    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c929e56e-b574-408c-8794-61850900ff63-plugins-config-volume\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:47:18.309045 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:18.308978    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mm7kw\" (UniqueName: \"kubernetes.io/projected/c929e56e-b574-408c-8794-61850900ff63-kube-api-access-mm7kw\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:47:18.309102 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:18.309052    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8c059a3e-25cd-4c4d-9822-3ee167badfc1-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8c059a3e-25cd-4c4d-9822-3ee167badfc1" (UID: "8c059a3e-25cd-4c4d-9822-3ee167badfc1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 02:47:18.310608 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:18.310586    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8c059a3e-25cd-4c4d-9822-3ee167badfc1-kube-api-access-xb4rk" (OuterVolumeSpecName: "kube-api-access-xb4rk") pod "8c059a3e-25cd-4c4d-9822-3ee167badfc1" (UID: "8c059a3e-25cd-4c4d-9822-3ee167badfc1"). InnerVolumeSpecName "kube-api-access-xb4rk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 02:47:18.409877 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:18.409836    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8c059a3e-25cd-4c4d-9822-3ee167badfc1-plugins-config-volume\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:47:18.409877 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:18.409875    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xb4rk\" (UniqueName: \"kubernetes.io/projected/8c059a3e-25cd-4c4d-9822-3ee167badfc1-kube-api-access-xb4rk\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:47:18.446167 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:18.446124    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:47:18.546180 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:18.546150    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:47:18.850768 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:18.850660    2579 generic.go:356] "Generic (PLEG): container finished" podID="8c059a3e-25cd-4c4d-9822-3ee167badfc1" containerID="f7387ae13d2f094945c6a3ff52717fa3d1e72ad20c25b750b008f7b7de1393bc" exitCode=0
Sep 16 02:47:18.850768 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:18.850721    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-5b136e-569bfb6474-s9pt6"
Sep 16 02:47:18.850768 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:18.850754    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-5b136e-569bfb6474-s9pt6" event={"ID":"8c059a3e-25cd-4c4d-9822-3ee167badfc1","Type":"ContainerDied","Data":"f7387ae13d2f094945c6a3ff52717fa3d1e72ad20c25b750b008f7b7de1393bc"}
Sep 16 02:47:18.851049 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:18.850821    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-5b136e-569bfb6474-s9pt6" event={"ID":"8c059a3e-25cd-4c4d-9822-3ee167badfc1","Type":"ContainerDied","Data":"8cb653cfa85906e5895b90b402cb1af3dc822e5bbf2c3534a47e9548ac977667"}
Sep 16 02:47:18.851049 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:18.850857    2579 scope.go:117] "RemoveContainer" containerID="f7387ae13d2f094945c6a3ff52717fa3d1e72ad20c25b750b008f7b7de1393bc"
Sep 16 02:47:18.852285 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:18.852268    2579 generic.go:356] "Generic (PLEG): container finished" podID="c929e56e-b574-408c-8794-61850900ff63" containerID="6dfc4dda31e0a441dcaa672d8b22ed22c6b9d7062728bc701d21ebe6cf92da43" exitCode=0
Sep 16 02:47:18.852599 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:18.852315    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-5b136e-5f4c46c655-w29bt"
Sep 16 02:47:18.852599 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:18.852328    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-5b136e-5f4c46c655-w29bt" event={"ID":"c929e56e-b574-408c-8794-61850900ff63","Type":"ContainerDied","Data":"6dfc4dda31e0a441dcaa672d8b22ed22c6b9d7062728bc701d21ebe6cf92da43"}
Sep 16 02:47:18.852599 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:18.852359    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-5b136e-5f4c46c655-w29bt" event={"ID":"c929e56e-b574-408c-8794-61850900ff63","Type":"ContainerDied","Data":"823ef249efdbd8a2c49f2260cc8d5ef880a6e9e3ae827fb11e7b50399b049f3a"}
Sep 16 02:47:18.860632 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:18.860609    2579 scope.go:117] "RemoveContainer" containerID="f7387ae13d2f094945c6a3ff52717fa3d1e72ad20c25b750b008f7b7de1393bc"
Sep 16 02:47:18.860931 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:47:18.860907    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f7387ae13d2f094945c6a3ff52717fa3d1e72ad20c25b750b008f7b7de1393bc\": container with ID starting with f7387ae13d2f094945c6a3ff52717fa3d1e72ad20c25b750b008f7b7de1393bc not found: ID does not exist" containerID="f7387ae13d2f094945c6a3ff52717fa3d1e72ad20c25b750b008f7b7de1393bc"
Sep 16 02:47:18.861055 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:18.860940    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f7387ae13d2f094945c6a3ff52717fa3d1e72ad20c25b750b008f7b7de1393bc"} err="failed to get container status \"f7387ae13d2f094945c6a3ff52717fa3d1e72ad20c25b750b008f7b7de1393bc\": rpc error: code = NotFound desc = could not find container \"f7387ae13d2f094945c6a3ff52717fa3d1e72ad20c25b750b008f7b7de1393bc\": container with ID starting with f7387ae13d2f094945c6a3ff52717fa3d1e72ad20c25b750b008f7b7de1393bc not found: ID does not exist"
Sep 16 02:47:18.861055 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:18.860959    2579 scope.go:117] "RemoveContainer" containerID="6dfc4dda31e0a441dcaa672d8b22ed22c6b9d7062728bc701d21ebe6cf92da43"
Sep 16 02:47:18.869952 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:18.869939    2579 scope.go:117] "RemoveContainer" containerID="6dfc4dda31e0a441dcaa672d8b22ed22c6b9d7062728bc701d21ebe6cf92da43"
Sep 16 02:47:18.870195 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:47:18.870176    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6dfc4dda31e0a441dcaa672d8b22ed22c6b9d7062728bc701d21ebe6cf92da43\": container with ID starting with 6dfc4dda31e0a441dcaa672d8b22ed22c6b9d7062728bc701d21ebe6cf92da43 not found: ID does not exist" containerID="6dfc4dda31e0a441dcaa672d8b22ed22c6b9d7062728bc701d21ebe6cf92da43"
Sep 16 02:47:18.870248 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:18.870200    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6dfc4dda31e0a441dcaa672d8b22ed22c6b9d7062728bc701d21ebe6cf92da43"} err="failed to get container status \"6dfc4dda31e0a441dcaa672d8b22ed22c6b9d7062728bc701d21ebe6cf92da43\": rpc error: code = NotFound desc = could not find container \"6dfc4dda31e0a441dcaa672d8b22ed22c6b9d7062728bc701d21ebe6cf92da43\": container with ID starting with 6dfc4dda31e0a441dcaa672d8b22ed22c6b9d7062728bc701d21ebe6cf92da43 not found: ID does not exist"
Sep 16 02:47:18.876987 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:18.876964    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-5b136e-569bfb6474-s9pt6"]
Sep 16 02:47:18.879060 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:18.879038    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-5b136e-569bfb6474-s9pt6"]
Sep 16 02:47:18.886802 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:18.886782    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-5b136e-5f4c46c655-w29bt"]
Sep 16 02:47:18.890117 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:18.890097    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-5b136e-5f4c46c655-w29bt"]
Sep 16 02:47:19.170518 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:19.170420    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:47:20.046701 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:20.046646    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:47:20.103868 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:20.103833    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8c059a3e-25cd-4c4d-9822-3ee167badfc1" path="/var/lib/kubelet/pods/8c059a3e-25cd-4c4d-9822-3ee167badfc1/volumes"
Sep 16 02:47:20.104250 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:20.104235    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c929e56e-b574-408c-8794-61850900ff63" path="/var/lib/kubelet/pods/c929e56e-b574-408c-8794-61850900ff63/volumes"
Sep 16 02:47:20.548804 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:20.548770    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:47:20.649896 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:20.649855    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:47:20.753394 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:20.753361    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:47:20.859322 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:20.859239    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:47:21.612726 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:21.612650    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:47:22.432118 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:22.432079    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:47:23.174246 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:23.174205    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:47:24.068365 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:24.068316    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:47:24.518422 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:24.518377    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:47:24.619140 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:24.619097    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:47:24.745710 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:24.745544    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:47:24.846592 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:24.846497    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:47:25.512190 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:25.512152    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:47:28.835456 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:28.835416    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:47:29.707824 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:29.707790    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:47:30.534419 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:30.534384    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:47:31.148113 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:31.148074    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:47:31.923343 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:31.923302    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:47:32.096709 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:32.096677    2579 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-pre-processing-6cd864bc95-mh4jh" secret="" err="secret \"payload-processing-dockercfg-zzx6j\" not found"
Sep 16 02:47:32.522295 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:32.522264    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:47:33.055825 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:33.055796    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-8e10f1-29825445-82xtd"]
Sep 16 02:47:33.059252 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:33.059222    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:47:33.059395 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:33.059376    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-8e10f1-29825445-82xtd"]
Sep 16 02:47:33.081768 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:33.081725    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w1-8e10f1-77cb4456c5-nscms"]
Sep 16 02:47:33.082080 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:33.082046    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-8e10f1-77cb4456c5-nscms" podUID="dde66f21-667a-4e8e-916f-04a8de158844" containerName="payload-processing" containerID="cri-o://15aa2f030a99e6ae41ef0501175612b4eb5213f59b0ef6eb89a37a2a8317f1d0" gracePeriod=30
Sep 16 02:47:33.087191 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:33.087152    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w1-8e10f1-84cb959658-664wr"]
Sep 16 02:47:33.087628 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:33.087598    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-8e10f1-84cb959658-664wr" podUID="b6610657-a9dc-45d4-ba19-1a71096a5121" containerName="payload-pre-processing" containerID="cri-o://152d9ce369af04ab5ca001f759b8045875151089967facf29c51dfa9094ecb49" gracePeriod=30
Sep 16 02:47:33.347928 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:33.347902    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-8e10f1-77cb4456c5-nscms"
Sep 16 02:47:33.386075 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:33.386045    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-8e10f1-84cb959658-664wr"
Sep 16 02:47:33.454495 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:33.454447    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dde66f21-667a-4e8e-916f-04a8de158844-plugins-config-volume\") pod \"dde66f21-667a-4e8e-916f-04a8de158844\" (UID: \"dde66f21-667a-4e8e-916f-04a8de158844\") "
Sep 16 02:47:33.454673 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:33.454557    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j2kn2\" (UniqueName: \"kubernetes.io/projected/b6610657-a9dc-45d4-ba19-1a71096a5121-kube-api-access-j2kn2\") pod \"b6610657-a9dc-45d4-ba19-1a71096a5121\" (UID: \"b6610657-a9dc-45d4-ba19-1a71096a5121\") "
Sep 16 02:47:33.454673 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:33.454624    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b6610657-a9dc-45d4-ba19-1a71096a5121-plugins-config-volume\") pod \"b6610657-a9dc-45d4-ba19-1a71096a5121\" (UID: \"b6610657-a9dc-45d4-ba19-1a71096a5121\") "
Sep 16 02:47:33.454789 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:33.454706    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r7nq8\" (UniqueName: \"kubernetes.io/projected/dde66f21-667a-4e8e-916f-04a8de158844-kube-api-access-r7nq8\") pod \"dde66f21-667a-4e8e-916f-04a8de158844\" (UID: \"dde66f21-667a-4e8e-916f-04a8de158844\") "
Sep 16 02:47:33.456928 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:33.456894    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b6610657-a9dc-45d4-ba19-1a71096a5121-kube-api-access-j2kn2" (OuterVolumeSpecName: "kube-api-access-j2kn2") pod "b6610657-a9dc-45d4-ba19-1a71096a5121" (UID: "b6610657-a9dc-45d4-ba19-1a71096a5121"). InnerVolumeSpecName "kube-api-access-j2kn2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 02:47:33.465730 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:33.465694    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b6610657-a9dc-45d4-ba19-1a71096a5121-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b6610657-a9dc-45d4-ba19-1a71096a5121" (UID: "b6610657-a9dc-45d4-ba19-1a71096a5121"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 02:47:33.465730 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:33.465711    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dde66f21-667a-4e8e-916f-04a8de158844-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "dde66f21-667a-4e8e-916f-04a8de158844" (UID: "dde66f21-667a-4e8e-916f-04a8de158844"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 02:47:33.465933 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:33.465705    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dde66f21-667a-4e8e-916f-04a8de158844-kube-api-access-r7nq8" (OuterVolumeSpecName: "kube-api-access-r7nq8") pod "dde66f21-667a-4e8e-916f-04a8de158844" (UID: "dde66f21-667a-4e8e-916f-04a8de158844"). InnerVolumeSpecName "kube-api-access-r7nq8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 02:47:33.551298 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:33.551256    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:47:33.556268 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:33.556243    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r7nq8\" (UniqueName: \"kubernetes.io/projected/dde66f21-667a-4e8e-916f-04a8de158844-kube-api-access-r7nq8\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:47:33.556268 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:33.556267    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dde66f21-667a-4e8e-916f-04a8de158844-plugins-config-volume\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:47:33.556462 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:33.556276    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j2kn2\" (UniqueName: \"kubernetes.io/projected/b6610657-a9dc-45d4-ba19-1a71096a5121-kube-api-access-j2kn2\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:47:33.556462 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:33.556284    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b6610657-a9dc-45d4-ba19-1a71096a5121-plugins-config-volume\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:47:33.604666 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:33.604587    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:47:33.663791 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:33.663758    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:47:33.950622 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:33.950591    2579 generic.go:356] "Generic (PLEG): container finished" podID="b6610657-a9dc-45d4-ba19-1a71096a5121" containerID="152d9ce369af04ab5ca001f759b8045875151089967facf29c51dfa9094ecb49" exitCode=0
Sep 16 02:47:33.950819 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:33.950641    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-8e10f1-84cb959658-664wr"
Sep 16 02:47:33.950819 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:33.950677    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-8e10f1-84cb959658-664wr" event={"ID":"b6610657-a9dc-45d4-ba19-1a71096a5121","Type":"ContainerDied","Data":"152d9ce369af04ab5ca001f759b8045875151089967facf29c51dfa9094ecb49"}
Sep 16 02:47:33.950819 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:33.950751    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-8e10f1-84cb959658-664wr" event={"ID":"b6610657-a9dc-45d4-ba19-1a71096a5121","Type":"ContainerDied","Data":"32058704802a32ffc6dcba0b4321802b5f7f8088b7b59aa79ddad35335250cef"}
Sep 16 02:47:33.950819 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:33.950778    2579 scope.go:117] "RemoveContainer" containerID="152d9ce369af04ab5ca001f759b8045875151089967facf29c51dfa9094ecb49"
Sep 16 02:47:33.952107 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:33.952080    2579 generic.go:356] "Generic (PLEG): container finished" podID="dde66f21-667a-4e8e-916f-04a8de158844" containerID="15aa2f030a99e6ae41ef0501175612b4eb5213f59b0ef6eb89a37a2a8317f1d0" exitCode=0
Sep 16 02:47:33.952337 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:33.952126    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-8e10f1-77cb4456c5-nscms" event={"ID":"dde66f21-667a-4e8e-916f-04a8de158844","Type":"ContainerDied","Data":"15aa2f030a99e6ae41ef0501175612b4eb5213f59b0ef6eb89a37a2a8317f1d0"}
Sep 16 02:47:33.952527 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:33.952170    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-8e10f1-77cb4456c5-nscms"
Sep 16 02:47:33.952714 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:33.952685    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-8e10f1-77cb4456c5-nscms" event={"ID":"dde66f21-667a-4e8e-916f-04a8de158844","Type":"ContainerDied","Data":"40d2eea8cdd1617890f1d5fa9327c7cf50655c13555222b32a24ff6793cdc256"}
Sep 16 02:47:33.961147 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:33.961131    2579 scope.go:117] "RemoveContainer" containerID="152d9ce369af04ab5ca001f759b8045875151089967facf29c51dfa9094ecb49"
Sep 16 02:47:33.961430 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:47:33.961407    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"152d9ce369af04ab5ca001f759b8045875151089967facf29c51dfa9094ecb49\": container with ID starting with 152d9ce369af04ab5ca001f759b8045875151089967facf29c51dfa9094ecb49 not found: ID does not exist" containerID="152d9ce369af04ab5ca001f759b8045875151089967facf29c51dfa9094ecb49"
Sep 16 02:47:33.961542 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:33.961439    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"152d9ce369af04ab5ca001f759b8045875151089967facf29c51dfa9094ecb49"} err="failed to get container status \"152d9ce369af04ab5ca001f759b8045875151089967facf29c51dfa9094ecb49\": rpc error: code = NotFound desc = could not find container \"152d9ce369af04ab5ca001f759b8045875151089967facf29c51dfa9094ecb49\": container with ID starting with 152d9ce369af04ab5ca001f759b8045875151089967facf29c51dfa9094ecb49 not found: ID does not exist"
Sep 16 02:47:33.961542 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:33.961457    2579 scope.go:117] "RemoveContainer" containerID="15aa2f030a99e6ae41ef0501175612b4eb5213f59b0ef6eb89a37a2a8317f1d0"
Sep 16 02:47:33.969871 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:33.969845    2579 scope.go:117] "RemoveContainer" containerID="15aa2f030a99e6ae41ef0501175612b4eb5213f59b0ef6eb89a37a2a8317f1d0"
Sep 16 02:47:33.970127 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:47:33.970107    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"15aa2f030a99e6ae41ef0501175612b4eb5213f59b0ef6eb89a37a2a8317f1d0\": container with ID starting with 15aa2f030a99e6ae41ef0501175612b4eb5213f59b0ef6eb89a37a2a8317f1d0 not found: ID does not exist" containerID="15aa2f030a99e6ae41ef0501175612b4eb5213f59b0ef6eb89a37a2a8317f1d0"
Sep 16 02:47:33.970180 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:33.970137    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"15aa2f030a99e6ae41ef0501175612b4eb5213f59b0ef6eb89a37a2a8317f1d0"} err="failed to get container status \"15aa2f030a99e6ae41ef0501175612b4eb5213f59b0ef6eb89a37a2a8317f1d0\": rpc error: code = NotFound desc = could not find container \"15aa2f030a99e6ae41ef0501175612b4eb5213f59b0ef6eb89a37a2a8317f1d0\": container with ID starting with 15aa2f030a99e6ae41ef0501175612b4eb5213f59b0ef6eb89a37a2a8317f1d0 not found: ID does not exist"
Sep 16 02:47:33.987088 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:33.987067    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w1-8e10f1-84cb959658-664wr"]
Sep 16 02:47:34.007120 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:34.007099    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w1-8e10f1-84cb959658-664wr"]
Sep 16 02:47:34.025052 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:34.025033    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w1-8e10f1-77cb4456c5-nscms"]
Sep 16 02:47:34.031077 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:34.031060    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w1-8e10f1-77cb4456c5-nscms"]
Sep 16 02:47:34.103197 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:34.103165    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="48ad0e64-5185-4129-90bc-49d15e2d9534" path="/var/lib/kubelet/pods/48ad0e64-5185-4129-90bc-49d15e2d9534/volumes"
Sep 16 02:47:34.103609 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:34.103593    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b6610657-a9dc-45d4-ba19-1a71096a5121" path="/var/lib/kubelet/pods/b6610657-a9dc-45d4-ba19-1a71096a5121/volumes"
Sep 16 02:47:34.103967 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:34.103952    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dde66f21-667a-4e8e-916f-04a8de158844" path="/var/lib/kubelet/pods/dde66f21-667a-4e8e-916f-04a8de158844/volumes"
Sep 16 02:47:34.332952 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:34.332852    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:47:35.138950 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:35.138914    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:47:35.931672 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:35.931631    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:47:37.731515 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:37.731458    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:47:38.716936 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:38.716902    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:47:39.810539 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:39.810502    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:47:39.872835 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:39.872797    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:47:40.078061 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:40.077968    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:47:44.167940 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:44.167908    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:47:44.172195 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:44.172170    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-fd614e-29825445-qbrbw"]
Sep 16 02:47:44.175831 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:44.175807    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-fd614e-29825445-qbrbw"]
Sep 16 02:47:44.195342 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:44.195315    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-fd614e-66fccddb69-57t8f"]
Sep 16 02:47:44.195590 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:44.195565    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-fd614e-66fccddb69-57t8f" podUID="c1ed1477-844b-4b24-8a3d-910f1eb44b2b" containerName="payload-processing" containerID="cri-o://420762a049b8fafe6d3ed41bc5009875a3c5a9613b0f8dbca84926e252299725" gracePeriod=30
Sep 16 02:47:44.203715 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:44.203628    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-fd614e-768875496b-mxqjp"]
Sep 16 02:47:44.204069 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:44.204041    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-fd614e-768875496b-mxqjp" podUID="9fefbf80-ce02-4fcf-ac3d-3af84b5b32ef" containerName="payload-pre-processing" containerID="cri-o://6d186c96140dfeb2e93764a14fec104a39dced8874e78a8c4fb6dd49c64702a1" gracePeriod=30
Sep 16 02:47:44.233029 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:44.232965    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:47:44.290801 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:44.290769    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:47:44.355906 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:44.355821    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:47:44.415178 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:44.415110    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:47:44.464064 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:44.464030    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-fd614e-66fccddb69-57t8f"
Sep 16 02:47:44.485259 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:44.485237    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-fd614e-768875496b-mxqjp"
Sep 16 02:47:44.558823 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:44.558790    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fqns4\" (UniqueName: \"kubernetes.io/projected/c1ed1477-844b-4b24-8a3d-910f1eb44b2b-kube-api-access-fqns4\") pod \"c1ed1477-844b-4b24-8a3d-910f1eb44b2b\" (UID: \"c1ed1477-844b-4b24-8a3d-910f1eb44b2b\") "
Sep 16 02:47:44.558989 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:44.558881    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c1ed1477-844b-4b24-8a3d-910f1eb44b2b-plugins-config-volume\") pod \"c1ed1477-844b-4b24-8a3d-910f1eb44b2b\" (UID: \"c1ed1477-844b-4b24-8a3d-910f1eb44b2b\") "
Sep 16 02:47:44.559331 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:44.559297    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c1ed1477-844b-4b24-8a3d-910f1eb44b2b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c1ed1477-844b-4b24-8a3d-910f1eb44b2b" (UID: "c1ed1477-844b-4b24-8a3d-910f1eb44b2b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 02:47:44.561050 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:44.561028    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c1ed1477-844b-4b24-8a3d-910f1eb44b2b-kube-api-access-fqns4" (OuterVolumeSpecName: "kube-api-access-fqns4") pod "c1ed1477-844b-4b24-8a3d-910f1eb44b2b" (UID: "c1ed1477-844b-4b24-8a3d-910f1eb44b2b"). InnerVolumeSpecName "kube-api-access-fqns4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 02:47:44.659620 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:44.659588    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-csh8t\" (UniqueName: \"kubernetes.io/projected/9fefbf80-ce02-4fcf-ac3d-3af84b5b32ef-kube-api-access-csh8t\") pod \"9fefbf80-ce02-4fcf-ac3d-3af84b5b32ef\" (UID: \"9fefbf80-ce02-4fcf-ac3d-3af84b5b32ef\") "
Sep 16 02:47:44.659776 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:44.659701    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9fefbf80-ce02-4fcf-ac3d-3af84b5b32ef-plugins-config-volume\") pod \"9fefbf80-ce02-4fcf-ac3d-3af84b5b32ef\" (UID: \"9fefbf80-ce02-4fcf-ac3d-3af84b5b32ef\") "
Sep 16 02:47:44.659885 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:44.659871    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fqns4\" (UniqueName: \"kubernetes.io/projected/c1ed1477-844b-4b24-8a3d-910f1eb44b2b-kube-api-access-fqns4\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:47:44.659927 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:44.659887    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c1ed1477-844b-4b24-8a3d-910f1eb44b2b-plugins-config-volume\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:47:44.660092 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:44.660068    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9fefbf80-ce02-4fcf-ac3d-3af84b5b32ef-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9fefbf80-ce02-4fcf-ac3d-3af84b5b32ef" (UID: "9fefbf80-ce02-4fcf-ac3d-3af84b5b32ef"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 02:47:44.661980 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:44.661950    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9fefbf80-ce02-4fcf-ac3d-3af84b5b32ef-kube-api-access-csh8t" (OuterVolumeSpecName: "kube-api-access-csh8t") pod "9fefbf80-ce02-4fcf-ac3d-3af84b5b32ef" (UID: "9fefbf80-ce02-4fcf-ac3d-3af84b5b32ef"). InnerVolumeSpecName "kube-api-access-csh8t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 02:47:44.761124 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:44.761039    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9fefbf80-ce02-4fcf-ac3d-3af84b5b32ef-plugins-config-volume\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:47:44.761124 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:44.761067    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-csh8t\" (UniqueName: \"kubernetes.io/projected/9fefbf80-ce02-4fcf-ac3d-3af84b5b32ef-kube-api-access-csh8t\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:47:45.015849 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:45.015741    2579 generic.go:356] "Generic (PLEG): container finished" podID="9fefbf80-ce02-4fcf-ac3d-3af84b5b32ef" containerID="6d186c96140dfeb2e93764a14fec104a39dced8874e78a8c4fb6dd49c64702a1" exitCode=0
Sep 16 02:47:45.015849 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:45.015805    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-fd614e-768875496b-mxqjp"
Sep 16 02:47:45.016070 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:45.015837    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-fd614e-768875496b-mxqjp" event={"ID":"9fefbf80-ce02-4fcf-ac3d-3af84b5b32ef","Type":"ContainerDied","Data":"6d186c96140dfeb2e93764a14fec104a39dced8874e78a8c4fb6dd49c64702a1"}
Sep 16 02:47:45.016070 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:45.015901    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-fd614e-768875496b-mxqjp" event={"ID":"9fefbf80-ce02-4fcf-ac3d-3af84b5b32ef","Type":"ContainerDied","Data":"99719d214f6f937ee74f1960b634fec72a7fd0a88dfcd4694c3851c5832dcc98"}
Sep 16 02:47:45.016070 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:45.015929    2579 scope.go:117] "RemoveContainer" containerID="6d186c96140dfeb2e93764a14fec104a39dced8874e78a8c4fb6dd49c64702a1"
Sep 16 02:47:45.017526 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:45.017498    2579 generic.go:356] "Generic (PLEG): container finished" podID="c1ed1477-844b-4b24-8a3d-910f1eb44b2b" containerID="420762a049b8fafe6d3ed41bc5009875a3c5a9613b0f8dbca84926e252299725" exitCode=0
Sep 16 02:47:45.017657 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:45.017596    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-fd614e-66fccddb69-57t8f"
Sep 16 02:47:45.017657 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:45.017644    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-fd614e-66fccddb69-57t8f" event={"ID":"c1ed1477-844b-4b24-8a3d-910f1eb44b2b","Type":"ContainerDied","Data":"420762a049b8fafe6d3ed41bc5009875a3c5a9613b0f8dbca84926e252299725"}
Sep 16 02:47:45.017772 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:45.017710    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-fd614e-66fccddb69-57t8f" event={"ID":"c1ed1477-844b-4b24-8a3d-910f1eb44b2b","Type":"ContainerDied","Data":"4754f3980e187cef53562100699cc88f71530313211962bad5932251dbf9a36f"}
Sep 16 02:47:45.034184 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:45.034167    2579 scope.go:117] "RemoveContainer" containerID="6d186c96140dfeb2e93764a14fec104a39dced8874e78a8c4fb6dd49c64702a1"
Sep 16 02:47:45.034495 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:47:45.034458    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6d186c96140dfeb2e93764a14fec104a39dced8874e78a8c4fb6dd49c64702a1\": container with ID starting with 6d186c96140dfeb2e93764a14fec104a39dced8874e78a8c4fb6dd49c64702a1 not found: ID does not exist" containerID="6d186c96140dfeb2e93764a14fec104a39dced8874e78a8c4fb6dd49c64702a1"
Sep 16 02:47:45.034557 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:45.034503    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d186c96140dfeb2e93764a14fec104a39dced8874e78a8c4fb6dd49c64702a1"} err="failed to get container status \"6d186c96140dfeb2e93764a14fec104a39dced8874e78a8c4fb6dd49c64702a1\": rpc error: code = NotFound desc = could not find container \"6d186c96140dfeb2e93764a14fec104a39dced8874e78a8c4fb6dd49c64702a1\": container with ID starting with 6d186c96140dfeb2e93764a14fec104a39dced8874e78a8c4fb6dd49c64702a1 not found: ID does not exist"
Sep 16 02:47:45.034557 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:45.034519    2579 scope.go:117] "RemoveContainer" containerID="420762a049b8fafe6d3ed41bc5009875a3c5a9613b0f8dbca84926e252299725"
Sep 16 02:47:45.036207 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:45.036185    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:47:45.041621 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:45.041603    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-fd614e-768875496b-mxqjp"]
Sep 16 02:47:45.043733 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:45.043716    2579 scope.go:117] "RemoveContainer" containerID="420762a049b8fafe6d3ed41bc5009875a3c5a9613b0f8dbca84926e252299725"
Sep 16 02:47:45.043984 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:47:45.043967    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"420762a049b8fafe6d3ed41bc5009875a3c5a9613b0f8dbca84926e252299725\": container with ID starting with 420762a049b8fafe6d3ed41bc5009875a3c5a9613b0f8dbca84926e252299725 not found: ID does not exist" containerID="420762a049b8fafe6d3ed41bc5009875a3c5a9613b0f8dbca84926e252299725"
Sep 16 02:47:45.044035 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:45.043992    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"420762a049b8fafe6d3ed41bc5009875a3c5a9613b0f8dbca84926e252299725"} err="failed to get container status \"420762a049b8fafe6d3ed41bc5009875a3c5a9613b0f8dbca84926e252299725\": rpc error: code = NotFound desc = could not find container \"420762a049b8fafe6d3ed41bc5009875a3c5a9613b0f8dbca84926e252299725\": container with ID starting with 420762a049b8fafe6d3ed41bc5009875a3c5a9613b0f8dbca84926e252299725 not found: ID does not exist"
Sep 16 02:47:45.047234 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:45.047211    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-fd614e-768875496b-mxqjp"]
Sep 16 02:47:45.056314 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:45.056291    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-fd614e-66fccddb69-57t8f"]
Sep 16 02:47:45.060002 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:45.059985    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-fd614e-66fccddb69-57t8f"]
Sep 16 02:47:45.137248 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:45.137211    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:47:45.717641 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:45.717599    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:47:46.102838 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:46.102764    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="93344fea-959d-47ba-8575-b3b8b1fec1be" path="/var/lib/kubelet/pods/93344fea-959d-47ba-8575-b3b8b1fec1be/volumes"
Sep 16 02:47:46.103103 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:46.103090    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9fefbf80-ce02-4fcf-ac3d-3af84b5b32ef" path="/var/lib/kubelet/pods/9fefbf80-ce02-4fcf-ac3d-3af84b5b32ef/volumes"
Sep 16 02:47:46.103400 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:46.103387    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c1ed1477-844b-4b24-8a3d-910f1eb44b2b" path="/var/lib/kubelet/pods/c1ed1477-844b-4b24-8a3d-910f1eb44b2b/volumes"
Sep 16 02:47:46.340050 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:46.340006    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:47:49.715413 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:49.715375    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-fd614e/ipp-route-fd614e-kserve-894d7456c-28xz5"]
Sep 16 02:47:49.818945 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:47:49.818907    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:48:00.235795 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:48:00.235764    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-c8q8z_ebf846fa-af7d-4fdd-9662-b7c533c36ddd/console-operator/2.log"
Sep 16 02:48:00.249878 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:48:00.249858    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-c8q8z_ebf846fa-af7d-4fdd-9662-b7c533c36ddd/console-operator/2.log"
Sep 16 02:48:04.735633 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:48:04.735590    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-ipp-a-fd614e/ipp-route-fd614e-kserve-894d7456c-28xz5" podUID="35e2f4db-dc20-4c83-9e37-058576e79472" containerName="main" containerID="cri-o://d28154072c7d83a8c3f65db292d5c81ad0949305f218d4ef8aec256954c7361f" gracePeriod=45
Sep 16 02:48:04.999159 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:48:04.999097    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-fd614e/ipp-route-fd614e-kserve-894d7456c-28xz5"
Sep 16 02:48:05.045817 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:48:05.045786    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/35e2f4db-dc20-4c83-9e37-058576e79472-dshm\") pod \"35e2f4db-dc20-4c83-9e37-058576e79472\" (UID: \"35e2f4db-dc20-4c83-9e37-058576e79472\") "
Sep 16 02:48:05.046020 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:48:05.045840    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s69bw\" (UniqueName: \"kubernetes.io/projected/35e2f4db-dc20-4c83-9e37-058576e79472-kube-api-access-s69bw\") pod \"35e2f4db-dc20-4c83-9e37-058576e79472\" (UID: \"35e2f4db-dc20-4c83-9e37-058576e79472\") "
Sep 16 02:48:05.046020 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:48:05.045931    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/35e2f4db-dc20-4c83-9e37-058576e79472-home\") pod \"35e2f4db-dc20-4c83-9e37-058576e79472\" (UID: \"35e2f4db-dc20-4c83-9e37-058576e79472\") "
Sep 16 02:48:05.046020 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:48:05.045969    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/35e2f4db-dc20-4c83-9e37-058576e79472-tls-certs\") pod \"35e2f4db-dc20-4c83-9e37-058576e79472\" (UID: \"35e2f4db-dc20-4c83-9e37-058576e79472\") "
Sep 16 02:48:05.046020 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:48:05.046013    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/35e2f4db-dc20-4c83-9e37-058576e79472-model-cache\") pod \"35e2f4db-dc20-4c83-9e37-058576e79472\" (UID: \"35e2f4db-dc20-4c83-9e37-058576e79472\") "
Sep 16 02:48:05.046231 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:48:05.046051    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/35e2f4db-dc20-4c83-9e37-058576e79472-tmp-dir\") pod \"35e2f4db-dc20-4c83-9e37-058576e79472\" (UID: \"35e2f4db-dc20-4c83-9e37-058576e79472\") "
Sep 16 02:48:05.046283 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:48:05.046225    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/35e2f4db-dc20-4c83-9e37-058576e79472-home" (OuterVolumeSpecName: "home") pod "35e2f4db-dc20-4c83-9e37-058576e79472" (UID: "35e2f4db-dc20-4c83-9e37-058576e79472"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 02:48:05.046380 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:48:05.046355    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/35e2f4db-dc20-4c83-9e37-058576e79472-model-cache" (OuterVolumeSpecName: "model-cache") pod "35e2f4db-dc20-4c83-9e37-058576e79472" (UID: "35e2f4db-dc20-4c83-9e37-058576e79472"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 02:48:05.046446 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:48:05.046381    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/35e2f4db-dc20-4c83-9e37-058576e79472-home\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:48:05.046529 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:48:05.046455    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/35e2f4db-dc20-4c83-9e37-058576e79472-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "35e2f4db-dc20-4c83-9e37-058576e79472" (UID: "35e2f4db-dc20-4c83-9e37-058576e79472"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 02:48:05.048442 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:48:05.048406    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/35e2f4db-dc20-4c83-9e37-058576e79472-dshm" (OuterVolumeSpecName: "dshm") pod "35e2f4db-dc20-4c83-9e37-058576e79472" (UID: "35e2f4db-dc20-4c83-9e37-058576e79472"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 02:48:05.048577 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:48:05.048519    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/35e2f4db-dc20-4c83-9e37-058576e79472-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "35e2f4db-dc20-4c83-9e37-058576e79472" (UID: "35e2f4db-dc20-4c83-9e37-058576e79472"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 02:48:05.048645 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:48:05.048625    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/35e2f4db-dc20-4c83-9e37-058576e79472-kube-api-access-s69bw" (OuterVolumeSpecName: "kube-api-access-s69bw") pod "35e2f4db-dc20-4c83-9e37-058576e79472" (UID: "35e2f4db-dc20-4c83-9e37-058576e79472"). InnerVolumeSpecName "kube-api-access-s69bw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 02:48:05.127975 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:48:05.127939    2579 generic.go:356] "Generic (PLEG): container finished" podID="35e2f4db-dc20-4c83-9e37-058576e79472" containerID="d28154072c7d83a8c3f65db292d5c81ad0949305f218d4ef8aec256954c7361f" exitCode=0
Sep 16 02:48:05.128163 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:48:05.127983    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-fd614e/ipp-route-fd614e-kserve-894d7456c-28xz5"
Sep 16 02:48:05.128163 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:48:05.128023    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-fd614e/ipp-route-fd614e-kserve-894d7456c-28xz5" event={"ID":"35e2f4db-dc20-4c83-9e37-058576e79472","Type":"ContainerDied","Data":"d28154072c7d83a8c3f65db292d5c81ad0949305f218d4ef8aec256954c7361f"}
Sep 16 02:48:05.128163 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:48:05.128066    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-fd614e/ipp-route-fd614e-kserve-894d7456c-28xz5" event={"ID":"35e2f4db-dc20-4c83-9e37-058576e79472","Type":"ContainerDied","Data":"28314c84d14962c8776419fa7d42f9b4de7caa9f25228ca34003278e9ceff002"}
Sep 16 02:48:05.128163 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:48:05.128083    2579 scope.go:117] "RemoveContainer" containerID="d28154072c7d83a8c3f65db292d5c81ad0949305f218d4ef8aec256954c7361f"
Sep 16 02:48:05.137860 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:48:05.137834    2579 scope.go:117] "RemoveContainer" containerID="d28154072c7d83a8c3f65db292d5c81ad0949305f218d4ef8aec256954c7361f"
Sep 16 02:48:05.138121 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:48:05.138102    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d28154072c7d83a8c3f65db292d5c81ad0949305f218d4ef8aec256954c7361f\": container with ID starting with d28154072c7d83a8c3f65db292d5c81ad0949305f218d4ef8aec256954c7361f not found: ID does not exist" containerID="d28154072c7d83a8c3f65db292d5c81ad0949305f218d4ef8aec256954c7361f"
Sep 16 02:48:05.138203 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:48:05.138130    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d28154072c7d83a8c3f65db292d5c81ad0949305f218d4ef8aec256954c7361f"} err="failed to get container status \"d28154072c7d83a8c3f65db292d5c81ad0949305f218d4ef8aec256954c7361f\": rpc error: code = NotFound desc = could not find container \"d28154072c7d83a8c3f65db292d5c81ad0949305f218d4ef8aec256954c7361f\": container with ID starting with d28154072c7d83a8c3f65db292d5c81ad0949305f218d4ef8aec256954c7361f not found: ID does not exist"
Sep 16 02:48:05.147451 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:48:05.147430    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/35e2f4db-dc20-4c83-9e37-058576e79472-model-cache\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:48:05.147451 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:48:05.147450    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/35e2f4db-dc20-4c83-9e37-058576e79472-tmp-dir\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:48:05.147629 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:48:05.147458    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/35e2f4db-dc20-4c83-9e37-058576e79472-dshm\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:48:05.147629 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:48:05.147483    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s69bw\" (UniqueName: \"kubernetes.io/projected/35e2f4db-dc20-4c83-9e37-058576e79472-kube-api-access-s69bw\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:48:05.147629 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:48:05.147497    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/35e2f4db-dc20-4c83-9e37-058576e79472-tls-certs\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:48:05.150786 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:48:05.150765    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-fd614e/ipp-route-fd614e-kserve-894d7456c-28xz5"]
Sep 16 02:48:05.153941 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:48:05.153921    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-ipp-a-fd614e/ipp-route-fd614e-kserve-894d7456c-28xz5"]
Sep 16 02:48:06.109818 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:48:06.109785    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="35e2f4db-dc20-4c83-9e37-058576e79472" path="/var/lib/kubelet/pods/35e2f4db-dc20-4c83-9e37-058576e79472/volumes"
Sep 16 02:48:26.582318 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:48:26.582280    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:48:27.301526 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:48:27.301492    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:48:32.688977 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:48:32.688938    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:48:46.784556 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:48:46.784520    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:48:46.885555 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:48:46.885518    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:48:47.068384 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:48:47.068290    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:48:47.097252 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:48:47.097225    2579 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-pre-processing-6cd864bc95-mh4jh" secret="" err="secret \"payload-processing-dockercfg-zzx6j\" not found"
Sep 16 02:48:48.265997 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:48:48.265946    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:48:49.123596 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:48:49.123543    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:48:49.636751 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:48:49.636712    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:48:57.866301 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:48:57.866218    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:48:57.872019 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:48:57.871982    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-d3d202-29825445-k57v7"]
Sep 16 02:48:57.873685 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:48:57.873660    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-d3d202-29825445-k57v7"]
Sep 16 02:48:57.894608 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:48:57.894585    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-d3d202-75996dd58f-kxn4n"]
Sep 16 02:48:57.894804 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:48:57.894779    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-d3d202-75996dd58f-kxn4n" podUID="c97c7775-4a18-4a8d-b068-8a625fc6a591" containerName="payload-processing" containerID="cri-o://1f2d62609f23e54b16ec971a7b531b1ff182dd85c90c755f4e1030a5bf43fb1f" gracePeriod=30
Sep 16 02:48:57.906962 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:48:57.906938    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-d3d202-558b6d7798-4mpfq"]
Sep 16 02:48:57.907293 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:48:57.907139    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d3d202-558b6d7798-4mpfq" podUID="c05721b2-9ebf-4250-9d7c-c0e464194f9c" containerName="payload-pre-processing" containerID="cri-o://a50de980cf97f663ca651f3565d7edaaf2b2ed13cc5da71a966d4bf445b0afbf" gracePeriod=30
Sep 16 02:48:57.933586 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:48:57.933545    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:48:57.989916 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:48:57.989889    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:48:58.058645 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:48:58.058572    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:48:58.102981 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:48:58.102950    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="39eea28c-9ffa-4790-810b-1474c29bc639" path="/var/lib/kubelet/pods/39eea28c-9ffa-4790-810b-1474c29bc639/volumes"
Sep 16 02:48:58.126096 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:48:58.126015    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:48:58.191423 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:48:58.191397    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-d3d202-75996dd58f-kxn4n"
Sep 16 02:48:58.194364 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:48:58.194346    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d3d202-558b6d7798-4mpfq"
Sep 16 02:48:58.317312 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:48:58.317282    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zn7hm\" (UniqueName: \"kubernetes.io/projected/c05721b2-9ebf-4250-9d7c-c0e464194f9c-kube-api-access-zn7hm\") pod \"c05721b2-9ebf-4250-9d7c-c0e464194f9c\" (UID: \"c05721b2-9ebf-4250-9d7c-c0e464194f9c\") "
Sep 16 02:48:58.317312 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:48:58.317317    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c05721b2-9ebf-4250-9d7c-c0e464194f9c-plugins-config-volume\") pod \"c05721b2-9ebf-4250-9d7c-c0e464194f9c\" (UID: \"c05721b2-9ebf-4250-9d7c-c0e464194f9c\") "
Sep 16 02:48:58.317543 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:48:58.317342    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c97c7775-4a18-4a8d-b068-8a625fc6a591-plugins-config-volume\") pod \"c97c7775-4a18-4a8d-b068-8a625fc6a591\" (UID: \"c97c7775-4a18-4a8d-b068-8a625fc6a591\") "
Sep 16 02:48:58.317543 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:48:58.317405    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zxkcz\" (UniqueName: \"kubernetes.io/projected/c97c7775-4a18-4a8d-b068-8a625fc6a591-kube-api-access-zxkcz\") pod \"c97c7775-4a18-4a8d-b068-8a625fc6a591\" (UID: \"c97c7775-4a18-4a8d-b068-8a625fc6a591\") "
Sep 16 02:48:58.317763 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:48:58.317739    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c05721b2-9ebf-4250-9d7c-c0e464194f9c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c05721b2-9ebf-4250-9d7c-c0e464194f9c" (UID: "c05721b2-9ebf-4250-9d7c-c0e464194f9c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 02:48:58.317763 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:48:58.317748    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c97c7775-4a18-4a8d-b068-8a625fc6a591-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c97c7775-4a18-4a8d-b068-8a625fc6a591" (UID: "c97c7775-4a18-4a8d-b068-8a625fc6a591"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 02:48:58.319594 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:48:58.319575    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c97c7775-4a18-4a8d-b068-8a625fc6a591-kube-api-access-zxkcz" (OuterVolumeSpecName: "kube-api-access-zxkcz") pod "c97c7775-4a18-4a8d-b068-8a625fc6a591" (UID: "c97c7775-4a18-4a8d-b068-8a625fc6a591"). InnerVolumeSpecName "kube-api-access-zxkcz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 02:48:58.319657 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:48:58.319628    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c05721b2-9ebf-4250-9d7c-c0e464194f9c-kube-api-access-zn7hm" (OuterVolumeSpecName: "kube-api-access-zn7hm") pod "c05721b2-9ebf-4250-9d7c-c0e464194f9c" (UID: "c05721b2-9ebf-4250-9d7c-c0e464194f9c"). InnerVolumeSpecName "kube-api-access-zn7hm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 02:48:58.407338 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:48:58.407261    2579 generic.go:356] "Generic (PLEG): container finished" podID="c97c7775-4a18-4a8d-b068-8a625fc6a591" containerID="1f2d62609f23e54b16ec971a7b531b1ff182dd85c90c755f4e1030a5bf43fb1f" exitCode=0
Sep 16 02:48:58.407338 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:48:58.407320    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-d3d202-75996dd58f-kxn4n"
Sep 16 02:48:58.407548 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:48:58.407347    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-d3d202-75996dd58f-kxn4n" event={"ID":"c97c7775-4a18-4a8d-b068-8a625fc6a591","Type":"ContainerDied","Data":"1f2d62609f23e54b16ec971a7b531b1ff182dd85c90c755f4e1030a5bf43fb1f"}
Sep 16 02:48:58.407548 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:48:58.407425    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-d3d202-75996dd58f-kxn4n" event={"ID":"c97c7775-4a18-4a8d-b068-8a625fc6a591","Type":"ContainerDied","Data":"eca5f7e4588a177170f6529a585ed4c5ac3b7819130d73cec32653ed1e30bae9"}
Sep 16 02:48:58.407548 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:48:58.407458    2579 scope.go:117] "RemoveContainer" containerID="1f2d62609f23e54b16ec971a7b531b1ff182dd85c90c755f4e1030a5bf43fb1f"
Sep 16 02:48:58.408891 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:48:58.408864    2579 generic.go:356] "Generic (PLEG): container finished" podID="c05721b2-9ebf-4250-9d7c-c0e464194f9c" containerID="a50de980cf97f663ca651f3565d7edaaf2b2ed13cc5da71a966d4bf445b0afbf" exitCode=0
Sep 16 02:48:58.409011 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:48:58.408910    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d3d202-558b6d7798-4mpfq"
Sep 16 02:48:58.409011 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:48:58.408938    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d3d202-558b6d7798-4mpfq" event={"ID":"c05721b2-9ebf-4250-9d7c-c0e464194f9c","Type":"ContainerDied","Data":"a50de980cf97f663ca651f3565d7edaaf2b2ed13cc5da71a966d4bf445b0afbf"}
Sep 16 02:48:58.409011 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:48:58.408977    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d3d202-558b6d7798-4mpfq" event={"ID":"c05721b2-9ebf-4250-9d7c-c0e464194f9c","Type":"ContainerDied","Data":"a00736f82066260c5165123c05a3b5e818f383567062c073bdfc017905892255"}
Sep 16 02:48:58.417263 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:48:58.417245    2579 scope.go:117] "RemoveContainer" containerID="1f2d62609f23e54b16ec971a7b531b1ff182dd85c90c755f4e1030a5bf43fb1f"
Sep 16 02:48:58.417556 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:48:58.417536    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1f2d62609f23e54b16ec971a7b531b1ff182dd85c90c755f4e1030a5bf43fb1f\": container with ID starting with 1f2d62609f23e54b16ec971a7b531b1ff182dd85c90c755f4e1030a5bf43fb1f not found: ID does not exist" containerID="1f2d62609f23e54b16ec971a7b531b1ff182dd85c90c755f4e1030a5bf43fb1f"
Sep 16 02:48:58.417636 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:48:58.417563    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f2d62609f23e54b16ec971a7b531b1ff182dd85c90c755f4e1030a5bf43fb1f"} err="failed to get container status \"1f2d62609f23e54b16ec971a7b531b1ff182dd85c90c755f4e1030a5bf43fb1f\": rpc error: code = NotFound desc = could not find container \"1f2d62609f23e54b16ec971a7b531b1ff182dd85c90c755f4e1030a5bf43fb1f\": container with ID starting with 1f2d62609f23e54b16ec971a7b531b1ff182dd85c90c755f4e1030a5bf43fb1f not found: ID does not exist"
Sep 16 02:48:58.417636 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:48:58.417581    2579 scope.go:117] "RemoveContainer" containerID="a50de980cf97f663ca651f3565d7edaaf2b2ed13cc5da71a966d4bf445b0afbf"
Sep 16 02:48:58.418160 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:48:58.418138    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zxkcz\" (UniqueName: \"kubernetes.io/projected/c97c7775-4a18-4a8d-b068-8a625fc6a591-kube-api-access-zxkcz\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:48:58.418348 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:48:58.418161    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zn7hm\" (UniqueName: \"kubernetes.io/projected/c05721b2-9ebf-4250-9d7c-c0e464194f9c-kube-api-access-zn7hm\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:48:58.418348 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:48:58.418175    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c05721b2-9ebf-4250-9d7c-c0e464194f9c-plugins-config-volume\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:48:58.418348 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:48:58.418189    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c97c7775-4a18-4a8d-b068-8a625fc6a591-plugins-config-volume\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:48:58.425629 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:48:58.425607    2579 scope.go:117] "RemoveContainer" containerID="a50de980cf97f663ca651f3565d7edaaf2b2ed13cc5da71a966d4bf445b0afbf"
Sep 16 02:48:58.425896 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:48:58.425875    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a50de980cf97f663ca651f3565d7edaaf2b2ed13cc5da71a966d4bf445b0afbf\": container with ID starting with a50de980cf97f663ca651f3565d7edaaf2b2ed13cc5da71a966d4bf445b0afbf not found: ID does not exist" containerID="a50de980cf97f663ca651f3565d7edaaf2b2ed13cc5da71a966d4bf445b0afbf"
Sep 16 02:48:58.425962 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:48:58.425906    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a50de980cf97f663ca651f3565d7edaaf2b2ed13cc5da71a966d4bf445b0afbf"} err="failed to get container status \"a50de980cf97f663ca651f3565d7edaaf2b2ed13cc5da71a966d4bf445b0afbf\": rpc error: code = NotFound desc = could not find container \"a50de980cf97f663ca651f3565d7edaaf2b2ed13cc5da71a966d4bf445b0afbf\": container with ID starting with a50de980cf97f663ca651f3565d7edaaf2b2ed13cc5da71a966d4bf445b0afbf not found: ID does not exist"
Sep 16 02:48:58.432748 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:48:58.432726    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-d3d202-75996dd58f-kxn4n"]
Sep 16 02:48:58.438496 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:48:58.438449    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-d3d202-75996dd58f-kxn4n"]
Sep 16 02:48:58.441549 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:48:58.441530    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:48:58.447404 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:48:58.447382    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-d3d202-558b6d7798-4mpfq"]
Sep 16 02:48:58.451031 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:48:58.451007    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-d3d202-558b6d7798-4mpfq"]
Sep 16 02:48:58.665897 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:48:58.665810    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:48:59.309737 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:48:59.309702    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:49:00.103605 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:00.103577    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c05721b2-9ebf-4250-9d7c-c0e464194f9c" path="/var/lib/kubelet/pods/c05721b2-9ebf-4250-9d7c-c0e464194f9c/volumes"
Sep 16 02:49:00.103942 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:00.103930    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c97c7775-4a18-4a8d-b068-8a625fc6a591" path="/var/lib/kubelet/pods/c97c7775-4a18-4a8d-b068-8a625fc6a591/volumes"
Sep 16 02:49:04.188388 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:04.188346    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:49:04.781998 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:04.781960    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-24aabe-kfn6t"]
Sep 16 02:49:04.782614 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:04.782596    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c929e56e-b574-408c-8794-61850900ff63" containerName="payload-processing"
Sep 16 02:49:04.782672 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:04.782618    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="c929e56e-b574-408c-8794-61850900ff63" containerName="payload-processing"
Sep 16 02:49:04.782672 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:04.782636    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a035cdb-0306-4400-b8fe-5f29fdd40f5c" containerName="revoke-keys"
Sep 16 02:49:04.782672 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:04.782644    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a035cdb-0306-4400-b8fe-5f29fdd40f5c" containerName="revoke-keys"
Sep 16 02:49:04.782672 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:04.782656    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c1ed1477-844b-4b24-8a3d-910f1eb44b2b" containerName="payload-processing"
Sep 16 02:49:04.782672 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:04.782664    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="c1ed1477-844b-4b24-8a3d-910f1eb44b2b" containerName="payload-processing"
Sep 16 02:49:04.782848 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:04.782673    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8c059a3e-25cd-4c4d-9822-3ee167badfc1" containerName="payload-pre-processing"
Sep 16 02:49:04.782848 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:04.782681    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="8c059a3e-25cd-4c4d-9822-3ee167badfc1" containerName="payload-pre-processing"
Sep 16 02:49:04.782848 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:04.782695    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dde66f21-667a-4e8e-916f-04a8de158844" containerName="payload-processing"
Sep 16 02:49:04.782848 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:04.782703    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="dde66f21-667a-4e8e-916f-04a8de158844" containerName="payload-processing"
Sep 16 02:49:04.782848 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:04.782715    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c97c7775-4a18-4a8d-b068-8a625fc6a591" containerName="payload-processing"
Sep 16 02:49:04.782848 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:04.782724    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="c97c7775-4a18-4a8d-b068-8a625fc6a591" containerName="payload-processing"
Sep 16 02:49:04.782848 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:04.782738    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b6610657-a9dc-45d4-ba19-1a71096a5121" containerName="payload-pre-processing"
Sep 16 02:49:04.782848 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:04.782746    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="b6610657-a9dc-45d4-ba19-1a71096a5121" containerName="payload-pre-processing"
Sep 16 02:49:04.782848 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:04.782761    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="35e2f4db-dc20-4c83-9e37-058576e79472" containerName="main"
Sep 16 02:49:04.782848 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:04.782769    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="35e2f4db-dc20-4c83-9e37-058576e79472" containerName="main"
Sep 16 02:49:04.782848 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:04.782788    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c05721b2-9ebf-4250-9d7c-c0e464194f9c" containerName="payload-pre-processing"
Sep 16 02:49:04.782848 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:04.782797    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="c05721b2-9ebf-4250-9d7c-c0e464194f9c" containerName="payload-pre-processing"
Sep 16 02:49:04.782848 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:04.782812    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9fefbf80-ce02-4fcf-ac3d-3af84b5b32ef" containerName="payload-pre-processing"
Sep 16 02:49:04.782848 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:04.782820    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="9fefbf80-ce02-4fcf-ac3d-3af84b5b32ef" containerName="payload-pre-processing"
Sep 16 02:49:04.783275 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:04.782920    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="c97c7775-4a18-4a8d-b068-8a625fc6a591" containerName="payload-processing"
Sep 16 02:49:04.783275 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:04.782932    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="8c059a3e-25cd-4c4d-9822-3ee167badfc1" containerName="payload-pre-processing"
Sep 16 02:49:04.783275 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:04.782942    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="dde66f21-667a-4e8e-916f-04a8de158844" containerName="payload-processing"
Sep 16 02:49:04.783275 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:04.782951    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="c05721b2-9ebf-4250-9d7c-c0e464194f9c" containerName="payload-pre-processing"
Sep 16 02:49:04.783275 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:04.782959    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="c929e56e-b574-408c-8794-61850900ff63" containerName="payload-processing"
Sep 16 02:49:04.783275 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:04.782967    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="9fefbf80-ce02-4fcf-ac3d-3af84b5b32ef" containerName="payload-pre-processing"
Sep 16 02:49:04.783275 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:04.782975    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="b6610657-a9dc-45d4-ba19-1a71096a5121" containerName="payload-pre-processing"
Sep 16 02:49:04.783275 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:04.782987    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="4a035cdb-0306-4400-b8fe-5f29fdd40f5c" containerName="revoke-keys"
Sep 16 02:49:04.783275 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:04.782997    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="c1ed1477-844b-4b24-8a3d-910f1eb44b2b" containerName="payload-processing"
Sep 16 02:49:04.783275 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:04.783006    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="35e2f4db-dc20-4c83-9e37-058576e79472" containerName="main"
Sep 16 02:49:04.786562 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:04.786545    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-24aabe-kfn6t"
Sep 16 02:49:04.788953 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:04.788930    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-jpd4f\""
Sep 16 02:49:04.790160 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:04.790135    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-24aabe-kfn6t"]
Sep 16 02:49:04.877233 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:04.877201    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/66cb243d-6470-40b3-8b58-cbef5146d7ca-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-24aabe-kfn6t\" (UID: \"66cb243d-6470-40b3-8b58-cbef5146d7ca\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-24aabe-kfn6t"
Sep 16 02:49:04.978770 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:04.978730    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/66cb243d-6470-40b3-8b58-cbef5146d7ca-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-24aabe-kfn6t\" (UID: \"66cb243d-6470-40b3-8b58-cbef5146d7ca\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-24aabe-kfn6t"
Sep 16 02:49:04.979571 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:04.979544    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/66cb243d-6470-40b3-8b58-cbef5146d7ca-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-24aabe-kfn6t\" (UID: \"66cb243d-6470-40b3-8b58-cbef5146d7ca\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-24aabe-kfn6t"
Sep 16 02:49:05.098677 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:05.098602    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-24aabe-kfn6t"
Sep 16 02:49:05.374772 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:05.374706    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-24aabe-kfn6t"]
Sep 16 02:49:05.377586 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:49:05.377555    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66cb243d_6470_40b3_8b58_cbef5146d7ca.slice/crio-9554f463fa94622b314044a83e36330ec5a48f2f2d3b4142b2791db7e84690d9 WatchSource:0}: Error finding container 9554f463fa94622b314044a83e36330ec5a48f2f2d3b4142b2791db7e84690d9: Status 404 returned error can't find the container with id 9554f463fa94622b314044a83e36330ec5a48f2f2d3b4142b2791db7e84690d9
Sep 16 02:49:05.448142 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:05.448109    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-24aabe-kfn6t" event={"ID":"66cb243d-6470-40b3-8b58-cbef5146d7ca","Type":"ContainerStarted","Data":"9554f463fa94622b314044a83e36330ec5a48f2f2d3b4142b2791db7e84690d9"}
Sep 16 02:49:05.831260 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:05.831220    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:49:05.931910 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:05.931802    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:49:06.455216 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:06.455181    2579 generic.go:356] "Generic (PLEG): container finished" podID="66cb243d-6470-40b3-8b58-cbef5146d7ca" containerID="4fe8d595462b39fa59353a068fec7d46e41201cccfcb491392f7fd5712e6f098" exitCode=0
Sep 16 02:49:06.455678 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:06.455267    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-24aabe-kfn6t" event={"ID":"66cb243d-6470-40b3-8b58-cbef5146d7ca","Type":"ContainerDied","Data":"4fe8d595462b39fa59353a068fec7d46e41201cccfcb491392f7fd5712e6f098"}
Sep 16 02:49:06.529519 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:06.529491    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:49:07.598053 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:07.598031    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-24aabe-kfn6t"
Sep 16 02:49:07.701460 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:07.701432    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/66cb243d-6470-40b3-8b58-cbef5146d7ca-maas-api-service-ca\") pod \"66cb243d-6470-40b3-8b58-cbef5146d7ca\" (UID: \"66cb243d-6470-40b3-8b58-cbef5146d7ca\") "
Sep 16 02:49:07.701761 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:07.701741    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/66cb243d-6470-40b3-8b58-cbef5146d7ca-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "66cb243d-6470-40b3-8b58-cbef5146d7ca" (UID: "66cb243d-6470-40b3-8b58-cbef5146d7ca"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 02:49:07.802842 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:07.802767    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/66cb243d-6470-40b3-8b58-cbef5146d7ca-maas-api-service-ca\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:49:08.465933 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:08.465895    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-24aabe-kfn6t" event={"ID":"66cb243d-6470-40b3-8b58-cbef5146d7ca","Type":"ContainerDied","Data":"9554f463fa94622b314044a83e36330ec5a48f2f2d3b4142b2791db7e84690d9"}
Sep 16 02:49:08.466113 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:08.465950    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9554f463fa94622b314044a83e36330ec5a48f2f2d3b4142b2791db7e84690d9"
Sep 16 02:49:08.466113 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:08.465911    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-24aabe-kfn6t"
Sep 16 02:49:15.235323 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:15.235292    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:49:15.253435 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:15.253401    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-24aabe-5d76d4564c-9blzj"]
Sep 16 02:49:15.253688 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:15.253628    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-24aabe-5d76d4564c-9blzj" podUID="0a019f4b-414a-44fa-9535-e1846e70866c" containerName="payload-processing" containerID="cri-o://107d9927529090744e2eda1749c1bd3806e35224e8b5e40992e39b9b5cdfb372" gracePeriod=30
Sep 16 02:49:15.300748 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:15.300720    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:49:15.397878 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:15.397771    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:49:15.409228 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:15.408942    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-24aabe-kfn6t"]
Sep 16 02:49:15.411766 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:15.411734    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-24aabe-kfn6t"]
Sep 16 02:49:15.459975 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:15.459931    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:49:15.500302 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:15.500272    2579 generic.go:356] "Generic (PLEG): container finished" podID="0a019f4b-414a-44fa-9535-e1846e70866c" containerID="107d9927529090744e2eda1749c1bd3806e35224e8b5e40992e39b9b5cdfb372" exitCode=0
Sep 16 02:49:15.500416 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:15.500347    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-24aabe-5d76d4564c-9blzj" event={"ID":"0a019f4b-414a-44fa-9535-e1846e70866c","Type":"ContainerDied","Data":"107d9927529090744e2eda1749c1bd3806e35224e8b5e40992e39b9b5cdfb372"}
Sep 16 02:49:15.500459 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:15.500422    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-24aabe-5d76d4564c-9blzj" event={"ID":"0a019f4b-414a-44fa-9535-e1846e70866c","Type":"ContainerDied","Data":"f469a6e69d22d6f098b4c820a38014a993888c3da2f2095390e8e5ef6ffa03fd"}
Sep 16 02:49:15.500459 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:15.500439    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f469a6e69d22d6f098b4c820a38014a993888c3da2f2095390e8e5ef6ffa03fd"
Sep 16 02:49:15.506509 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:15.506489    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-24aabe-5d76d4564c-9blzj"
Sep 16 02:49:15.670191 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:15.670159    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2lpjk\" (UniqueName: \"kubernetes.io/projected/0a019f4b-414a-44fa-9535-e1846e70866c-kube-api-access-2lpjk\") pod \"0a019f4b-414a-44fa-9535-e1846e70866c\" (UID: \"0a019f4b-414a-44fa-9535-e1846e70866c\") "
Sep 16 02:49:15.670377 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:15.670207    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0a019f4b-414a-44fa-9535-e1846e70866c-plugins-config-volume\") pod \"0a019f4b-414a-44fa-9535-e1846e70866c\" (UID: \"0a019f4b-414a-44fa-9535-e1846e70866c\") "
Sep 16 02:49:15.670718 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:15.670693    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0a019f4b-414a-44fa-9535-e1846e70866c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "0a019f4b-414a-44fa-9535-e1846e70866c" (UID: "0a019f4b-414a-44fa-9535-e1846e70866c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 02:49:15.672634 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:15.672603    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a019f4b-414a-44fa-9535-e1846e70866c-kube-api-access-2lpjk" (OuterVolumeSpecName: "kube-api-access-2lpjk") pod "0a019f4b-414a-44fa-9535-e1846e70866c" (UID: "0a019f4b-414a-44fa-9535-e1846e70866c"). InnerVolumeSpecName "kube-api-access-2lpjk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 02:49:15.771314 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:15.771233    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2lpjk\" (UniqueName: \"kubernetes.io/projected/0a019f4b-414a-44fa-9535-e1846e70866c-kube-api-access-2lpjk\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:49:15.771314 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:15.771261    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0a019f4b-414a-44fa-9535-e1846e70866c-plugins-config-volume\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:49:15.778950 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:15.778922    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:49:16.033309 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:16.033201    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:49:16.102087 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:16.102043    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="66cb243d-6470-40b3-8b58-cbef5146d7ca" path="/var/lib/kubelet/pods/66cb243d-6470-40b3-8b58-cbef5146d7ca/volumes"
Sep 16 02:49:16.505124 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:16.505095    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-24aabe-5d76d4564c-9blzj"
Sep 16 02:49:16.521045 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:16.521020    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-24aabe-5d76d4564c-9blzj"]
Sep 16 02:49:16.526862 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:16.526834    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-24aabe-5d76d4564c-9blzj"]
Sep 16 02:49:18.102418 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:18.102383    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0a019f4b-414a-44fa-9535-e1846e70866c" path="/var/lib/kubelet/pods/0a019f4b-414a-44fa-9535-e1846e70866c/volumes"
Sep 16 02:49:21.723037 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:21.722999    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:49:21.822665 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:21.822629    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:49:22.059761 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:22.059674    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:49:22.333662 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:22.333568    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-a53a44-jckf7"]
Sep 16 02:49:22.333953 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:22.333938    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a019f4b-414a-44fa-9535-e1846e70866c" containerName="payload-processing"
Sep 16 02:49:22.333953 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:22.333953    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a019f4b-414a-44fa-9535-e1846e70866c" containerName="payload-processing"
Sep 16 02:49:22.334094 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:22.333965    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="66cb243d-6470-40b3-8b58-cbef5146d7ca" containerName="revoke-keys"
Sep 16 02:49:22.334094 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:22.333971    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="66cb243d-6470-40b3-8b58-cbef5146d7ca" containerName="revoke-keys"
Sep 16 02:49:22.334094 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:22.334031    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="66cb243d-6470-40b3-8b58-cbef5146d7ca" containerName="revoke-keys"
Sep 16 02:49:22.334094 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:22.334039    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a019f4b-414a-44fa-9535-e1846e70866c" containerName="payload-processing"
Sep 16 02:49:22.338261 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:22.338241    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-a53a44-jckf7"
Sep 16 02:49:22.341095 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:22.340640    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-jpd4f\""
Sep 16 02:49:22.344016 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:22.343992    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-a53a44-jckf7"]
Sep 16 02:49:22.434723 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:22.434689    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6045a693-3494-4367-beb9-60a78e7fd1d3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-a53a44-jckf7\" (UID: \"6045a693-3494-4367-beb9-60a78e7fd1d3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-a53a44-jckf7"
Sep 16 02:49:22.535794 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:22.535756    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6045a693-3494-4367-beb9-60a78e7fd1d3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-a53a44-jckf7\" (UID: \"6045a693-3494-4367-beb9-60a78e7fd1d3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-a53a44-jckf7"
Sep 16 02:49:22.536414 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:22.536395    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6045a693-3494-4367-beb9-60a78e7fd1d3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-a53a44-jckf7\" (UID: \"6045a693-3494-4367-beb9-60a78e7fd1d3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-a53a44-jckf7"
Sep 16 02:49:22.650620 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:22.650539    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-a53a44-jckf7"
Sep 16 02:49:22.923889 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:22.923864    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-a53a44-jckf7"]
Sep 16 02:49:22.926014 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:49:22.925986    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6045a693_3494_4367_beb9_60a78e7fd1d3.slice/crio-29d99e07bdb080f03f24350cbd951da3501f6b0e2fd7571f16955e213be075db WatchSource:0}: Error finding container 29d99e07bdb080f03f24350cbd951da3501f6b0e2fd7571f16955e213be075db: Status 404 returned error can't find the container with id 29d99e07bdb080f03f24350cbd951da3501f6b0e2fd7571f16955e213be075db
Sep 16 02:49:23.544670 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:23.544589    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-a53a44-jckf7" event={"ID":"6045a693-3494-4367-beb9-60a78e7fd1d3","Type":"ContainerStarted","Data":"57f7335f2ac8414543d5a92f64556f5e5d56555c68ebd0120c15770ab8c2da99"}
Sep 16 02:49:23.544670 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:23.544651    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-a53a44-jckf7" event={"ID":"6045a693-3494-4367-beb9-60a78e7fd1d3","Type":"ContainerStarted","Data":"29d99e07bdb080f03f24350cbd951da3501f6b0e2fd7571f16955e213be075db"}
Sep 16 02:49:23.559509 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:23.559442    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-a53a44-jckf7" podStartSLOduration=1.5594248670000002 podStartE2EDuration="1.559424867s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 02:49:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 02:49:23.557617833 +0000 UTC m=+2183.941133078" watchObservedRunningTime="2026-09-16 02:49:23.559424867 +0000 UTC m=+2183.942940111"
Sep 16 02:49:24.552102 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:24.552071    2579 generic.go:356] "Generic (PLEG): container finished" podID="6045a693-3494-4367-beb9-60a78e7fd1d3" containerID="57f7335f2ac8414543d5a92f64556f5e5d56555c68ebd0120c15770ab8c2da99" exitCode=0
Sep 16 02:49:24.552510 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:24.552139    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-a53a44-jckf7" event={"ID":"6045a693-3494-4367-beb9-60a78e7fd1d3","Type":"ContainerDied","Data":"57f7335f2ac8414543d5a92f64556f5e5d56555c68ebd0120c15770ab8c2da99"}
Sep 16 02:49:25.691067 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:25.691044    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-a53a44-jckf7"
Sep 16 02:49:25.769815 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:25.769785    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6045a693-3494-4367-beb9-60a78e7fd1d3-maas-api-service-ca\") pod \"6045a693-3494-4367-beb9-60a78e7fd1d3\" (UID: \"6045a693-3494-4367-beb9-60a78e7fd1d3\") "
Sep 16 02:49:25.770127 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:25.770104    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6045a693-3494-4367-beb9-60a78e7fd1d3-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "6045a693-3494-4367-beb9-60a78e7fd1d3" (UID: "6045a693-3494-4367-beb9-60a78e7fd1d3"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 02:49:25.871331 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:25.871253    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6045a693-3494-4367-beb9-60a78e7fd1d3-maas-api-service-ca\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:49:26.563416 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:26.563385    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-a53a44-jckf7"
Sep 16 02:49:26.563604 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:26.563422    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-a53a44-jckf7" event={"ID":"6045a693-3494-4367-beb9-60a78e7fd1d3","Type":"ContainerDied","Data":"29d99e07bdb080f03f24350cbd951da3501f6b0e2fd7571f16955e213be075db"}
Sep 16 02:49:26.563604 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:26.563488    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="29d99e07bdb080f03f24350cbd951da3501f6b0e2fd7571f16955e213be075db"
Sep 16 02:49:32.914629 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:32.914589    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:49:32.943951 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:32.943911    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-a53a44-6c64b45999-td78d"]
Sep 16 02:49:32.944195 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:32.944158    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-a53a44-6c64b45999-td78d" podUID="b8e5853d-4887-4741-ae00-33e0959aafea" containerName="payload-processing" containerID="cri-o://6e2a1ce2afa3400410d4b499318a5410483d2c714b91a86eaad4067fd94ec6ad" gracePeriod=30
Sep 16 02:49:32.951566 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:32.951542    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-a53a44-566f584d6-hnt5w"]
Sep 16 02:49:32.951793 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:32.951752    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-a53a44-566f584d6-hnt5w" podUID="b2097872-a162-49ab-95f4-06bd1aa0ed40" containerName="payload-pre-processing" containerID="cri-o://83f341696678104e73c1015b323ebe754055ba138e035a18efa155f2f47db180" gracePeriod=30
Sep 16 02:49:32.984622 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:32.984587    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:49:33.050716 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:33.050679    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:49:33.099713 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:33.099679    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-a53a44-jckf7"]
Sep 16 02:49:33.102641 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:33.101298    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-a53a44-jckf7"]
Sep 16 02:49:33.112664 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:33.112627    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:49:33.215855 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:33.215831    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-a53a44-6c64b45999-td78d"
Sep 16 02:49:33.237956 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:33.237932    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-a53a44-566f584d6-hnt5w"
Sep 16 02:49:33.339069 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:33.339041    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b8e5853d-4887-4741-ae00-33e0959aafea-plugins-config-volume\") pod \"b8e5853d-4887-4741-ae00-33e0959aafea\" (UID: \"b8e5853d-4887-4741-ae00-33e0959aafea\") "
Sep 16 02:49:33.339069 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:33.339084    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rxffg\" (UniqueName: \"kubernetes.io/projected/b2097872-a162-49ab-95f4-06bd1aa0ed40-kube-api-access-rxffg\") pod \"b2097872-a162-49ab-95f4-06bd1aa0ed40\" (UID: \"b2097872-a162-49ab-95f4-06bd1aa0ed40\") "
Sep 16 02:49:33.339311 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:33.339200    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4rvrp\" (UniqueName: \"kubernetes.io/projected/b8e5853d-4887-4741-ae00-33e0959aafea-kube-api-access-4rvrp\") pod \"b8e5853d-4887-4741-ae00-33e0959aafea\" (UID: \"b8e5853d-4887-4741-ae00-33e0959aafea\") "
Sep 16 02:49:33.339311 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:33.339223    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b2097872-a162-49ab-95f4-06bd1aa0ed40-plugins-config-volume\") pod \"b2097872-a162-49ab-95f4-06bd1aa0ed40\" (UID: \"b2097872-a162-49ab-95f4-06bd1aa0ed40\") "
Sep 16 02:49:33.339454 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:33.339426    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b8e5853d-4887-4741-ae00-33e0959aafea-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b8e5853d-4887-4741-ae00-33e0959aafea" (UID: "b8e5853d-4887-4741-ae00-33e0959aafea"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 02:49:33.339715 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:33.339688    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b2097872-a162-49ab-95f4-06bd1aa0ed40-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b2097872-a162-49ab-95f4-06bd1aa0ed40" (UID: "b2097872-a162-49ab-95f4-06bd1aa0ed40"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 02:49:33.341388 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:33.341367    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b8e5853d-4887-4741-ae00-33e0959aafea-kube-api-access-4rvrp" (OuterVolumeSpecName: "kube-api-access-4rvrp") pod "b8e5853d-4887-4741-ae00-33e0959aafea" (UID: "b8e5853d-4887-4741-ae00-33e0959aafea"). InnerVolumeSpecName "kube-api-access-4rvrp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 02:49:33.341487 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:33.341399    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b2097872-a162-49ab-95f4-06bd1aa0ed40-kube-api-access-rxffg" (OuterVolumeSpecName: "kube-api-access-rxffg") pod "b2097872-a162-49ab-95f4-06bd1aa0ed40" (UID: "b2097872-a162-49ab-95f4-06bd1aa0ed40"). InnerVolumeSpecName "kube-api-access-rxffg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 02:49:33.440792 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:33.440761    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4rvrp\" (UniqueName: \"kubernetes.io/projected/b8e5853d-4887-4741-ae00-33e0959aafea-kube-api-access-4rvrp\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:49:33.440792 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:33.440788    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b2097872-a162-49ab-95f4-06bd1aa0ed40-plugins-config-volume\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:49:33.440792 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:33.440797    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b8e5853d-4887-4741-ae00-33e0959aafea-plugins-config-volume\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:49:33.441024 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:33.440806    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rxffg\" (UniqueName: \"kubernetes.io/projected/b2097872-a162-49ab-95f4-06bd1aa0ed40-kube-api-access-rxffg\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:49:33.461988 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:33.461956    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:49:33.600660 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:33.600628    2579 generic.go:356] "Generic (PLEG): container finished" podID="b8e5853d-4887-4741-ae00-33e0959aafea" containerID="6e2a1ce2afa3400410d4b499318a5410483d2c714b91a86eaad4067fd94ec6ad" exitCode=0
Sep 16 02:49:33.600832 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:33.600672    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-a53a44-6c64b45999-td78d"
Sep 16 02:49:33.600832 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:33.600706    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-a53a44-6c64b45999-td78d" event={"ID":"b8e5853d-4887-4741-ae00-33e0959aafea","Type":"ContainerDied","Data":"6e2a1ce2afa3400410d4b499318a5410483d2c714b91a86eaad4067fd94ec6ad"}
Sep 16 02:49:33.600832 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:33.600750    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-a53a44-6c64b45999-td78d" event={"ID":"b8e5853d-4887-4741-ae00-33e0959aafea","Type":"ContainerDied","Data":"0207e9c8e613ba111440f37c96bce593c96fc472ff9aeb5a083e6b1ffeee9c6f"}
Sep 16 02:49:33.600832 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:33.600772    2579 scope.go:117] "RemoveContainer" containerID="6e2a1ce2afa3400410d4b499318a5410483d2c714b91a86eaad4067fd94ec6ad"
Sep 16 02:49:33.602224 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:33.602200    2579 generic.go:356] "Generic (PLEG): container finished" podID="b2097872-a162-49ab-95f4-06bd1aa0ed40" containerID="83f341696678104e73c1015b323ebe754055ba138e035a18efa155f2f47db180" exitCode=0
Sep 16 02:49:33.602327 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:33.602256    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-a53a44-566f584d6-hnt5w"
Sep 16 02:49:33.602394 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:33.602278    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-a53a44-566f584d6-hnt5w" event={"ID":"b2097872-a162-49ab-95f4-06bd1aa0ed40","Type":"ContainerDied","Data":"83f341696678104e73c1015b323ebe754055ba138e035a18efa155f2f47db180"}
Sep 16 02:49:33.602440 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:33.602421    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-a53a44-566f584d6-hnt5w" event={"ID":"b2097872-a162-49ab-95f4-06bd1aa0ed40","Type":"ContainerDied","Data":"aa227b7b8972906bfffacb913e23747a96b2bc310938f253d87aa78f6eaf3903"}
Sep 16 02:49:33.609572 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:33.609552    2579 scope.go:117] "RemoveContainer" containerID="6e2a1ce2afa3400410d4b499318a5410483d2c714b91a86eaad4067fd94ec6ad"
Sep 16 02:49:33.609895 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:49:33.609840    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6e2a1ce2afa3400410d4b499318a5410483d2c714b91a86eaad4067fd94ec6ad\": container with ID starting with 6e2a1ce2afa3400410d4b499318a5410483d2c714b91a86eaad4067fd94ec6ad not found: ID does not exist" containerID="6e2a1ce2afa3400410d4b499318a5410483d2c714b91a86eaad4067fd94ec6ad"
Sep 16 02:49:33.609997 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:33.609906    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e2a1ce2afa3400410d4b499318a5410483d2c714b91a86eaad4067fd94ec6ad"} err="failed to get container status \"6e2a1ce2afa3400410d4b499318a5410483d2c714b91a86eaad4067fd94ec6ad\": rpc error: code = NotFound desc = could not find container \"6e2a1ce2afa3400410d4b499318a5410483d2c714b91a86eaad4067fd94ec6ad\": container with ID starting with 6e2a1ce2afa3400410d4b499318a5410483d2c714b91a86eaad4067fd94ec6ad not found: ID does not exist"
Sep 16 02:49:33.609997 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:33.609928    2579 scope.go:117] "RemoveContainer" containerID="83f341696678104e73c1015b323ebe754055ba138e035a18efa155f2f47db180"
Sep 16 02:49:33.618272 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:33.618257    2579 scope.go:117] "RemoveContainer" containerID="83f341696678104e73c1015b323ebe754055ba138e035a18efa155f2f47db180"
Sep 16 02:49:33.619251 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:49:33.619233    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"83f341696678104e73c1015b323ebe754055ba138e035a18efa155f2f47db180\": container with ID starting with 83f341696678104e73c1015b323ebe754055ba138e035a18efa155f2f47db180 not found: ID does not exist" containerID="83f341696678104e73c1015b323ebe754055ba138e035a18efa155f2f47db180"
Sep 16 02:49:33.619316 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:33.619258    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"83f341696678104e73c1015b323ebe754055ba138e035a18efa155f2f47db180"} err="failed to get container status \"83f341696678104e73c1015b323ebe754055ba138e035a18efa155f2f47db180\": rpc error: code = NotFound desc = could not find container \"83f341696678104e73c1015b323ebe754055ba138e035a18efa155f2f47db180\": container with ID starting with 83f341696678104e73c1015b323ebe754055ba138e035a18efa155f2f47db180 not found: ID does not exist"
Sep 16 02:49:33.625257 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:33.625233    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-a53a44-6c64b45999-td78d"]
Sep 16 02:49:33.628717 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:33.628698    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-a53a44-6c64b45999-td78d"]
Sep 16 02:49:33.639807 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:33.639786    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-a53a44-566f584d6-hnt5w"]
Sep 16 02:49:33.643205 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:33.643180    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-a53a44-566f584d6-hnt5w"]
Sep 16 02:49:34.104877 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:34.104831    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6045a693-3494-4367-beb9-60a78e7fd1d3" path="/var/lib/kubelet/pods/6045a693-3494-4367-beb9-60a78e7fd1d3/volumes"
Sep 16 02:49:34.105411 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:34.105388    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b2097872-a162-49ab-95f4-06bd1aa0ed40" path="/var/lib/kubelet/pods/b2097872-a162-49ab-95f4-06bd1aa0ed40/volumes"
Sep 16 02:49:34.105939 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:34.105915    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b8e5853d-4887-4741-ae00-33e0959aafea" path="/var/lib/kubelet/pods/b8e5853d-4887-4741-ae00-33e0959aafea/volumes"
Sep 16 02:49:39.270562 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:39.270523    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:49:39.381367 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:39.381332    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:49:39.565228 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:39.565153    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:49:41.662686 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:41.662651    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-b878dcb6c-6kg9c"]
Sep 16 02:49:41.663308 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:41.662913    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-b878dcb6c-6kg9c" podUID="8e36faf3-4f71-4f46-8928-45795654eea8" containerName="manager" containerID="cri-o://c53b9cd4475773d7cb95f4d0c9f8caa3c387555c88e24a769c7bd86557b0709f" gracePeriod=30
Sep 16 02:49:41.904108 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:41.904071    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-b878dcb6c-6kg9c"
Sep 16 02:49:42.018517 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:42.018444    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bzlg5\" (UniqueName: \"kubernetes.io/projected/8e36faf3-4f71-4f46-8928-45795654eea8-kube-api-access-bzlg5\") pod \"8e36faf3-4f71-4f46-8928-45795654eea8\" (UID: \"8e36faf3-4f71-4f46-8928-45795654eea8\") "
Sep 16 02:49:42.018719 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:42.018542    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8e36faf3-4f71-4f46-8928-45795654eea8-metrics-certs\") pod \"8e36faf3-4f71-4f46-8928-45795654eea8\" (UID: \"8e36faf3-4f71-4f46-8928-45795654eea8\") "
Sep 16 02:49:42.018719 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:42.018581    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8e36faf3-4f71-4f46-8928-45795654eea8-webhook-cert\") pod \"8e36faf3-4f71-4f46-8928-45795654eea8\" (UID: \"8e36faf3-4f71-4f46-8928-45795654eea8\") "
Sep 16 02:49:42.020765 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:42.020732    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8e36faf3-4f71-4f46-8928-45795654eea8-kube-api-access-bzlg5" (OuterVolumeSpecName: "kube-api-access-bzlg5") pod "8e36faf3-4f71-4f46-8928-45795654eea8" (UID: "8e36faf3-4f71-4f46-8928-45795654eea8"). InnerVolumeSpecName "kube-api-access-bzlg5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 02:49:42.020893 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:42.020776    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8e36faf3-4f71-4f46-8928-45795654eea8-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "8e36faf3-4f71-4f46-8928-45795654eea8" (UID: "8e36faf3-4f71-4f46-8928-45795654eea8"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 02:49:42.020893 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:42.020797    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8e36faf3-4f71-4f46-8928-45795654eea8-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "8e36faf3-4f71-4f46-8928-45795654eea8" (UID: "8e36faf3-4f71-4f46-8928-45795654eea8"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 02:49:42.119714 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:42.119687    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bzlg5\" (UniqueName: \"kubernetes.io/projected/8e36faf3-4f71-4f46-8928-45795654eea8-kube-api-access-bzlg5\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:49:42.119714 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:42.119716    2579 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8e36faf3-4f71-4f46-8928-45795654eea8-metrics-certs\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:49:42.119878 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:42.119728    2579 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8e36faf3-4f71-4f46-8928-45795654eea8-webhook-cert\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:49:42.649782 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:42.649751    2579 generic.go:356] "Generic (PLEG): container finished" podID="8e36faf3-4f71-4f46-8928-45795654eea8" containerID="c53b9cd4475773d7cb95f4d0c9f8caa3c387555c88e24a769c7bd86557b0709f" exitCode=0
Sep 16 02:49:42.649977 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:42.649791    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-b878dcb6c-6kg9c"
Sep 16 02:49:42.649977 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:42.649843    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-b878dcb6c-6kg9c" event={"ID":"8e36faf3-4f71-4f46-8928-45795654eea8","Type":"ContainerDied","Data":"c53b9cd4475773d7cb95f4d0c9f8caa3c387555c88e24a769c7bd86557b0709f"}
Sep 16 02:49:42.649977 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:42.649902    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-b878dcb6c-6kg9c" event={"ID":"8e36faf3-4f71-4f46-8928-45795654eea8","Type":"ContainerDied","Data":"e2b1defc93cb0a6520f2148ae924ef032ad6965ddf9a1e3a5978d39f49d72944"}
Sep 16 02:49:42.649977 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:42.649924    2579 scope.go:117] "RemoveContainer" containerID="c53b9cd4475773d7cb95f4d0c9f8caa3c387555c88e24a769c7bd86557b0709f"
Sep 16 02:49:42.659497 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:42.659459    2579 scope.go:117] "RemoveContainer" containerID="c53b9cd4475773d7cb95f4d0c9f8caa3c387555c88e24a769c7bd86557b0709f"
Sep 16 02:49:42.659771 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:49:42.659754    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c53b9cd4475773d7cb95f4d0c9f8caa3c387555c88e24a769c7bd86557b0709f\": container with ID starting with c53b9cd4475773d7cb95f4d0c9f8caa3c387555c88e24a769c7bd86557b0709f not found: ID does not exist" containerID="c53b9cd4475773d7cb95f4d0c9f8caa3c387555c88e24a769c7bd86557b0709f"
Sep 16 02:49:42.659832 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:42.659777    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c53b9cd4475773d7cb95f4d0c9f8caa3c387555c88e24a769c7bd86557b0709f"} err="failed to get container status \"c53b9cd4475773d7cb95f4d0c9f8caa3c387555c88e24a769c7bd86557b0709f\": rpc error: code = NotFound desc = could not find container \"c53b9cd4475773d7cb95f4d0c9f8caa3c387555c88e24a769c7bd86557b0709f\": container with ID starting with c53b9cd4475773d7cb95f4d0c9f8caa3c387555c88e24a769c7bd86557b0709f not found: ID does not exist"
Sep 16 02:49:42.664886 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:42.664863    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-b878dcb6c-6kg9c"]
Sep 16 02:49:42.668040 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:42.668018    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-b878dcb6c-6kg9c"]
Sep 16 02:49:44.102561 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:49:44.102522    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8e36faf3-4f71-4f46-8928-45795654eea8" path="/var/lib/kubelet/pods/8e36faf3-4f71-4f46-8928-45795654eea8/volumes"
Sep 16 02:50:00.468485 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:50:00.468440    2579 scope.go:117] "RemoveContainer" containerID="107d9927529090744e2eda1749c1bd3806e35224e8b5e40992e39b9b5cdfb372"
Sep 16 02:50:04.102339 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:50:04.102303    2579 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-pre-processing-6cd864bc95-mh4jh" secret="" err="secret \"payload-processing-dockercfg-zzx6j\" not found"
Sep 16 02:50:30.778112 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:50:30.778024    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:50:43.476547 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:50:43.476506    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-b878dcb6c-pzfrm"]
Sep 16 02:50:43.476961 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:50:43.476873    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6045a693-3494-4367-beb9-60a78e7fd1d3" containerName="revoke-keys"
Sep 16 02:50:43.476961 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:50:43.476885    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="6045a693-3494-4367-beb9-60a78e7fd1d3" containerName="revoke-keys"
Sep 16 02:50:43.476961 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:50:43.476900    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8e36faf3-4f71-4f46-8928-45795654eea8" containerName="manager"
Sep 16 02:50:43.476961 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:50:43.476906    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="8e36faf3-4f71-4f46-8928-45795654eea8" containerName="manager"
Sep 16 02:50:43.476961 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:50:43.476921    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b2097872-a162-49ab-95f4-06bd1aa0ed40" containerName="payload-pre-processing"
Sep 16 02:50:43.476961 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:50:43.476928    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="b2097872-a162-49ab-95f4-06bd1aa0ed40" containerName="payload-pre-processing"
Sep 16 02:50:43.476961 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:50:43.476944    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b8e5853d-4887-4741-ae00-33e0959aafea" containerName="payload-processing"
Sep 16 02:50:43.476961 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:50:43.476949    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="b8e5853d-4887-4741-ae00-33e0959aafea" containerName="payload-processing"
Sep 16 02:50:43.477259 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:50:43.477003    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="b2097872-a162-49ab-95f4-06bd1aa0ed40" containerName="payload-pre-processing"
Sep 16 02:50:43.477259 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:50:43.477013    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="b8e5853d-4887-4741-ae00-33e0959aafea" containerName="payload-processing"
Sep 16 02:50:43.477259 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:50:43.477019    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="8e36faf3-4f71-4f46-8928-45795654eea8" containerName="manager"
Sep 16 02:50:43.477259 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:50:43.477027    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="6045a693-3494-4367-beb9-60a78e7fd1d3" containerName="revoke-keys"
Sep 16 02:50:43.480171 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:50:43.480154    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-b878dcb6c-pzfrm"
Sep 16 02:50:43.484848 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:50:43.484813    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 16 02:50:43.484981 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:50:43.484852    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 16 02:50:43.484981 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:50:43.484863    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 16 02:50:43.484981 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:50:43.484861    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-cs9kk\""
Sep 16 02:50:43.488370 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:50:43.488345    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-b878dcb6c-pzfrm"]
Sep 16 02:50:43.666527 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:50:43.666495    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fb9b4d6e-95d8-4616-be9b-afa2046b4dbe-metrics-certs\") pod \"maas-controller-b878dcb6c-pzfrm\" (UID: \"fb9b4d6e-95d8-4616-be9b-afa2046b4dbe\") " pod="opendatahub/maas-controller-b878dcb6c-pzfrm"
Sep 16 02:50:43.666687 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:50:43.666534    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fb9b4d6e-95d8-4616-be9b-afa2046b4dbe-webhook-cert\") pod \"maas-controller-b878dcb6c-pzfrm\" (UID: \"fb9b4d6e-95d8-4616-be9b-afa2046b4dbe\") " pod="opendatahub/maas-controller-b878dcb6c-pzfrm"
Sep 16 02:50:43.666687 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:50:43.666572    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ch5km\" (UniqueName: \"kubernetes.io/projected/fb9b4d6e-95d8-4616-be9b-afa2046b4dbe-kube-api-access-ch5km\") pod \"maas-controller-b878dcb6c-pzfrm\" (UID: \"fb9b4d6e-95d8-4616-be9b-afa2046b4dbe\") " pod="opendatahub/maas-controller-b878dcb6c-pzfrm"
Sep 16 02:50:43.768099 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:50:43.768011    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ch5km\" (UniqueName: \"kubernetes.io/projected/fb9b4d6e-95d8-4616-be9b-afa2046b4dbe-kube-api-access-ch5km\") pod \"maas-controller-b878dcb6c-pzfrm\" (UID: \"fb9b4d6e-95d8-4616-be9b-afa2046b4dbe\") " pod="opendatahub/maas-controller-b878dcb6c-pzfrm"
Sep 16 02:50:43.768265 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:50:43.768119    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fb9b4d6e-95d8-4616-be9b-afa2046b4dbe-metrics-certs\") pod \"maas-controller-b878dcb6c-pzfrm\" (UID: \"fb9b4d6e-95d8-4616-be9b-afa2046b4dbe\") " pod="opendatahub/maas-controller-b878dcb6c-pzfrm"
Sep 16 02:50:43.768265 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:50:43.768150    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fb9b4d6e-95d8-4616-be9b-afa2046b4dbe-webhook-cert\") pod \"maas-controller-b878dcb6c-pzfrm\" (UID: \"fb9b4d6e-95d8-4616-be9b-afa2046b4dbe\") " pod="opendatahub/maas-controller-b878dcb6c-pzfrm"
Sep 16 02:50:43.770727 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:50:43.770704    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fb9b4d6e-95d8-4616-be9b-afa2046b4dbe-webhook-cert\") pod \"maas-controller-b878dcb6c-pzfrm\" (UID: \"fb9b4d6e-95d8-4616-be9b-afa2046b4dbe\") " pod="opendatahub/maas-controller-b878dcb6c-pzfrm"
Sep 16 02:50:43.770832 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:50:43.770734    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fb9b4d6e-95d8-4616-be9b-afa2046b4dbe-metrics-certs\") pod \"maas-controller-b878dcb6c-pzfrm\" (UID: \"fb9b4d6e-95d8-4616-be9b-afa2046b4dbe\") " pod="opendatahub/maas-controller-b878dcb6c-pzfrm"
Sep 16 02:50:43.775376 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:50:43.775359    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ch5km\" (UniqueName: \"kubernetes.io/projected/fb9b4d6e-95d8-4616-be9b-afa2046b4dbe-kube-api-access-ch5km\") pod \"maas-controller-b878dcb6c-pzfrm\" (UID: \"fb9b4d6e-95d8-4616-be9b-afa2046b4dbe\") " pod="opendatahub/maas-controller-b878dcb6c-pzfrm"
Sep 16 02:50:43.791196 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:50:43.791177    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-b878dcb6c-pzfrm"
Sep 16 02:50:44.061216 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:50:44.061191    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-b878dcb6c-pzfrm"]
Sep 16 02:50:44.062897 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:50:44.062866    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb9b4d6e_95d8_4616_be9b_afa2046b4dbe.slice/crio-09a1410a1a9ea16e2864e1d0e43b26457549b3873f40df497f7f87d393dd2085 WatchSource:0}: Error finding container 09a1410a1a9ea16e2864e1d0e43b26457549b3873f40df497f7f87d393dd2085: Status 404 returned error can't find the container with id 09a1410a1a9ea16e2864e1d0e43b26457549b3873f40df497f7f87d393dd2085
Sep 16 02:50:44.064202 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:50:44.064178    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 16 02:50:44.963776 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:50:44.963744    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-b878dcb6c-pzfrm" event={"ID":"fb9b4d6e-95d8-4616-be9b-afa2046b4dbe","Type":"ContainerStarted","Data":"13f7d3eadc7f75fc5acbd2d33fd0b98fb9c510156dca41b8a03a6d22f5adf2f5"}
Sep 16 02:50:44.964223 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:50:44.963799    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-b878dcb6c-pzfrm" event={"ID":"fb9b4d6e-95d8-4616-be9b-afa2046b4dbe","Type":"ContainerStarted","Data":"09a1410a1a9ea16e2864e1d0e43b26457549b3873f40df497f7f87d393dd2085"}
Sep 16 02:50:44.964223 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:50:44.963825    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-b878dcb6c-pzfrm"
Sep 16 02:50:44.981990 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:50:44.981943    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-b878dcb6c-pzfrm" podStartSLOduration=1.574007366 podStartE2EDuration="1.981929045s" totalImagesPullingTime="407.921679ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 02:50:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 02:50:44.979984897 +0000 UTC m=+2265.363500166" watchObservedRunningTime="2026-09-16 02:50:44.981929045 +0000 UTC m=+2265.365444290"
Sep 16 02:50:55.971127 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:50:55.971092    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-b878dcb6c-pzfrm"
Sep 16 02:51:00.502000 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:51:00.501968    2579 scope.go:117] "RemoveContainer" containerID="cf4292e5cf32ed491f0b05d9283e3a27d791a40dee750b1f29a42732ed98b664"
Sep 16 02:51:00.510559 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:51:00.510539    2579 scope.go:117] "RemoveContainer" containerID="9fa31aa345bc7cef5511bf9df1a05e7189e43a0e55ebbc1b7d4dc09d67b487a0"
Sep 16 02:51:03.065849 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:51:03.065813    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:51:03.163115 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:51:03.163089    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-b878dcb6c-pzfrm"]
Sep 16 02:51:03.163381 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:51:03.163289    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-b878dcb6c-pzfrm" podUID="fb9b4d6e-95d8-4616-be9b-afa2046b4dbe" containerName="manager" containerID="cri-o://13f7d3eadc7f75fc5acbd2d33fd0b98fb9c510156dca41b8a03a6d22f5adf2f5" gracePeriod=30
Sep 16 02:51:03.409169 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:51:03.409143    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-b878dcb6c-pzfrm"
Sep 16 02:51:03.543309 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:51:03.543266    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ch5km\" (UniqueName: \"kubernetes.io/projected/fb9b4d6e-95d8-4616-be9b-afa2046b4dbe-kube-api-access-ch5km\") pod \"fb9b4d6e-95d8-4616-be9b-afa2046b4dbe\" (UID: \"fb9b4d6e-95d8-4616-be9b-afa2046b4dbe\") "
Sep 16 02:51:03.543309 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:51:03.543312    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fb9b4d6e-95d8-4616-be9b-afa2046b4dbe-webhook-cert\") pod \"fb9b4d6e-95d8-4616-be9b-afa2046b4dbe\" (UID: \"fb9b4d6e-95d8-4616-be9b-afa2046b4dbe\") "
Sep 16 02:51:03.543549 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:51:03.543374    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fb9b4d6e-95d8-4616-be9b-afa2046b4dbe-metrics-certs\") pod \"fb9b4d6e-95d8-4616-be9b-afa2046b4dbe\" (UID: \"fb9b4d6e-95d8-4616-be9b-afa2046b4dbe\") "
Sep 16 02:51:03.545555 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:51:03.545516    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fb9b4d6e-95d8-4616-be9b-afa2046b4dbe-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "fb9b4d6e-95d8-4616-be9b-afa2046b4dbe" (UID: "fb9b4d6e-95d8-4616-be9b-afa2046b4dbe"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 02:51:03.545699 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:51:03.545564    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fb9b4d6e-95d8-4616-be9b-afa2046b4dbe-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "fb9b4d6e-95d8-4616-be9b-afa2046b4dbe" (UID: "fb9b4d6e-95d8-4616-be9b-afa2046b4dbe"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 02:51:03.545699 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:51:03.545564    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb9b4d6e-95d8-4616-be9b-afa2046b4dbe-kube-api-access-ch5km" (OuterVolumeSpecName: "kube-api-access-ch5km") pod "fb9b4d6e-95d8-4616-be9b-afa2046b4dbe" (UID: "fb9b4d6e-95d8-4616-be9b-afa2046b4dbe"). InnerVolumeSpecName "kube-api-access-ch5km". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 02:51:03.644733 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:51:03.644663    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ch5km\" (UniqueName: \"kubernetes.io/projected/fb9b4d6e-95d8-4616-be9b-afa2046b4dbe-kube-api-access-ch5km\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:51:03.644733 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:51:03.644686    2579 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fb9b4d6e-95d8-4616-be9b-afa2046b4dbe-webhook-cert\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:51:03.644733 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:51:03.644695    2579 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fb9b4d6e-95d8-4616-be9b-afa2046b4dbe-metrics-certs\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:51:04.060737 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:51:04.060704    2579 generic.go:356] "Generic (PLEG): container finished" podID="fb9b4d6e-95d8-4616-be9b-afa2046b4dbe" containerID="13f7d3eadc7f75fc5acbd2d33fd0b98fb9c510156dca41b8a03a6d22f5adf2f5" exitCode=0
Sep 16 02:51:04.060920 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:51:04.060756    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-b878dcb6c-pzfrm"
Sep 16 02:51:04.060920 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:51:04.060761    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-b878dcb6c-pzfrm" event={"ID":"fb9b4d6e-95d8-4616-be9b-afa2046b4dbe","Type":"ContainerDied","Data":"13f7d3eadc7f75fc5acbd2d33fd0b98fb9c510156dca41b8a03a6d22f5adf2f5"}
Sep 16 02:51:04.060920 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:51:04.060813    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-b878dcb6c-pzfrm" event={"ID":"fb9b4d6e-95d8-4616-be9b-afa2046b4dbe","Type":"ContainerDied","Data":"09a1410a1a9ea16e2864e1d0e43b26457549b3873f40df497f7f87d393dd2085"}
Sep 16 02:51:04.060920 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:51:04.060845    2579 scope.go:117] "RemoveContainer" containerID="13f7d3eadc7f75fc5acbd2d33fd0b98fb9c510156dca41b8a03a6d22f5adf2f5"
Sep 16 02:51:04.070051 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:51:04.070034    2579 scope.go:117] "RemoveContainer" containerID="13f7d3eadc7f75fc5acbd2d33fd0b98fb9c510156dca41b8a03a6d22f5adf2f5"
Sep 16 02:51:04.070321 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:51:04.070303    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"13f7d3eadc7f75fc5acbd2d33fd0b98fb9c510156dca41b8a03a6d22f5adf2f5\": container with ID starting with 13f7d3eadc7f75fc5acbd2d33fd0b98fb9c510156dca41b8a03a6d22f5adf2f5 not found: ID does not exist" containerID="13f7d3eadc7f75fc5acbd2d33fd0b98fb9c510156dca41b8a03a6d22f5adf2f5"
Sep 16 02:51:04.070362 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:51:04.070329    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"13f7d3eadc7f75fc5acbd2d33fd0b98fb9c510156dca41b8a03a6d22f5adf2f5"} err="failed to get container status \"13f7d3eadc7f75fc5acbd2d33fd0b98fb9c510156dca41b8a03a6d22f5adf2f5\": rpc error: code = NotFound desc = could not find container \"13f7d3eadc7f75fc5acbd2d33fd0b98fb9c510156dca41b8a03a6d22f5adf2f5\": container with ID starting with 13f7d3eadc7f75fc5acbd2d33fd0b98fb9c510156dca41b8a03a6d22f5adf2f5 not found: ID does not exist"
Sep 16 02:51:04.081323 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:51:04.081302    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-b878dcb6c-pzfrm"]
Sep 16 02:51:04.086487 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:51:04.086453    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-b878dcb6c-pzfrm"]
Sep 16 02:51:04.101876 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:51:04.101844    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fb9b4d6e-95d8-4616-be9b-afa2046b4dbe" path="/var/lib/kubelet/pods/fb9b4d6e-95d8-4616-be9b-afa2046b4dbe/volumes"
Sep 16 02:51:04.920722 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:51:04.920682    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:51:05.027448 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:51:05.027408    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-b878dcb6c-8j8cx"]
Sep 16 02:51:05.027874 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:51:05.027855    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb9b4d6e-95d8-4616-be9b-afa2046b4dbe" containerName="manager"
Sep 16 02:51:05.028042 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:51:05.027878    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb9b4d6e-95d8-4616-be9b-afa2046b4dbe" containerName="manager"
Sep 16 02:51:05.028042 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:51:05.027967    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="fb9b4d6e-95d8-4616-be9b-afa2046b4dbe" containerName="manager"
Sep 16 02:51:05.030869 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:51:05.030850    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-b878dcb6c-8j8cx"
Sep 16 02:51:05.032893 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:51:05.032872    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-cs9kk\""
Sep 16 02:51:05.033715 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:51:05.033694    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 16 02:51:05.033819 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:51:05.033757    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 16 02:51:05.033894 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:51:05.033844    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 16 02:51:05.041434 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:51:05.041408    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-b878dcb6c-8j8cx"]
Sep 16 02:51:05.157682 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:51:05.157648    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-76rzv\" (UniqueName: \"kubernetes.io/projected/1338df10-6da2-451a-8370-b7078c273476-kube-api-access-76rzv\") pod \"maas-controller-b878dcb6c-8j8cx\" (UID: \"1338df10-6da2-451a-8370-b7078c273476\") " pod="opendatahub/maas-controller-b878dcb6c-8j8cx"
Sep 16 02:51:05.158083 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:51:05.157713    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1338df10-6da2-451a-8370-b7078c273476-webhook-cert\") pod \"maas-controller-b878dcb6c-8j8cx\" (UID: \"1338df10-6da2-451a-8370-b7078c273476\") " pod="opendatahub/maas-controller-b878dcb6c-8j8cx"
Sep 16 02:51:05.158083 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:51:05.157758    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1338df10-6da2-451a-8370-b7078c273476-metrics-certs\") pod \"maas-controller-b878dcb6c-8j8cx\" (UID: \"1338df10-6da2-451a-8370-b7078c273476\") " pod="opendatahub/maas-controller-b878dcb6c-8j8cx"
Sep 16 02:51:05.258340 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:51:05.258307    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-76rzv\" (UniqueName: \"kubernetes.io/projected/1338df10-6da2-451a-8370-b7078c273476-kube-api-access-76rzv\") pod \"maas-controller-b878dcb6c-8j8cx\" (UID: \"1338df10-6da2-451a-8370-b7078c273476\") " pod="opendatahub/maas-controller-b878dcb6c-8j8cx"
Sep 16 02:51:05.258520 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:51:05.258376    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1338df10-6da2-451a-8370-b7078c273476-webhook-cert\") pod \"maas-controller-b878dcb6c-8j8cx\" (UID: \"1338df10-6da2-451a-8370-b7078c273476\") " pod="opendatahub/maas-controller-b878dcb6c-8j8cx"
Sep 16 02:51:05.258520 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:51:05.258413    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1338df10-6da2-451a-8370-b7078c273476-metrics-certs\") pod \"maas-controller-b878dcb6c-8j8cx\" (UID: \"1338df10-6da2-451a-8370-b7078c273476\") " pod="opendatahub/maas-controller-b878dcb6c-8j8cx"
Sep 16 02:51:05.260966 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:51:05.260943    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1338df10-6da2-451a-8370-b7078c273476-metrics-certs\") pod \"maas-controller-b878dcb6c-8j8cx\" (UID: \"1338df10-6da2-451a-8370-b7078c273476\") " pod="opendatahub/maas-controller-b878dcb6c-8j8cx"
Sep 16 02:51:05.261074 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:51:05.260975    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1338df10-6da2-451a-8370-b7078c273476-webhook-cert\") pod \"maas-controller-b878dcb6c-8j8cx\" (UID: \"1338df10-6da2-451a-8370-b7078c273476\") " pod="opendatahub/maas-controller-b878dcb6c-8j8cx"
Sep 16 02:51:05.265493 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:51:05.265450    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-76rzv\" (UniqueName: \"kubernetes.io/projected/1338df10-6da2-451a-8370-b7078c273476-kube-api-access-76rzv\") pod \"maas-controller-b878dcb6c-8j8cx\" (UID: \"1338df10-6da2-451a-8370-b7078c273476\") " pod="opendatahub/maas-controller-b878dcb6c-8j8cx"
Sep 16 02:51:05.341270 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:51:05.341242    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-b878dcb6c-8j8cx"
Sep 16 02:51:05.612749 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:51:05.612669    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-b878dcb6c-8j8cx"]
Sep 16 02:51:05.615866 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:51:05.615838    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1338df10_6da2_451a_8370_b7078c273476.slice/crio-0c6dde60d662ff9a721bf77bceb160bf06ac843aa401e22498d9e0a65c8faaad WatchSource:0}: Error finding container 0c6dde60d662ff9a721bf77bceb160bf06ac843aa401e22498d9e0a65c8faaad: Status 404 returned error can't find the container with id 0c6dde60d662ff9a721bf77bceb160bf06ac843aa401e22498d9e0a65c8faaad
Sep 16 02:51:06.081218 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:51:06.081188    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-b878dcb6c-8j8cx" event={"ID":"1338df10-6da2-451a-8370-b7078c273476","Type":"ContainerStarted","Data":"0c6dde60d662ff9a721bf77bceb160bf06ac843aa401e22498d9e0a65c8faaad"}
Sep 16 02:51:07.089192 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:51:07.089157    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-b878dcb6c-8j8cx" event={"ID":"1338df10-6da2-451a-8370-b7078c273476","Type":"ContainerStarted","Data":"c508686d8e3d62518be2e19d83d0bd1f2c02cb4dac703bad9cd60d855079fcdf"}
Sep 16 02:51:07.089671 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:51:07.089216    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-b878dcb6c-8j8cx"
Sep 16 02:51:07.107585 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:51:07.107542    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-b878dcb6c-8j8cx" podStartSLOduration=1.676994683 podStartE2EDuration="2.10752647s" totalImagesPullingTime="430.531787ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 02:51:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 02:51:07.105640473 +0000 UTC m=+2287.489155722" watchObservedRunningTime="2026-09-16 02:51:07.10752647 +0000 UTC m=+2287.491041714"
Sep 16 02:51:18.102361 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:51:18.102332    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-b878dcb6c-8j8cx"
Sep 16 02:51:34.097231 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:51:34.097195    2579 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-pre-processing-6cd864bc95-mh4jh" secret="" err="secret \"payload-processing-dockercfg-zzx6j\" not found"
Sep 16 02:51:52.173050 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:51:52.172967    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:52:00.533029 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:52:00.532994    2579 scope.go:117] "RemoveContainer" containerID="e6071188f1162f62c4e6ea7869326b9a1c77b7dd53902df4b26a870447bf3ef2"
Sep 16 02:52:00.541270 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:52:00.541250    2579 scope.go:117] "RemoveContainer" containerID="9c714fd65bc6ea2178df0dcda8caeb9805750aaeaba4e441b372a08fc34899aa"
Sep 16 02:52:41.096224 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:52:41.096190    2579 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-pre-processing-6cd864bc95-mh4jh" secret="" err="secret \"payload-processing-dockercfg-zzx6j\" not found"
Sep 16 02:53:00.266916 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:53:00.266890    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-c8q8z_ebf846fa-af7d-4fdd-9662-b7c533c36ddd/console-operator/2.log"
Sep 16 02:53:00.279350 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:53:00.279325    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-c8q8z_ebf846fa-af7d-4fdd-9662-b7c533c36ddd/console-operator/2.log"
Sep 16 02:53:00.568038 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:53:00.567964    2579 scope.go:117] "RemoveContainer" containerID="52156b8ad30b96431d4551cfa1aa4900d1918d93c64f3b7c5ca280df60db7d37"
Sep 16 02:53:20.581244 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:53:20.581203    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:53:20.681331 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:53:20.681294    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:53:52.318698 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:53:52.318659    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:53:52.419347 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:53:52.419307    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:54:00.586762 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:54:00.586729    2579 scope.go:117] "RemoveContainer" containerID="8f9a7d22d51294604cf5f2c1ac6357285db4a5beb528430c641b7553459a4ff9"
Sep 16 02:54:04.078724 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:54:04.078688    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-d058eb81-kserve-88b98c779-jm28k"]
Sep 16 02:54:04.083211 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:54:04.083193    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-d058eb81-kserve-88b98c779-jm28k"
Sep 16 02:54:04.085452 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:54:04.085428    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-d058eb81-kserve-self-signed-certs\""
Sep 16 02:54:04.089998 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:54:04.089973    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-d058eb81-kserve-88b98c779-jm28k"]
Sep 16 02:54:04.097257 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:54:04.097235    2579 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-pre-processing-6cd864bc95-mh4jh" secret="" err="secret \"payload-processing-dockercfg-zzx6j\" not found"
Sep 16 02:54:04.112919 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:54:04.112886    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:54:04.218139 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:54:04.218105    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c89ddb2d-657a-48f8-9eae-9d085b15b183-tls-certs\") pod \"e2e-same-id-a-d058eb81-kserve-88b98c779-jm28k\" (UID: \"c89ddb2d-657a-48f8-9eae-9d085b15b183\") " pod="llm/e2e-same-id-a-d058eb81-kserve-88b98c779-jm28k"
Sep 16 02:54:04.218280 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:54:04.218164    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c89ddb2d-657a-48f8-9eae-9d085b15b183-tmp-dir\") pod \"e2e-same-id-a-d058eb81-kserve-88b98c779-jm28k\" (UID: \"c89ddb2d-657a-48f8-9eae-9d085b15b183\") " pod="llm/e2e-same-id-a-d058eb81-kserve-88b98c779-jm28k"
Sep 16 02:54:04.218393 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:54:04.218296    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jwh8f\" (UniqueName: \"kubernetes.io/projected/c89ddb2d-657a-48f8-9eae-9d085b15b183-kube-api-access-jwh8f\") pod \"e2e-same-id-a-d058eb81-kserve-88b98c779-jm28k\" (UID: \"c89ddb2d-657a-48f8-9eae-9d085b15b183\") " pod="llm/e2e-same-id-a-d058eb81-kserve-88b98c779-jm28k"
Sep 16 02:54:04.218393 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:54:04.218340    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c89ddb2d-657a-48f8-9eae-9d085b15b183-model-cache\") pod \"e2e-same-id-a-d058eb81-kserve-88b98c779-jm28k\" (UID: \"c89ddb2d-657a-48f8-9eae-9d085b15b183\") " pod="llm/e2e-same-id-a-d058eb81-kserve-88b98c779-jm28k"
Sep 16 02:54:04.218393 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:54:04.218379    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c89ddb2d-657a-48f8-9eae-9d085b15b183-dshm\") pod \"e2e-same-id-a-d058eb81-kserve-88b98c779-jm28k\" (UID: \"c89ddb2d-657a-48f8-9eae-9d085b15b183\") " pod="llm/e2e-same-id-a-d058eb81-kserve-88b98c779-jm28k"
Sep 16 02:54:04.218800 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:54:04.218542    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c89ddb2d-657a-48f8-9eae-9d085b15b183-home\") pod \"e2e-same-id-a-d058eb81-kserve-88b98c779-jm28k\" (UID: \"c89ddb2d-657a-48f8-9eae-9d085b15b183\") " pod="llm/e2e-same-id-a-d058eb81-kserve-88b98c779-jm28k"
Sep 16 02:54:04.319690 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:54:04.319655    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c89ddb2d-657a-48f8-9eae-9d085b15b183-home\") pod \"e2e-same-id-a-d058eb81-kserve-88b98c779-jm28k\" (UID: \"c89ddb2d-657a-48f8-9eae-9d085b15b183\") " pod="llm/e2e-same-id-a-d058eb81-kserve-88b98c779-jm28k"
Sep 16 02:54:04.319844 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:54:04.319703    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c89ddb2d-657a-48f8-9eae-9d085b15b183-tls-certs\") pod \"e2e-same-id-a-d058eb81-kserve-88b98c779-jm28k\" (UID: \"c89ddb2d-657a-48f8-9eae-9d085b15b183\") " pod="llm/e2e-same-id-a-d058eb81-kserve-88b98c779-jm28k"
Sep 16 02:54:04.319844 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:54:04.319831    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c89ddb2d-657a-48f8-9eae-9d085b15b183-tmp-dir\") pod \"e2e-same-id-a-d058eb81-kserve-88b98c779-jm28k\" (UID: \"c89ddb2d-657a-48f8-9eae-9d085b15b183\") " pod="llm/e2e-same-id-a-d058eb81-kserve-88b98c779-jm28k"
Sep 16 02:54:04.319958 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:54:04.319935    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jwh8f\" (UniqueName: \"kubernetes.io/projected/c89ddb2d-657a-48f8-9eae-9d085b15b183-kube-api-access-jwh8f\") pod \"e2e-same-id-a-d058eb81-kserve-88b98c779-jm28k\" (UID: \"c89ddb2d-657a-48f8-9eae-9d085b15b183\") " pod="llm/e2e-same-id-a-d058eb81-kserve-88b98c779-jm28k"
Sep 16 02:54:04.320016 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:54:04.319968    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c89ddb2d-657a-48f8-9eae-9d085b15b183-model-cache\") pod \"e2e-same-id-a-d058eb81-kserve-88b98c779-jm28k\" (UID: \"c89ddb2d-657a-48f8-9eae-9d085b15b183\") " pod="llm/e2e-same-id-a-d058eb81-kserve-88b98c779-jm28k"
Sep 16 02:54:04.320016 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:54:04.320007    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c89ddb2d-657a-48f8-9eae-9d085b15b183-dshm\") pod \"e2e-same-id-a-d058eb81-kserve-88b98c779-jm28k\" (UID: \"c89ddb2d-657a-48f8-9eae-9d085b15b183\") " pod="llm/e2e-same-id-a-d058eb81-kserve-88b98c779-jm28k"
Sep 16 02:54:04.320127 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:54:04.320087    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c89ddb2d-657a-48f8-9eae-9d085b15b183-home\") pod \"e2e-same-id-a-d058eb81-kserve-88b98c779-jm28k\" (UID: \"c89ddb2d-657a-48f8-9eae-9d085b15b183\") " pod="llm/e2e-same-id-a-d058eb81-kserve-88b98c779-jm28k"
Sep 16 02:54:04.320188 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:54:04.320155    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c89ddb2d-657a-48f8-9eae-9d085b15b183-tmp-dir\") pod \"e2e-same-id-a-d058eb81-kserve-88b98c779-jm28k\" (UID: \"c89ddb2d-657a-48f8-9eae-9d085b15b183\") " pod="llm/e2e-same-id-a-d058eb81-kserve-88b98c779-jm28k"
Sep 16 02:54:04.320380 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:54:04.320359    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c89ddb2d-657a-48f8-9eae-9d085b15b183-model-cache\") pod \"e2e-same-id-a-d058eb81-kserve-88b98c779-jm28k\" (UID: \"c89ddb2d-657a-48f8-9eae-9d085b15b183\") " pod="llm/e2e-same-id-a-d058eb81-kserve-88b98c779-jm28k"
Sep 16 02:54:04.322220 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:54:04.322198    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c89ddb2d-657a-48f8-9eae-9d085b15b183-dshm\") pod \"e2e-same-id-a-d058eb81-kserve-88b98c779-jm28k\" (UID: \"c89ddb2d-657a-48f8-9eae-9d085b15b183\") " pod="llm/e2e-same-id-a-d058eb81-kserve-88b98c779-jm28k"
Sep 16 02:54:04.322380 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:54:04.322362    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c89ddb2d-657a-48f8-9eae-9d085b15b183-tls-certs\") pod \"e2e-same-id-a-d058eb81-kserve-88b98c779-jm28k\" (UID: \"c89ddb2d-657a-48f8-9eae-9d085b15b183\") " pod="llm/e2e-same-id-a-d058eb81-kserve-88b98c779-jm28k"
Sep 16 02:54:04.327219 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:54:04.327199    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jwh8f\" (UniqueName: \"kubernetes.io/projected/c89ddb2d-657a-48f8-9eae-9d085b15b183-kube-api-access-jwh8f\") pod \"e2e-same-id-a-d058eb81-kserve-88b98c779-jm28k\" (UID: \"c89ddb2d-657a-48f8-9eae-9d085b15b183\") " pod="llm/e2e-same-id-a-d058eb81-kserve-88b98c779-jm28k"
Sep 16 02:54:04.394492 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:54:04.394405    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-d058eb81-kserve-88b98c779-jm28k"
Sep 16 02:54:04.669572 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:54:04.669548    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-d058eb81-kserve-88b98c779-jm28k"]
Sep 16 02:54:04.671516 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:54:04.671484    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc89ddb2d_657a_48f8_9eae_9d085b15b183.slice/crio-7c5766f527cca374541836b39be7bc3d0915064eaeb1ead0cd0924ea13d27b90 WatchSource:0}: Error finding container 7c5766f527cca374541836b39be7bc3d0915064eaeb1ead0cd0924ea13d27b90: Status 404 returned error can't find the container with id 7c5766f527cca374541836b39be7bc3d0915064eaeb1ead0cd0924ea13d27b90
Sep 16 02:54:04.956594 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:54:04.956563    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-d058eb81-kserve-88b98c779-jm28k" event={"ID":"c89ddb2d-657a-48f8-9eae-9d085b15b183","Type":"ContainerStarted","Data":"d9b86d306fef27f9119ac52b00039db2df8eb3cd7b4d15f2af9d4062395b6ca3"}
Sep 16 02:54:04.956780 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:54:04.956603    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-d058eb81-kserve-88b98c779-jm28k" event={"ID":"c89ddb2d-657a-48f8-9eae-9d085b15b183","Type":"ContainerStarted","Data":"7c5766f527cca374541836b39be7bc3d0915064eaeb1ead0cd0924ea13d27b90"}
Sep 16 02:54:04.973527 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:54:04.973446    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-d058eb81-kserve-88b98c779-jm28k" podStartSLOduration=0.9734311 podStartE2EDuration="973.4311ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 02:54:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 02:54:04.972008118 +0000 UTC m=+2465.355523365" watchObservedRunningTime="2026-09-16 02:54:04.9734311 +0000 UTC m=+2465.356946344"
Sep 16 02:54:10.770726 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:54:10.770696    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-d058eb81-kserve-8dcf765cf-h2vqf"]
Sep 16 02:54:10.774526 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:54:10.774500    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-d058eb81-kserve-8dcf765cf-h2vqf"
Sep 16 02:54:10.777015 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:54:10.776989    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-d058eb81-kserve-self-signed-certs\""
Sep 16 02:54:10.785570 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:54:10.785547    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-d058eb81-kserve-8dcf765cf-h2vqf"]
Sep 16 02:54:10.806980 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:54:10.806951    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:54:10.870310 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:54:10.870278    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fdf2e873-f961-4e2b-85fd-123f2a7c3de8-dshm\") pod \"e2e-same-id-b-d058eb81-kserve-8dcf765cf-h2vqf\" (UID: \"fdf2e873-f961-4e2b-85fd-123f2a7c3de8\") " pod="llm/e2e-same-id-b-d058eb81-kserve-8dcf765cf-h2vqf"
Sep 16 02:54:10.870448 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:54:10.870314    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fdf2e873-f961-4e2b-85fd-123f2a7c3de8-tls-certs\") pod \"e2e-same-id-b-d058eb81-kserve-8dcf765cf-h2vqf\" (UID: \"fdf2e873-f961-4e2b-85fd-123f2a7c3de8\") " pod="llm/e2e-same-id-b-d058eb81-kserve-8dcf765cf-h2vqf"
Sep 16 02:54:10.870448 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:54:10.870350    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7l95b\" (UniqueName: \"kubernetes.io/projected/fdf2e873-f961-4e2b-85fd-123f2a7c3de8-kube-api-access-7l95b\") pod \"e2e-same-id-b-d058eb81-kserve-8dcf765cf-h2vqf\" (UID: \"fdf2e873-f961-4e2b-85fd-123f2a7c3de8\") " pod="llm/e2e-same-id-b-d058eb81-kserve-8dcf765cf-h2vqf"
Sep 16 02:54:10.870448 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:54:10.870400    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fdf2e873-f961-4e2b-85fd-123f2a7c3de8-home\") pod \"e2e-same-id-b-d058eb81-kserve-8dcf765cf-h2vqf\" (UID: \"fdf2e873-f961-4e2b-85fd-123f2a7c3de8\") " pod="llm/e2e-same-id-b-d058eb81-kserve-8dcf765cf-h2vqf"
Sep 16 02:54:10.870586 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:54:10.870490    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fdf2e873-f961-4e2b-85fd-123f2a7c3de8-tmp-dir\") pod \"e2e-same-id-b-d058eb81-kserve-8dcf765cf-h2vqf\" (UID: \"fdf2e873-f961-4e2b-85fd-123f2a7c3de8\") " pod="llm/e2e-same-id-b-d058eb81-kserve-8dcf765cf-h2vqf"
Sep 16 02:54:10.870586 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:54:10.870555    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fdf2e873-f961-4e2b-85fd-123f2a7c3de8-model-cache\") pod \"e2e-same-id-b-d058eb81-kserve-8dcf765cf-h2vqf\" (UID: \"fdf2e873-f961-4e2b-85fd-123f2a7c3de8\") " pod="llm/e2e-same-id-b-d058eb81-kserve-8dcf765cf-h2vqf"
Sep 16 02:54:10.971572 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:54:10.971541    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fdf2e873-f961-4e2b-85fd-123f2a7c3de8-model-cache\") pod \"e2e-same-id-b-d058eb81-kserve-8dcf765cf-h2vqf\" (UID: \"fdf2e873-f961-4e2b-85fd-123f2a7c3de8\") " pod="llm/e2e-same-id-b-d058eb81-kserve-8dcf765cf-h2vqf"
Sep 16 02:54:10.971712 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:54:10.971590    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fdf2e873-f961-4e2b-85fd-123f2a7c3de8-dshm\") pod \"e2e-same-id-b-d058eb81-kserve-8dcf765cf-h2vqf\" (UID: \"fdf2e873-f961-4e2b-85fd-123f2a7c3de8\") " pod="llm/e2e-same-id-b-d058eb81-kserve-8dcf765cf-h2vqf"
Sep 16 02:54:10.971712 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:54:10.971608    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fdf2e873-f961-4e2b-85fd-123f2a7c3de8-tls-certs\") pod \"e2e-same-id-b-d058eb81-kserve-8dcf765cf-h2vqf\" (UID: \"fdf2e873-f961-4e2b-85fd-123f2a7c3de8\") " pod="llm/e2e-same-id-b-d058eb81-kserve-8dcf765cf-h2vqf"
Sep 16 02:54:10.971712 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:54:10.971633    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7l95b\" (UniqueName: \"kubernetes.io/projected/fdf2e873-f961-4e2b-85fd-123f2a7c3de8-kube-api-access-7l95b\") pod \"e2e-same-id-b-d058eb81-kserve-8dcf765cf-h2vqf\" (UID: \"fdf2e873-f961-4e2b-85fd-123f2a7c3de8\") " pod="llm/e2e-same-id-b-d058eb81-kserve-8dcf765cf-h2vqf"
Sep 16 02:54:10.971712 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:54:10.971656    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fdf2e873-f961-4e2b-85fd-123f2a7c3de8-home\") pod \"e2e-same-id-b-d058eb81-kserve-8dcf765cf-h2vqf\" (UID: \"fdf2e873-f961-4e2b-85fd-123f2a7c3de8\") " pod="llm/e2e-same-id-b-d058eb81-kserve-8dcf765cf-h2vqf"
Sep 16 02:54:10.971712 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:54:10.971694    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fdf2e873-f961-4e2b-85fd-123f2a7c3de8-tmp-dir\") pod \"e2e-same-id-b-d058eb81-kserve-8dcf765cf-h2vqf\" (UID: \"fdf2e873-f961-4e2b-85fd-123f2a7c3de8\") " pod="llm/e2e-same-id-b-d058eb81-kserve-8dcf765cf-h2vqf"
Sep 16 02:54:10.972018 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:54:10.971992    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fdf2e873-f961-4e2b-85fd-123f2a7c3de8-model-cache\") pod \"e2e-same-id-b-d058eb81-kserve-8dcf765cf-h2vqf\" (UID: \"fdf2e873-f961-4e2b-85fd-123f2a7c3de8\") " pod="llm/e2e-same-id-b-d058eb81-kserve-8dcf765cf-h2vqf"
Sep 16 02:54:10.972095 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:54:10.972070    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fdf2e873-f961-4e2b-85fd-123f2a7c3de8-tmp-dir\") pod \"e2e-same-id-b-d058eb81-kserve-8dcf765cf-h2vqf\" (UID: \"fdf2e873-f961-4e2b-85fd-123f2a7c3de8\") " pod="llm/e2e-same-id-b-d058eb81-kserve-8dcf765cf-h2vqf"
Sep 16 02:54:10.972144 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:54:10.972109    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fdf2e873-f961-4e2b-85fd-123f2a7c3de8-home\") pod \"e2e-same-id-b-d058eb81-kserve-8dcf765cf-h2vqf\" (UID: \"fdf2e873-f961-4e2b-85fd-123f2a7c3de8\") " pod="llm/e2e-same-id-b-d058eb81-kserve-8dcf765cf-h2vqf"
Sep 16 02:54:10.974078 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:54:10.974055    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fdf2e873-f961-4e2b-85fd-123f2a7c3de8-dshm\") pod \"e2e-same-id-b-d058eb81-kserve-8dcf765cf-h2vqf\" (UID: \"fdf2e873-f961-4e2b-85fd-123f2a7c3de8\") " pod="llm/e2e-same-id-b-d058eb81-kserve-8dcf765cf-h2vqf"
Sep 16 02:54:10.974240 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:54:10.974221    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fdf2e873-f961-4e2b-85fd-123f2a7c3de8-tls-certs\") pod \"e2e-same-id-b-d058eb81-kserve-8dcf765cf-h2vqf\" (UID: \"fdf2e873-f961-4e2b-85fd-123f2a7c3de8\") " pod="llm/e2e-same-id-b-d058eb81-kserve-8dcf765cf-h2vqf"
Sep 16 02:54:10.978753 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:54:10.978731    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7l95b\" (UniqueName: \"kubernetes.io/projected/fdf2e873-f961-4e2b-85fd-123f2a7c3de8-kube-api-access-7l95b\") pod \"e2e-same-id-b-d058eb81-kserve-8dcf765cf-h2vqf\" (UID: \"fdf2e873-f961-4e2b-85fd-123f2a7c3de8\") " pod="llm/e2e-same-id-b-d058eb81-kserve-8dcf765cf-h2vqf"
Sep 16 02:54:11.085962 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:54:11.085882    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-d058eb81-kserve-8dcf765cf-h2vqf"
Sep 16 02:54:11.356240 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:54:11.356135    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-d058eb81-kserve-8dcf765cf-h2vqf"]
Sep 16 02:54:11.358349 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:54:11.358319    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfdf2e873_f961_4e2b_85fd_123f2a7c3de8.slice/crio-7548b0a2316c23a23f95ad39a206bace9f93f54f027b91aace124fa724e44cb4 WatchSource:0}: Error finding container 7548b0a2316c23a23f95ad39a206bace9f93f54f027b91aace124fa724e44cb4: Status 404 returned error can't find the container with id 7548b0a2316c23a23f95ad39a206bace9f93f54f027b91aace124fa724e44cb4
Sep 16 02:54:11.994194 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:54:11.994160    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-d058eb81-kserve-8dcf765cf-h2vqf" event={"ID":"fdf2e873-f961-4e2b-85fd-123f2a7c3de8","Type":"ContainerStarted","Data":"4401cb2d24b175243ecf35b2a5c1e06f3479189c36d72186a1a1259393692fad"}
Sep 16 02:54:11.994581 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:54:11.994206    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-d058eb81-kserve-8dcf765cf-h2vqf" event={"ID":"fdf2e873-f961-4e2b-85fd-123f2a7c3de8","Type":"ContainerStarted","Data":"7548b0a2316c23a23f95ad39a206bace9f93f54f027b91aace124fa724e44cb4"}
Sep 16 02:54:12.009515 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:54:12.009451    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-d058eb81-kserve-8dcf765cf-h2vqf" podStartSLOduration=2.009433906 podStartE2EDuration="2.009433906s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 02:54:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 02:54:12.007863041 +0000 UTC m=+2472.391378278" watchObservedRunningTime="2026-09-16 02:54:12.009433906 +0000 UTC m=+2472.392949151"
Sep 16 02:54:14.395522 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:54:14.395459    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-d058eb81-kserve-88b98c779-jm28k"
Sep 16 02:54:14.395907 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:54:14.395545    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-d058eb81-kserve-88b98c779-jm28k"
Sep 16 02:54:14.411371 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:54:14.411337    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-d058eb81-kserve-88b98c779-jm28k"
Sep 16 02:54:15.019160 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:54:15.019127    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-d058eb81-kserve-88b98c779-jm28k"
Sep 16 02:54:21.086280 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:54:21.086240    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-d058eb81-kserve-8dcf765cf-h2vqf"
Sep 16 02:54:21.086816 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:54:21.086299    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-d058eb81-kserve-8dcf765cf-h2vqf"
Sep 16 02:54:21.098632 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:54:21.098606    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-d058eb81-kserve-8dcf765cf-h2vqf"
Sep 16 02:54:22.052386 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:54:22.052354    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-d058eb81-kserve-8dcf765cf-h2vqf"
Sep 16 02:54:29.996905 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:54:29.996855    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:54:49.132202 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:54:49.132168    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-d058eb81-kserve-88b98c779-jm28k"]
Sep 16 02:54:49.151050 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:54:49.151017    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:54:50.983326 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:54:50.983288    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-d058eb81-kserve-8dcf765cf-h2vqf"]
Sep 16 02:54:51.006968 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:54:51.006937    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:55:04.153657 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:55:04.153594    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-d058eb81-kserve-88b98c779-jm28k" podUID="c89ddb2d-657a-48f8-9eae-9d085b15b183" containerName="main" containerID="cri-o://d9b86d306fef27f9119ac52b00039db2df8eb3cd7b4d15f2af9d4062395b6ca3" gracePeriod=45
Sep 16 02:55:04.406295 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:55:04.406229    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-d058eb81-kserve-88b98c779-jm28k"
Sep 16 02:55:04.434624 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:55:04.434594    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c89ddb2d-657a-48f8-9eae-9d085b15b183-tmp-dir\") pod \"c89ddb2d-657a-48f8-9eae-9d085b15b183\" (UID: \"c89ddb2d-657a-48f8-9eae-9d085b15b183\") "
Sep 16 02:55:04.434771 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:55:04.434639    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c89ddb2d-657a-48f8-9eae-9d085b15b183-dshm\") pod \"c89ddb2d-657a-48f8-9eae-9d085b15b183\" (UID: \"c89ddb2d-657a-48f8-9eae-9d085b15b183\") "
Sep 16 02:55:04.434771 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:55:04.434668    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c89ddb2d-657a-48f8-9eae-9d085b15b183-home\") pod \"c89ddb2d-657a-48f8-9eae-9d085b15b183\" (UID: \"c89ddb2d-657a-48f8-9eae-9d085b15b183\") "
Sep 16 02:55:04.434771 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:55:04.434695    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c89ddb2d-657a-48f8-9eae-9d085b15b183-tls-certs\") pod \"c89ddb2d-657a-48f8-9eae-9d085b15b183\" (UID: \"c89ddb2d-657a-48f8-9eae-9d085b15b183\") "
Sep 16 02:55:04.434771 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:55:04.434721    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jwh8f\" (UniqueName: \"kubernetes.io/projected/c89ddb2d-657a-48f8-9eae-9d085b15b183-kube-api-access-jwh8f\") pod \"c89ddb2d-657a-48f8-9eae-9d085b15b183\" (UID: \"c89ddb2d-657a-48f8-9eae-9d085b15b183\") "
Sep 16 02:55:04.434992 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:55:04.434794    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c89ddb2d-657a-48f8-9eae-9d085b15b183-model-cache\") pod \"c89ddb2d-657a-48f8-9eae-9d085b15b183\" (UID: \"c89ddb2d-657a-48f8-9eae-9d085b15b183\") "
Sep 16 02:55:04.434992 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:55:04.434839    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c89ddb2d-657a-48f8-9eae-9d085b15b183-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "c89ddb2d-657a-48f8-9eae-9d085b15b183" (UID: "c89ddb2d-657a-48f8-9eae-9d085b15b183"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 02:55:04.435101 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:55:04.435012    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c89ddb2d-657a-48f8-9eae-9d085b15b183-home" (OuterVolumeSpecName: "home") pod "c89ddb2d-657a-48f8-9eae-9d085b15b183" (UID: "c89ddb2d-657a-48f8-9eae-9d085b15b183"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 02:55:04.435147 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:55:04.435119    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c89ddb2d-657a-48f8-9eae-9d085b15b183-model-cache" (OuterVolumeSpecName: "model-cache") pod "c89ddb2d-657a-48f8-9eae-9d085b15b183" (UID: "c89ddb2d-657a-48f8-9eae-9d085b15b183"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 02:55:04.435204 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:55:04.435140    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c89ddb2d-657a-48f8-9eae-9d085b15b183-tmp-dir\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:55:04.435204 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:55:04.435159    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c89ddb2d-657a-48f8-9eae-9d085b15b183-home\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:55:04.437127 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:55:04.437098    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c89ddb2d-657a-48f8-9eae-9d085b15b183-kube-api-access-jwh8f" (OuterVolumeSpecName: "kube-api-access-jwh8f") pod "c89ddb2d-657a-48f8-9eae-9d085b15b183" (UID: "c89ddb2d-657a-48f8-9eae-9d085b15b183"). InnerVolumeSpecName "kube-api-access-jwh8f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 02:55:04.437226 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:55:04.437210    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c89ddb2d-657a-48f8-9eae-9d085b15b183-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c89ddb2d-657a-48f8-9eae-9d085b15b183" (UID: "c89ddb2d-657a-48f8-9eae-9d085b15b183"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 02:55:04.437275 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:55:04.437254    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c89ddb2d-657a-48f8-9eae-9d085b15b183-dshm" (OuterVolumeSpecName: "dshm") pod "c89ddb2d-657a-48f8-9eae-9d085b15b183" (UID: "c89ddb2d-657a-48f8-9eae-9d085b15b183"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 02:55:04.536324 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:55:04.536290    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c89ddb2d-657a-48f8-9eae-9d085b15b183-dshm\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:55:04.536324 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:55:04.536317    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c89ddb2d-657a-48f8-9eae-9d085b15b183-tls-certs\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:55:04.536324 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:55:04.536332    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jwh8f\" (UniqueName: \"kubernetes.io/projected/c89ddb2d-657a-48f8-9eae-9d085b15b183-kube-api-access-jwh8f\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:55:04.536591 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:55:04.536344    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c89ddb2d-657a-48f8-9eae-9d085b15b183-model-cache\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:55:05.248636 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:55:05.248599    2579 generic.go:356] "Generic (PLEG): container finished" podID="c89ddb2d-657a-48f8-9eae-9d085b15b183" containerID="d9b86d306fef27f9119ac52b00039db2df8eb3cd7b4d15f2af9d4062395b6ca3" exitCode=0
Sep 16 02:55:05.248636 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:55:05.248642    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-d058eb81-kserve-88b98c779-jm28k" event={"ID":"c89ddb2d-657a-48f8-9eae-9d085b15b183","Type":"ContainerDied","Data":"d9b86d306fef27f9119ac52b00039db2df8eb3cd7b4d15f2af9d4062395b6ca3"}
Sep 16 02:55:05.249102 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:55:05.248659    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-d058eb81-kserve-88b98c779-jm28k"
Sep 16 02:55:05.249102 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:55:05.248669    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-d058eb81-kserve-88b98c779-jm28k" event={"ID":"c89ddb2d-657a-48f8-9eae-9d085b15b183","Type":"ContainerDied","Data":"7c5766f527cca374541836b39be7bc3d0915064eaeb1ead0cd0924ea13d27b90"}
Sep 16 02:55:05.249102 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:55:05.248685    2579 scope.go:117] "RemoveContainer" containerID="d9b86d306fef27f9119ac52b00039db2df8eb3cd7b4d15f2af9d4062395b6ca3"
Sep 16 02:55:05.261170 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:55:05.261152    2579 scope.go:117] "RemoveContainer" containerID="d9b86d306fef27f9119ac52b00039db2df8eb3cd7b4d15f2af9d4062395b6ca3"
Sep 16 02:55:05.261487 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:55:05.261450    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d9b86d306fef27f9119ac52b00039db2df8eb3cd7b4d15f2af9d4062395b6ca3\": container with ID starting with d9b86d306fef27f9119ac52b00039db2df8eb3cd7b4d15f2af9d4062395b6ca3 not found: ID does not exist" containerID="d9b86d306fef27f9119ac52b00039db2df8eb3cd7b4d15f2af9d4062395b6ca3"
Sep 16 02:55:05.261559 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:55:05.261498    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d9b86d306fef27f9119ac52b00039db2df8eb3cd7b4d15f2af9d4062395b6ca3"} err="failed to get container status \"d9b86d306fef27f9119ac52b00039db2df8eb3cd7b4d15f2af9d4062395b6ca3\": rpc error: code = NotFound desc = could not find container \"d9b86d306fef27f9119ac52b00039db2df8eb3cd7b4d15f2af9d4062395b6ca3\": container with ID starting with d9b86d306fef27f9119ac52b00039db2df8eb3cd7b4d15f2af9d4062395b6ca3 not found: ID does not exist"
Sep 16 02:55:05.269436 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:55:05.269413    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-d058eb81-kserve-88b98c779-jm28k"]
Sep 16 02:55:05.272302 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:55:05.272280    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-d058eb81-kserve-88b98c779-jm28k"]
Sep 16 02:55:06.005505 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:55:06.005410    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-d058eb81-kserve-8dcf765cf-h2vqf" podUID="fdf2e873-f961-4e2b-85fd-123f2a7c3de8" containerName="main" containerID="cri-o://4401cb2d24b175243ecf35b2a5c1e06f3479189c36d72186a1a1259393692fad" gracePeriod=45
Sep 16 02:55:06.103237 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:55:06.103202    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c89ddb2d-657a-48f8-9eae-9d085b15b183" path="/var/lib/kubelet/pods/c89ddb2d-657a-48f8-9eae-9d085b15b183/volumes"
Sep 16 02:55:06.253701 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:55:06.253675    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-d058eb81-kserve-8dcf765cf-h2vqf"
Sep 16 02:55:06.255161 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:55:06.255133    2579 generic.go:356] "Generic (PLEG): container finished" podID="fdf2e873-f961-4e2b-85fd-123f2a7c3de8" containerID="4401cb2d24b175243ecf35b2a5c1e06f3479189c36d72186a1a1259393692fad" exitCode=0
Sep 16 02:55:06.255281 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:55:06.255169    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-d058eb81-kserve-8dcf765cf-h2vqf"
Sep 16 02:55:06.255281 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:55:06.255223    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-d058eb81-kserve-8dcf765cf-h2vqf" event={"ID":"fdf2e873-f961-4e2b-85fd-123f2a7c3de8","Type":"ContainerDied","Data":"4401cb2d24b175243ecf35b2a5c1e06f3479189c36d72186a1a1259393692fad"}
Sep 16 02:55:06.255281 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:55:06.255262    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-d058eb81-kserve-8dcf765cf-h2vqf" event={"ID":"fdf2e873-f961-4e2b-85fd-123f2a7c3de8","Type":"ContainerDied","Data":"7548b0a2316c23a23f95ad39a206bace9f93f54f027b91aace124fa724e44cb4"}
Sep 16 02:55:06.255426 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:55:06.255285    2579 scope.go:117] "RemoveContainer" containerID="4401cb2d24b175243ecf35b2a5c1e06f3479189c36d72186a1a1259393692fad"
Sep 16 02:55:06.263452 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:55:06.263435    2579 scope.go:117] "RemoveContainer" containerID="4401cb2d24b175243ecf35b2a5c1e06f3479189c36d72186a1a1259393692fad"
Sep 16 02:55:06.263750 ip-10-0-136-118 kubenswrapper[2579]: E0916 02:55:06.263731    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4401cb2d24b175243ecf35b2a5c1e06f3479189c36d72186a1a1259393692fad\": container with ID starting with 4401cb2d24b175243ecf35b2a5c1e06f3479189c36d72186a1a1259393692fad not found: ID does not exist" containerID="4401cb2d24b175243ecf35b2a5c1e06f3479189c36d72186a1a1259393692fad"
Sep 16 02:55:06.263806 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:55:06.263760    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4401cb2d24b175243ecf35b2a5c1e06f3479189c36d72186a1a1259393692fad"} err="failed to get container status \"4401cb2d24b175243ecf35b2a5c1e06f3479189c36d72186a1a1259393692fad\": rpc error: code = NotFound desc = could not find container \"4401cb2d24b175243ecf35b2a5c1e06f3479189c36d72186a1a1259393692fad\": container with ID starting with 4401cb2d24b175243ecf35b2a5c1e06f3479189c36d72186a1a1259393692fad not found: ID does not exist"
Sep 16 02:55:06.351614 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:55:06.351577    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fdf2e873-f961-4e2b-85fd-123f2a7c3de8-dshm\") pod \"fdf2e873-f961-4e2b-85fd-123f2a7c3de8\" (UID: \"fdf2e873-f961-4e2b-85fd-123f2a7c3de8\") "
Sep 16 02:55:06.351808 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:55:06.351652    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fdf2e873-f961-4e2b-85fd-123f2a7c3de8-model-cache\") pod \"fdf2e873-f961-4e2b-85fd-123f2a7c3de8\" (UID: \"fdf2e873-f961-4e2b-85fd-123f2a7c3de8\") "
Sep 16 02:55:06.351808 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:55:06.351685    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fdf2e873-f961-4e2b-85fd-123f2a7c3de8-tmp-dir\") pod \"fdf2e873-f961-4e2b-85fd-123f2a7c3de8\" (UID: \"fdf2e873-f961-4e2b-85fd-123f2a7c3de8\") "
Sep 16 02:55:06.351808 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:55:06.351757    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fdf2e873-f961-4e2b-85fd-123f2a7c3de8-home\") pod \"fdf2e873-f961-4e2b-85fd-123f2a7c3de8\" (UID: \"fdf2e873-f961-4e2b-85fd-123f2a7c3de8\") "
Sep 16 02:55:06.351808 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:55:06.351788    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7l95b\" (UniqueName: \"kubernetes.io/projected/fdf2e873-f961-4e2b-85fd-123f2a7c3de8-kube-api-access-7l95b\") pod \"fdf2e873-f961-4e2b-85fd-123f2a7c3de8\" (UID: \"fdf2e873-f961-4e2b-85fd-123f2a7c3de8\") "
Sep 16 02:55:06.352036 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:55:06.351815    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fdf2e873-f961-4e2b-85fd-123f2a7c3de8-tls-certs\") pod \"fdf2e873-f961-4e2b-85fd-123f2a7c3de8\" (UID: \"fdf2e873-f961-4e2b-85fd-123f2a7c3de8\") "
Sep 16 02:55:06.352036 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:55:06.351934    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fdf2e873-f961-4e2b-85fd-123f2a7c3de8-model-cache" (OuterVolumeSpecName: "model-cache") pod "fdf2e873-f961-4e2b-85fd-123f2a7c3de8" (UID: "fdf2e873-f961-4e2b-85fd-123f2a7c3de8"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 02:55:06.352036 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:55:06.351950    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fdf2e873-f961-4e2b-85fd-123f2a7c3de8-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "fdf2e873-f961-4e2b-85fd-123f2a7c3de8" (UID: "fdf2e873-f961-4e2b-85fd-123f2a7c3de8"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 02:55:06.352195 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:55:06.352032    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fdf2e873-f961-4e2b-85fd-123f2a7c3de8-home" (OuterVolumeSpecName: "home") pod "fdf2e873-f961-4e2b-85fd-123f2a7c3de8" (UID: "fdf2e873-f961-4e2b-85fd-123f2a7c3de8"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 02:55:06.352195 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:55:06.352156    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fdf2e873-f961-4e2b-85fd-123f2a7c3de8-home\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:55:06.352195 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:55:06.352175    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fdf2e873-f961-4e2b-85fd-123f2a7c3de8-model-cache\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:55:06.352195 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:55:06.352185    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fdf2e873-f961-4e2b-85fd-123f2a7c3de8-tmp-dir\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:55:06.354011 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:55:06.353989    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fdf2e873-f961-4e2b-85fd-123f2a7c3de8-dshm" (OuterVolumeSpecName: "dshm") pod "fdf2e873-f961-4e2b-85fd-123f2a7c3de8" (UID: "fdf2e873-f961-4e2b-85fd-123f2a7c3de8"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 02:55:06.354331 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:55:06.354028    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fdf2e873-f961-4e2b-85fd-123f2a7c3de8-kube-api-access-7l95b" (OuterVolumeSpecName: "kube-api-access-7l95b") pod "fdf2e873-f961-4e2b-85fd-123f2a7c3de8" (UID: "fdf2e873-f961-4e2b-85fd-123f2a7c3de8"). InnerVolumeSpecName "kube-api-access-7l95b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 02:55:06.354331 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:55:06.354094    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fdf2e873-f961-4e2b-85fd-123f2a7c3de8-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "fdf2e873-f961-4e2b-85fd-123f2a7c3de8" (UID: "fdf2e873-f961-4e2b-85fd-123f2a7c3de8"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 02:55:06.453182 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:55:06.453154    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7l95b\" (UniqueName: \"kubernetes.io/projected/fdf2e873-f961-4e2b-85fd-123f2a7c3de8-kube-api-access-7l95b\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:55:06.453182 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:55:06.453180    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fdf2e873-f961-4e2b-85fd-123f2a7c3de8-tls-certs\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:55:06.453182 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:55:06.453190    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fdf2e873-f961-4e2b-85fd-123f2a7c3de8-dshm\") on node \"ip-10-0-136-118.ec2.internal\" DevicePath \"\""
Sep 16 02:55:06.575461 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:55:06.575433    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-d058eb81-kserve-8dcf765cf-h2vqf"]
Sep 16 02:55:06.578503 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:55:06.578461    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-d058eb81-kserve-8dcf765cf-h2vqf"]
Sep 16 02:55:08.102726 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:55:08.102691    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fdf2e873-f961-4e2b-85fd-123f2a7c3de8" path="/var/lib/kubelet/pods/fdf2e873-f961-4e2b-85fd-123f2a7c3de8/volumes"
Sep 16 02:55:14.096404 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:55:14.096359    2579 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-pre-processing-6cd864bc95-mh4jh" secret="" err="secret \"payload-processing-dockercfg-zzx6j\" not found"
Sep 16 02:56:00.612595 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:56:00.612566    2579 scope.go:117] "RemoveContainer" containerID="4fe8d595462b39fa59353a068fec7d46e41201cccfcb491392f7fd5712e6f098"
Sep 16 02:56:00.620797 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:56:00.620776    2579 scope.go:117] "RemoveContainer" containerID="57f7335f2ac8414543d5a92f64556f5e5d56555c68ebd0120c15770ab8c2da99"
Sep 16 02:56:36.103190 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:56:36.103116    2579 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-pre-processing-6cd864bc95-mh4jh" secret="" err="secret \"payload-processing-dockercfg-zzx6j\" not found"
Sep 16 02:57:52.096776 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:57:52.096689    2579 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-pre-processing-6cd864bc95-mh4jh" secret="" err="secret \"payload-processing-dockercfg-zzx6j\" not found"
Sep 16 02:58:00.294364 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:58:00.294336    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-c8q8z_ebf846fa-af7d-4fdd-9662-b7c533c36ddd/console-operator/2.log"
Sep 16 02:58:00.308459 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:58:00.308437    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-c8q8z_ebf846fa-af7d-4fdd-9662-b7c533c36ddd/console-operator/2.log"
Sep 16 02:58:42.837513 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:58:42.837463    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-x7gcl/must-gather-fcfk7"]
Sep 16 02:58:42.838053 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:58:42.837825    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fdf2e873-f961-4e2b-85fd-123f2a7c3de8" containerName="main"
Sep 16 02:58:42.838053 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:58:42.837835    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="fdf2e873-f961-4e2b-85fd-123f2a7c3de8" containerName="main"
Sep 16 02:58:42.838053 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:58:42.837864    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c89ddb2d-657a-48f8-9eae-9d085b15b183" containerName="main"
Sep 16 02:58:42.838053 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:58:42.837870    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="c89ddb2d-657a-48f8-9eae-9d085b15b183" containerName="main"
Sep 16 02:58:42.838053 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:58:42.837933    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="fdf2e873-f961-4e2b-85fd-123f2a7c3de8" containerName="main"
Sep 16 02:58:42.838053 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:58:42.837943    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="c89ddb2d-657a-48f8-9eae-9d085b15b183" containerName="main"
Sep 16 02:58:42.841014 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:58:42.840998    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-x7gcl/must-gather-fcfk7"
Sep 16 02:58:42.843185 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:58:42.843165    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-x7gcl\"/\"default-dockercfg-zwm8t\""
Sep 16 02:58:42.844082 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:58:42.844063    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-x7gcl\"/\"kube-root-ca.crt\""
Sep 16 02:58:42.844159 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:58:42.844081    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-x7gcl\"/\"openshift-service-ca.crt\""
Sep 16 02:58:42.856548 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:58:42.856520    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-x7gcl/must-gather-fcfk7"]
Sep 16 02:58:42.880245 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:58:42.880205    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:58:42.939325 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:58:42.939287    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log" limit=1024
Sep 16 02:58:42.955541 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:58:42.955513    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vbfmb\" (UniqueName: \"kubernetes.io/projected/339249ca-ddb5-4a35-b182-b811bbe56495-kube-api-access-vbfmb\") pod \"must-gather-fcfk7\" (UID: \"339249ca-ddb5-4a35-b182-b811bbe56495\") " pod="openshift-must-gather-x7gcl/must-gather-fcfk7"
Sep 16 02:58:42.955652 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:58:42.955599    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/339249ca-ddb5-4a35-b182-b811bbe56495-must-gather-output\") pod \"must-gather-fcfk7\" (UID: \"339249ca-ddb5-4a35-b182-b811bbe56495\") " pod="openshift-must-gather-x7gcl/must-gather-fcfk7"
Sep 16 02:58:43.056917 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:58:43.056887    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/339249ca-ddb5-4a35-b182-b811bbe56495-must-gather-output\") pod \"must-gather-fcfk7\" (UID: \"339249ca-ddb5-4a35-b182-b811bbe56495\") " pod="openshift-must-gather-x7gcl/must-gather-fcfk7"
Sep 16 02:58:43.057070 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:58:43.056940    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vbfmb\" (UniqueName: \"kubernetes.io/projected/339249ca-ddb5-4a35-b182-b811bbe56495-kube-api-access-vbfmb\") pod \"must-gather-fcfk7\" (UID: \"339249ca-ddb5-4a35-b182-b811bbe56495\") " pod="openshift-must-gather-x7gcl/must-gather-fcfk7"
Sep 16 02:58:43.057223 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:58:43.057200    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/339249ca-ddb5-4a35-b182-b811bbe56495-must-gather-output\") pod \"must-gather-fcfk7\" (UID: \"339249ca-ddb5-4a35-b182-b811bbe56495\") " pod="openshift-must-gather-x7gcl/must-gather-fcfk7"
Sep 16 02:58:43.064441 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:58:43.064414    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vbfmb\" (UniqueName: \"kubernetes.io/projected/339249ca-ddb5-4a35-b182-b811bbe56495-kube-api-access-vbfmb\") pod \"must-gather-fcfk7\" (UID: \"339249ca-ddb5-4a35-b182-b811bbe56495\") " pod="openshift-must-gather-x7gcl/must-gather-fcfk7"
Sep 16 02:58:43.150549 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:58:43.150439    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-x7gcl/must-gather-fcfk7"
Sep 16 02:58:43.423531 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:58:43.423504    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-x7gcl/must-gather-fcfk7"]
Sep 16 02:58:43.426014 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:58:43.425984    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod339249ca_ddb5_4a35_b182_b811bbe56495.slice/crio-b016d0fd463fb57742aa4e8d3fd94e55e6f43fd73ecb5367d92fa05612e5e14c WatchSource:0}: Error finding container b016d0fd463fb57742aa4e8d3fd94e55e6f43fd73ecb5367d92fa05612e5e14c: Status 404 returned error can't find the container with id b016d0fd463fb57742aa4e8d3fd94e55e6f43fd73ecb5367d92fa05612e5e14c
Sep 16 02:58:43.427857 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:58:43.427840    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 16 02:58:44.277324 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:58:44.277288    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-x7gcl/must-gather-fcfk7" event={"ID":"339249ca-ddb5-4a35-b182-b811bbe56495","Type":"ContainerStarted","Data":"b016d0fd463fb57742aa4e8d3fd94e55e6f43fd73ecb5367d92fa05612e5e14c"}
Sep 16 02:58:45.285492 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:58:45.285424    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-x7gcl/must-gather-fcfk7" event={"ID":"339249ca-ddb5-4a35-b182-b811bbe56495","Type":"ContainerStarted","Data":"6e267500769ca3b4da8f120807aaeb06b20a47a72f090ee6db3c8523a278ee34"}
Sep 16 02:58:45.285492 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:58:45.285483    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-x7gcl/must-gather-fcfk7" event={"ID":"339249ca-ddb5-4a35-b182-b811bbe56495","Type":"ContainerStarted","Data":"9441e4239a4002221185830514d9887486905c45a7322f35afb98c60019511c0"}
Sep 16 02:58:45.306511 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:58:45.306442    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-x7gcl/must-gather-fcfk7" podStartSLOduration=2.414560355 podStartE2EDuration="3.306425747s" totalImagesPullingTime="891.865392ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 02:58:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 02:58:45.304646806 +0000 UTC m=+2745.688162051" watchObservedRunningTime="2026-09-16 02:58:45.306425747 +0000 UTC m=+2745.689940993"
Sep 16 02:58:45.794145 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:58:45.794103    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-cdwh7_2b6aaa85-d6dd-420d-a818-f56d06ce6467/global-pull-secret-syncer/0.log"
Sep 16 02:58:45.942004 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:58:45.941972    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-bhd9l_98489f25-0768-47e2-b8f0-e2a7c7af2063/konnectivity-agent/0.log"
Sep 16 02:58:46.035024 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:58:46.034987    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-136-118.ec2.internal_126583346e4bbf13f4d978c46ad24fd3/haproxy/0.log"
Sep 16 02:58:50.074614 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:58:50.074580    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6784f86975-8tkcv_b05ade60-6d3a-493e-a720-923ee22c57cb/authorino/0.log"
Sep 16 02:58:52.367142 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:58:52.367017    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-6cd954b897-s5pxd_e74b93a9-2fe3-43bd-81a4-b928b46db246/cluster-monitoring-operator/0.log"
Sep 16 02:58:52.395964 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:58:52.395921    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-59fbdd77d-sv5sm_e75b8c92-b447-4b4a-847a-6529c04af4cb/kube-state-metrics/0.log"
Sep 16 02:58:52.422401 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:58:52.422344    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-59fbdd77d-sv5sm_e75b8c92-b447-4b4a-847a-6529c04af4cb/kube-rbac-proxy-main/0.log"
Sep 16 02:58:52.442608 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:58:52.442575    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-59fbdd77d-sv5sm_e75b8c92-b447-4b4a-847a-6529c04af4cb/kube-rbac-proxy-self/0.log"
Sep 16 02:58:52.529011 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:58:52.528982    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-87q8v_3bd0fffa-d049-4ea6-ae0b-3e70c1645a12/node-exporter/0.log"
Sep 16 02:58:52.550983 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:58:52.550952    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-87q8v_3bd0fffa-d049-4ea6-ae0b-3e70c1645a12/kube-rbac-proxy/0.log"
Sep 16 02:58:52.573321 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:58:52.573294    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-87q8v_3bd0fffa-d049-4ea6-ae0b-3e70c1645a12/init-textfile/0.log"
Sep 16 02:58:52.996829 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:58:52.996794    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-68bcd7f987-kg6j9_ec276e5b-0368-4ee8-a433-b0941cf8bc16/prometheus-operator/0.log"
Sep 16 02:58:53.021463 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:58:53.021436    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-68bcd7f987-kg6j9_ec276e5b-0368-4ee8-a433-b0941cf8bc16/kube-rbac-proxy/0.log"
Sep 16 02:58:53.045144 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:58:53.045118    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-5864cd99d7-p2tqn_c46f41bf-3230-4da8-9ec8-45759ed3551d/prometheus-operator-admission-webhook/0.log"
Sep 16 02:58:55.013927 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:58:55.013893    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-5dcbc7df6b-jtz8l_8ff5e4be-2aa6-4203-83ac-e715f2f0f565/networking-console-plugin/0.log"
Sep 16 02:58:55.636938 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:58:55.636908    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-c8q8z_ebf846fa-af7d-4fdd-9662-b7c533c36ddd/console-operator/2.log"
Sep 16 02:58:55.643749 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:58:55.643715    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-c8q8z_ebf846fa-af7d-4fdd-9662-b7c533c36ddd/console-operator/3.log"
Sep 16 02:58:56.223822 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:58:56.223789    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-76f99584f4-tknsv_03b72ca3-a7e2-469f-85d4-281646ff1a2a/download-server/0.log"
Sep 16 02:58:56.562447 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:58:56.562352    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-x7gcl/perf-node-gather-daemonset-lm92f"]
Sep 16 02:58:56.569008 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:58:56.568976    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-x7gcl/perf-node-gather-daemonset-lm92f"
Sep 16 02:58:56.573502 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:58:56.573452    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-x7gcl/perf-node-gather-daemonset-lm92f"]
Sep 16 02:58:56.598492 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:58:56.597756    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/98c5af09-1847-42d6-8c8c-cef1eede3234-lib-modules\") pod \"perf-node-gather-daemonset-lm92f\" (UID: \"98c5af09-1847-42d6-8c8c-cef1eede3234\") " pod="openshift-must-gather-x7gcl/perf-node-gather-daemonset-lm92f"
Sep 16 02:58:56.598492 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:58:56.597843    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/98c5af09-1847-42d6-8c8c-cef1eede3234-proc\") pod \"perf-node-gather-daemonset-lm92f\" (UID: \"98c5af09-1847-42d6-8c8c-cef1eede3234\") " pod="openshift-must-gather-x7gcl/perf-node-gather-daemonset-lm92f"
Sep 16 02:58:56.598492 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:58:56.597938    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2b9cz\" (UniqueName: \"kubernetes.io/projected/98c5af09-1847-42d6-8c8c-cef1eede3234-kube-api-access-2b9cz\") pod \"perf-node-gather-daemonset-lm92f\" (UID: \"98c5af09-1847-42d6-8c8c-cef1eede3234\") " pod="openshift-must-gather-x7gcl/perf-node-gather-daemonset-lm92f"
Sep 16 02:58:56.598492 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:58:56.597986    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/98c5af09-1847-42d6-8c8c-cef1eede3234-sys\") pod \"perf-node-gather-daemonset-lm92f\" (UID: \"98c5af09-1847-42d6-8c8c-cef1eede3234\") " pod="openshift-must-gather-x7gcl/perf-node-gather-daemonset-lm92f"
Sep 16 02:58:56.598492 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:58:56.598046    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/98c5af09-1847-42d6-8c8c-cef1eede3234-podres\") pod \"perf-node-gather-daemonset-lm92f\" (UID: \"98c5af09-1847-42d6-8c8c-cef1eede3234\") " pod="openshift-must-gather-x7gcl/perf-node-gather-daemonset-lm92f"
Sep 16 02:58:56.699203 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:58:56.699161    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2b9cz\" (UniqueName: \"kubernetes.io/projected/98c5af09-1847-42d6-8c8c-cef1eede3234-kube-api-access-2b9cz\") pod \"perf-node-gather-daemonset-lm92f\" (UID: \"98c5af09-1847-42d6-8c8c-cef1eede3234\") " pod="openshift-must-gather-x7gcl/perf-node-gather-daemonset-lm92f"
Sep 16 02:58:56.699395 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:58:56.699218    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/98c5af09-1847-42d6-8c8c-cef1eede3234-sys\") pod \"perf-node-gather-daemonset-lm92f\" (UID: \"98c5af09-1847-42d6-8c8c-cef1eede3234\") " pod="openshift-must-gather-x7gcl/perf-node-gather-daemonset-lm92f"
Sep 16 02:58:56.699395 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:58:56.699264    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/98c5af09-1847-42d6-8c8c-cef1eede3234-podres\") pod \"perf-node-gather-daemonset-lm92f\" (UID: \"98c5af09-1847-42d6-8c8c-cef1eede3234\") " pod="openshift-must-gather-x7gcl/perf-node-gather-daemonset-lm92f"
Sep 16 02:58:56.699395 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:58:56.699309    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/98c5af09-1847-42d6-8c8c-cef1eede3234-lib-modules\") pod \"perf-node-gather-daemonset-lm92f\" (UID: \"98c5af09-1847-42d6-8c8c-cef1eede3234\") " pod="openshift-must-gather-x7gcl/perf-node-gather-daemonset-lm92f"
Sep 16 02:58:56.699395 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:58:56.699366    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/98c5af09-1847-42d6-8c8c-cef1eede3234-proc\") pod \"perf-node-gather-daemonset-lm92f\" (UID: \"98c5af09-1847-42d6-8c8c-cef1eede3234\") " pod="openshift-must-gather-x7gcl/perf-node-gather-daemonset-lm92f"
Sep 16 02:58:56.699642 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:58:56.699512    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/98c5af09-1847-42d6-8c8c-cef1eede3234-proc\") pod \"perf-node-gather-daemonset-lm92f\" (UID: \"98c5af09-1847-42d6-8c8c-cef1eede3234\") " pod="openshift-must-gather-x7gcl/perf-node-gather-daemonset-lm92f"
Sep 16 02:58:56.699930 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:58:56.699898    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/98c5af09-1847-42d6-8c8c-cef1eede3234-sys\") pod \"perf-node-gather-daemonset-lm92f\" (UID: \"98c5af09-1847-42d6-8c8c-cef1eede3234\") " pod="openshift-must-gather-x7gcl/perf-node-gather-daemonset-lm92f"
Sep 16 02:58:56.700049 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:58:56.700029    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/98c5af09-1847-42d6-8c8c-cef1eede3234-podres\") pod \"perf-node-gather-daemonset-lm92f\" (UID: \"98c5af09-1847-42d6-8c8c-cef1eede3234\") " pod="openshift-must-gather-x7gcl/perf-node-gather-daemonset-lm92f"
Sep 16 02:58:56.700154 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:58:56.700136    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/98c5af09-1847-42d6-8c8c-cef1eede3234-lib-modules\") pod \"perf-node-gather-daemonset-lm92f\" (UID: \"98c5af09-1847-42d6-8c8c-cef1eede3234\") " pod="openshift-must-gather-x7gcl/perf-node-gather-daemonset-lm92f"
Sep 16 02:58:56.709016 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:58:56.707398    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2b9cz\" (UniqueName: \"kubernetes.io/projected/98c5af09-1847-42d6-8c8c-cef1eede3234-kube-api-access-2b9cz\") pod \"perf-node-gather-daemonset-lm92f\" (UID: \"98c5af09-1847-42d6-8c8c-cef1eede3234\") " pod="openshift-must-gather-x7gcl/perf-node-gather-daemonset-lm92f"
Sep 16 02:58:56.885728 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:58:56.885625    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-x7gcl/perf-node-gather-daemonset-lm92f"
Sep 16 02:58:57.172633 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:58:57.172601    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-x7gcl/perf-node-gather-daemonset-lm92f"]
Sep 16 02:58:57.174997 ip-10-0-136-118 kubenswrapper[2579]: W0916 02:58:57.174949    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod98c5af09_1847_42d6_8c8c_cef1eede3234.slice/crio-5cdd50aacfafacfe5b18c8f3775f784cc6fd7c433173d9ffd25f7c93cbe417c7 WatchSource:0}: Error finding container 5cdd50aacfafacfe5b18c8f3775f784cc6fd7c433173d9ffd25f7c93cbe417c7: Status 404 returned error can't find the container with id 5cdd50aacfafacfe5b18c8f3775f784cc6fd7c433173d9ffd25f7c93cbe417c7
Sep 16 02:58:57.372602 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:58:57.372567    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-x7gcl/perf-node-gather-daemonset-lm92f" event={"ID":"98c5af09-1847-42d6-8c8c-cef1eede3234","Type":"ContainerStarted","Data":"d3b229bf7a0b51eeb836496eaa727483bb82617ad94a089fd95e102835ef1ff1"}
Sep 16 02:58:57.373142 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:58:57.373119    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-x7gcl/perf-node-gather-daemonset-lm92f" event={"ID":"98c5af09-1847-42d6-8c8c-cef1eede3234","Type":"ContainerStarted","Data":"5cdd50aacfafacfe5b18c8f3775f784cc6fd7c433173d9ffd25f7c93cbe417c7"}
Sep 16 02:58:57.373280 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:58:57.373267    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-x7gcl/perf-node-gather-daemonset-lm92f"
Sep 16 02:58:57.388651 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:58:57.388596    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-x7gcl/perf-node-gather-daemonset-lm92f" podStartSLOduration=1.388575461 podStartE2EDuration="1.388575461s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 02:58:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 02:58:57.387323327 +0000 UTC m=+2757.770838573" watchObservedRunningTime="2026-09-16 02:58:57.388575461 +0000 UTC m=+2757.772090707"
Sep 16 02:58:58.140268 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:58:58.140245    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-q6tmq_8e7a4943-f231-4531-9e5d-4bcf2a844cf9/dns/0.log"
Sep 16 02:58:58.161923 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:58:58.161634    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-q6tmq_8e7a4943-f231-4531-9e5d-4bcf2a844cf9/kube-rbac-proxy/0.log"
Sep 16 02:58:58.290576 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:58:58.290546    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-9522z_835b007c-0c9a-4c8f-94dd-68f8a3b3a137/dns-node-resolver/0.log"
Sep 16 02:58:58.878724 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:58:58.878693    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-kwdns_496c15b3-4d67-473f-b894-2e66b47df562/node-ca/0.log"
Sep 16 02:59:00.527712 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:59:00.527678    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7cff6fbc6c-q989h_60cf8c38-8ee9-493d-a871-22678ad8b6d0/kube-auth-proxy/0.log"
Sep 16 02:59:00.579916 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:59:00.579875    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-6cd864bc95-mh4jh_8c3aa767-5631-4c03-bbe4-9cb9fa96b7f8/payload-pre-processing/0.log"
Sep 16 02:59:00.626314 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:59:00.626283    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-758df9dfb-pnlzd_91fb29dc-53d2-4a7e-adc8-365d8791dfbc/router/0.log"
Sep 16 02:59:01.564089 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:59:01.564060    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-vbpq2_5caaf390-c372-40a3-8303-e99ab52b762b/serve-healthcheck-canary/0.log"
Sep 16 02:59:02.004535 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:59:02.004504    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5844b856db-hdtv6_d9ffb572-8b90-4503-82c1-84deef377ca7/insights-operator/0.log"
Sep 16 02:59:02.004870 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:59:02.004855    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5844b856db-hdtv6_d9ffb572-8b90-4503-82c1-84deef377ca7/insights-operator/1.log"
Sep 16 02:59:02.029222 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:59:02.029192    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-84f2w_aefd9717-5d49-4029-96d1-e195cfc682ff/kube-rbac-proxy/0.log"
Sep 16 02:59:02.048622 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:59:02.048552    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-84f2w_aefd9717-5d49-4029-96d1-e195cfc682ff/exporter/0.log"
Sep 16 02:59:02.069193 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:59:02.069167    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-84f2w_aefd9717-5d49-4029-96d1-e195cfc682ff/extractor/0.log"
Sep 16 02:59:04.401999 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:59:04.401972    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-x7gcl/perf-node-gather-daemonset-lm92f"
Sep 16 02:59:04.514646 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:59:04.514604    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-b878dcb6c-8j8cx_1338df10-6da2-451a-8370-b7078c273476/manager/0.log"
Sep 16 02:59:04.800656 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:59:04.800621    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-74fbd865d6-5jt27_f3048123-5d9f-4e0c-a3e6-56d2bbd04f5f/manager/0.log"
Sep 16 02:59:04.821761 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:59:04.821732    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-74fbd865d6-5jt27_f3048123-5d9f-4e0c-a3e6-56d2bbd04f5f/copy-manifests/0.log"
Sep 16 02:59:06.389158 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:59:06.389084    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-79c8b99f94-2wfnm_b7970ae7-f890-4baa-825b-8dedd185dce5/manager/0.log"
Sep 16 02:59:06.441025 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:59:06.440996    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-jmk65_cdf71ee6-224a-42a8-a95a-58c7486d2241/openshift-lws-operator/0.log"
Sep 16 02:59:06.966167 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:59:06.966096    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7cff6fbc6c-q989h_60cf8c38-8ee9-493d-a871-22678ad8b6d0/kube-auth-proxy/0.log"
Sep 16 02:59:06.997744 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:59:06.997720    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-6cd864bc95-mh4jh_8c3aa767-5631-4c03-bbe4-9cb9fa96b7f8/payload-pre-processing/0.log"
Sep 16 02:59:07.029186 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:59:07.029159    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-758df9dfb-pnlzd_91fb29dc-53d2-4a7e-adc8-365d8791dfbc/router/0.log"
Sep 16 02:59:09.203661 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:59:09.203624    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7cff6fbc6c-q989h_60cf8c38-8ee9-493d-a871-22678ad8b6d0/kube-auth-proxy/0.log"
Sep 16 02:59:09.237716 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:59:09.237688    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-6cd864bc95-mh4jh_8c3aa767-5631-4c03-bbe4-9cb9fa96b7f8/payload-pre-processing/0.log"
Sep 16 02:59:09.271283 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:59:09.271261    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-758df9dfb-pnlzd_91fb29dc-53d2-4a7e-adc8-365d8791dfbc/router/0.log"
Sep 16 02:59:11.096978 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:59:11.096951    2579 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-pre-processing-6cd864bc95-mh4jh" secret="" err="secret \"payload-processing-dockercfg-zzx6j\" not found"
Sep 16 02:59:12.879380 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:59:12.879351    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9g4cg_34a0d09d-a96b-4242-9d17-61fc3b8d88bd/kube-multus-additional-cni-plugins/0.log"
Sep 16 02:59:12.902721 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:59:12.902698    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9g4cg_34a0d09d-a96b-4242-9d17-61fc3b8d88bd/egress-router-binary-copy/0.log"
Sep 16 02:59:12.924606 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:59:12.924577    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9g4cg_34a0d09d-a96b-4242-9d17-61fc3b8d88bd/cni-plugins/0.log"
Sep 16 02:59:12.944814 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:59:12.944765    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9g4cg_34a0d09d-a96b-4242-9d17-61fc3b8d88bd/bond-cni-plugin/0.log"
Sep 16 02:59:12.966847 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:59:12.966822    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9g4cg_34a0d09d-a96b-4242-9d17-61fc3b8d88bd/routeoverride-cni/0.log"
Sep 16 02:59:12.986421 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:59:12.986401    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9g4cg_34a0d09d-a96b-4242-9d17-61fc3b8d88bd/whereabouts-cni-bincopy/0.log"
Sep 16 02:59:13.013663 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:59:13.013642    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9g4cg_34a0d09d-a96b-4242-9d17-61fc3b8d88bd/whereabouts-cni/0.log"
Sep 16 02:59:13.201654 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:59:13.201624    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-ncht9_bc90ef77-c833-4991-9fe8-656fdf2f7457/kube-multus/0.log"
Sep 16 02:59:13.265921 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:59:13.265890    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-6lwr9_e3472392-891d-49a1-83a7-57f1f99f4b79/network-metrics-daemon/0.log"
Sep 16 02:59:13.285996 ip-10-0-136-118 kubenswrapper[2579]: I0916 02:59:13.285970    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-6lwr9_e3472392-891d-49a1-83a7-57f1f99f4b79/kube-rbac-proxy/0.log"