Sep 20 01:50:31.250842 ip-10-0-130-159 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Sep 20 01:50:31.250857 ip-10-0-130-159 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Sep 20 01:50:31.250879 ip-10-0-130-159 systemd[1]: kubelet.service: Failed with result 'resources'.
Sep 20 01:50:31.251221 ip-10-0-130-159 systemd[1]: Failed to start Kubernetes Kubelet.
Sep 20 01:50:41.375460 ip-10-0-130-159 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Sep 20 01:50:41.375481 ip-10-0-130-159 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 4da959aa09fd4bc3bf3b1f68a0b801f7 --
Sep 20 01:52:21.868789 ip-10-0-130-159 systemd[1]: Starting Kubernetes Kubelet...
Sep 20 01:52:24.926257 ip-10-0-130-159 kubenswrapper[2584]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 20 01:52:24.926257 ip-10-0-130-159 kubenswrapper[2584]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 20 01:52:24.926257 ip-10-0-130-159 kubenswrapper[2584]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 20 01:52:24.926257 ip-10-0-130-159 kubenswrapper[2584]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 20 01:52:24.926257 ip-10-0-130-159 kubenswrapper[2584]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 20 01:52:24.926974 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.926690    2584 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 20 01:52:24.930339 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.930323    2584 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 20 01:52:24.930339 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.930339    2584 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 20 01:52:24.930409 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.930343    2584 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 20 01:52:24.930409 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.930346    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 20 01:52:24.930409 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.930349    2584 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 20 01:52:24.930409 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.930352    2584 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 20 01:52:24.930409 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.930355    2584 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 20 01:52:24.930409 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.930358    2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 20 01:52:24.930409 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.930360    2584 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 20 01:52:24.930409 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.930363    2584 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 20 01:52:24.930409 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.930366    2584 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 20 01:52:24.930409 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.930368    2584 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 20 01:52:24.930409 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.930371    2584 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 20 01:52:24.930409 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.930373    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 20 01:52:24.930409 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.930377    2584 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 20 01:52:24.930409 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.930385    2584 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 20 01:52:24.930409 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.930387    2584 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 20 01:52:24.930409 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.930390    2584 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 20 01:52:24.930409 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.930393    2584 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 20 01:52:24.930409 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.930396    2584 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 20 01:52:24.930409 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.930398    2584 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 20 01:52:24.930409 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.930401    2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 20 01:52:24.930900 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.930403    2584 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 20 01:52:24.930900 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.930406    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 20 01:52:24.930900 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.930409    2584 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 20 01:52:24.930900 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.930411    2584 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 20 01:52:24.930900 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.930414    2584 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 20 01:52:24.930900 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.930417    2584 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 20 01:52:24.930900 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.930420    2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 20 01:52:24.930900 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.930423    2584 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 20 01:52:24.930900 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.930426    2584 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 20 01:52:24.930900 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.930429    2584 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 20 01:52:24.930900 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.930431    2584 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 20 01:52:24.930900 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.930434    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 20 01:52:24.930900 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.930436    2584 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 20 01:52:24.930900 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.930439    2584 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 20 01:52:24.930900 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.930441    2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 20 01:52:24.930900 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.930444    2584 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 20 01:52:24.930900 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.930446    2584 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 20 01:52:24.930900 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.930449    2584 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 20 01:52:24.930900 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.930451    2584 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 20 01:52:24.931385 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.930455    2584 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 20 01:52:24.931385 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.930458    2584 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 20 01:52:24.931385 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.930461    2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 20 01:52:24.931385 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.930464    2584 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 20 01:52:24.931385 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.930468    2584 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 20 01:52:24.931385 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.930471    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 20 01:52:24.931385 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.930475    2584 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 20 01:52:24.931385 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.930478    2584 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 20 01:52:24.931385 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.930480    2584 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 20 01:52:24.931385 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.930483    2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 20 01:52:24.931385 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.930491    2584 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 20 01:52:24.931385 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.930494    2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 20 01:52:24.931385 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.930496    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 20 01:52:24.931385 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.930499    2584 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 20 01:52:24.931385 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.930503    2584 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 20 01:52:24.931385 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.930506    2584 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 20 01:52:24.931385 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.930509    2584 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 20 01:52:24.931385 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.930512    2584 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 20 01:52:24.931840 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.930527    2584 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 20 01:52:24.931840 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.930531    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 20 01:52:24.931840 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.930534    2584 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 20 01:52:24.931840 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.930536    2584 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 20 01:52:24.931840 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.930539    2584 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 20 01:52:24.931840 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.930542    2584 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 20 01:52:24.931840 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.930544    2584 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 20 01:52:24.931840 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.930547    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 20 01:52:24.931840 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.930549    2584 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 20 01:52:24.931840 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.930552    2584 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 20 01:52:24.931840 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.930554    2584 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 20 01:52:24.931840 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.930557    2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 20 01:52:24.931840 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.930561    2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 20 01:52:24.931840 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.930563    2584 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 20 01:52:24.931840 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.930566    2584 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 20 01:52:24.931840 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.930568    2584 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 20 01:52:24.931840 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.930571    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 20 01:52:24.931840 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.930573    2584 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 20 01:52:24.931840 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.930576    2584 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 20 01:52:24.931840 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.930578    2584 feature_gate.go:328] unrecognized feature gate: Example2
Sep 20 01:52:24.932325 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.930581    2584 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 20 01:52:24.932325 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.930584    2584 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 20 01:52:24.932325 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.930586    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 20 01:52:24.932325 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.930589    2584 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 20 01:52:24.932325 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.930591    2584 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 20 01:52:24.932325 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.930594    2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 20 01:52:24.932325 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.930596    2584 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 20 01:52:24.932325 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.930599    2584 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 20 01:52:24.932325 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.930603    2584 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 20 01:52:24.932325 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.930606    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 20 01:52:24.932325 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.930608    2584 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 20 01:52:24.932325 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.930611    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 20 01:52:24.932325 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.930613    2584 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 20 01:52:24.932325 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.930616    2584 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 20 01:52:24.932325 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.930618    2584 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 20 01:52:24.932325 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.930621    2584 feature_gate.go:328] unrecognized feature gate: Example
Sep 20 01:52:24.932325 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.930623    2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 20 01:52:24.932325 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.930626    2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 20 01:52:24.932325 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.930628    2584 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 20 01:52:24.932325 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.930630    2584 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 20 01:52:24.932857 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.931293    2584 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 20 01:52:24.932857 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.931299    2584 feature_gate.go:328] unrecognized feature gate: Example2
Sep 20 01:52:24.932857 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.931302    2584 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 20 01:52:24.932857 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.931307    2584 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 20 01:52:24.932857 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.931311    2584 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 20 01:52:24.932857 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.931315    2584 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 20 01:52:24.932857 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.931317    2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 20 01:52:24.932857 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.931320    2584 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 20 01:52:24.932857 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.931323    2584 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 20 01:52:24.932857 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.931326    2584 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 20 01:52:24.932857 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.931329    2584 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 20 01:52:24.932857 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.931332    2584 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 20 01:52:24.932857 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.931335    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 20 01:52:24.932857 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.931339    2584 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 20 01:52:24.932857 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.931343    2584 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 20 01:52:24.932857 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.931346    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 20 01:52:24.932857 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.931349    2584 feature_gate.go:328] unrecognized feature gate: Example
Sep 20 01:52:24.932857 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.931352    2584 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 20 01:52:24.932857 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.931355    2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 20 01:52:24.933360 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.931358    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 20 01:52:24.933360 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.931361    2584 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 20 01:52:24.933360 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.931364    2584 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 20 01:52:24.933360 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.931367    2584 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 20 01:52:24.933360 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.931369    2584 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 20 01:52:24.933360 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.931372    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 20 01:52:24.933360 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.931375    2584 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 20 01:52:24.933360 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.931377    2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 20 01:52:24.933360 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.931380    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 20 01:52:24.933360 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.931382    2584 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 20 01:52:24.933360 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.931385    2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 20 01:52:24.933360 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.931388    2584 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 20 01:52:24.933360 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.931390    2584 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 20 01:52:24.933360 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.931393    2584 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 20 01:52:24.933360 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.931396    2584 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 20 01:52:24.933360 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.931398    2584 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 20 01:52:24.933360 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.931401    2584 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 20 01:52:24.933360 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.931403    2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 20 01:52:24.933360 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.931406    2584 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 20 01:52:24.933360 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.931409    2584 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 20 01:52:24.933900 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.931412    2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 20 01:52:24.933900 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.931414    2584 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 20 01:52:24.933900 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.931417    2584 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 20 01:52:24.933900 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.931419    2584 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 20 01:52:24.933900 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.931422    2584 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 20 01:52:24.933900 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.931426    2584 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 20 01:52:24.933900 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.931429    2584 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 20 01:52:24.933900 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.931431    2584 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 20 01:52:24.933900 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.931435    2584 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 20 01:52:24.933900 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.931437    2584 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 20 01:52:24.933900 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.931440    2584 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 20 01:52:24.933900 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.931443    2584 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 20 01:52:24.933900 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.931445    2584 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 20 01:52:24.933900 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.931448    2584 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 20 01:52:24.933900 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.931450    2584 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 20 01:52:24.933900 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.931453    2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 20 01:52:24.933900 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.931455    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 20 01:52:24.933900 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.931458    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 20 01:52:24.933900 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.931460    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 20 01:52:24.933900 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.931463    2584 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 20 01:52:24.934387 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.931466    2584 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 20 01:52:24.934387 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.931468    2584 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 20 01:52:24.934387 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.931471    2584 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 20 01:52:24.934387 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.931473    2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 20 01:52:24.934387 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.931476    2584 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 20 01:52:24.934387 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.931478    2584 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 20 01:52:24.934387 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.931481    2584 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 20 01:52:24.934387 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.931483    2584 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 20 01:52:24.934387 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.931486    2584 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 20 01:52:24.934387 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.931488    2584 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 20 01:52:24.934387 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.931491    2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 20 01:52:24.934387 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.931494    2584 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 20 01:52:24.934387 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.931497    2584 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 20 01:52:24.934387 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.931499    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 20 01:52:24.934387 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.931502    2584 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 20 01:52:24.934387 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.931505    2584 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 20 01:52:24.934387 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.931507    2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 20 01:52:24.934387 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.931510    2584 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 20 01:52:24.934387 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.931513    2584 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 20 01:52:24.934387 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.931531    2584 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 20 01:52:24.934885 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.931534    2584 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 20 01:52:24.934885 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.931537    2584 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 20 01:52:24.934885 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.931539    2584 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 20 01:52:24.934885 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.931542    2584 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 20 01:52:24.934885 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.931545    2584 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 20 01:52:24.934885 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.931551    2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 20 01:52:24.934885 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.931553    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 20 01:52:24.934885 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.931556    2584 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 20 01:52:24.934885 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.931559    2584 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 20 01:52:24.934885 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.931561    2584 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 20 01:52:24.934885 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.931564    2584 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 20 01:52:24.934885 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.931567    2584 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 20 01:52:24.934885 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.931569    2584 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 20 01:52:24.934885 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.931572    2584 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 20 01:52:24.934885 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.931574    2584 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 20 01:52:24.934885 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.931577    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 20 01:52:24.934885 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.931579    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 20 01:52:24.934885 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.931582    2584 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 20 01:52:24.934885 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.931584    2584 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 20 01:52:24.934885 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.931587    2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 20 01:52:24.935374 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.931941    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 20 01:52:24.935374 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.931945    2584 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 20 01:52:24.935374 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.931949    2584 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 20 01:52:24.935374 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.931951    2584 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 20 01:52:24.935374 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.931954    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 20 01:52:24.935374 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.931958    2584 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 20 01:52:24.935374 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.931961    2584 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 20 01:52:24.935374 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.931964    2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 20 01:52:24.935374 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.931968    2584 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 20 01:52:24.935374 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.931972    2584 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 20 01:52:24.935374 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.931975    2584 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 20 01:52:24.935374 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.931978    2584 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 20 01:52:24.935374 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.931981    2584 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 20 01:52:24.935374 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.931983    2584 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 20 01:52:24.935374 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.931986    2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 20 01:52:24.935374 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.931989    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 20 01:52:24.935374 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.931991    2584 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 20 01:52:24.935374 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.931994    2584 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 20 01:52:24.935374 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.931996    2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 20 01:52:24.935374 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.931999    2584 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 20 01:52:24.935877 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.932002    2584 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 20 01:52:24.935877 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.932004    2584 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 20 01:52:24.935877 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.932007    2584 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 20 01:52:24.935877 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.932009    2584 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 20 01:52:24.935877 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.932012    2584 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 20 01:52:24.935877 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.932014    2584 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 20 01:52:24.935877 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.932017    2584 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 20 01:52:24.935877 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.932020    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 20 01:52:24.935877 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.932023    2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 20 01:52:24.935877 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.932025    2584 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 20 01:52:24.935877 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.932028    2584 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 20 01:52:24.935877 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.932030    2584 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 20 01:52:24.935877 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.932036    2584 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 20 01:52:24.935877 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.932039    2584 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 20 01:52:24.935877 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.932041    2584 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 20 01:52:24.935877 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.932044    2584 feature_gate.go:328] unrecognized feature gate: Example2
Sep 20 01:52:24.935877 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.932047    2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 20 01:52:24.935877 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.932050    2584 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 20 01:52:24.935877 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.932053    2584 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 20 01:52:24.936348 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.932056    2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 20 01:52:24.936348 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.932059    2584 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 20 01:52:24.936348 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.932061    2584 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 20 01:52:24.936348 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.932066    2584 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 20 01:52:24.936348 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.932069    2584 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 20 01:52:24.936348 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.932072    2584 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 20 01:52:24.936348 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.932075    2584 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 20 01:52:24.936348 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.932078    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 20 01:52:24.936348 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.932080    2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 20 01:52:24.936348 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.932083    2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 20 01:52:24.936348 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.932086    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 20 01:52:24.936348 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.932089    2584 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 20 01:52:24.936348 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.932091    2584 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 20 01:52:24.936348 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.932094    2584 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 20 01:52:24.936348 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.932096    2584 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 20 01:52:24.936348 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.932099    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 20 01:52:24.936348 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.932101    2584 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 20 01:52:24.936348 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.932104    2584 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 20 01:52:24.936348 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.932106    2584 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 20 01:52:24.936827 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.932109    2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 20 01:52:24.936827 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.932112    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 20 01:52:24.936827 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.932114    2584 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 20 01:52:24.936827 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.932117    2584 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 20 01:52:24.936827 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.932119    2584 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 20 01:52:24.936827 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.932122    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 20 01:52:24.936827 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.932126    2584 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 20 01:52:24.936827 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.932128    2584 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 20 01:52:24.936827 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.932131    2584 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 20 01:52:24.936827 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.932133    2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 20 01:52:24.936827 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.932136    2584 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 20 01:52:24.936827 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.932138    2584 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 20 01:52:24.936827 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.932141    2584 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 20 01:52:24.936827 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.932144    2584 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 20 01:52:24.936827 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.932146    2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 20 01:52:24.936827 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.932149    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 20 01:52:24.936827 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.932152    2584 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 20 01:52:24.936827 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.932154    2584 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 20 01:52:24.936827 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.932157    2584 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 20 01:52:24.936827 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.932159    2584 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 20 01:52:24.937303 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.932162    2584 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 20 01:52:24.937303 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.932164    2584 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 20 01:52:24.937303 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.932166    2584 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 20 01:52:24.937303 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.932169    2584 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 20 01:52:24.937303 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.932171    2584 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 20 01:52:24.937303 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.932175    2584 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 20 01:52:24.937303 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.932193    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 20 01:52:24.937303 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.932197    2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 20 01:52:24.937303 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.932200    2584 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 20 01:52:24.937303 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.932210    2584 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 20 01:52:24.937303 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.932214    2584 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 20 01:52:24.937303 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.932217    2584 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 20 01:52:24.937303 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.932220    2584 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 20 01:52:24.937303 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.932223    2584 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 20 01:52:24.937303 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.932226    2584 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 20 01:52:24.937303 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.932229    2584 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 20 01:52:24.937303 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.932232    2584 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 20 01:52:24.937303 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.932235    2584 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 20 01:52:24.937303 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.932239    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 20 01:52:24.937303 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.932241    2584 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 20 01:52:24.937789 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.932244    2584 feature_gate.go:328] unrecognized feature gate: Example
Sep 20 01:52:24.937789 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932309    2584 flags.go:64] FLAG: --address="0.0.0.0"
Sep 20 01:52:24.937789 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932348    2584 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 20 01:52:24.937789 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932355    2584 flags.go:64] FLAG: --anonymous-auth="true"
Sep 20 01:52:24.937789 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932360    2584 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 20 01:52:24.937789 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932364    2584 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 20 01:52:24.937789 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932367    2584 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 20 01:52:24.937789 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932371    2584 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 20 01:52:24.937789 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932375    2584 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 20 01:52:24.937789 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932378    2584 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 20 01:52:24.937789 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932381    2584 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 20 01:52:24.937789 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932385    2584 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 20 01:52:24.937789 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932388    2584 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 20 01:52:24.937789 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932391    2584 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 20 01:52:24.937789 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932394    2584 flags.go:64] FLAG: --cgroup-root=""
Sep 20 01:52:24.937789 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932397    2584 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 20 01:52:24.937789 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932400    2584 flags.go:64] FLAG: --client-ca-file=""
Sep 20 01:52:24.937789 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932403    2584 flags.go:64] FLAG: --cloud-provider="external"
Sep 20 01:52:24.937789 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932405    2584 flags.go:64] FLAG: --cluster-dns="[]"
Sep 20 01:52:24.937789 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932409    2584 flags.go:64] FLAG: --cluster-domain=""
Sep 20 01:52:24.937789 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932412    2584 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 20 01:52:24.937789 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932415    2584 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 20 01:52:24.937789 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932418    2584 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 20 01:52:24.937789 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932421    2584 flags.go:64] FLAG: --container-log-max-files="5"
Sep 20 01:52:24.938355 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932426    2584 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 20 01:52:24.938355 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932429    2584 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 20 01:52:24.938355 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932432    2584 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 20 01:52:24.938355 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932436    2584 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 20 01:52:24.938355 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932439    2584 flags.go:64] FLAG: --contention-profiling="false"
Sep 20 01:52:24.938355 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932442    2584 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 20 01:52:24.938355 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932446    2584 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 20 01:52:24.938355 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932449    2584 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 20 01:52:24.938355 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932452    2584 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 20 01:52:24.938355 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932456    2584 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 20 01:52:24.938355 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932459    2584 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 20 01:52:24.938355 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932462    2584 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 20 01:52:24.938355 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932464    2584 flags.go:64] FLAG: --enable-load-reader="false"
Sep 20 01:52:24.938355 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932467    2584 flags.go:64] FLAG: --enable-server="true"
Sep 20 01:52:24.938355 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932471    2584 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 20 01:52:24.938355 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932476    2584 flags.go:64] FLAG: --event-burst="100"
Sep 20 01:52:24.938355 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932479    2584 flags.go:64] FLAG: --event-qps="50"
Sep 20 01:52:24.938355 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932482    2584 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 20 01:52:24.938355 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932485    2584 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 20 01:52:24.938355 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932487    2584 flags.go:64] FLAG: --eviction-hard=""
Sep 20 01:52:24.938355 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932491    2584 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 20 01:52:24.938355 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932494    2584 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 20 01:52:24.938355 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932497    2584 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 20 01:52:24.938355 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932500    2584 flags.go:64] FLAG: --eviction-soft=""
Sep 20 01:52:24.938355 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932502    2584 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 20 01:52:24.939086 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932505    2584 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 20 01:52:24.939086 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932508    2584 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 20 01:52:24.939086 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932511    2584 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 20 01:52:24.939086 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932514    2584 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 20 01:52:24.939086 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932529    2584 flags.go:64] FLAG: --fail-swap-on="true"
Sep 20 01:52:24.939086 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932532    2584 flags.go:64] FLAG: --feature-gates=""
Sep 20 01:52:24.939086 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932536    2584 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 20 01:52:24.939086 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932541    2584 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 20 01:52:24.939086 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932544    2584 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 20 01:52:24.939086 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932547    2584 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 20 01:52:24.939086 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932550    2584 flags.go:64] FLAG: --healthz-port="10248"
Sep 20 01:52:24.939086 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932553    2584 flags.go:64] FLAG: --help="false"
Sep 20 01:52:24.939086 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932556    2584 flags.go:64] FLAG: --hostname-override="ip-10-0-130-159.ec2.internal"
Sep 20 01:52:24.939086 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932560    2584 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 20 01:52:24.939086 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932563    2584 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 20 01:52:24.939086 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932566    2584 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 20 01:52:24.939086 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932570    2584 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 20 01:52:24.939086 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932573    2584 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 20 01:52:24.939086 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932576    2584 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 20 01:52:24.939086 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932579    2584 flags.go:64] FLAG: --image-service-endpoint=""
Sep 20 01:52:24.939086 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932582    2584 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 20 01:52:24.939086 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932585    2584 flags.go:64] FLAG: --kube-api-burst="100"
Sep 20 01:52:24.939086 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932589    2584 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 20 01:52:24.940108 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932592    2584 flags.go:64] FLAG: --kube-api-qps="50"
Sep 20 01:52:24.940108 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932595    2584 flags.go:64] FLAG: --kube-reserved=""
Sep 20 01:52:24.940108 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932598    2584 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 20 01:52:24.940108 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932601    2584 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 20 01:52:24.940108 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932604    2584 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 20 01:52:24.940108 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932607    2584 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 20 01:52:24.940108 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932610    2584 flags.go:64] FLAG: --lock-file=""
Sep 20 01:52:24.940108 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932613    2584 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 20 01:52:24.940108 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932616    2584 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 20 01:52:24.940108 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932619    2584 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 20 01:52:24.940108 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932624    2584 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 20 01:52:24.940108 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932627    2584 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 20 01:52:24.940108 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932630    2584 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 20 01:52:24.940108 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932633    2584 flags.go:64] FLAG: --logging-format="text"
Sep 20 01:52:24.940108 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932636    2584 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 20 01:52:24.940108 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932639    2584 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 20 01:52:24.940108 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932644    2584 flags.go:64] FLAG: --manifest-url=""
Sep 20 01:52:24.940108 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932646    2584 flags.go:64] FLAG: --manifest-url-header=""
Sep 20 01:52:24.940108 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932651    2584 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 20 01:52:24.940108 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932653    2584 flags.go:64] FLAG: --max-open-files="1000000"
Sep 20 01:52:24.940108 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932662    2584 flags.go:64] FLAG: --max-pods="110"
Sep 20 01:52:24.940108 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932665    2584 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 20 01:52:24.940108 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932669    2584 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 20 01:52:24.940108 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932672    2584 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 20 01:52:24.940108 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932675    2584 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 20 01:52:24.941183 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932678    2584 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 20 01:52:24.941183 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932681    2584 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 20 01:52:24.941183 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932684    2584 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 20 01:52:24.941183 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932690    2584 flags.go:64] FLAG: --node-status-max-images="50"
Sep 20 01:52:24.941183 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932693    2584 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 20 01:52:24.941183 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932696    2584 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 20 01:52:24.941183 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932700    2584 flags.go:64] FLAG: --pod-cidr=""
Sep 20 01:52:24.941183 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932703    2584 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a6abe77af61c5f2e3985a83e7d43538aefc38b1b0444127da14ee0e8cf8f76fc"
Sep 20 01:52:24.941183 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932708    2584 flags.go:64] FLAG: --pod-manifest-path=""
Sep 20 01:52:24.941183 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932712    2584 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 20 01:52:24.941183 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932715    2584 flags.go:64] FLAG: --pods-per-core="0"
Sep 20 01:52:24.941183 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932718    2584 flags.go:64] FLAG: --port="10250"
Sep 20 01:52:24.941183 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932721    2584 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 20 01:52:24.941183 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932724    2584 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-048ace93dd515164b"
Sep 20 01:52:24.941183 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932727    2584 flags.go:64] FLAG: --qos-reserved=""
Sep 20 01:52:24.941183 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932730    2584 flags.go:64] FLAG: --read-only-port="10255"
Sep 20 01:52:24.941183 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932733    2584 flags.go:64] FLAG: --register-node="true"
Sep 20 01:52:24.941183 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932736    2584 flags.go:64] FLAG: --register-with-taints=""
Sep 20 01:52:24.941183 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932740    2584 flags.go:64] FLAG: --registry-burst="10"
Sep 20 01:52:24.941183 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932742    2584 flags.go:64] FLAG: --registry-qps="5"
Sep 20 01:52:24.941183 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932745    2584 flags.go:64] FLAG: --reserved-cpus=""
Sep 20 01:52:24.941183 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932748    2584 flags.go:64] FLAG: --reserved-memory=""
Sep 20 01:52:24.941183 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932751    2584 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 20 01:52:24.941183 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932755    2584 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 20 01:52:24.941183 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932758    2584 flags.go:64] FLAG: --rotate-certificates="false"
Sep 20 01:52:24.942281 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932761    2584 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 20 01:52:24.942281 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932764    2584 flags.go:64] FLAG: --runonce="false"
Sep 20 01:52:24.942281 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932767    2584 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 20 01:52:24.942281 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932770    2584 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 20 01:52:24.942281 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932773    2584 flags.go:64] FLAG: --seccomp-default="false"
Sep 20 01:52:24.942281 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932777    2584 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 20 01:52:24.942281 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932780    2584 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 20 01:52:24.942281 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932783    2584 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 20 01:52:24.942281 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932786    2584 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 20 01:52:24.942281 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932789    2584 flags.go:64] FLAG: --storage-driver-password="root"
Sep 20 01:52:24.942281 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932793    2584 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 20 01:52:24.942281 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932795    2584 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 20 01:52:24.942281 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932798    2584 flags.go:64] FLAG: --storage-driver-user="root"
Sep 20 01:52:24.942281 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932801    2584 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 20 01:52:24.942281 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932804    2584 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 20 01:52:24.942281 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932808    2584 flags.go:64] FLAG: --system-cgroups=""
Sep 20 01:52:24.942281 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932811    2584 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 20 01:52:24.942281 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932816    2584 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 20 01:52:24.942281 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932819    2584 flags.go:64] FLAG: --tls-cert-file=""
Sep 20 01:52:24.942281 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932821    2584 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 20 01:52:24.942281 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932825    2584 flags.go:64] FLAG: --tls-min-version=""
Sep 20 01:52:24.942281 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932828    2584 flags.go:64] FLAG: --tls-private-key-file=""
Sep 20 01:52:24.942281 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932831    2584 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 20 01:52:24.942281 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932834    2584 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 20 01:52:24.942281 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932836    2584 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 20 01:52:24.943088 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932839    2584 flags.go:64] FLAG: --v="2"
Sep 20 01:52:24.943088 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932843    2584 flags.go:64] FLAG: --version="false"
Sep 20 01:52:24.943088 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932847    2584 flags.go:64] FLAG: --vmodule=""
Sep 20 01:52:24.943088 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932850    2584 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 20 01:52:24.943088 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.932854    2584 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 20 01:52:24.943088 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.932981    2584 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 20 01:52:24.943088 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.932985    2584 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 20 01:52:24.943088 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.932989    2584 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 20 01:52:24.943088 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.932992    2584 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 20 01:52:24.943088 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.932995    2584 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 20 01:52:24.943088 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.932998    2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 20 01:52:24.943088 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.933001    2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 20 01:52:24.943088 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.933004    2584 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 20 01:52:24.943088 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.933007    2584 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 20 01:52:24.943088 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.933009    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 20 01:52:24.943088 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.933012    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 20 01:52:24.943088 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.933015    2584 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 20 01:52:24.943088 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.933018    2584 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 20 01:52:24.943088 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.933020    2584 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 20 01:52:24.943088 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.933023    2584 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 20 01:52:24.943088 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.933026    2584 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 20 01:52:24.943964 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.933030    2584 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 20 01:52:24.943964 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.933033    2584 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 20 01:52:24.943964 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.933038    2584 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 20 01:52:24.943964 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.933041    2584 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 20 01:52:24.943964 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.933044    2584 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 20 01:52:24.943964 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.933047    2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 20 01:52:24.943964 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.933050    2584 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 20 01:52:24.943964 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.933052    2584 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 20 01:52:24.943964 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.933055    2584 feature_gate.go:328] unrecognized feature gate: Example
Sep 20 01:52:24.943964 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.933058    2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 20 01:52:24.943964 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.933060    2584 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 20 01:52:24.943964 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.933063    2584 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 20 01:52:24.943964 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.933066    2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 20 01:52:24.943964 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.933068    2584 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 20 01:52:24.943964 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.933071    2584 feature_gate.go:328] unrecognized feature gate: Example2
Sep 20 01:52:24.943964 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.933074    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 20 01:52:24.943964 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.933080    2584 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 20 01:52:24.943964 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.933083    2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 20 01:52:24.943964 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.933086    2584 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 20 01:52:24.943964 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.933088    2584 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 20 01:52:24.944714 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.933091    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 20 01:52:24.944714 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.933094    2584 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 20 01:52:24.944714 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.933097    2584 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 20 01:52:24.944714 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.933099    2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 20 01:52:24.944714 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.933102    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 20 01:52:24.944714 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.933104    2584 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 20 01:52:24.944714 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.933107    2584 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 20 01:52:24.944714 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.933110    2584 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 20 01:52:24.944714 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.933112    2584 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 20 01:52:24.944714 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.933116    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 20 01:52:24.944714 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.933120    2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 20 01:52:24.944714 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.933123    2584 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 20 01:52:24.944714 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.933128    2584 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 20 01:52:24.944714 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.933131    2584 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 20 01:52:24.944714 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.933136    2584 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 20 01:52:24.944714 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.933141    2584 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 20 01:52:24.944714 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.933145    2584 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 20 01:52:24.944714 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.933149    2584 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 20 01:52:24.944714 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.933152    2584 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 20 01:52:24.944714 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.933155    2584 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 20 01:52:24.945344 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.933157    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 20 01:52:24.945344 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.933160    2584 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 20 01:52:24.945344 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.933163    2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 20 01:52:24.945344 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.933166    2584 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 20 01:52:24.945344 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.933168    2584 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 20 01:52:24.945344 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.933170    2584 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 20 01:52:24.945344 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.933173    2584 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 20 01:52:24.945344 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.933176    2584 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 20 01:52:24.945344 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.933180    2584 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 20 01:52:24.945344 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.933182    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 20 01:52:24.945344 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.933185    2584 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 20 01:52:24.945344 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.933188    2584 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 20 01:52:24.945344 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.933190    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 20 01:52:24.945344 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.933193    2584 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 20 01:52:24.945344 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.933196    2584 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 20 01:52:24.945344 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.933198    2584 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 20 01:52:24.945344 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.933201    2584 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 20 01:52:24.945344 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.933203    2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 20 01:52:24.945344 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.933206    2584 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 20 01:52:24.945860 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.933209    2584 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 20 01:52:24.945860 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.933212    2584 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 20 01:52:24.945860 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.933215    2584 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 20 01:52:24.945860 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.933218    2584 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 20 01:52:24.945860 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.933221    2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 20 01:52:24.945860 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.933224    2584 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 20 01:52:24.945860 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.933227    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 20 01:52:24.945860 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.933230    2584 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 20 01:52:24.945860 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.933233    2584 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 20 01:52:24.945860 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.933237    2584 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 20 01:52:24.945860 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.933240    2584 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 20 01:52:24.945860 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.933242    2584 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 20 01:52:24.945860 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.933245    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 20 01:52:24.945860 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.933248    2584 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 20 01:52:24.945860 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.933250    2584 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 20 01:52:24.945860 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.933253    2584 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 20 01:52:24.945860 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.933255    2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 20 01:52:24.945860 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.933258    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 20 01:52:24.945860 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.933260    2584 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 20 01:52:24.945860 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.933263    2584 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 20 01:52:24.946364 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.933266    2584 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 20 01:52:24.946364 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.933270    2584 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 20 01:52:24.946364 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.933272    2584 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 20 01:52:24.946364 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.933275    2584 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 20 01:52:24.946364 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.933708    2584 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 20 01:52:24.946364 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.941091    2584 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 20 01:52:24.946364 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.941109    2584 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 20 01:52:24.946364 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.941136    2584 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 20 01:52:24.946364 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.941144    2584 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 20 01:52:24.946364 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.941226    2584 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 20 01:52:24.946364 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.941233    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 20 01:52:24.946364 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.941238    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 20 01:52:24.946364 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.941242    2584 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 20 01:52:24.946364 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.941247    2584 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 20 01:52:24.946364 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.941251    2584 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 20 01:52:24.946810 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.941255    2584 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 20 01:52:24.946810 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.941259    2584 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 20 01:52:24.946810 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.941263    2584 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 20 01:52:24.946810 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.941267    2584 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 20 01:52:24.946810 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.941272    2584 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 20 01:52:24.946810 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.941276    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 20 01:52:24.946810 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.941281    2584 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 20 01:52:24.946810 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.941285    2584 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 20 01:52:24.946810 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.941289    2584 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 20 01:52:24.946810 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.941292    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 20 01:52:24.946810 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.941297    2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 20 01:52:24.946810 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.941301    2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 20 01:52:24.946810 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.941305    2584 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 20 01:52:24.946810 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.941310    2584 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 20 01:52:24.946810 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.941314    2584 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 20 01:52:24.946810 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.941319    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 20 01:52:24.946810 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.941323    2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 20 01:52:24.946810 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.941328    2584 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 20 01:52:24.946810 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.941332    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 20 01:52:24.946810 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.941336    2584 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 20 01:52:24.947321 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.941340    2584 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 20 01:52:24.947321 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.941344    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 20 01:52:24.947321 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.941351    2584 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 20 01:52:24.947321 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.941358    2584 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 20 01:52:24.947321 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.941364    2584 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 20 01:52:24.947321 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.941369    2584 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 20 01:52:24.947321 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.941373    2584 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 20 01:52:24.947321 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.941378    2584 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 20 01:52:24.947321 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.941382    2584 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 20 01:52:24.947321 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.941387    2584 feature_gate.go:328] unrecognized feature gate: Example
Sep 20 01:52:24.947321 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.941391    2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 20 01:52:24.947321 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.941395    2584 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 20 01:52:24.947321 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.941399    2584 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 20 01:52:24.947321 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.941403    2584 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 20 01:52:24.947321 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.941407    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 20 01:52:24.947321 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.941412    2584 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 20 01:52:24.947321 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.941416    2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 20 01:52:24.947321 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.941419    2584 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 20 01:52:24.947321 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.941424    2584 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 20 01:52:24.947321 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.941428    2584 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 20 01:52:24.948124 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.941432    2584 feature_gate.go:328] unrecognized feature gate: Example2
Sep 20 01:52:24.948124 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.941436    2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 20 01:52:24.948124 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.941441    2584 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 20 01:52:24.948124 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.941445    2584 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 20 01:52:24.948124 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.941449    2584 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 20 01:52:24.948124 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.941453    2584 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 20 01:52:24.948124 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.941458    2584 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 20 01:52:24.948124 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.941462    2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 20 01:52:24.948124 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.941466    2584 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 20 01:52:24.948124 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.941470    2584 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 20 01:52:24.948124 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.941474    2584 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 20 01:52:24.948124 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.941478    2584 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 20 01:52:24.948124 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.941482    2584 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 20 01:52:24.948124 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.941486    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 20 01:52:24.948124 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.941490    2584 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 20 01:52:24.948124 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.941495    2584 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 20 01:52:24.948124 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.941500    2584 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 20 01:52:24.948124 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.941504    2584 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 20 01:52:24.948124 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.941508    2584 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 20 01:52:24.948124 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.941513    2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 20 01:52:24.948763 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.941534    2584 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 20 01:52:24.948763 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.941540    2584 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 20 01:52:24.948763 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.941544    2584 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 20 01:52:24.948763 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.941548    2584 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 20 01:52:24.948763 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.941552    2584 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 20 01:52:24.948763 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.941557    2584 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 20 01:52:24.948763 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.941561    2584 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 20 01:52:24.948763 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.941567    2584 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 20 01:52:24.948763 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.941573    2584 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 20 01:52:24.948763 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.941577    2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 20 01:52:24.948763 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.941582    2584 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 20 01:52:24.948763 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.941587    2584 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 20 01:52:24.948763 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.941592    2584 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 20 01:52:24.948763 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.941597    2584 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 20 01:52:24.948763 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.941602    2584 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 20 01:52:24.948763 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.941607    2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 20 01:52:24.948763 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.941613    2584 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 20 01:52:24.948763 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.941619    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 20 01:52:24.948763 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.941623    2584 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 20 01:52:24.949403 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.941628    2584 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 20 01:52:24.949403 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.941633    2584 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 20 01:52:24.949403 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.941638    2584 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 20 01:52:24.949403 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.941642    2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 20 01:52:24.949403 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.941646    2584 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 20 01:52:24.949403 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.941650    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 20 01:52:24.949403 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.941654    2584 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 20 01:52:24.949403 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.941659    2584 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 20 01:52:24.949403 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.941663    2584 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 20 01:52:24.949403 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.941667    2584 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 20 01:52:24.949403 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.941672    2584 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 20 01:52:24.949403 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.941677    2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 20 01:52:24.949403 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.941681    2584 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 20 01:52:24.949403 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.941685    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 20 01:52:24.949403 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.941693    2584 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 20 01:52:24.949907 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.941865    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 20 01:52:24.949907 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.941874    2584 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 20 01:52:24.949907 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.941880    2584 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 20 01:52:24.949907 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.941884    2584 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 20 01:52:24.949907 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.941889    2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 20 01:52:24.949907 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.941893    2584 feature_gate.go:328] unrecognized feature gate: Example
Sep 20 01:52:24.949907 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.941898    2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 20 01:52:24.949907 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.941902    2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 20 01:52:24.949907 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.941907    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 20 01:52:24.949907 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.941911    2584 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 20 01:52:24.949907 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.941915    2584 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 20 01:52:24.949907 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.941920    2584 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 20 01:52:24.949907 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.941925    2584 feature_gate.go:328] unrecognized feature gate: Example2
Sep 20 01:52:24.949907 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.941929    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 20 01:52:24.949907 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.941933    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 20 01:52:24.949907 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.941937    2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 20 01:52:24.949907 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.941943    2584 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 20 01:52:24.949907 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.941949    2584 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 20 01:52:24.949907 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.941954    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 20 01:52:24.949907 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.941958    2584 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 20 01:52:24.950467 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.941963    2584 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 20 01:52:24.950467 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.941968    2584 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 20 01:52:24.950467 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.941972    2584 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 20 01:52:24.950467 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.941976    2584 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 20 01:52:24.950467 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.941980    2584 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 20 01:52:24.950467 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.941984    2584 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 20 01:52:24.950467 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.941989    2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 20 01:52:24.950467 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.941996    2584 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 20 01:52:24.950467 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.942002    2584 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 20 01:52:24.950467 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.942007    2584 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 20 01:52:24.950467 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.942011    2584 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 20 01:52:24.950467 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.942015    2584 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 20 01:52:24.950467 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.942020    2584 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 20 01:52:24.950467 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.942024    2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 20 01:52:24.950467 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.942028    2584 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 20 01:52:24.950467 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.942032    2584 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 20 01:52:24.950467 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.942036    2584 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 20 01:52:24.950467 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.942040    2584 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 20 01:52:24.950467 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.942045    2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 20 01:52:24.951193 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.942049    2584 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 20 01:52:24.951193 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.942053    2584 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 20 01:52:24.951193 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.942057    2584 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 20 01:52:24.951193 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.942061    2584 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 20 01:52:24.951193 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.942065    2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 20 01:52:24.951193 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.942070    2584 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 20 01:52:24.951193 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.942074    2584 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 20 01:52:24.951193 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.942078    2584 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 20 01:52:24.951193 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.942082    2584 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 20 01:52:24.951193 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.942086    2584 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 20 01:52:24.951193 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.942090    2584 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 20 01:52:24.951193 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.942094    2584 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 20 01:52:24.951193 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.942098    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 20 01:52:24.951193 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.942102    2584 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 20 01:52:24.951193 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.942106    2584 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 20 01:52:24.951193 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.942110    2584 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 20 01:52:24.951193 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.942114    2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 20 01:52:24.951193 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.942119    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 20 01:52:24.951193 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.942123    2584 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 20 01:52:24.951193 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.942127    2584 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 20 01:52:24.951933 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.942132    2584 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 20 01:52:24.951933 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.942135    2584 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 20 01:52:24.951933 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.942141    2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 20 01:52:24.951933 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.942145    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 20 01:52:24.951933 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.942149    2584 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 20 01:52:24.951933 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.942153    2584 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 20 01:52:24.951933 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.942157    2584 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 20 01:52:24.951933 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.942162    2584 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 20 01:52:24.951933 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.942166    2584 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 20 01:52:24.951933 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.942170    2584 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 20 01:52:24.951933 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.942174    2584 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 20 01:52:24.951933 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.942178    2584 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 20 01:52:24.951933 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.942182    2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 20 01:52:24.951933 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.942186    2584 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 20 01:52:24.951933 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.942190    2584 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 20 01:52:24.951933 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.942194    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 20 01:52:24.951933 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.942198    2584 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 20 01:52:24.951933 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.942202    2584 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 20 01:52:24.951933 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.942206    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 20 01:52:24.951933 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.942210    2584 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 20 01:52:24.952422 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.942215    2584 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 20 01:52:24.952422 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.942219    2584 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 20 01:52:24.952422 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.942223    2584 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 20 01:52:24.952422 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.942228    2584 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 20 01:52:24.952422 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.942232    2584 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 20 01:52:24.952422 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.942236    2584 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 20 01:52:24.952422 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.942240    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 20 01:52:24.952422 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.942244    2584 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 20 01:52:24.952422 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.942248    2584 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 20 01:52:24.952422 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.942252    2584 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 20 01:52:24.952422 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.942256    2584 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 20 01:52:24.952422 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.942261    2584 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 20 01:52:24.952422 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.942265    2584 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 20 01:52:24.952422 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.942269    2584 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 20 01:52:24.952422 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.942273    2584 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 20 01:52:24.952422 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.942277    2584 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 20 01:52:24.952422 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.942281    2584 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 20 01:52:24.952422 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.942285    2584 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 20 01:52:24.952422 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.942289    2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 20 01:52:24.952422 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:24.942293    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 20 01:52:24.953015 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.942300    2584 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 20 01:52:24.953015 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.942445    2584 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 20 01:52:24.953015 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.945032    2584 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 20 01:52:24.953015 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.945818    2584 server.go:1019] "Starting client certificate rotation"
Sep 20 01:52:24.953015 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.945921    2584 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 20 01:52:24.953015 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.945971    2584 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 20 01:52:24.953015 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.948270    2584 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 20 01:52:24.953420 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.953400    2584 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 20 01:52:24.960354 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.960139    2584 log.go:25] "Validated CRI v1 runtime API"
Sep 20 01:52:24.961938 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.961922    2584 log.go:25] "Validated CRI v1 image API"
Sep 20 01:52:24.962759 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.962743    2584 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 20 01:52:24.964965 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.964945    2584 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 ac44616c-a779-4209-bd2f-5f48e5010c92:/dev/nvme0n1p3 cf7f85b0-f22e-4639-a743-e4f91fc32702:/dev/nvme0n1p4]
Sep 20 01:52:24.965022 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.964968    2584 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 20 01:52:24.969403 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.969286    2584 manager.go:216] Machine: {Timestamp:2026-09-20 01:52:24.968813188 +0000 UTC m=+1.711492384 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3103512 MemoryCapacity:33164476416 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2c71152afea61390286f48ef1b1c8a SystemUUID:ec2c7115-2afe-a613-9028-6f48ef1b1c8a BootID:4da959aa-09fd-4bc3-bf3b-1f68a0b801f7 Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582238208 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:fa:08:34:df Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:fa:08:34:df Speed:0 Mtu:9001} {Name:ovs-system MacAddress:0a:8b:27:87:ab:44 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164476416 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 20 01:52:24.969403 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.969395    2584 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 20 01:52:24.969543 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.969494    2584 manager.go:232] Version: {KernelVersion:5.14.0-570.141.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260914-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 20 01:52:24.970763 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.970742    2584 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 20 01:52:24.970924 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.970765    2584 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-130-159.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 20 01:52:24.971833 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.971821    2584 topology_manager.go:138] "Creating topology manager with none policy"
Sep 20 01:52:24.971870 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.971837    2584 container_manager_linux.go:309] "Creating device plugin manager"
Sep 20 01:52:24.971870 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.971845    2584 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 20 01:52:24.971870 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.971856    2584 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 20 01:52:24.971953 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.971926    2584 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 20 01:52:24.975591 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.975580    2584 state_mem.go:36] "Initialized new in-memory state store"
Sep 20 01:52:24.975700 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.975691    2584 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 20 01:52:24.975754 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.975722    2584 kubelet.go:486] "Attempting to sync node with API server"
Sep 20 01:52:24.975754 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.975736    2584 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 20 01:52:24.975754 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.975747    2584 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 20 01:52:24.975863 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.975757    2584 kubelet.go:398] "Adding apiserver pod source"
Sep 20 01:52:24.975863 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.975766    2584 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 20 01:52:24.975950 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.975894    2584 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 20 01:52:24.976122 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.976110    2584 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 20 01:52:24.976209 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.976128    2584 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 20 01:52:24.977985 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.977964    2584 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.13-6.rhaos4.21.git6253162.el9" apiVersion="v1"
Sep 20 01:52:24.979944 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.979931    2584 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 20 01:52:24.979993 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.979962    2584 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 20 01:52:24.981106 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.981084    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 20 01:52:24.981106 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.981102    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 20 01:52:24.981106 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.981108    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 20 01:52:24.981246 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.981114    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 20 01:52:24.981246 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.981123    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 20 01:52:24.981246 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.981129    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 20 01:52:24.981246 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.981142    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 20 01:52:24.981246 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.981148    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 20 01:52:24.981246 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.981154    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 20 01:52:24.981246 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.981159    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 20 01:52:24.981246 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.981168    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 20 01:52:24.981246 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.981173    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 20 01:52:24.981246 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.981197    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 20 01:52:24.981246 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.981203    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 20 01:52:24.984610 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.984598    2584 server.go:1268] "Started kubelet"
Sep 20 01:52:24.984730 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.984702    2584 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 20 01:52:24.984794 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.984752    2584 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 20 01:52:24.984842 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:24.984814    2584 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 20 01:52:24.985384 ip-10-0-130-159 systemd[1]: Started Kubernetes Kubelet.
Sep 20 01:52:25.013483 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:25.013461    2584 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 20 01:52:25.013584 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:25.013556    2584 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-130-159.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 20 01:52:25.013770 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:25.013751    2584 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 20 01:52:25.014043 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:25.014026    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-130-159.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 20 01:52:25.014645 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:25.014627    2584 server.go:310] "Adding debug handlers to kubelet server"
Sep 20 01:52:25.016575 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:25.016556    2584 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 20 01:52:25.019569 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:25.018874    2584 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-159.ec2.internal.18d6e481153cd69c  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-159.ec2.internal,UID:ip-10-0-130-159.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-130-159.ec2.internal,},FirstTimestamp:2026-09-20 01:52:24.984573596 +0000 UTC m=+1.727252792,LastTimestamp:2026-09-20 01:52:24.984573596 +0000 UTC m=+1.727252792,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-159.ec2.internal,}"
Sep 20 01:52:25.020734 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:25.020717    2584 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 20 01:52:25.020850 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:25.020830    2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-159.ec2.internal\" not found"
Sep 20 01:52:25.020916 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:25.020832    2584 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 20 01:52:25.021203 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:25.021183    2584 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-64lbj"
Sep 20 01:52:25.021297 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:25.021220    2584 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 20 01:52:25.021297 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:25.021236    2584 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 20 01:52:25.021400 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:25.021315    2584 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 20 01:52:25.021400 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:25.021354    2584 reconstruct.go:97] "Volume reconstruction finished"
Sep 20 01:52:25.021400 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:25.021362    2584 reconciler.go:29] "Reconciler: start to sync state"
Sep 20 01:52:25.021696 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:25.021676    2584 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 20 01:52:25.021758 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:25.021698    2584 factory.go:55] Registering systemd factory
Sep 20 01:52:25.021758 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:25.021708    2584 factory.go:223] Registration of the systemd container factory successfully
Sep 20 01:52:25.021954 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:25.021936    2584 factory.go:153] Registering CRI-O factory
Sep 20 01:52:25.021954 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:25.021957    2584 factory.go:223] Registration of the crio container factory successfully
Sep 20 01:52:25.022101 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:25.021974    2584 factory.go:103] Registering Raw factory
Sep 20 01:52:25.022101 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:25.021992    2584 manager.go:1196] Started watching for new ooms in manager
Sep 20 01:52:25.022411 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:25.022398    2584 manager.go:349] Starting recovery of all containers
Sep 20 01:52:25.023367 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:25.023333    2584 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 20 01:52:25.023469 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:25.023436    2584 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-130-159.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 20 01:52:25.028506 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:25.028488    2584 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-64lbj"
Sep 20 01:52:25.031701 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:25.031686    2584 manager.go:354] Recovery completed
Sep 20 01:52:25.035937 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:25.035923    2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 20 01:52:25.037772 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:25.037757    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-159.ec2.internal" event="NodeHasSufficientMemory"
Sep 20 01:52:25.037857 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:25.037788    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-159.ec2.internal" event="NodeHasNoDiskPressure"
Sep 20 01:52:25.037857 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:25.037803    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-159.ec2.internal" event="NodeHasSufficientPID"
Sep 20 01:52:25.038301 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:25.038289    2584 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 20 01:52:25.038301 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:25.038301    2584 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 20 01:52:25.038407 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:25.038289    2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 20 01:52:25.038407 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:25.038318    2584 state_mem.go:36] "Initialized new in-memory state store"
Sep 20 01:52:25.038968 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:25.038955    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-159.ec2.internal" event="NodeHasSufficientMemory"
Sep 20 01:52:25.039023 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:25.038977    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-159.ec2.internal" event="NodeHasNoDiskPressure"
Sep 20 01:52:25.039023 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:25.038988    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-159.ec2.internal" event="NodeHasSufficientPID"
Sep 20 01:52:25.039802 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:25.039739    2584 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-159.ec2.internal.18d6e481186895d7  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-159.ec2.internal,UID:ip-10-0-130-159.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-130-159.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-130-159.ec2.internal,},FirstTimestamp:2026-09-20 01:52:25.037772247 +0000 UTC m=+1.780451452,LastTimestamp:2026-09-20 01:52:25.037772247 +0000 UTC m=+1.780451452,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-159.ec2.internal,}"
Sep 20 01:52:25.040437 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:25.040418    2584 policy_none.go:49] "None policy: Start"
Sep 20 01:52:25.040437 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:25.040435    2584 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 20 01:52:25.040553 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:25.040444    2584 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 20 01:52:25.042456 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:25.042445    2584 policy_none.go:47] "Start"
Sep 20 01:52:25.084705 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:25.084673    2584 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 20 01:52:25.095571 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:25.085867    2584 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 20 01:52:25.095571 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:25.085912    2584 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 20 01:52:25.095571 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:25.085933    2584 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 20 01:52:25.095571 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:25.085980    2584 kubelet.go:2472] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]"
Sep 20 01:52:25.095571 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:25.087369    2584 manager.go:337] "Starting Device Plugin manager"
Sep 20 01:52:25.095571 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:25.087400    2584 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 20 01:52:25.095571 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:25.087409    2584 server.go:87] "Starting device plugin registration server"
Sep 20 01:52:25.095571 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:25.087653    2584 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 20 01:52:25.095571 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:25.087678    2584 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 20 01:52:25.095571 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:25.087801    2584 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 20 01:52:25.095571 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:25.087893    2584 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 20 01:52:25.095571 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:25.087998    2584 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 20 01:52:25.095571 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:25.088024    2584 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 20 01:52:25.095571 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:25.088457    2584 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 20 01:52:25.095571 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:25.088497    2584 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-130-159.ec2.internal\" not found"
Sep 20 01:52:25.186452 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:25.186380    2584 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-159.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-130-159.ec2.internal"]
Sep 20 01:52:25.186452 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:25.186441    2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 20 01:52:25.187272 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:25.187256    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-159.ec2.internal" event="NodeHasSufficientMemory"
Sep 20 01:52:25.187326 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:25.187286    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-159.ec2.internal" event="NodeHasNoDiskPressure"
Sep 20 01:52:25.187326 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:25.187299    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-159.ec2.internal" event="NodeHasSufficientPID"
Sep 20 01:52:25.188337 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:25.188323    2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 20 01:52:25.188995 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:25.188982    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-159.ec2.internal" event="NodeHasSufficientMemory"
Sep 20 01:52:25.189073 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:25.189006    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-159.ec2.internal" event="NodeHasNoDiskPressure"
Sep 20 01:52:25.189073 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:25.189020    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-159.ec2.internal" event="NodeHasSufficientPID"
Sep 20 01:52:25.189073 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:25.189040    2584 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-130-159.ec2.internal"
Sep 20 01:52:25.189668 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:25.189657    2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 20 01:52:25.189797 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:25.189785    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-159.ec2.internal"
Sep 20 01:52:25.189836 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:25.189811    2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 20 01:52:25.192654 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:25.192638    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-159.ec2.internal" event="NodeHasSufficientMemory"
Sep 20 01:52:25.192750 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:25.192666    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-159.ec2.internal" event="NodeHasNoDiskPressure"
Sep 20 01:52:25.192750 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:25.192679    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-159.ec2.internal" event="NodeHasSufficientPID"
Sep 20 01:52:25.192750 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:25.192667    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-159.ec2.internal" event="NodeHasSufficientMemory"
Sep 20 01:52:25.192750 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:25.192751    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-159.ec2.internal" event="NodeHasNoDiskPressure"
Sep 20 01:52:25.192897 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:25.192762    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-159.ec2.internal" event="NodeHasSufficientPID"
Sep 20 01:52:25.195026 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:25.195012    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-159.ec2.internal"
Sep 20 01:52:25.195092 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:25.195036    2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 20 01:52:25.195730 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:25.195716    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-159.ec2.internal" event="NodeHasSufficientMemory"
Sep 20 01:52:25.195802 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:25.195740    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-159.ec2.internal" event="NodeHasNoDiskPressure"
Sep 20 01:52:25.195802 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:25.195752    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-159.ec2.internal" event="NodeHasSufficientPID"
Sep 20 01:52:25.197370 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:25.197356    2584 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-130-159.ec2.internal"
Sep 20 01:52:25.197420 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:25.197375    2584 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-130-159.ec2.internal\": node \"ip-10-0-130-159.ec2.internal\" not found"
Sep 20 01:52:25.222453 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:25.222435    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4f739e7563e31c6ca101623c66d03faf-config\") pod \"kube-apiserver-proxy-ip-10-0-130-159.ec2.internal\" (UID: \"4f739e7563e31c6ca101623c66d03faf\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-159.ec2.internal"
Sep 20 01:52:25.222543 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:25.222458    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/55f1df651e44bec56d534fcb469f62e8-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-159.ec2.internal\" (UID: \"55f1df651e44bec56d534fcb469f62e8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-159.ec2.internal"
Sep 20 01:52:25.222543 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:25.222476    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/55f1df651e44bec56d534fcb469f62e8-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-159.ec2.internal\" (UID: \"55f1df651e44bec56d534fcb469f62e8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-159.ec2.internal"
Sep 20 01:52:25.223136 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:25.223117    2584 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-159.ec2.internal\" not found" node="ip-10-0-130-159.ec2.internal"
Sep 20 01:52:25.225409 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:25.225392    2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-159.ec2.internal\" not found"
Sep 20 01:52:25.227478 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:25.227465    2584 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-159.ec2.internal\" not found" node="ip-10-0-130-159.ec2.internal"
Sep 20 01:52:25.322624 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:25.322599    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/55f1df651e44bec56d534fcb469f62e8-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-159.ec2.internal\" (UID: \"55f1df651e44bec56d534fcb469f62e8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-159.ec2.internal"
Sep 20 01:52:25.322731 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:25.322666    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/55f1df651e44bec56d534fcb469f62e8-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-159.ec2.internal\" (UID: \"55f1df651e44bec56d534fcb469f62e8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-159.ec2.internal"
Sep 20 01:52:25.322731 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:25.322706    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/55f1df651e44bec56d534fcb469f62e8-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-159.ec2.internal\" (UID: \"55f1df651e44bec56d534fcb469f62e8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-159.ec2.internal"
Sep 20 01:52:25.322808 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:25.322731    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4f739e7563e31c6ca101623c66d03faf-config\") pod \"kube-apiserver-proxy-ip-10-0-130-159.ec2.internal\" (UID: \"4f739e7563e31c6ca101623c66d03faf\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-159.ec2.internal"
Sep 20 01:52:25.322808 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:25.322771    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4f739e7563e31c6ca101623c66d03faf-config\") pod \"kube-apiserver-proxy-ip-10-0-130-159.ec2.internal\" (UID: \"4f739e7563e31c6ca101623c66d03faf\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-159.ec2.internal"
Sep 20 01:52:25.322808 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:25.322773    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/55f1df651e44bec56d534fcb469f62e8-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-159.ec2.internal\" (UID: \"55f1df651e44bec56d534fcb469f62e8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-159.ec2.internal"
Sep 20 01:52:25.325715 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:25.325700    2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-159.ec2.internal\" not found"
Sep 20 01:52:25.426156 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:25.426130    2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-159.ec2.internal\" not found"
Sep 20 01:52:25.526808 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:25.526779    2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-159.ec2.internal\" not found"
Sep 20 01:52:25.526808 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:25.526791    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-159.ec2.internal"
Sep 20 01:52:25.530479 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:25.530462    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-159.ec2.internal"
Sep 20 01:52:25.626906 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:25.626881    2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-159.ec2.internal\" not found"
Sep 20 01:52:25.727438 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:25.727413    2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-159.ec2.internal\" not found"
Sep 20 01:52:25.822461 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:25.822397    2584 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 20 01:52:25.873200 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:25.873170    2584 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 20 01:52:25.920917 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:25.920888    2584 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-159.ec2.internal"
Sep 20 01:52:25.933415 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:25.933396    2584 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 20 01:52:25.934209 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:25.934197    2584 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-159.ec2.internal"
Sep 20 01:52:25.941871 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:25.941855    2584 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 20 01:52:25.945992 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:25.945980    2584 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 20 01:52:25.946090 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:25.946076    2584 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 20 01:52:25.946132 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:25.946113    2584 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 20 01:52:25.946132 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:25.946122    2584 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 20 01:52:25.976286 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:25.976266    2584 apiserver.go:52] "Watching apiserver"
Sep 20 01:52:25.990662 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:25.990641    2584 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 20 01:52:25.990970 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:25.990950    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/node-resolver-l9ssk","openshift-multus/multus-k8rn8","openshift-multus/network-metrics-daemon-n4dk7","openshift-network-diagnostics/network-check-target-47994","openshift-network-operator/iptables-alerter-chcxk","openshift-ovn-kubernetes/ovnkube-node-p2wkn","kube-system/global-pull-secret-syncer-llv6h","kube-system/konnectivity-agent-ftlkq","kube-system/kube-apiserver-proxy-ip-10-0-130-159.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-27wj7","openshift-cluster-node-tuning-operator/tuned-m4zgq","openshift-image-registry/node-ca-v6d2k","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-159.ec2.internal","openshift-multus/multus-additional-cni-plugins-f94fv"]
Sep 20 01:52:25.993241 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:25.993211    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-l9ssk"
Sep 20 01:52:25.995949 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:25.995923    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 20 01:52:25.995949 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:25.995923    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 20 01:52:25.996114 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:25.996020    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-sjbdm\""
Sep 20 01:52:25.996308 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:25.996291    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-k8rn8"
Sep 20 01:52:25.998215 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:25.998196    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 20 01:52:25.998370 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:25.998347    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 20 01:52:25.998484 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:25.998401    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n4dk7"
Sep 20 01:52:25.998484 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:25.998351    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 20 01:52:25.998484 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:25.998446    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 20 01:52:25.998655 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:25.998487    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-mtz2t\""
Sep 20 01:52:25.998655 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:25.998480    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n4dk7" podUID="c9c657b9-b508-4bb7-99f3-aed2502cfba4"
Sep 20 01:52:26.000607 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.000588    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-47994"
Sep 20 01:52:26.000683 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:26.000663    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-47994" podUID="9b376ae6-8f6a-4139-ab2d-96dc32e1201f"
Sep 20 01:52:26.003574 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.003554    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-chcxk"
Sep 20 01:52:26.005406 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.005387    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 20 01:52:26.005734 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.005718    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 20 01:52:26.005810 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.005743    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 20 01:52:26.005860 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.005744    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-c4f4k\""
Sep 20 01:52:26.005996 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.005982    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-p2wkn"
Sep 20 01:52:26.007844 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.007825    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 20 01:52:26.007954 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.007938    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 20 01:52:26.008040 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.008025    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 20 01:52:26.008199 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.008185    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 20 01:52:26.008199 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.008208    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 20 01:52:26.008334 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.008318    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-b8kn9\""
Sep 20 01:52:26.008452 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.008434    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-llv6h"
Sep 20 01:52:26.008569 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:26.008491    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-llv6h" podUID="05a98348-758f-4a4a-b150-35bbdeb5b359"
Sep 20 01:52:26.008635 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.008575    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 20 01:52:26.010742 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.010726    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-ftlkq"
Sep 20 01:52:26.012626 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.012608    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 20 01:52:26.012760 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.012745    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 20 01:52:26.012821 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.012804    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-r4nnl\""
Sep 20 01:52:26.013605 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.013590    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-27wj7"
Sep 20 01:52:26.015426 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.015410    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 20 01:52:26.015496 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.015464    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 20 01:52:26.015559 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.015541    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 20 01:52:26.015690 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.015674    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-762g8\""
Sep 20 01:52:26.016241 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.016228    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-m4zgq"
Sep 20 01:52:26.018020 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.017997    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 20 01:52:26.018332 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.018317    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 20 01:52:26.018668 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.018655    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-8g865\""
Sep 20 01:52:26.019076 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.019062    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-v6d2k"
Sep 20 01:52:26.020826 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.020813    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 20 01:52:26.020906 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.020883    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 20 01:52:26.020944 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.020924    2584 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 20 01:52:26.021073 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.021056    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 20 01:52:26.021121 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.021072    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-khpdd\""
Sep 20 01:52:26.021625 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.021606    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-f94fv"
Sep 20 01:52:26.024619 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.024600    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-5r49k\""
Sep 20 01:52:26.024859 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.024833    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 20 01:52:26.025201 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.025178    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 20 01:52:26.026289 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.026269    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f9800646-e98f-45d8-aa4e-5eb53056f5e8-host-run-netns\") pod \"multus-k8rn8\" (UID: \"f9800646-e98f-45d8-aa4e-5eb53056f5e8\") " pod="openshift-multus/multus-k8rn8"
Sep 20 01:52:26.026353 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.026298    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f9800646-e98f-45d8-aa4e-5eb53056f5e8-etc-kubernetes\") pod \"multus-k8rn8\" (UID: \"f9800646-e98f-45d8-aa4e-5eb53056f5e8\") " pod="openshift-multus/multus-k8rn8"
Sep 20 01:52:26.026353 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.026316    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/43dd43c0-ff48-4f9a-a1c5-9020836702aa-hosts-file\") pod \"node-resolver-l9ssk\" (UID: \"43dd43c0-ff48-4f9a-a1c5-9020836702aa\") " pod="openshift-dns/node-resolver-l9ssk"
Sep 20 01:52:26.026353 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.026329    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d08af64d-ba39-4d61-9b95-c6febd32cbbb-konnectivity-ca\") pod \"konnectivity-agent-ftlkq\" (UID: \"d08af64d-ba39-4d61-9b95-c6febd32cbbb\") " pod="kube-system/konnectivity-agent-ftlkq"
Sep 20 01:52:26.026353 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.026345    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/92615b78-f30d-4687-a822-6261a2d9ae55-etc-sysconfig\") pod \"tuned-m4zgq\" (UID: \"92615b78-f30d-4687-a822-6261a2d9ae55\") " pod="openshift-cluster-node-tuning-operator/tuned-m4zgq"
Sep 20 01:52:26.026540 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.026387    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f9800646-e98f-45d8-aa4e-5eb53056f5e8-multus-cni-dir\") pod \"multus-k8rn8\" (UID: \"f9800646-e98f-45d8-aa4e-5eb53056f5e8\") " pod="openshift-multus/multus-k8rn8"
Sep 20 01:52:26.026540 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.026416    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f9800646-e98f-45d8-aa4e-5eb53056f5e8-host-var-lib-cni-bin\") pod \"multus-k8rn8\" (UID: \"f9800646-e98f-45d8-aa4e-5eb53056f5e8\") " pod="openshift-multus/multus-k8rn8"
Sep 20 01:52:26.026540 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.026470    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mrb2z\" (UniqueName: \"kubernetes.io/projected/9b376ae6-8f6a-4139-ab2d-96dc32e1201f-kube-api-access-mrb2z\") pod \"network-check-target-47994\" (UID: \"9b376ae6-8f6a-4139-ab2d-96dc32e1201f\") " pod="openshift-network-diagnostics/network-check-target-47994"
Sep 20 01:52:26.026540 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.026503    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/147f9cc3-d8b1-4c83-9a97-4b29246cd454-host-cni-netd\") pod \"ovnkube-node-p2wkn\" (UID: \"147f9cc3-d8b1-4c83-9a97-4b29246cd454\") " pod="openshift-ovn-kubernetes/ovnkube-node-p2wkn"
Sep 20 01:52:26.027165 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.026918    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/147f9cc3-d8b1-4c83-9a97-4b29246cd454-ovn-node-metrics-cert\") pod \"ovnkube-node-p2wkn\" (UID: \"147f9cc3-d8b1-4c83-9a97-4b29246cd454\") " pod="openshift-ovn-kubernetes/ovnkube-node-p2wkn"
Sep 20 01:52:26.027165 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.026978    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d0aee250-9aee-4473-ae29-83ffdbaeb920-sys-fs\") pod \"aws-ebs-csi-driver-node-27wj7\" (UID: \"d0aee250-9aee-4473-ae29-83ffdbaeb920\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-27wj7"
Sep 20 01:52:26.027165 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.027025    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f9800646-e98f-45d8-aa4e-5eb53056f5e8-multus-socket-dir-parent\") pod \"multus-k8rn8\" (UID: \"f9800646-e98f-45d8-aa4e-5eb53056f5e8\") " pod="openshift-multus/multus-k8rn8"
Sep 20 01:52:26.027165 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.027064    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f9800646-e98f-45d8-aa4e-5eb53056f5e8-multus-conf-dir\") pod \"multus-k8rn8\" (UID: \"f9800646-e98f-45d8-aa4e-5eb53056f5e8\") " pod="openshift-multus/multus-k8rn8"
Sep 20 01:52:26.027165 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.027099    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jjdvt\" (UniqueName: \"kubernetes.io/projected/c9c657b9-b508-4bb7-99f3-aed2502cfba4-kube-api-access-jjdvt\") pod \"network-metrics-daemon-n4dk7\" (UID: \"c9c657b9-b508-4bb7-99f3-aed2502cfba4\") " pod="openshift-multus/network-metrics-daemon-n4dk7"
Sep 20 01:52:26.027165 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.027133    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/147f9cc3-d8b1-4c83-9a97-4b29246cd454-log-socket\") pod \"ovnkube-node-p2wkn\" (UID: \"147f9cc3-d8b1-4c83-9a97-4b29246cd454\") " pod="openshift-ovn-kubernetes/ovnkube-node-p2wkn"
Sep 20 01:52:26.027503 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.027198    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/92615b78-f30d-4687-a822-6261a2d9ae55-etc-kubernetes\") pod \"tuned-m4zgq\" (UID: \"92615b78-f30d-4687-a822-6261a2d9ae55\") " pod="openshift-cluster-node-tuning-operator/tuned-m4zgq"
Sep 20 01:52:26.027503 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.027239    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/147f9cc3-d8b1-4c83-9a97-4b29246cd454-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-p2wkn\" (UID: \"147f9cc3-d8b1-4c83-9a97-4b29246cd454\") " pod="openshift-ovn-kubernetes/ovnkube-node-p2wkn"
Sep 20 01:52:26.027503 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.027271    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/147f9cc3-d8b1-4c83-9a97-4b29246cd454-ovnkube-config\") pod \"ovnkube-node-p2wkn\" (UID: \"147f9cc3-d8b1-4c83-9a97-4b29246cd454\") " pod="openshift-ovn-kubernetes/ovnkube-node-p2wkn"
Sep 20 01:52:26.027503 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.027295    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lxxxz\" (UniqueName: \"kubernetes.io/projected/147f9cc3-d8b1-4c83-9a97-4b29246cd454-kube-api-access-lxxxz\") pod \"ovnkube-node-p2wkn\" (UID: \"147f9cc3-d8b1-4c83-9a97-4b29246cd454\") " pod="openshift-ovn-kubernetes/ovnkube-node-p2wkn"
Sep 20 01:52:26.027503 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.027318    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/96a97560-011d-4806-ae10-4d53880eb9ff-cnibin\") pod \"multus-additional-cni-plugins-f94fv\" (UID: \"96a97560-011d-4806-ae10-4d53880eb9ff\") " pod="openshift-multus/multus-additional-cni-plugins-f94fv"
Sep 20 01:52:26.027503 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.027343    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f9800646-e98f-45d8-aa4e-5eb53056f5e8-host-run-multus-certs\") pod \"multus-k8rn8\" (UID: \"f9800646-e98f-45d8-aa4e-5eb53056f5e8\") " pod="openshift-multus/multus-k8rn8"
Sep 20 01:52:26.027503 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.027374    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/147f9cc3-d8b1-4c83-9a97-4b29246cd454-host-kubelet\") pod \"ovnkube-node-p2wkn\" (UID: \"147f9cc3-d8b1-4c83-9a97-4b29246cd454\") " pod="openshift-ovn-kubernetes/ovnkube-node-p2wkn"
Sep 20 01:52:26.027503 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.027402    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/147f9cc3-d8b1-4c83-9a97-4b29246cd454-systemd-units\") pod \"ovnkube-node-p2wkn\" (UID: \"147f9cc3-d8b1-4c83-9a97-4b29246cd454\") " pod="openshift-ovn-kubernetes/ovnkube-node-p2wkn"
Sep 20 01:52:26.029924 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.028653    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/147f9cc3-d8b1-4c83-9a97-4b29246cd454-var-lib-openvswitch\") pod \"ovnkube-node-p2wkn\" (UID: \"147f9cc3-d8b1-4c83-9a97-4b29246cd454\") " pod="openshift-ovn-kubernetes/ovnkube-node-p2wkn"
Sep 20 01:52:26.029924 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.028697    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4fa3e7e2-7940-4d7d-af23-63ff07edba43-serviceca\") pod \"node-ca-v6d2k\" (UID: \"4fa3e7e2-7940-4d7d-af23-63ff07edba43\") " pod="openshift-image-registry/node-ca-v6d2k"
Sep 20 01:52:26.029924 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.028724    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nm94k\" (UniqueName: \"kubernetes.io/projected/43dd43c0-ff48-4f9a-a1c5-9020836702aa-kube-api-access-nm94k\") pod \"node-resolver-l9ssk\" (UID: \"43dd43c0-ff48-4f9a-a1c5-9020836702aa\") " pod="openshift-dns/node-resolver-l9ssk"
Sep 20 01:52:26.029924 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.028753    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d0aee250-9aee-4473-ae29-83ffdbaeb920-etc-selinux\") pod \"aws-ebs-csi-driver-node-27wj7\" (UID: \"d0aee250-9aee-4473-ae29-83ffdbaeb920\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-27wj7"
Sep 20 01:52:26.029924 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.028784    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/92615b78-f30d-4687-a822-6261a2d9ae55-etc-tuned\") pod \"tuned-m4zgq\" (UID: \"92615b78-f30d-4687-a822-6261a2d9ae55\") " pod="openshift-cluster-node-tuning-operator/tuned-m4zgq"
Sep 20 01:52:26.029924 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.028818    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d94badda-e0c0-4b65-a95f-64f524896542-host-slash\") pod \"iptables-alerter-chcxk\" (UID: \"d94badda-e0c0-4b65-a95f-64f524896542\") " pod="openshift-network-operator/iptables-alerter-chcxk"
Sep 20 01:52:26.029924 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.028849    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/05a98348-758f-4a4a-b150-35bbdeb5b359-original-pull-secret\") pod \"global-pull-secret-syncer-llv6h\" (UID: \"05a98348-758f-4a4a-b150-35bbdeb5b359\") " pod="kube-system/global-pull-secret-syncer-llv6h"
Sep 20 01:52:26.029924 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.028874    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d0aee250-9aee-4473-ae29-83ffdbaeb920-device-dir\") pod \"aws-ebs-csi-driver-node-27wj7\" (UID: \"d0aee250-9aee-4473-ae29-83ffdbaeb920\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-27wj7"
Sep 20 01:52:26.029924 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.028903    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/92615b78-f30d-4687-a822-6261a2d9ae55-sys\") pod \"tuned-m4zgq\" (UID: \"92615b78-f30d-4687-a822-6261a2d9ae55\") " pod="openshift-cluster-node-tuning-operator/tuned-m4zgq"
Sep 20 01:52:26.029924 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.028931    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f9800646-e98f-45d8-aa4e-5eb53056f5e8-cnibin\") pod \"multus-k8rn8\" (UID: \"f9800646-e98f-45d8-aa4e-5eb53056f5e8\") " pod="openshift-multus/multus-k8rn8"
Sep 20 01:52:26.029924 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.028958    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tsj66\" (UniqueName: \"kubernetes.io/projected/96a97560-011d-4806-ae10-4d53880eb9ff-kube-api-access-tsj66\") pod \"multus-additional-cni-plugins-f94fv\" (UID: \"96a97560-011d-4806-ae10-4d53880eb9ff\") " pod="openshift-multus/multus-additional-cni-plugins-f94fv"
Sep 20 01:52:26.029924 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.028987    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/92615b78-f30d-4687-a822-6261a2d9ae55-lib-modules\") pod \"tuned-m4zgq\" (UID: \"92615b78-f30d-4687-a822-6261a2d9ae55\") " pod="openshift-cluster-node-tuning-operator/tuned-m4zgq"
Sep 20 01:52:26.029924 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.029009    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f9800646-e98f-45d8-aa4e-5eb53056f5e8-host-var-lib-cni-multus\") pod \"multus-k8rn8\" (UID: \"f9800646-e98f-45d8-aa4e-5eb53056f5e8\") " pod="openshift-multus/multus-k8rn8"
Sep 20 01:52:26.029924 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.029036    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f9800646-e98f-45d8-aa4e-5eb53056f5e8-host-var-lib-kubelet\") pod \"multus-k8rn8\" (UID: \"f9800646-e98f-45d8-aa4e-5eb53056f5e8\") " pod="openshift-multus/multus-k8rn8"
Sep 20 01:52:26.029924 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.029062    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f9800646-e98f-45d8-aa4e-5eb53056f5e8-hostroot\") pod \"multus-k8rn8\" (UID: \"f9800646-e98f-45d8-aa4e-5eb53056f5e8\") " pod="openshift-multus/multus-k8rn8"
Sep 20 01:52:26.029924 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.029097    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wjsfw\" (UniqueName: \"kubernetes.io/projected/d94badda-e0c0-4b65-a95f-64f524896542-kube-api-access-wjsfw\") pod \"iptables-alerter-chcxk\" (UID: \"d94badda-e0c0-4b65-a95f-64f524896542\") " pod="openshift-network-operator/iptables-alerter-chcxk"
Sep 20 01:52:26.030603 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.029123    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/147f9cc3-d8b1-4c83-9a97-4b29246cd454-host-slash\") pod \"ovnkube-node-p2wkn\" (UID: \"147f9cc3-d8b1-4c83-9a97-4b29246cd454\") " pod="openshift-ovn-kubernetes/ovnkube-node-p2wkn"
Sep 20 01:52:26.030603 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.029146    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/147f9cc3-d8b1-4c83-9a97-4b29246cd454-run-systemd\") pod \"ovnkube-node-p2wkn\" (UID: \"147f9cc3-d8b1-4c83-9a97-4b29246cd454\") " pod="openshift-ovn-kubernetes/ovnkube-node-p2wkn"
Sep 20 01:52:26.030603 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.029175    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/96a97560-011d-4806-ae10-4d53880eb9ff-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-f94fv\" (UID: \"96a97560-011d-4806-ae10-4d53880eb9ff\") " pod="openshift-multus/multus-additional-cni-plugins-f94fv"
Sep 20 01:52:26.030603 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.029204    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/96a97560-011d-4806-ae10-4d53880eb9ff-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-f94fv\" (UID: \"96a97560-011d-4806-ae10-4d53880eb9ff\") " pod="openshift-multus/multus-additional-cni-plugins-f94fv"
Sep 20 01:52:26.030603 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.029234    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/05a98348-758f-4a4a-b150-35bbdeb5b359-dbus\") pod \"global-pull-secret-syncer-llv6h\" (UID: \"05a98348-758f-4a4a-b150-35bbdeb5b359\") " pod="kube-system/global-pull-secret-syncer-llv6h"
Sep 20 01:52:26.030603 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.029263    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d0aee250-9aee-4473-ae29-83ffdbaeb920-socket-dir\") pod \"aws-ebs-csi-driver-node-27wj7\" (UID: \"d0aee250-9aee-4473-ae29-83ffdbaeb920\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-27wj7"
Sep 20 01:52:26.030603 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.029286    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/92615b78-f30d-4687-a822-6261a2d9ae55-etc-modprobe-d\") pod \"tuned-m4zgq\" (UID: \"92615b78-f30d-4687-a822-6261a2d9ae55\") " pod="openshift-cluster-node-tuning-operator/tuned-m4zgq"
Sep 20 01:52:26.030603 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.029314    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f9800646-e98f-45d8-aa4e-5eb53056f5e8-host-run-k8s-cni-cncf-io\") pod \"multus-k8rn8\" (UID: \"f9800646-e98f-45d8-aa4e-5eb53056f5e8\") " pod="openshift-multus/multus-k8rn8"
Sep 20 01:52:26.030603 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.029342    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f9800646-e98f-45d8-aa4e-5eb53056f5e8-cni-binary-copy\") pod \"multus-k8rn8\" (UID: \"f9800646-e98f-45d8-aa4e-5eb53056f5e8\") " pod="openshift-multus/multus-k8rn8"
Sep 20 01:52:26.030603 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.029369    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/147f9cc3-d8b1-4c83-9a97-4b29246cd454-host-run-netns\") pod \"ovnkube-node-p2wkn\" (UID: \"147f9cc3-d8b1-4c83-9a97-4b29246cd454\") " pod="openshift-ovn-kubernetes/ovnkube-node-p2wkn"
Sep 20 01:52:26.030603 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.029399    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/147f9cc3-d8b1-4c83-9a97-4b29246cd454-host-cni-bin\") pod \"ovnkube-node-p2wkn\" (UID: \"147f9cc3-d8b1-4c83-9a97-4b29246cd454\") " pod="openshift-ovn-kubernetes/ovnkube-node-p2wkn"
Sep 20 01:52:26.030603 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.029423    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/147f9cc3-d8b1-4c83-9a97-4b29246cd454-env-overrides\") pod \"ovnkube-node-p2wkn\" (UID: \"147f9cc3-d8b1-4c83-9a97-4b29246cd454\") " pod="openshift-ovn-kubernetes/ovnkube-node-p2wkn"
Sep 20 01:52:26.030603 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.029452    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/05a98348-758f-4a4a-b150-35bbdeb5b359-kubelet-config\") pod \"global-pull-secret-syncer-llv6h\" (UID: \"05a98348-758f-4a4a-b150-35bbdeb5b359\") " pod="kube-system/global-pull-secret-syncer-llv6h"
Sep 20 01:52:26.030603 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.029485    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/43dd43c0-ff48-4f9a-a1c5-9020836702aa-tmp-dir\") pod \"node-resolver-l9ssk\" (UID: \"43dd43c0-ff48-4f9a-a1c5-9020836702aa\") " pod="openshift-dns/node-resolver-l9ssk"
Sep 20 01:52:26.030603 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.029512    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d0aee250-9aee-4473-ae29-83ffdbaeb920-kubelet-dir\") pod \"aws-ebs-csi-driver-node-27wj7\" (UID: \"d0aee250-9aee-4473-ae29-83ffdbaeb920\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-27wj7"
Sep 20 01:52:26.030603 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.029560    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ngtsk\" (UniqueName: \"kubernetes.io/projected/d0aee250-9aee-4473-ae29-83ffdbaeb920-kube-api-access-ngtsk\") pod \"aws-ebs-csi-driver-node-27wj7\" (UID: \"d0aee250-9aee-4473-ae29-83ffdbaeb920\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-27wj7"
Sep 20 01:52:26.031126 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.029599    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6kt7d\" (UniqueName: \"kubernetes.io/projected/f9800646-e98f-45d8-aa4e-5eb53056f5e8-kube-api-access-6kt7d\") pod \"multus-k8rn8\" (UID: \"f9800646-e98f-45d8-aa4e-5eb53056f5e8\") " pod="openshift-multus/multus-k8rn8"
Sep 20 01:52:26.031126 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.029630    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/96a97560-011d-4806-ae10-4d53880eb9ff-cni-binary-copy\") pod \"multus-additional-cni-plugins-f94fv\" (UID: \"96a97560-011d-4806-ae10-4d53880eb9ff\") " pod="openshift-multus/multus-additional-cni-plugins-f94fv"
Sep 20 01:52:26.031126 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.029661    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/96a97560-011d-4806-ae10-4d53880eb9ff-tuning-conf-dir\") pod \"multus-additional-cni-plugins-f94fv\" (UID: \"96a97560-011d-4806-ae10-4d53880eb9ff\") " pod="openshift-multus/multus-additional-cni-plugins-f94fv"
Sep 20 01:52:26.031126 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.029688    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f9800646-e98f-45d8-aa4e-5eb53056f5e8-system-cni-dir\") pod \"multus-k8rn8\" (UID: \"f9800646-e98f-45d8-aa4e-5eb53056f5e8\") " pod="openshift-multus/multus-k8rn8"
Sep 20 01:52:26.031126 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.029721    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/92615b78-f30d-4687-a822-6261a2d9ae55-etc-sysctl-conf\") pod \"tuned-m4zgq\" (UID: \"92615b78-f30d-4687-a822-6261a2d9ae55\") " pod="openshift-cluster-node-tuning-operator/tuned-m4zgq"
Sep 20 01:52:26.031126 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.029748    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m5r9c\" (UniqueName: \"kubernetes.io/projected/4fa3e7e2-7940-4d7d-af23-63ff07edba43-kube-api-access-m5r9c\") pod \"node-ca-v6d2k\" (UID: \"4fa3e7e2-7940-4d7d-af23-63ff07edba43\") " pod="openshift-image-registry/node-ca-v6d2k"
Sep 20 01:52:26.031126 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.029772    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/92615b78-f30d-4687-a822-6261a2d9ae55-tmp\") pod \"tuned-m4zgq\" (UID: \"92615b78-f30d-4687-a822-6261a2d9ae55\") " pod="openshift-cluster-node-tuning-operator/tuned-m4zgq"
Sep 20 01:52:26.031126 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.029801    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/96a97560-011d-4806-ae10-4d53880eb9ff-system-cni-dir\") pod \"multus-additional-cni-plugins-f94fv\" (UID: \"96a97560-011d-4806-ae10-4d53880eb9ff\") " pod="openshift-multus/multus-additional-cni-plugins-f94fv"
Sep 20 01:52:26.031126 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.029862    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c9c657b9-b508-4bb7-99f3-aed2502cfba4-metrics-certs\") pod \"network-metrics-daemon-n4dk7\" (UID: \"c9c657b9-b508-4bb7-99f3-aed2502cfba4\") " pod="openshift-multus/network-metrics-daemon-n4dk7"
Sep 20 01:52:26.031126 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.030029    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d94badda-e0c0-4b65-a95f-64f524896542-iptables-alerter-script\") pod \"iptables-alerter-chcxk\" (UID: \"d94badda-e0c0-4b65-a95f-64f524896542\") " pod="openshift-network-operator/iptables-alerter-chcxk"
Sep 20 01:52:26.031126 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.030060    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/147f9cc3-d8b1-4c83-9a97-4b29246cd454-run-openvswitch\") pod \"ovnkube-node-p2wkn\" (UID: \"147f9cc3-d8b1-4c83-9a97-4b29246cd454\") " pod="openshift-ovn-kubernetes/ovnkube-node-p2wkn"
Sep 20 01:52:26.031126 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.030169    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/147f9cc3-d8b1-4c83-9a97-4b29246cd454-host-run-ovn-kubernetes\") pod \"ovnkube-node-p2wkn\" (UID: \"147f9cc3-d8b1-4c83-9a97-4b29246cd454\") " pod="openshift-ovn-kubernetes/ovnkube-node-p2wkn"
Sep 20 01:52:26.031126 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.030197    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d0aee250-9aee-4473-ae29-83ffdbaeb920-registration-dir\") pod \"aws-ebs-csi-driver-node-27wj7\" (UID: \"d0aee250-9aee-4473-ae29-83ffdbaeb920\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-27wj7"
Sep 20 01:52:26.031126 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.030240    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/92615b78-f30d-4687-a822-6261a2d9ae55-etc-sysctl-d\") pod \"tuned-m4zgq\" (UID: \"92615b78-f30d-4687-a822-6261a2d9ae55\") " pod="openshift-cluster-node-tuning-operator/tuned-m4zgq"
Sep 20 01:52:26.031126 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.030277    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/147f9cc3-d8b1-4c83-9a97-4b29246cd454-etc-openvswitch\") pod \"ovnkube-node-p2wkn\" (UID: \"147f9cc3-d8b1-4c83-9a97-4b29246cd454\") " pod="openshift-ovn-kubernetes/ovnkube-node-p2wkn"
Sep 20 01:52:26.031126 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.030304    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/147f9cc3-d8b1-4c83-9a97-4b29246cd454-run-ovn\") pod \"ovnkube-node-p2wkn\" (UID: \"147f9cc3-d8b1-4c83-9a97-4b29246cd454\") " pod="openshift-ovn-kubernetes/ovnkube-node-p2wkn"
Sep 20 01:52:26.031836 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.030342    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/147f9cc3-d8b1-4c83-9a97-4b29246cd454-node-log\") pod \"ovnkube-node-p2wkn\" (UID: \"147f9cc3-d8b1-4c83-9a97-4b29246cd454\") " pod="openshift-ovn-kubernetes/ovnkube-node-p2wkn"
Sep 20 01:52:26.031836 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.030383    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d08af64d-ba39-4d61-9b95-c6febd32cbbb-agent-certs\") pod \"konnectivity-agent-ftlkq\" (UID: \"d08af64d-ba39-4d61-9b95-c6febd32cbbb\") " pod="kube-system/konnectivity-agent-ftlkq"
Sep 20 01:52:26.031836 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.030723    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/92615b78-f30d-4687-a822-6261a2d9ae55-etc-systemd\") pod \"tuned-m4zgq\" (UID: \"92615b78-f30d-4687-a822-6261a2d9ae55\") " pod="openshift-cluster-node-tuning-operator/tuned-m4zgq"
Sep 20 01:52:26.031836 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.030750    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/92615b78-f30d-4687-a822-6261a2d9ae55-run\") pod \"tuned-m4zgq\" (UID: \"92615b78-f30d-4687-a822-6261a2d9ae55\") " pod="openshift-cluster-node-tuning-operator/tuned-m4zgq"
Sep 20 01:52:26.031836 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.030770    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/92615b78-f30d-4687-a822-6261a2d9ae55-host\") pod \"tuned-m4zgq\" (UID: \"92615b78-f30d-4687-a822-6261a2d9ae55\") " pod="openshift-cluster-node-tuning-operator/tuned-m4zgq"
Sep 20 01:52:26.031836 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.030790    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h9rmf\" (UniqueName: \"kubernetes.io/projected/92615b78-f30d-4687-a822-6261a2d9ae55-kube-api-access-h9rmf\") pod \"tuned-m4zgq\" (UID: \"92615b78-f30d-4687-a822-6261a2d9ae55\") " pod="openshift-cluster-node-tuning-operator/tuned-m4zgq"
Sep 20 01:52:26.031836 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.030811    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4fa3e7e2-7940-4d7d-af23-63ff07edba43-host\") pod \"node-ca-v6d2k\" (UID: \"4fa3e7e2-7940-4d7d-af23-63ff07edba43\") " pod="openshift-image-registry/node-ca-v6d2k"
Sep 20 01:52:26.031836 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.030830    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f9800646-e98f-45d8-aa4e-5eb53056f5e8-multus-daemon-config\") pod \"multus-k8rn8\" (UID: \"f9800646-e98f-45d8-aa4e-5eb53056f5e8\") " pod="openshift-multus/multus-k8rn8"
Sep 20 01:52:26.031836 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.030849    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/147f9cc3-d8b1-4c83-9a97-4b29246cd454-ovnkube-script-lib\") pod \"ovnkube-node-p2wkn\" (UID: \"147f9cc3-d8b1-4c83-9a97-4b29246cd454\") " pod="openshift-ovn-kubernetes/ovnkube-node-p2wkn"
Sep 20 01:52:26.031836 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.030869    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/92615b78-f30d-4687-a822-6261a2d9ae55-var-lib-kubelet\") pod \"tuned-m4zgq\" (UID: \"92615b78-f30d-4687-a822-6261a2d9ae55\") " pod="openshift-cluster-node-tuning-operator/tuned-m4zgq"
Sep 20 01:52:26.033259 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.033217    2584 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-19 01:47:25 +0000 UTC" deadline="2028-03-12 22:09:14.24121251 +0000 UTC"
Sep 20 01:52:26.033259 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.033258    2584 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12956h16m48.207957189s"
Sep 20 01:52:26.035845 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.035827    2584 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 20 01:52:26.059304 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.059286    2584 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-kt5kr"
Sep 20 01:52:26.068280 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.068263    2584 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-kt5kr"
Sep 20 01:52:26.122113 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.121957    2584 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 20 01:52:26.131083 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.131059    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/96a97560-011d-4806-ae10-4d53880eb9ff-tuning-conf-dir\") pod \"multus-additional-cni-plugins-f94fv\" (UID: \"96a97560-011d-4806-ae10-4d53880eb9ff\") " pod="openshift-multus/multus-additional-cni-plugins-f94fv"
Sep 20 01:52:26.131189 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.131100    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f9800646-e98f-45d8-aa4e-5eb53056f5e8-system-cni-dir\") pod \"multus-k8rn8\" (UID: \"f9800646-e98f-45d8-aa4e-5eb53056f5e8\") " pod="openshift-multus/multus-k8rn8"
Sep 20 01:52:26.131189 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.131128    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/92615b78-f30d-4687-a822-6261a2d9ae55-etc-sysctl-conf\") pod \"tuned-m4zgq\" (UID: \"92615b78-f30d-4687-a822-6261a2d9ae55\") " pod="openshift-cluster-node-tuning-operator/tuned-m4zgq"
Sep 20 01:52:26.131189 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.131151    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m5r9c\" (UniqueName: \"kubernetes.io/projected/4fa3e7e2-7940-4d7d-af23-63ff07edba43-kube-api-access-m5r9c\") pod \"node-ca-v6d2k\" (UID: \"4fa3e7e2-7940-4d7d-af23-63ff07edba43\") " pod="openshift-image-registry/node-ca-v6d2k"
Sep 20 01:52:26.131189 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.131172    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/92615b78-f30d-4687-a822-6261a2d9ae55-tmp\") pod \"tuned-m4zgq\" (UID: \"92615b78-f30d-4687-a822-6261a2d9ae55\") " pod="openshift-cluster-node-tuning-operator/tuned-m4zgq"
Sep 20 01:52:26.131374 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.131194    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/96a97560-011d-4806-ae10-4d53880eb9ff-system-cni-dir\") pod \"multus-additional-cni-plugins-f94fv\" (UID: \"96a97560-011d-4806-ae10-4d53880eb9ff\") " pod="openshift-multus/multus-additional-cni-plugins-f94fv"
Sep 20 01:52:26.131374 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.131218    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c9c657b9-b508-4bb7-99f3-aed2502cfba4-metrics-certs\") pod \"network-metrics-daemon-n4dk7\" (UID: \"c9c657b9-b508-4bb7-99f3-aed2502cfba4\") " pod="openshift-multus/network-metrics-daemon-n4dk7"
Sep 20 01:52:26.131374 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.131241    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d94badda-e0c0-4b65-a95f-64f524896542-iptables-alerter-script\") pod \"iptables-alerter-chcxk\" (UID: \"d94badda-e0c0-4b65-a95f-64f524896542\") " pod="openshift-network-operator/iptables-alerter-chcxk"
Sep 20 01:52:26.131374 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.131278    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/96a97560-011d-4806-ae10-4d53880eb9ff-tuning-conf-dir\") pod \"multus-additional-cni-plugins-f94fv\" (UID: \"96a97560-011d-4806-ae10-4d53880eb9ff\") " pod="openshift-multus/multus-additional-cni-plugins-f94fv"
Sep 20 01:52:26.131374 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.131311    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/92615b78-f30d-4687-a822-6261a2d9ae55-etc-sysctl-conf\") pod \"tuned-m4zgq\" (UID: \"92615b78-f30d-4687-a822-6261a2d9ae55\") " pod="openshift-cluster-node-tuning-operator/tuned-m4zgq"
Sep 20 01:52:26.131374 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.131339    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/96a97560-011d-4806-ae10-4d53880eb9ff-system-cni-dir\") pod \"multus-additional-cni-plugins-f94fv\" (UID: \"96a97560-011d-4806-ae10-4d53880eb9ff\") " pod="openshift-multus/multus-additional-cni-plugins-f94fv"
Sep 20 01:52:26.131688 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:26.131419    2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 20 01:52:26.131688 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.131422    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f9800646-e98f-45d8-aa4e-5eb53056f5e8-system-cni-dir\") pod \"multus-k8rn8\" (UID: \"f9800646-e98f-45d8-aa4e-5eb53056f5e8\") " pod="openshift-multus/multus-k8rn8"
Sep 20 01:52:26.131688 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:26.131501    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c9c657b9-b508-4bb7-99f3-aed2502cfba4-metrics-certs podName:c9c657b9-b508-4bb7-99f3-aed2502cfba4 nodeName:}" failed. No retries permitted until 2026-09-20 01:52:26.631464811 +0000 UTC m=+3.374144017 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c9c657b9-b508-4bb7-99f3-aed2502cfba4-metrics-certs") pod "network-metrics-daemon-n4dk7" (UID: "c9c657b9-b508-4bb7-99f3-aed2502cfba4") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 20 01:52:26.131853 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.131707    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/147f9cc3-d8b1-4c83-9a97-4b29246cd454-run-openvswitch\") pod \"ovnkube-node-p2wkn\" (UID: \"147f9cc3-d8b1-4c83-9a97-4b29246cd454\") " pod="openshift-ovn-kubernetes/ovnkube-node-p2wkn"
Sep 20 01:52:26.131853 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.131738    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/147f9cc3-d8b1-4c83-9a97-4b29246cd454-host-run-ovn-kubernetes\") pod \"ovnkube-node-p2wkn\" (UID: \"147f9cc3-d8b1-4c83-9a97-4b29246cd454\") " pod="openshift-ovn-kubernetes/ovnkube-node-p2wkn"
Sep 20 01:52:26.131853 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.131763    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d0aee250-9aee-4473-ae29-83ffdbaeb920-registration-dir\") pod \"aws-ebs-csi-driver-node-27wj7\" (UID: \"d0aee250-9aee-4473-ae29-83ffdbaeb920\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-27wj7"
Sep 20 01:52:26.131853 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.131788    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/92615b78-f30d-4687-a822-6261a2d9ae55-etc-sysctl-d\") pod \"tuned-m4zgq\" (UID: \"92615b78-f30d-4687-a822-6261a2d9ae55\") " pod="openshift-cluster-node-tuning-operator/tuned-m4zgq"
Sep 20 01:52:26.131853 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.131808    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/147f9cc3-d8b1-4c83-9a97-4b29246cd454-run-openvswitch\") pod \"ovnkube-node-p2wkn\" (UID: \"147f9cc3-d8b1-4c83-9a97-4b29246cd454\") " pod="openshift-ovn-kubernetes/ovnkube-node-p2wkn"
Sep 20 01:52:26.131853 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.131811    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/147f9cc3-d8b1-4c83-9a97-4b29246cd454-etc-openvswitch\") pod \"ovnkube-node-p2wkn\" (UID: \"147f9cc3-d8b1-4c83-9a97-4b29246cd454\") " pod="openshift-ovn-kubernetes/ovnkube-node-p2wkn"
Sep 20 01:52:26.131853 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.131836    2584 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 20 01:52:26.131853 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.131848    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/147f9cc3-d8b1-4c83-9a97-4b29246cd454-etc-openvswitch\") pod \"ovnkube-node-p2wkn\" (UID: \"147f9cc3-d8b1-4c83-9a97-4b29246cd454\") " pod="openshift-ovn-kubernetes/ovnkube-node-p2wkn"
Sep 20 01:52:26.132253 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.131860    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/147f9cc3-d8b1-4c83-9a97-4b29246cd454-run-ovn\") pod \"ovnkube-node-p2wkn\" (UID: \"147f9cc3-d8b1-4c83-9a97-4b29246cd454\") " pod="openshift-ovn-kubernetes/ovnkube-node-p2wkn"
Sep 20 01:52:26.132253 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.131884    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/147f9cc3-d8b1-4c83-9a97-4b29246cd454-node-log\") pod \"ovnkube-node-p2wkn\" (UID: \"147f9cc3-d8b1-4c83-9a97-4b29246cd454\") " pod="openshift-ovn-kubernetes/ovnkube-node-p2wkn"
Sep 20 01:52:26.132253 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.131906    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d08af64d-ba39-4d61-9b95-c6febd32cbbb-agent-certs\") pod \"konnectivity-agent-ftlkq\" (UID: \"d08af64d-ba39-4d61-9b95-c6febd32cbbb\") " pod="kube-system/konnectivity-agent-ftlkq"
Sep 20 01:52:26.132253 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.131915    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/147f9cc3-d8b1-4c83-9a97-4b29246cd454-host-run-ovn-kubernetes\") pod \"ovnkube-node-p2wkn\" (UID: \"147f9cc3-d8b1-4c83-9a97-4b29246cd454\") " pod="openshift-ovn-kubernetes/ovnkube-node-p2wkn"
Sep 20 01:52:26.132253 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.131927    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/92615b78-f30d-4687-a822-6261a2d9ae55-etc-systemd\") pod \"tuned-m4zgq\" (UID: \"92615b78-f30d-4687-a822-6261a2d9ae55\") " pod="openshift-cluster-node-tuning-operator/tuned-m4zgq"
Sep 20 01:52:26.132253 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.131947    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/92615b78-f30d-4687-a822-6261a2d9ae55-run\") pod \"tuned-m4zgq\" (UID: \"92615b78-f30d-4687-a822-6261a2d9ae55\") " pod="openshift-cluster-node-tuning-operator/tuned-m4zgq"
Sep 20 01:52:26.132253 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.131966    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d0aee250-9aee-4473-ae29-83ffdbaeb920-registration-dir\") pod \"aws-ebs-csi-driver-node-27wj7\" (UID: \"d0aee250-9aee-4473-ae29-83ffdbaeb920\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-27wj7"
Sep 20 01:52:26.132253 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.131968    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/92615b78-f30d-4687-a822-6261a2d9ae55-host\") pod \"tuned-m4zgq\" (UID: \"92615b78-f30d-4687-a822-6261a2d9ae55\") " pod="openshift-cluster-node-tuning-operator/tuned-m4zgq"
Sep 20 01:52:26.132253 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.132005    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h9rmf\" (UniqueName: \"kubernetes.io/projected/92615b78-f30d-4687-a822-6261a2d9ae55-kube-api-access-h9rmf\") pod \"tuned-m4zgq\" (UID: \"92615b78-f30d-4687-a822-6261a2d9ae55\") " pod="openshift-cluster-node-tuning-operator/tuned-m4zgq"
Sep 20 01:52:26.132253 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.132010    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/92615b78-f30d-4687-a822-6261a2d9ae55-host\") pod \"tuned-m4zgq\" (UID: \"92615b78-f30d-4687-a822-6261a2d9ae55\") " pod="openshift-cluster-node-tuning-operator/tuned-m4zgq"
Sep 20 01:52:26.132253 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.132046    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d94badda-e0c0-4b65-a95f-64f524896542-iptables-alerter-script\") pod \"iptables-alerter-chcxk\" (UID: \"d94badda-e0c0-4b65-a95f-64f524896542\") " pod="openshift-network-operator/iptables-alerter-chcxk"
Sep 20 01:52:26.132253 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.132110    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/92615b78-f30d-4687-a822-6261a2d9ae55-etc-sysctl-d\") pod \"tuned-m4zgq\" (UID: \"92615b78-f30d-4687-a822-6261a2d9ae55\") " pod="openshift-cluster-node-tuning-operator/tuned-m4zgq"
Sep 20 01:52:26.132253 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.132180    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/92615b78-f30d-4687-a822-6261a2d9ae55-run\") pod \"tuned-m4zgq\" (UID: \"92615b78-f30d-4687-a822-6261a2d9ae55\") " pod="openshift-cluster-node-tuning-operator/tuned-m4zgq"
Sep 20 01:52:26.132253 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.132224    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4fa3e7e2-7940-4d7d-af23-63ff07edba43-host\") pod \"node-ca-v6d2k\" (UID: \"4fa3e7e2-7940-4d7d-af23-63ff07edba43\") " pod="openshift-image-registry/node-ca-v6d2k"
Sep 20 01:52:26.132253 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.132235    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/92615b78-f30d-4687-a822-6261a2d9ae55-etc-systemd\") pod \"tuned-m4zgq\" (UID: \"92615b78-f30d-4687-a822-6261a2d9ae55\") " pod="openshift-cluster-node-tuning-operator/tuned-m4zgq"
Sep 20 01:52:26.132253 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.132256    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f9800646-e98f-45d8-aa4e-5eb53056f5e8-multus-daemon-config\") pod \"multus-k8rn8\" (UID: \"f9800646-e98f-45d8-aa4e-5eb53056f5e8\") " pod="openshift-multus/multus-k8rn8"
Sep 20 01:52:26.133197 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.132285    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/147f9cc3-d8b1-4c83-9a97-4b29246cd454-ovnkube-script-lib\") pod \"ovnkube-node-p2wkn\" (UID: \"147f9cc3-d8b1-4c83-9a97-4b29246cd454\") " pod="openshift-ovn-kubernetes/ovnkube-node-p2wkn"
Sep 20 01:52:26.133197 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.132308    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/92615b78-f30d-4687-a822-6261a2d9ae55-var-lib-kubelet\") pod \"tuned-m4zgq\" (UID: \"92615b78-f30d-4687-a822-6261a2d9ae55\") " pod="openshift-cluster-node-tuning-operator/tuned-m4zgq"
Sep 20 01:52:26.133197 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.132330    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f9800646-e98f-45d8-aa4e-5eb53056f5e8-host-run-netns\") pod \"multus-k8rn8\" (UID: \"f9800646-e98f-45d8-aa4e-5eb53056f5e8\") " pod="openshift-multus/multus-k8rn8"
Sep 20 01:52:26.133197 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.132351    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f9800646-e98f-45d8-aa4e-5eb53056f5e8-etc-kubernetes\") pod \"multus-k8rn8\" (UID: \"f9800646-e98f-45d8-aa4e-5eb53056f5e8\") " pod="openshift-multus/multus-k8rn8"
Sep 20 01:52:26.133197 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.132370    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/43dd43c0-ff48-4f9a-a1c5-9020836702aa-hosts-file\") pod \"node-resolver-l9ssk\" (UID: \"43dd43c0-ff48-4f9a-a1c5-9020836702aa\") " pod="openshift-dns/node-resolver-l9ssk"
Sep 20 01:52:26.133197 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.132388    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d08af64d-ba39-4d61-9b95-c6febd32cbbb-konnectivity-ca\") pod \"konnectivity-agent-ftlkq\" (UID: \"d08af64d-ba39-4d61-9b95-c6febd32cbbb\") " pod="kube-system/konnectivity-agent-ftlkq"
Sep 20 01:52:26.133197 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.132408    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/92615b78-f30d-4687-a822-6261a2d9ae55-etc-sysconfig\") pod \"tuned-m4zgq\" (UID: \"92615b78-f30d-4687-a822-6261a2d9ae55\") " pod="openshift-cluster-node-tuning-operator/tuned-m4zgq"
Sep 20 01:52:26.133197 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.132428    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f9800646-e98f-45d8-aa4e-5eb53056f5e8-multus-cni-dir\") pod \"multus-k8rn8\" (UID: \"f9800646-e98f-45d8-aa4e-5eb53056f5e8\") " pod="openshift-multus/multus-k8rn8"
Sep 20 01:52:26.133197 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.132448    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f9800646-e98f-45d8-aa4e-5eb53056f5e8-host-var-lib-cni-bin\") pod \"multus-k8rn8\" (UID: \"f9800646-e98f-45d8-aa4e-5eb53056f5e8\") " pod="openshift-multus/multus-k8rn8"
Sep 20 01:52:26.133197 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.132471    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mrb2z\" (UniqueName: \"kubernetes.io/projected/9b376ae6-8f6a-4139-ab2d-96dc32e1201f-kube-api-access-mrb2z\") pod \"network-check-target-47994\" (UID: \"9b376ae6-8f6a-4139-ab2d-96dc32e1201f\") " pod="openshift-network-diagnostics/network-check-target-47994"
Sep 20 01:52:26.133197 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.132491    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/147f9cc3-d8b1-4c83-9a97-4b29246cd454-host-cni-netd\") pod \"ovnkube-node-p2wkn\" (UID: \"147f9cc3-d8b1-4c83-9a97-4b29246cd454\") " pod="openshift-ovn-kubernetes/ovnkube-node-p2wkn"
Sep 20 01:52:26.133197 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.132510    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/147f9cc3-d8b1-4c83-9a97-4b29246cd454-ovn-node-metrics-cert\") pod \"ovnkube-node-p2wkn\" (UID: \"147f9cc3-d8b1-4c83-9a97-4b29246cd454\") " pod="openshift-ovn-kubernetes/ovnkube-node-p2wkn"
Sep 20 01:52:26.133197 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.132536    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f9800646-e98f-45d8-aa4e-5eb53056f5e8-host-run-netns\") pod \"multus-k8rn8\" (UID: \"f9800646-e98f-45d8-aa4e-5eb53056f5e8\") " pod="openshift-multus/multus-k8rn8"
Sep 20 01:52:26.133197 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.132549    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d0aee250-9aee-4473-ae29-83ffdbaeb920-sys-fs\") pod \"aws-ebs-csi-driver-node-27wj7\" (UID: \"d0aee250-9aee-4473-ae29-83ffdbaeb920\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-27wj7"
Sep 20 01:52:26.133197 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.132565    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/147f9cc3-d8b1-4c83-9a97-4b29246cd454-run-ovn\") pod \"ovnkube-node-p2wkn\" (UID: \"147f9cc3-d8b1-4c83-9a97-4b29246cd454\") " pod="openshift-ovn-kubernetes/ovnkube-node-p2wkn"
Sep 20 01:52:26.133197 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.132569    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f9800646-e98f-45d8-aa4e-5eb53056f5e8-multus-socket-dir-parent\") pod \"multus-k8rn8\" (UID: \"f9800646-e98f-45d8-aa4e-5eb53056f5e8\") " pod="openshift-multus/multus-k8rn8"
Sep 20 01:52:26.133197 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.132604    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f9800646-e98f-45d8-aa4e-5eb53056f5e8-multus-conf-dir\") pod \"multus-k8rn8\" (UID: \"f9800646-e98f-45d8-aa4e-5eb53056f5e8\") " pod="openshift-multus/multus-k8rn8"
Sep 20 01:52:26.133197 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.132632    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jjdvt\" (UniqueName: \"kubernetes.io/projected/c9c657b9-b508-4bb7-99f3-aed2502cfba4-kube-api-access-jjdvt\") pod \"network-metrics-daemon-n4dk7\" (UID: \"c9c657b9-b508-4bb7-99f3-aed2502cfba4\") " pod="openshift-multus/network-metrics-daemon-n4dk7"
Sep 20 01:52:26.134049 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.132656    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/147f9cc3-d8b1-4c83-9a97-4b29246cd454-host-cni-netd\") pod \"ovnkube-node-p2wkn\" (UID: \"147f9cc3-d8b1-4c83-9a97-4b29246cd454\") " pod="openshift-ovn-kubernetes/ovnkube-node-p2wkn"
Sep 20 01:52:26.134049 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.132506    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f9800646-e98f-45d8-aa4e-5eb53056f5e8-etc-kubernetes\") pod \"multus-k8rn8\" (UID: \"f9800646-e98f-45d8-aa4e-5eb53056f5e8\") " pod="openshift-multus/multus-k8rn8"
Sep 20 01:52:26.134049 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.132672    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f9800646-e98f-45d8-aa4e-5eb53056f5e8-multus-socket-dir-parent\") pod \"multus-k8rn8\" (UID: \"f9800646-e98f-45d8-aa4e-5eb53056f5e8\") " pod="openshift-multus/multus-k8rn8"
Sep 20 01:52:26.134049 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.132660    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/147f9cc3-d8b1-4c83-9a97-4b29246cd454-log-socket\") pod \"ovnkube-node-p2wkn\" (UID: \"147f9cc3-d8b1-4c83-9a97-4b29246cd454\") " pod="openshift-ovn-kubernetes/ovnkube-node-p2wkn"
Sep 20 01:52:26.134049 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.132693    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/147f9cc3-d8b1-4c83-9a97-4b29246cd454-log-socket\") pod \"ovnkube-node-p2wkn\" (UID: \"147f9cc3-d8b1-4c83-9a97-4b29246cd454\") " pod="openshift-ovn-kubernetes/ovnkube-node-p2wkn"
Sep 20 01:52:26.134049 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.132711    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/92615b78-f30d-4687-a822-6261a2d9ae55-etc-kubernetes\") pod \"tuned-m4zgq\" (UID: \"92615b78-f30d-4687-a822-6261a2d9ae55\") " pod="openshift-cluster-node-tuning-operator/tuned-m4zgq"
Sep 20 01:52:26.134049 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.132736    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/147f9cc3-d8b1-4c83-9a97-4b29246cd454-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-p2wkn\" (UID: \"147f9cc3-d8b1-4c83-9a97-4b29246cd454\") " pod="openshift-ovn-kubernetes/ovnkube-node-p2wkn"
Sep 20 01:52:26.134049 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.132744    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/92615b78-f30d-4687-a822-6261a2d9ae55-var-lib-kubelet\") pod \"tuned-m4zgq\" (UID: \"92615b78-f30d-4687-a822-6261a2d9ae55\") " pod="openshift-cluster-node-tuning-operator/tuned-m4zgq"
Sep 20 01:52:26.134049 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.132761    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/147f9cc3-d8b1-4c83-9a97-4b29246cd454-ovnkube-config\") pod \"ovnkube-node-p2wkn\" (UID: \"147f9cc3-d8b1-4c83-9a97-4b29246cd454\") " pod="openshift-ovn-kubernetes/ovnkube-node-p2wkn"
Sep 20 01:52:26.134049 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.132778    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f9800646-e98f-45d8-aa4e-5eb53056f5e8-host-var-lib-cni-bin\") pod \"multus-k8rn8\" (UID: \"f9800646-e98f-45d8-aa4e-5eb53056f5e8\") " pod="openshift-multus/multus-k8rn8"
Sep 20 01:52:26.134049 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.132786    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lxxxz\" (UniqueName: \"kubernetes.io/projected/147f9cc3-d8b1-4c83-9a97-4b29246cd454-kube-api-access-lxxxz\") pod \"ovnkube-node-p2wkn\" (UID: \"147f9cc3-d8b1-4c83-9a97-4b29246cd454\") " pod="openshift-ovn-kubernetes/ovnkube-node-p2wkn"
Sep 20 01:52:26.134049 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.132813    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/96a97560-011d-4806-ae10-4d53880eb9ff-cnibin\") pod \"multus-additional-cni-plugins-f94fv\" (UID: \"96a97560-011d-4806-ae10-4d53880eb9ff\") " pod="openshift-multus/multus-additional-cni-plugins-f94fv"
Sep 20 01:52:26.134049 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.132820    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f9800646-e98f-45d8-aa4e-5eb53056f5e8-multus-conf-dir\") pod \"multus-k8rn8\" (UID: \"f9800646-e98f-45d8-aa4e-5eb53056f5e8\") " pod="openshift-multus/multus-k8rn8"
Sep 20 01:52:26.134049 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.132836    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f9800646-e98f-45d8-aa4e-5eb53056f5e8-host-run-multus-certs\") pod \"multus-k8rn8\" (UID: \"f9800646-e98f-45d8-aa4e-5eb53056f5e8\") " pod="openshift-multus/multus-k8rn8"
Sep 20 01:52:26.134049 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.132857    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/147f9cc3-d8b1-4c83-9a97-4b29246cd454-host-kubelet\") pod \"ovnkube-node-p2wkn\" (UID: \"147f9cc3-d8b1-4c83-9a97-4b29246cd454\") " pod="openshift-ovn-kubernetes/ovnkube-node-p2wkn"
Sep 20 01:52:26.134049 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.132879    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/147f9cc3-d8b1-4c83-9a97-4b29246cd454-systemd-units\") pod \"ovnkube-node-p2wkn\" (UID: \"147f9cc3-d8b1-4c83-9a97-4b29246cd454\") " pod="openshift-ovn-kubernetes/ovnkube-node-p2wkn"
Sep 20 01:52:26.134049 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.132899    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/147f9cc3-d8b1-4c83-9a97-4b29246cd454-var-lib-openvswitch\") pod \"ovnkube-node-p2wkn\" (UID: \"147f9cc3-d8b1-4c83-9a97-4b29246cd454\") " pod="openshift-ovn-kubernetes/ovnkube-node-p2wkn"
Sep 20 01:52:26.134829 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.132942    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/147f9cc3-d8b1-4c83-9a97-4b29246cd454-var-lib-openvswitch\") pod \"ovnkube-node-p2wkn\" (UID: \"147f9cc3-d8b1-4c83-9a97-4b29246cd454\") " pod="openshift-ovn-kubernetes/ovnkube-node-p2wkn"
Sep 20 01:52:26.134829 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.132950    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f9800646-e98f-45d8-aa4e-5eb53056f5e8-multus-cni-dir\") pod \"multus-k8rn8\" (UID: \"f9800646-e98f-45d8-aa4e-5eb53056f5e8\") " pod="openshift-multus/multus-k8rn8"
Sep 20 01:52:26.134829 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.132985    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/43dd43c0-ff48-4f9a-a1c5-9020836702aa-hosts-file\") pod \"node-resolver-l9ssk\" (UID: \"43dd43c0-ff48-4f9a-a1c5-9020836702aa\") " pod="openshift-dns/node-resolver-l9ssk"
Sep 20 01:52:26.134829 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.132979    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4fa3e7e2-7940-4d7d-af23-63ff07edba43-serviceca\") pod \"node-ca-v6d2k\" (UID: \"4fa3e7e2-7940-4d7d-af23-63ff07edba43\") " pod="openshift-image-registry/node-ca-v6d2k"
Sep 20 01:52:26.134829 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.132635    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/92615b78-f30d-4687-a822-6261a2d9ae55-etc-sysconfig\") pod \"tuned-m4zgq\" (UID: \"92615b78-f30d-4687-a822-6261a2d9ae55\") " pod="openshift-cluster-node-tuning-operator/tuned-m4zgq"
Sep 20 01:52:26.134829 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.133021    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nm94k\" (UniqueName: \"kubernetes.io/projected/43dd43c0-ff48-4f9a-a1c5-9020836702aa-kube-api-access-nm94k\") pod \"node-resolver-l9ssk\" (UID: \"43dd43c0-ff48-4f9a-a1c5-9020836702aa\") " pod="openshift-dns/node-resolver-l9ssk"
Sep 20 01:52:26.134829 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.133046    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d0aee250-9aee-4473-ae29-83ffdbaeb920-etc-selinux\") pod \"aws-ebs-csi-driver-node-27wj7\" (UID: \"d0aee250-9aee-4473-ae29-83ffdbaeb920\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-27wj7"
Sep 20 01:52:26.134829 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.133050    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/147f9cc3-d8b1-4c83-9a97-4b29246cd454-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-p2wkn\" (UID: \"147f9cc3-d8b1-4c83-9a97-4b29246cd454\") " pod="openshift-ovn-kubernetes/ovnkube-node-p2wkn"
Sep 20 01:52:26.134829 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.132712    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d0aee250-9aee-4473-ae29-83ffdbaeb920-sys-fs\") pod \"aws-ebs-csi-driver-node-27wj7\" (UID: \"d0aee250-9aee-4473-ae29-83ffdbaeb920\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-27wj7"
Sep 20 01:52:26.134829 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.133060    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/147f9cc3-d8b1-4c83-9a97-4b29246cd454-node-log\") pod \"ovnkube-node-p2wkn\" (UID: \"147f9cc3-d8b1-4c83-9a97-4b29246cd454\") " pod="openshift-ovn-kubernetes/ovnkube-node-p2wkn"
Sep 20 01:52:26.134829 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.133067    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f9800646-e98f-45d8-aa4e-5eb53056f5e8-multus-daemon-config\") pod \"multus-k8rn8\" (UID: \"f9800646-e98f-45d8-aa4e-5eb53056f5e8\") " pod="openshift-multus/multus-k8rn8"
Sep 20 01:52:26.134829 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.133025    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/92615b78-f30d-4687-a822-6261a2d9ae55-etc-kubernetes\") pod \"tuned-m4zgq\" (UID: \"92615b78-f30d-4687-a822-6261a2d9ae55\") " pod="openshift-cluster-node-tuning-operator/tuned-m4zgq"
Sep 20 01:52:26.134829 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.133076    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/92615b78-f30d-4687-a822-6261a2d9ae55-etc-tuned\") pod \"tuned-m4zgq\" (UID: \"92615b78-f30d-4687-a822-6261a2d9ae55\") " pod="openshift-cluster-node-tuning-operator/tuned-m4zgq"
Sep 20 01:52:26.134829 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.133114    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d94badda-e0c0-4b65-a95f-64f524896542-host-slash\") pod \"iptables-alerter-chcxk\" (UID: \"d94badda-e0c0-4b65-a95f-64f524896542\") " pod="openshift-network-operator/iptables-alerter-chcxk"
Sep 20 01:52:26.134829 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.133142    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/05a98348-758f-4a4a-b150-35bbdeb5b359-original-pull-secret\") pod \"global-pull-secret-syncer-llv6h\" (UID: \"05a98348-758f-4a4a-b150-35bbdeb5b359\") " pod="kube-system/global-pull-secret-syncer-llv6h"
Sep 20 01:52:26.134829 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.133168    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d0aee250-9aee-4473-ae29-83ffdbaeb920-device-dir\") pod \"aws-ebs-csi-driver-node-27wj7\" (UID: \"d0aee250-9aee-4473-ae29-83ffdbaeb920\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-27wj7"
Sep 20 01:52:26.134829 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.133192    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/92615b78-f30d-4687-a822-6261a2d9ae55-sys\") pod \"tuned-m4zgq\" (UID: \"92615b78-f30d-4687-a822-6261a2d9ae55\") " pod="openshift-cluster-node-tuning-operator/tuned-m4zgq"
Sep 20 01:52:26.135629 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.133213    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/96a97560-011d-4806-ae10-4d53880eb9ff-cnibin\") pod \"multus-additional-cni-plugins-f94fv\" (UID: \"96a97560-011d-4806-ae10-4d53880eb9ff\") " pod="openshift-multus/multus-additional-cni-plugins-f94fv"
Sep 20 01:52:26.135629 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.133217    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f9800646-e98f-45d8-aa4e-5eb53056f5e8-cnibin\") pod \"multus-k8rn8\" (UID: \"f9800646-e98f-45d8-aa4e-5eb53056f5e8\") " pod="openshift-multus/multus-k8rn8"
Sep 20 01:52:26.135629 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.133252    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tsj66\" (UniqueName: \"kubernetes.io/projected/96a97560-011d-4806-ae10-4d53880eb9ff-kube-api-access-tsj66\") pod \"multus-additional-cni-plugins-f94fv\" (UID: \"96a97560-011d-4806-ae10-4d53880eb9ff\") " pod="openshift-multus/multus-additional-cni-plugins-f94fv"
Sep 20 01:52:26.135629 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.133260    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f9800646-e98f-45d8-aa4e-5eb53056f5e8-cnibin\") pod \"multus-k8rn8\" (UID: \"f9800646-e98f-45d8-aa4e-5eb53056f5e8\") " pod="openshift-multus/multus-k8rn8"
Sep 20 01:52:26.135629 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.133276    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/92615b78-f30d-4687-a822-6261a2d9ae55-lib-modules\") pod \"tuned-m4zgq\" (UID: \"92615b78-f30d-4687-a822-6261a2d9ae55\") " pod="openshift-cluster-node-tuning-operator/tuned-m4zgq"
Sep 20 01:52:26.135629 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.133305    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f9800646-e98f-45d8-aa4e-5eb53056f5e8-host-var-lib-cni-multus\") pod \"multus-k8rn8\" (UID: \"f9800646-e98f-45d8-aa4e-5eb53056f5e8\") " pod="openshift-multus/multus-k8rn8"
Sep 20 01:52:26.135629 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.133311    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/147f9cc3-d8b1-4c83-9a97-4b29246cd454-systemd-units\") pod \"ovnkube-node-p2wkn\" (UID: \"147f9cc3-d8b1-4c83-9a97-4b29246cd454\") " pod="openshift-ovn-kubernetes/ovnkube-node-p2wkn"
Sep 20 01:52:26.135629 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.133339    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f9800646-e98f-45d8-aa4e-5eb53056f5e8-host-var-lib-kubelet\") pod \"multus-k8rn8\" (UID: \"f9800646-e98f-45d8-aa4e-5eb53056f5e8\") " pod="openshift-multus/multus-k8rn8"
Sep 20 01:52:26.135629 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.133344    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f9800646-e98f-45d8-aa4e-5eb53056f5e8-host-var-lib-cni-multus\") pod \"multus-k8rn8\" (UID: \"f9800646-e98f-45d8-aa4e-5eb53056f5e8\") " pod="openshift-multus/multus-k8rn8"
Sep 20 01:52:26.135629 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.133365    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f9800646-e98f-45d8-aa4e-5eb53056f5e8-hostroot\") pod \"multus-k8rn8\" (UID: \"f9800646-e98f-45d8-aa4e-5eb53056f5e8\") " pod="openshift-multus/multus-k8rn8"
Sep 20 01:52:26.135629 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.133381    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d94badda-e0c0-4b65-a95f-64f524896542-host-slash\") pod \"iptables-alerter-chcxk\" (UID: \"d94badda-e0c0-4b65-a95f-64f524896542\") " pod="openshift-network-operator/iptables-alerter-chcxk"
Sep 20 01:52:26.135629 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.133391    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wjsfw\" (UniqueName: \"kubernetes.io/projected/d94badda-e0c0-4b65-a95f-64f524896542-kube-api-access-wjsfw\") pod \"iptables-alerter-chcxk\" (UID: \"d94badda-e0c0-4b65-a95f-64f524896542\") " pod="openshift-network-operator/iptables-alerter-chcxk"
Sep 20 01:52:26.135629 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.133417    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/147f9cc3-d8b1-4c83-9a97-4b29246cd454-host-slash\") pod \"ovnkube-node-p2wkn\" (UID: \"147f9cc3-d8b1-4c83-9a97-4b29246cd454\") " pod="openshift-ovn-kubernetes/ovnkube-node-p2wkn"
Sep 20 01:52:26.135629 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.133441    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/147f9cc3-d8b1-4c83-9a97-4b29246cd454-run-systemd\") pod \"ovnkube-node-p2wkn\" (UID: \"147f9cc3-d8b1-4c83-9a97-4b29246cd454\") " pod="openshift-ovn-kubernetes/ovnkube-node-p2wkn"
Sep 20 01:52:26.135629 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:26.133447    2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 20 01:52:26.135629 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.133465    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/96a97560-011d-4806-ae10-4d53880eb9ff-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-f94fv\" (UID: \"96a97560-011d-4806-ae10-4d53880eb9ff\") " pod="openshift-multus/multus-additional-cni-plugins-f94fv"
Sep 20 01:52:26.135629 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:26.133501    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/05a98348-758f-4a4a-b150-35bbdeb5b359-original-pull-secret podName:05a98348-758f-4a4a-b150-35bbdeb5b359 nodeName:}" failed. No retries permitted until 2026-09-20 01:52:26.633485524 +0000 UTC m=+3.376164721 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/05a98348-758f-4a4a-b150-35bbdeb5b359-original-pull-secret") pod "global-pull-secret-syncer-llv6h" (UID: "05a98348-758f-4a4a-b150-35bbdeb5b359") : object "kube-system"/"original-pull-secret" not registered
Sep 20 01:52:26.136346 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.133540    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/96a97560-011d-4806-ae10-4d53880eb9ff-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-f94fv\" (UID: \"96a97560-011d-4806-ae10-4d53880eb9ff\") " pod="openshift-multus/multus-additional-cni-plugins-f94fv"
Sep 20 01:52:26.136346 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.133569    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/05a98348-758f-4a4a-b150-35bbdeb5b359-dbus\") pod \"global-pull-secret-syncer-llv6h\" (UID: \"05a98348-758f-4a4a-b150-35bbdeb5b359\") " pod="kube-system/global-pull-secret-syncer-llv6h"
Sep 20 01:52:26.136346 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.133591    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d0aee250-9aee-4473-ae29-83ffdbaeb920-socket-dir\") pod \"aws-ebs-csi-driver-node-27wj7\" (UID: \"d0aee250-9aee-4473-ae29-83ffdbaeb920\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-27wj7"
Sep 20 01:52:26.136346 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.133636    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/92615b78-f30d-4687-a822-6261a2d9ae55-etc-modprobe-d\") pod \"tuned-m4zgq\" (UID: \"92615b78-f30d-4687-a822-6261a2d9ae55\") " pod="openshift-cluster-node-tuning-operator/tuned-m4zgq"
Sep 20 01:52:26.136346 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.133662    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f9800646-e98f-45d8-aa4e-5eb53056f5e8-host-run-k8s-cni-cncf-io\") pod \"multus-k8rn8\" (UID: \"f9800646-e98f-45d8-aa4e-5eb53056f5e8\") " pod="openshift-multus/multus-k8rn8"
Sep 20 01:52:26.136346 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.133684    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f9800646-e98f-45d8-aa4e-5eb53056f5e8-cni-binary-copy\") pod \"multus-k8rn8\" (UID: \"f9800646-e98f-45d8-aa4e-5eb53056f5e8\") " pod="openshift-multus/multus-k8rn8"
Sep 20 01:52:26.136346 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.133709    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/147f9cc3-d8b1-4c83-9a97-4b29246cd454-host-run-netns\") pod \"ovnkube-node-p2wkn\" (UID: \"147f9cc3-d8b1-4c83-9a97-4b29246cd454\") " pod="openshift-ovn-kubernetes/ovnkube-node-p2wkn"
Sep 20 01:52:26.136346 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.133735    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/147f9cc3-d8b1-4c83-9a97-4b29246cd454-host-cni-bin\") pod \"ovnkube-node-p2wkn\" (UID: \"147f9cc3-d8b1-4c83-9a97-4b29246cd454\") " pod="openshift-ovn-kubernetes/ovnkube-node-p2wkn"
Sep 20 01:52:26.136346 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.133760    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/147f9cc3-d8b1-4c83-9a97-4b29246cd454-env-overrides\") pod \"ovnkube-node-p2wkn\" (UID: \"147f9cc3-d8b1-4c83-9a97-4b29246cd454\") " pod="openshift-ovn-kubernetes/ovnkube-node-p2wkn"
Sep 20 01:52:26.136346 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.133786    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/05a98348-758f-4a4a-b150-35bbdeb5b359-kubelet-config\") pod \"global-pull-secret-syncer-llv6h\" (UID: \"05a98348-758f-4a4a-b150-35bbdeb5b359\") " pod="kube-system/global-pull-secret-syncer-llv6h"
Sep 20 01:52:26.136346 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.133809    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/43dd43c0-ff48-4f9a-a1c5-9020836702aa-tmp-dir\") pod \"node-resolver-l9ssk\" (UID: \"43dd43c0-ff48-4f9a-a1c5-9020836702aa\") " pod="openshift-dns/node-resolver-l9ssk"
Sep 20 01:52:26.136346 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.133833    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d0aee250-9aee-4473-ae29-83ffdbaeb920-kubelet-dir\") pod \"aws-ebs-csi-driver-node-27wj7\" (UID: \"d0aee250-9aee-4473-ae29-83ffdbaeb920\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-27wj7"
Sep 20 01:52:26.136346 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.133859    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ngtsk\" (UniqueName: \"kubernetes.io/projected/d0aee250-9aee-4473-ae29-83ffdbaeb920-kube-api-access-ngtsk\") pod \"aws-ebs-csi-driver-node-27wj7\" (UID: \"d0aee250-9aee-4473-ae29-83ffdbaeb920\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-27wj7"
Sep 20 01:52:26.136346 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.133872    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/147f9cc3-d8b1-4c83-9a97-4b29246cd454-ovnkube-config\") pod \"ovnkube-node-p2wkn\" (UID: \"147f9cc3-d8b1-4c83-9a97-4b29246cd454\") " pod="openshift-ovn-kubernetes/ovnkube-node-p2wkn"
Sep 20 01:52:26.136346 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.133884    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6kt7d\" (UniqueName: \"kubernetes.io/projected/f9800646-e98f-45d8-aa4e-5eb53056f5e8-kube-api-access-6kt7d\") pod \"multus-k8rn8\" (UID: \"f9800646-e98f-45d8-aa4e-5eb53056f5e8\") " pod="openshift-multus/multus-k8rn8"
Sep 20 01:52:26.136346 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.133919    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/96a97560-011d-4806-ae10-4d53880eb9ff-cni-binary-copy\") pod \"multus-additional-cni-plugins-f94fv\" (UID: \"96a97560-011d-4806-ae10-4d53880eb9ff\") " pod="openshift-multus/multus-additional-cni-plugins-f94fv"
Sep 20 01:52:26.136346 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.133960    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d0aee250-9aee-4473-ae29-83ffdbaeb920-etc-selinux\") pod \"aws-ebs-csi-driver-node-27wj7\" (UID: \"d0aee250-9aee-4473-ae29-83ffdbaeb920\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-27wj7"
Sep 20 01:52:26.137088 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.134159    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f9800646-e98f-45d8-aa4e-5eb53056f5e8-hostroot\") pod \"multus-k8rn8\" (UID: \"f9800646-e98f-45d8-aa4e-5eb53056f5e8\") " pod="openshift-multus/multus-k8rn8"
Sep 20 01:52:26.137088 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.134210    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f9800646-e98f-45d8-aa4e-5eb53056f5e8-host-run-multus-certs\") pod \"multus-k8rn8\" (UID: \"f9800646-e98f-45d8-aa4e-5eb53056f5e8\") " pod="openshift-multus/multus-k8rn8"
Sep 20 01:52:26.137088 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.132660    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4fa3e7e2-7940-4d7d-af23-63ff07edba43-host\") pod \"node-ca-v6d2k\" (UID: \"4fa3e7e2-7940-4d7d-af23-63ff07edba43\") " pod="openshift-image-registry/node-ca-v6d2k"
Sep 20 01:52:26.137088 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.134262    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/147f9cc3-d8b1-4c83-9a97-4b29246cd454-host-kubelet\") pod \"ovnkube-node-p2wkn\" (UID: \"147f9cc3-d8b1-4c83-9a97-4b29246cd454\") " pod="openshift-ovn-kubernetes/ovnkube-node-p2wkn"
Sep 20 01:52:26.137088 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.134340    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/92615b78-f30d-4687-a822-6261a2d9ae55-lib-modules\") pod \"tuned-m4zgq\" (UID: \"92615b78-f30d-4687-a822-6261a2d9ae55\") " pod="openshift-cluster-node-tuning-operator/tuned-m4zgq"
Sep 20 01:52:26.137088 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.134374    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f9800646-e98f-45d8-aa4e-5eb53056f5e8-host-var-lib-kubelet\") pod \"multus-k8rn8\" (UID: \"f9800646-e98f-45d8-aa4e-5eb53056f5e8\") " pod="openshift-multus/multus-k8rn8"
Sep 20 01:52:26.137088 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.134490    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/96a97560-011d-4806-ae10-4d53880eb9ff-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-f94fv\" (UID: \"96a97560-011d-4806-ae10-4d53880eb9ff\") " pod="openshift-multus/multus-additional-cni-plugins-f94fv"
Sep 20 01:52:26.137088 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.134676    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d08af64d-ba39-4d61-9b95-c6febd32cbbb-konnectivity-ca\") pod \"konnectivity-agent-ftlkq\" (UID: \"d08af64d-ba39-4d61-9b95-c6febd32cbbb\") " pod="kube-system/konnectivity-agent-ftlkq"
Sep 20 01:52:26.137088 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.134694    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f9800646-e98f-45d8-aa4e-5eb53056f5e8-cni-binary-copy\") pod \"multus-k8rn8\" (UID: \"f9800646-e98f-45d8-aa4e-5eb53056f5e8\") " pod="openshift-multus/multus-k8rn8"
Sep 20 01:52:26.137088 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.134759    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d0aee250-9aee-4473-ae29-83ffdbaeb920-device-dir\") pod \"aws-ebs-csi-driver-node-27wj7\" (UID: \"d0aee250-9aee-4473-ae29-83ffdbaeb920\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-27wj7"
Sep 20 01:52:26.137088 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.134809    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/92615b78-f30d-4687-a822-6261a2d9ae55-sys\") pod \"tuned-m4zgq\" (UID: \"92615b78-f30d-4687-a822-6261a2d9ae55\") " pod="openshift-cluster-node-tuning-operator/tuned-m4zgq"
Sep 20 01:52:26.137088 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.134853    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/147f9cc3-d8b1-4c83-9a97-4b29246cd454-host-run-netns\") pod \"ovnkube-node-p2wkn\" (UID: \"147f9cc3-d8b1-4c83-9a97-4b29246cd454\") " pod="openshift-ovn-kubernetes/ovnkube-node-p2wkn"
Sep 20 01:52:26.137088 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.134966    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/96a97560-011d-4806-ae10-4d53880eb9ff-cni-binary-copy\") pod \"multus-additional-cni-plugins-f94fv\" (UID: \"96a97560-011d-4806-ae10-4d53880eb9ff\") " pod="openshift-multus/multus-additional-cni-plugins-f94fv"
Sep 20 01:52:26.137088 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.134953    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/147f9cc3-d8b1-4c83-9a97-4b29246cd454-host-cni-bin\") pod \"ovnkube-node-p2wkn\" (UID: \"147f9cc3-d8b1-4c83-9a97-4b29246cd454\") " pod="openshift-ovn-kubernetes/ovnkube-node-p2wkn"
Sep 20 01:52:26.137088 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.133417    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4fa3e7e2-7940-4d7d-af23-63ff07edba43-serviceca\") pod \"node-ca-v6d2k\" (UID: \"4fa3e7e2-7940-4d7d-af23-63ff07edba43\") " pod="openshift-image-registry/node-ca-v6d2k"
Sep 20 01:52:26.137088 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.135035    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/05a98348-758f-4a4a-b150-35bbdeb5b359-kubelet-config\") pod \"global-pull-secret-syncer-llv6h\" (UID: \"05a98348-758f-4a4a-b150-35bbdeb5b359\") " pod="kube-system/global-pull-secret-syncer-llv6h"
Sep 20 01:52:26.137088 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.135241    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/96a97560-011d-4806-ae10-4d53880eb9ff-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-f94fv\" (UID: \"96a97560-011d-4806-ae10-4d53880eb9ff\") " pod="openshift-multus/multus-additional-cni-plugins-f94fv"
Sep 20 01:52:26.137088 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.135296    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/43dd43c0-ff48-4f9a-a1c5-9020836702aa-tmp-dir\") pod \"node-resolver-l9ssk\" (UID: \"43dd43c0-ff48-4f9a-a1c5-9020836702aa\") " pod="openshift-dns/node-resolver-l9ssk"
Sep 20 01:52:26.137694 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.135314    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/147f9cc3-d8b1-4c83-9a97-4b29246cd454-env-overrides\") pod \"ovnkube-node-p2wkn\" (UID: \"147f9cc3-d8b1-4c83-9a97-4b29246cd454\") " pod="openshift-ovn-kubernetes/ovnkube-node-p2wkn"
Sep 20 01:52:26.137694 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.133713    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/147f9cc3-d8b1-4c83-9a97-4b29246cd454-ovnkube-script-lib\") pod \"ovnkube-node-p2wkn\" (UID: \"147f9cc3-d8b1-4c83-9a97-4b29246cd454\") " pod="openshift-ovn-kubernetes/ovnkube-node-p2wkn"
Sep 20 01:52:26.137694 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.135338    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/92615b78-f30d-4687-a822-6261a2d9ae55-etc-modprobe-d\") pod \"tuned-m4zgq\" (UID: \"92615b78-f30d-4687-a822-6261a2d9ae55\") " pod="openshift-cluster-node-tuning-operator/tuned-m4zgq"
Sep 20 01:52:26.137694 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.134980    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/147f9cc3-d8b1-4c83-9a97-4b29246cd454-host-slash\") pod \"ovnkube-node-p2wkn\" (UID: \"147f9cc3-d8b1-4c83-9a97-4b29246cd454\") " pod="openshift-ovn-kubernetes/ovnkube-node-p2wkn"
Sep 20 01:52:26.137694 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.135383    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d0aee250-9aee-4473-ae29-83ffdbaeb920-kubelet-dir\") pod \"aws-ebs-csi-driver-node-27wj7\" (UID: \"d0aee250-9aee-4473-ae29-83ffdbaeb920\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-27wj7"
Sep 20 01:52:26.137694 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.135424    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f9800646-e98f-45d8-aa4e-5eb53056f5e8-host-run-k8s-cni-cncf-io\") pod \"multus-k8rn8\" (UID: \"f9800646-e98f-45d8-aa4e-5eb53056f5e8\") " pod="openshift-multus/multus-k8rn8"
Sep 20 01:52:26.137694 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.135479    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d0aee250-9aee-4473-ae29-83ffdbaeb920-socket-dir\") pod \"aws-ebs-csi-driver-node-27wj7\" (UID: \"d0aee250-9aee-4473-ae29-83ffdbaeb920\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-27wj7"
Sep 20 01:52:26.137694 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.135486    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/05a98348-758f-4a4a-b150-35bbdeb5b359-dbus\") pod \"global-pull-secret-syncer-llv6h\" (UID: \"05a98348-758f-4a4a-b150-35bbdeb5b359\") " pod="kube-system/global-pull-secret-syncer-llv6h"
Sep 20 01:52:26.137694 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.135542    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/147f9cc3-d8b1-4c83-9a97-4b29246cd454-run-systemd\") pod \"ovnkube-node-p2wkn\" (UID: \"147f9cc3-d8b1-4c83-9a97-4b29246cd454\") " pod="openshift-ovn-kubernetes/ovnkube-node-p2wkn"
Sep 20 01:52:26.137694 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.136356    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/92615b78-f30d-4687-a822-6261a2d9ae55-tmp\") pod \"tuned-m4zgq\" (UID: \"92615b78-f30d-4687-a822-6261a2d9ae55\") " pod="openshift-cluster-node-tuning-operator/tuned-m4zgq"
Sep 20 01:52:26.137694 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.136490    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/147f9cc3-d8b1-4c83-9a97-4b29246cd454-ovn-node-metrics-cert\") pod \"ovnkube-node-p2wkn\" (UID: \"147f9cc3-d8b1-4c83-9a97-4b29246cd454\") " pod="openshift-ovn-kubernetes/ovnkube-node-p2wkn"
Sep 20 01:52:26.137694 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.136710    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d08af64d-ba39-4d61-9b95-c6febd32cbbb-agent-certs\") pod \"konnectivity-agent-ftlkq\" (UID: \"d08af64d-ba39-4d61-9b95-c6febd32cbbb\") " pod="kube-system/konnectivity-agent-ftlkq"
Sep 20 01:52:26.137694 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.137037    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/92615b78-f30d-4687-a822-6261a2d9ae55-etc-tuned\") pod \"tuned-m4zgq\" (UID: \"92615b78-f30d-4687-a822-6261a2d9ae55\") " pod="openshift-cluster-node-tuning-operator/tuned-m4zgq"
Sep 20 01:52:26.143770 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:26.143746    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 20 01:52:26.143878 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:26.143775    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 20 01:52:26.143878 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:26.143789    2584 projected.go:196] Error preparing data for projected volume kube-api-access-mrb2z for pod openshift-network-diagnostics/network-check-target-47994: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 20 01:52:26.143878 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:26.143857    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9b376ae6-8f6a-4139-ab2d-96dc32e1201f-kube-api-access-mrb2z podName:9b376ae6-8f6a-4139-ab2d-96dc32e1201f nodeName:}" failed. No retries permitted until 2026-09-20 01:52:26.643839301 +0000 UTC m=+3.386518487 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-mrb2z" (UniqueName: "kubernetes.io/projected/9b376ae6-8f6a-4139-ab2d-96dc32e1201f-kube-api-access-mrb2z") pod "network-check-target-47994" (UID: "9b376ae6-8f6a-4139-ab2d-96dc32e1201f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 20 01:52:26.144540 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.144464    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m5r9c\" (UniqueName: \"kubernetes.io/projected/4fa3e7e2-7940-4d7d-af23-63ff07edba43-kube-api-access-m5r9c\") pod \"node-ca-v6d2k\" (UID: \"4fa3e7e2-7940-4d7d-af23-63ff07edba43\") " pod="openshift-image-registry/node-ca-v6d2k"
Sep 20 01:52:26.145226 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.145203    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h9rmf\" (UniqueName: \"kubernetes.io/projected/92615b78-f30d-4687-a822-6261a2d9ae55-kube-api-access-h9rmf\") pod \"tuned-m4zgq\" (UID: \"92615b78-f30d-4687-a822-6261a2d9ae55\") " pod="openshift-cluster-node-tuning-operator/tuned-m4zgq"
Sep 20 01:52:26.145718 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.145680    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tsj66\" (UniqueName: \"kubernetes.io/projected/96a97560-011d-4806-ae10-4d53880eb9ff-kube-api-access-tsj66\") pod \"multus-additional-cni-plugins-f94fv\" (UID: \"96a97560-011d-4806-ae10-4d53880eb9ff\") " pod="openshift-multus/multus-additional-cni-plugins-f94fv"
Sep 20 01:52:26.145718 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.145680    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6kt7d\" (UniqueName: \"kubernetes.io/projected/f9800646-e98f-45d8-aa4e-5eb53056f5e8-kube-api-access-6kt7d\") pod \"multus-k8rn8\" (UID: \"f9800646-e98f-45d8-aa4e-5eb53056f5e8\") " pod="openshift-multus/multus-k8rn8"
Sep 20 01:52:26.145848 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.145710    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nm94k\" (UniqueName: \"kubernetes.io/projected/43dd43c0-ff48-4f9a-a1c5-9020836702aa-kube-api-access-nm94k\") pod \"node-resolver-l9ssk\" (UID: \"43dd43c0-ff48-4f9a-a1c5-9020836702aa\") " pod="openshift-dns/node-resolver-l9ssk"
Sep 20 01:52:26.146423 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.146401    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wjsfw\" (UniqueName: \"kubernetes.io/projected/d94badda-e0c0-4b65-a95f-64f524896542-kube-api-access-wjsfw\") pod \"iptables-alerter-chcxk\" (UID: \"d94badda-e0c0-4b65-a95f-64f524896542\") " pod="openshift-network-operator/iptables-alerter-chcxk"
Sep 20 01:52:26.146683 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.146653    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jjdvt\" (UniqueName: \"kubernetes.io/projected/c9c657b9-b508-4bb7-99f3-aed2502cfba4-kube-api-access-jjdvt\") pod \"network-metrics-daemon-n4dk7\" (UID: \"c9c657b9-b508-4bb7-99f3-aed2502cfba4\") " pod="openshift-multus/network-metrics-daemon-n4dk7"
Sep 20 01:52:26.146996 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.146980    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ngtsk\" (UniqueName: \"kubernetes.io/projected/d0aee250-9aee-4473-ae29-83ffdbaeb920-kube-api-access-ngtsk\") pod \"aws-ebs-csi-driver-node-27wj7\" (UID: \"d0aee250-9aee-4473-ae29-83ffdbaeb920\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-27wj7"
Sep 20 01:52:26.147052 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.147034    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lxxxz\" (UniqueName: \"kubernetes.io/projected/147f9cc3-d8b1-4c83-9a97-4b29246cd454-kube-api-access-lxxxz\") pod \"ovnkube-node-p2wkn\" (UID: \"147f9cc3-d8b1-4c83-9a97-4b29246cd454\") " pod="openshift-ovn-kubernetes/ovnkube-node-p2wkn"
Sep 20 01:52:26.218017 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:26.217982    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55f1df651e44bec56d534fcb469f62e8.slice/crio-97a5f513021557ee533f3a5ade3f3c7d85b81cdee53f3a489da05d1c7f8f2db2 WatchSource:0}: Error finding container 97a5f513021557ee533f3a5ade3f3c7d85b81cdee53f3a489da05d1c7f8f2db2: Status 404 returned error can't find the container with id 97a5f513021557ee533f3a5ade3f3c7d85b81cdee53f3a489da05d1c7f8f2db2
Sep 20 01:52:26.218278 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:26.218262    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f739e7563e31c6ca101623c66d03faf.slice/crio-9f5587d258c05b58398869ad040e10b1bd5e64232fa5fa448e4b8e461b8ffe2d WatchSource:0}: Error finding container 9f5587d258c05b58398869ad040e10b1bd5e64232fa5fa448e4b8e461b8ffe2d: Status 404 returned error can't find the container with id 9f5587d258c05b58398869ad040e10b1bd5e64232fa5fa448e4b8e461b8ffe2d
Sep 20 01:52:26.223398 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.223383    2584 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 20 01:52:26.329064 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.328988    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-l9ssk"
Sep 20 01:52:26.338559 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:26.338537    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43dd43c0_ff48_4f9a_a1c5_9020836702aa.slice/crio-cd335d42f50daf39de2c12397e8157591cd0a78df20742942590813720c48060 WatchSource:0}: Error finding container cd335d42f50daf39de2c12397e8157591cd0a78df20742942590813720c48060: Status 404 returned error can't find the container with id cd335d42f50daf39de2c12397e8157591cd0a78df20742942590813720c48060
Sep 20 01:52:26.339765 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.339706    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-k8rn8"
Sep 20 01:52:26.349090 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:26.349073    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9800646_e98f_45d8_aa4e_5eb53056f5e8.slice/crio-df242c6c8d0c9e7affcbfbb8bf1c8ada24874c97b8337e5cdda17ed6d0b3cef4 WatchSource:0}: Error finding container df242c6c8d0c9e7affcbfbb8bf1c8ada24874c97b8337e5cdda17ed6d0b3cef4: Status 404 returned error can't find the container with id df242c6c8d0c9e7affcbfbb8bf1c8ada24874c97b8337e5cdda17ed6d0b3cef4
Sep 20 01:52:26.357073 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.357058    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-chcxk"
Sep 20 01:52:26.364383 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:26.364353    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd94badda_e0c0_4b65_a95f_64f524896542.slice/crio-f85e38c462845fdcf12b07a86d5636559b2026da8f10bf04285390cb8d5d3f80 WatchSource:0}: Error finding container f85e38c462845fdcf12b07a86d5636559b2026da8f10bf04285390cb8d5d3f80: Status 404 returned error can't find the container with id f85e38c462845fdcf12b07a86d5636559b2026da8f10bf04285390cb8d5d3f80
Sep 20 01:52:26.370301 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.370285    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-p2wkn"
Sep 20 01:52:26.377504 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:26.377487    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod147f9cc3_d8b1_4c83_9a97_4b29246cd454.slice/crio-2c73e5236fa3b141f0d2c70da8d10943cf4d04a5d4169651eadea67dc0c88ceb WatchSource:0}: Error finding container 2c73e5236fa3b141f0d2c70da8d10943cf4d04a5d4169651eadea67dc0c88ceb: Status 404 returned error can't find the container with id 2c73e5236fa3b141f0d2c70da8d10943cf4d04a5d4169651eadea67dc0c88ceb
Sep 20 01:52:26.383568 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.383551    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-ftlkq"
Sep 20 01:52:26.390605 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:26.390586    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd08af64d_ba39_4d61_9b95_c6febd32cbbb.slice/crio-be494be3460ef2ab675f4d2468703172790638445cd529b2336d7a32e6d820bd WatchSource:0}: Error finding container be494be3460ef2ab675f4d2468703172790638445cd529b2336d7a32e6d820bd: Status 404 returned error can't find the container with id be494be3460ef2ab675f4d2468703172790638445cd529b2336d7a32e6d820bd
Sep 20 01:52:26.398376 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.398355    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-27wj7"
Sep 20 01:52:26.405593 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:26.405572    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0aee250_9aee_4473_ae29_83ffdbaeb920.slice/crio-c30864928a553d9d1d6a33c2dda08ce60d21e09f952908ed4e17fde2362c8cef WatchSource:0}: Error finding container c30864928a553d9d1d6a33c2dda08ce60d21e09f952908ed4e17fde2362c8cef: Status 404 returned error can't find the container with id c30864928a553d9d1d6a33c2dda08ce60d21e09f952908ed4e17fde2362c8cef
Sep 20 01:52:26.413602 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.413588    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-m4zgq"
Sep 20 01:52:26.421589 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:26.421569    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92615b78_f30d_4687_a822_6261a2d9ae55.slice/crio-0fdfb4e01a096998ec06cf33ad6955275f17a6f615f33835bfb48c4dec27b27b WatchSource:0}: Error finding container 0fdfb4e01a096998ec06cf33ad6955275f17a6f615f33835bfb48c4dec27b27b: Status 404 returned error can't find the container with id 0fdfb4e01a096998ec06cf33ad6955275f17a6f615f33835bfb48c4dec27b27b
Sep 20 01:52:26.429602 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.429588    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-v6d2k"
Sep 20 01:52:26.437897 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.437879    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-f94fv"
Sep 20 01:52:26.438113 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:26.438093    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4fa3e7e2_7940_4d7d_af23_63ff07edba43.slice/crio-b25093e03b69089ff41d7faefd0f2638915b7134e1b71114e765caa911d8d9af WatchSource:0}: Error finding container b25093e03b69089ff41d7faefd0f2638915b7134e1b71114e765caa911d8d9af: Status 404 returned error can't find the container with id b25093e03b69089ff41d7faefd0f2638915b7134e1b71114e765caa911d8d9af
Sep 20 01:52:26.446828 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:26.446802    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96a97560_011d_4806_ae10_4d53880eb9ff.slice/crio-9fd4e57f6cd6f8e39a37582f3e456963e4488154b0672134624e8fa5a97bc62c WatchSource:0}: Error finding container 9fd4e57f6cd6f8e39a37582f3e456963e4488154b0672134624e8fa5a97bc62c: Status 404 returned error can't find the container with id 9fd4e57f6cd6f8e39a37582f3e456963e4488154b0672134624e8fa5a97bc62c
Sep 20 01:52:26.565988 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.565961    2584 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 20 01:52:26.638224 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.638149    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/05a98348-758f-4a4a-b150-35bbdeb5b359-original-pull-secret\") pod \"global-pull-secret-syncer-llv6h\" (UID: \"05a98348-758f-4a4a-b150-35bbdeb5b359\") " pod="kube-system/global-pull-secret-syncer-llv6h"
Sep 20 01:52:26.638224 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.638196    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c9c657b9-b508-4bb7-99f3-aed2502cfba4-metrics-certs\") pod \"network-metrics-daemon-n4dk7\" (UID: \"c9c657b9-b508-4bb7-99f3-aed2502cfba4\") " pod="openshift-multus/network-metrics-daemon-n4dk7"
Sep 20 01:52:26.638423 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:26.638293    2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 20 01:52:26.638423 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:26.638360    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c9c657b9-b508-4bb7-99f3-aed2502cfba4-metrics-certs podName:c9c657b9-b508-4bb7-99f3-aed2502cfba4 nodeName:}" failed. No retries permitted until 2026-09-20 01:52:27.638335697 +0000 UTC m=+4.381014880 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c9c657b9-b508-4bb7-99f3-aed2502cfba4-metrics-certs") pod "network-metrics-daemon-n4dk7" (UID: "c9c657b9-b508-4bb7-99f3-aed2502cfba4") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 20 01:52:26.638423 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:26.638294    2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 20 01:52:26.638602 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:26.638426    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/05a98348-758f-4a4a-b150-35bbdeb5b359-original-pull-secret podName:05a98348-758f-4a4a-b150-35bbdeb5b359 nodeName:}" failed. No retries permitted until 2026-09-20 01:52:27.638413721 +0000 UTC m=+4.381092904 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/05a98348-758f-4a4a-b150-35bbdeb5b359-original-pull-secret") pod "global-pull-secret-syncer-llv6h" (UID: "05a98348-758f-4a4a-b150-35bbdeb5b359") : object "kube-system"/"original-pull-secret" not registered
Sep 20 01:52:26.739349 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.739316    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mrb2z\" (UniqueName: \"kubernetes.io/projected/9b376ae6-8f6a-4139-ab2d-96dc32e1201f-kube-api-access-mrb2z\") pod \"network-check-target-47994\" (UID: \"9b376ae6-8f6a-4139-ab2d-96dc32e1201f\") " pod="openshift-network-diagnostics/network-check-target-47994"
Sep 20 01:52:26.739532 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:26.739448    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 20 01:52:26.739532 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:26.739461    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 20 01:52:26.739532 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:26.739470    2584 projected.go:196] Error preparing data for projected volume kube-api-access-mrb2z for pod openshift-network-diagnostics/network-check-target-47994: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 20 01:52:26.739532 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:26.739532    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9b376ae6-8f6a-4139-ab2d-96dc32e1201f-kube-api-access-mrb2z podName:9b376ae6-8f6a-4139-ab2d-96dc32e1201f nodeName:}" failed. No retries permitted until 2026-09-20 01:52:27.739504679 +0000 UTC m=+4.482183862 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-mrb2z" (UniqueName: "kubernetes.io/projected/9b376ae6-8f6a-4139-ab2d-96dc32e1201f-kube-api-access-mrb2z") pod "network-check-target-47994" (UID: "9b376ae6-8f6a-4139-ab2d-96dc32e1201f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 20 01:52:26.926291 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:26.926054    2584 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 20 01:52:27.071930 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:27.071838    2584 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-19 01:47:26 +0000 UTC" deadline="2028-05-06 16:37:35.257957886 +0000 UTC"
Sep 20 01:52:27.071930 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:27.071874    2584 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14270h45m8.186089193s"
Sep 20 01:52:27.091489 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:27.091463    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n4dk7"
Sep 20 01:52:27.091714 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:27.091643    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n4dk7" podUID="c9c657b9-b508-4bb7-99f3-aed2502cfba4"
Sep 20 01:52:27.108271 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:27.108195    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-p2wkn" event={"ID":"147f9cc3-d8b1-4c83-9a97-4b29246cd454","Type":"ContainerStarted","Data":"2c73e5236fa3b141f0d2c70da8d10943cf4d04a5d4169651eadea67dc0c88ceb"}
Sep 20 01:52:27.114662 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:27.114594    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-chcxk" event={"ID":"d94badda-e0c0-4b65-a95f-64f524896542","Type":"ContainerStarted","Data":"f85e38c462845fdcf12b07a86d5636559b2026da8f10bf04285390cb8d5d3f80"}
Sep 20 01:52:27.123995 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:27.123920    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-l9ssk" event={"ID":"43dd43c0-ff48-4f9a-a1c5-9020836702aa","Type":"ContainerStarted","Data":"cd335d42f50daf39de2c12397e8157591cd0a78df20742942590813720c48060"}
Sep 20 01:52:27.127293 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:27.127253    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-159.ec2.internal" event={"ID":"4f739e7563e31c6ca101623c66d03faf","Type":"ContainerStarted","Data":"9f5587d258c05b58398869ad040e10b1bd5e64232fa5fa448e4b8e461b8ffe2d"}
Sep 20 01:52:27.136908 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:27.136828    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-159.ec2.internal" event={"ID":"55f1df651e44bec56d534fcb469f62e8","Type":"ContainerStarted","Data":"97a5f513021557ee533f3a5ade3f3c7d85b81cdee53f3a489da05d1c7f8f2db2"}
Sep 20 01:52:27.144428 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:27.144403    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-f94fv" event={"ID":"96a97560-011d-4806-ae10-4d53880eb9ff","Type":"ContainerStarted","Data":"9fd4e57f6cd6f8e39a37582f3e456963e4488154b0672134624e8fa5a97bc62c"}
Sep 20 01:52:27.157717 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:27.157691    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-v6d2k" event={"ID":"4fa3e7e2-7940-4d7d-af23-63ff07edba43","Type":"ContainerStarted","Data":"b25093e03b69089ff41d7faefd0f2638915b7134e1b71114e765caa911d8d9af"}
Sep 20 01:52:27.177213 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:27.177132    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-ftlkq" event={"ID":"d08af64d-ba39-4d61-9b95-c6febd32cbbb","Type":"ContainerStarted","Data":"be494be3460ef2ab675f4d2468703172790638445cd529b2336d7a32e6d820bd"}
Sep 20 01:52:27.201456 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:27.201416    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-k8rn8" event={"ID":"f9800646-e98f-45d8-aa4e-5eb53056f5e8","Type":"ContainerStarted","Data":"df242c6c8d0c9e7affcbfbb8bf1c8ada24874c97b8337e5cdda17ed6d0b3cef4"}
Sep 20 01:52:27.208928 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:27.208866    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-m4zgq" event={"ID":"92615b78-f30d-4687-a822-6261a2d9ae55","Type":"ContainerStarted","Data":"0fdfb4e01a096998ec06cf33ad6955275f17a6f615f33835bfb48c4dec27b27b"}
Sep 20 01:52:27.232379 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:27.232344    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-27wj7" event={"ID":"d0aee250-9aee-4473-ae29-83ffdbaeb920","Type":"ContainerStarted","Data":"c30864928a553d9d1d6a33c2dda08ce60d21e09f952908ed4e17fde2362c8cef"}
Sep 20 01:52:27.647223 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:27.647183    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/05a98348-758f-4a4a-b150-35bbdeb5b359-original-pull-secret\") pod \"global-pull-secret-syncer-llv6h\" (UID: \"05a98348-758f-4a4a-b150-35bbdeb5b359\") " pod="kube-system/global-pull-secret-syncer-llv6h"
Sep 20 01:52:27.647394 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:27.647265    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c9c657b9-b508-4bb7-99f3-aed2502cfba4-metrics-certs\") pod \"network-metrics-daemon-n4dk7\" (UID: \"c9c657b9-b508-4bb7-99f3-aed2502cfba4\") " pod="openshift-multus/network-metrics-daemon-n4dk7"
Sep 20 01:52:27.647394 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:27.647382    2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 20 01:52:27.647540 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:27.647444    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c9c657b9-b508-4bb7-99f3-aed2502cfba4-metrics-certs podName:c9c657b9-b508-4bb7-99f3-aed2502cfba4 nodeName:}" failed. No retries permitted until 2026-09-20 01:52:29.647424844 +0000 UTC m=+6.390104030 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c9c657b9-b508-4bb7-99f3-aed2502cfba4-metrics-certs") pod "network-metrics-daemon-n4dk7" (UID: "c9c657b9-b508-4bb7-99f3-aed2502cfba4") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 20 01:52:27.647897 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:27.647870    2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 20 01:52:27.647984 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:27.647924    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/05a98348-758f-4a4a-b150-35bbdeb5b359-original-pull-secret podName:05a98348-758f-4a4a-b150-35bbdeb5b359 nodeName:}" failed. No retries permitted until 2026-09-20 01:52:29.647909658 +0000 UTC m=+6.390588847 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/05a98348-758f-4a4a-b150-35bbdeb5b359-original-pull-secret") pod "global-pull-secret-syncer-llv6h" (UID: "05a98348-758f-4a4a-b150-35bbdeb5b359") : object "kube-system"/"original-pull-secret" not registered
Sep 20 01:52:27.748364 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:27.748158    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mrb2z\" (UniqueName: \"kubernetes.io/projected/9b376ae6-8f6a-4139-ab2d-96dc32e1201f-kube-api-access-mrb2z\") pod \"network-check-target-47994\" (UID: \"9b376ae6-8f6a-4139-ab2d-96dc32e1201f\") " pod="openshift-network-diagnostics/network-check-target-47994"
Sep 20 01:52:27.748555 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:27.748381    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 20 01:52:27.748555 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:27.748401    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 20 01:52:27.748555 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:27.748413    2584 projected.go:196] Error preparing data for projected volume kube-api-access-mrb2z for pod openshift-network-diagnostics/network-check-target-47994: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 20 01:52:27.748555 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:27.748469    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9b376ae6-8f6a-4139-ab2d-96dc32e1201f-kube-api-access-mrb2z podName:9b376ae6-8f6a-4139-ab2d-96dc32e1201f nodeName:}" failed. No retries permitted until 2026-09-20 01:52:29.748450266 +0000 UTC m=+6.491129455 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-mrb2z" (UniqueName: "kubernetes.io/projected/9b376ae6-8f6a-4139-ab2d-96dc32e1201f-kube-api-access-mrb2z") pod "network-check-target-47994" (UID: "9b376ae6-8f6a-4139-ab2d-96dc32e1201f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 20 01:52:27.795099 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:27.794907    2584 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 20 01:52:28.072280 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:28.072213    2584 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-19 01:47:26 +0000 UTC" deadline="2028-03-07 12:47:31.703426776 +0000 UTC"
Sep 20 01:52:28.072280 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:28.072277    2584 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12826h55m3.631154143s"
Sep 20 01:52:28.087214 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:28.086625    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-llv6h"
Sep 20 01:52:28.087214 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:28.086753    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-llv6h" podUID="05a98348-758f-4a4a-b150-35bbdeb5b359"
Sep 20 01:52:28.087214 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:28.087141    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-47994"
Sep 20 01:52:28.087461 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:28.087227    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-47994" podUID="9b376ae6-8f6a-4139-ab2d-96dc32e1201f"
Sep 20 01:52:28.761475 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:28.761442    2584 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 20 01:52:29.086955 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:29.086888    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n4dk7"
Sep 20 01:52:29.087392 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:29.086986    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n4dk7" podUID="c9c657b9-b508-4bb7-99f3-aed2502cfba4"
Sep 20 01:52:29.666776 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:29.666729    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/05a98348-758f-4a4a-b150-35bbdeb5b359-original-pull-secret\") pod \"global-pull-secret-syncer-llv6h\" (UID: \"05a98348-758f-4a4a-b150-35bbdeb5b359\") " pod="kube-system/global-pull-secret-syncer-llv6h"
Sep 20 01:52:29.666961 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:29.666821    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c9c657b9-b508-4bb7-99f3-aed2502cfba4-metrics-certs\") pod \"network-metrics-daemon-n4dk7\" (UID: \"c9c657b9-b508-4bb7-99f3-aed2502cfba4\") " pod="openshift-multus/network-metrics-daemon-n4dk7"
Sep 20 01:52:29.667031 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:29.667007    2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 20 01:52:29.667088 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:29.667073    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c9c657b9-b508-4bb7-99f3-aed2502cfba4-metrics-certs podName:c9c657b9-b508-4bb7-99f3-aed2502cfba4 nodeName:}" failed. No retries permitted until 2026-09-20 01:52:33.667052423 +0000 UTC m=+10.409731606 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c9c657b9-b508-4bb7-99f3-aed2502cfba4-metrics-certs") pod "network-metrics-daemon-n4dk7" (UID: "c9c657b9-b508-4bb7-99f3-aed2502cfba4") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 20 01:52:29.667640 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:29.667455    2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 20 01:52:29.667640 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:29.667504    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/05a98348-758f-4a4a-b150-35bbdeb5b359-original-pull-secret podName:05a98348-758f-4a4a-b150-35bbdeb5b359 nodeName:}" failed. No retries permitted until 2026-09-20 01:52:33.667490857 +0000 UTC m=+10.410170040 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/05a98348-758f-4a4a-b150-35bbdeb5b359-original-pull-secret") pod "global-pull-secret-syncer-llv6h" (UID: "05a98348-758f-4a4a-b150-35bbdeb5b359") : object "kube-system"/"original-pull-secret" not registered
Sep 20 01:52:29.767761 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:29.767727    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mrb2z\" (UniqueName: \"kubernetes.io/projected/9b376ae6-8f6a-4139-ab2d-96dc32e1201f-kube-api-access-mrb2z\") pod \"network-check-target-47994\" (UID: \"9b376ae6-8f6a-4139-ab2d-96dc32e1201f\") " pod="openshift-network-diagnostics/network-check-target-47994"
Sep 20 01:52:29.767947 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:29.767913    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 20 01:52:29.767947 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:29.767933    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 20 01:52:29.767947 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:29.767947    2584 projected.go:196] Error preparing data for projected volume kube-api-access-mrb2z for pod openshift-network-diagnostics/network-check-target-47994: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 20 01:52:29.768116 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:29.768001    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9b376ae6-8f6a-4139-ab2d-96dc32e1201f-kube-api-access-mrb2z podName:9b376ae6-8f6a-4139-ab2d-96dc32e1201f nodeName:}" failed. No retries permitted until 2026-09-20 01:52:33.767984448 +0000 UTC m=+10.510663644 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-mrb2z" (UniqueName: "kubernetes.io/projected/9b376ae6-8f6a-4139-ab2d-96dc32e1201f-kube-api-access-mrb2z") pod "network-check-target-47994" (UID: "9b376ae6-8f6a-4139-ab2d-96dc32e1201f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 20 01:52:30.087233 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:30.087094    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-llv6h"
Sep 20 01:52:30.087233 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:30.087144    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-47994"
Sep 20 01:52:30.087768 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:30.087230    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-llv6h" podUID="05a98348-758f-4a4a-b150-35bbdeb5b359"
Sep 20 01:52:30.087768 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:30.087336    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-47994" podUID="9b376ae6-8f6a-4139-ab2d-96dc32e1201f"
Sep 20 01:52:31.090545 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:31.090038    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n4dk7"
Sep 20 01:52:31.090545 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:31.090170    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n4dk7" podUID="c9c657b9-b508-4bb7-99f3-aed2502cfba4"
Sep 20 01:52:32.086575 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:32.086532    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-llv6h"
Sep 20 01:52:32.086575 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:32.086558    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-47994"
Sep 20 01:52:32.086821 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:32.086671    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-llv6h" podUID="05a98348-758f-4a4a-b150-35bbdeb5b359"
Sep 20 01:52:32.086821 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:32.086777    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-47994" podUID="9b376ae6-8f6a-4139-ab2d-96dc32e1201f"
Sep 20 01:52:33.090695 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:33.090654    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n4dk7"
Sep 20 01:52:33.091095 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:33.090814    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n4dk7" podUID="c9c657b9-b508-4bb7-99f3-aed2502cfba4"
Sep 20 01:52:33.698679 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:33.698504    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c9c657b9-b508-4bb7-99f3-aed2502cfba4-metrics-certs\") pod \"network-metrics-daemon-n4dk7\" (UID: \"c9c657b9-b508-4bb7-99f3-aed2502cfba4\") " pod="openshift-multus/network-metrics-daemon-n4dk7"
Sep 20 01:52:33.698679 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:33.698624    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/05a98348-758f-4a4a-b150-35bbdeb5b359-original-pull-secret\") pod \"global-pull-secret-syncer-llv6h\" (UID: \"05a98348-758f-4a4a-b150-35bbdeb5b359\") " pod="kube-system/global-pull-secret-syncer-llv6h"
Sep 20 01:52:33.698679 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:33.698641    2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 20 01:52:33.698949 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:33.698714    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c9c657b9-b508-4bb7-99f3-aed2502cfba4-metrics-certs podName:c9c657b9-b508-4bb7-99f3-aed2502cfba4 nodeName:}" failed. No retries permitted until 2026-09-20 01:52:41.698694162 +0000 UTC m=+18.441373360 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c9c657b9-b508-4bb7-99f3-aed2502cfba4-metrics-certs") pod "network-metrics-daemon-n4dk7" (UID: "c9c657b9-b508-4bb7-99f3-aed2502cfba4") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 20 01:52:33.698949 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:33.698768    2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 20 01:52:33.698949 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:33.698838    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/05a98348-758f-4a4a-b150-35bbdeb5b359-original-pull-secret podName:05a98348-758f-4a4a-b150-35bbdeb5b359 nodeName:}" failed. No retries permitted until 2026-09-20 01:52:41.698819577 +0000 UTC m=+18.441498784 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/05a98348-758f-4a4a-b150-35bbdeb5b359-original-pull-secret") pod "global-pull-secret-syncer-llv6h" (UID: "05a98348-758f-4a4a-b150-35bbdeb5b359") : object "kube-system"/"original-pull-secret" not registered
Sep 20 01:52:33.800032 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:33.799928    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mrb2z\" (UniqueName: \"kubernetes.io/projected/9b376ae6-8f6a-4139-ab2d-96dc32e1201f-kube-api-access-mrb2z\") pod \"network-check-target-47994\" (UID: \"9b376ae6-8f6a-4139-ab2d-96dc32e1201f\") " pod="openshift-network-diagnostics/network-check-target-47994"
Sep 20 01:52:33.800192 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:33.800078    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 20 01:52:33.800192 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:33.800097    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 20 01:52:33.800192 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:33.800108    2584 projected.go:196] Error preparing data for projected volume kube-api-access-mrb2z for pod openshift-network-diagnostics/network-check-target-47994: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 20 01:52:33.800192 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:33.800168    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9b376ae6-8f6a-4139-ab2d-96dc32e1201f-kube-api-access-mrb2z podName:9b376ae6-8f6a-4139-ab2d-96dc32e1201f nodeName:}" failed. No retries permitted until 2026-09-20 01:52:41.800151168 +0000 UTC m=+18.542830364 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-mrb2z" (UniqueName: "kubernetes.io/projected/9b376ae6-8f6a-4139-ab2d-96dc32e1201f-kube-api-access-mrb2z") pod "network-check-target-47994" (UID: "9b376ae6-8f6a-4139-ab2d-96dc32e1201f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 20 01:52:34.086896 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:34.086808    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-llv6h"
Sep 20 01:52:34.086896 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:34.086832    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-47994"
Sep 20 01:52:34.087107 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:34.086926    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-llv6h" podUID="05a98348-758f-4a4a-b150-35bbdeb5b359"
Sep 20 01:52:34.087107 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:34.087038    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-47994" podUID="9b376ae6-8f6a-4139-ab2d-96dc32e1201f"
Sep 20 01:52:35.090537 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:35.090493    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n4dk7"
Sep 20 01:52:35.090947 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:35.090639    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n4dk7" podUID="c9c657b9-b508-4bb7-99f3-aed2502cfba4"
Sep 20 01:52:36.086722 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:36.086684    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-llv6h"
Sep 20 01:52:36.086908 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:36.086815    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-llv6h" podUID="05a98348-758f-4a4a-b150-35bbdeb5b359"
Sep 20 01:52:36.086908 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:36.086875    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-47994"
Sep 20 01:52:36.087032 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:36.086987    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-47994" podUID="9b376ae6-8f6a-4139-ab2d-96dc32e1201f"
Sep 20 01:52:37.088340 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:37.088314    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n4dk7"
Sep 20 01:52:37.088815 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:37.088408    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n4dk7" podUID="c9c657b9-b508-4bb7-99f3-aed2502cfba4"
Sep 20 01:52:38.086784 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:38.086746    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-llv6h"
Sep 20 01:52:38.086784 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:38.086765    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-47994"
Sep 20 01:52:38.087010 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:38.086881    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-llv6h" podUID="05a98348-758f-4a4a-b150-35bbdeb5b359"
Sep 20 01:52:38.087063 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:38.087027    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-47994" podUID="9b376ae6-8f6a-4139-ab2d-96dc32e1201f"
Sep 20 01:52:39.086737 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:39.086705    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n4dk7"
Sep 20 01:52:39.087165 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:39.086850    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n4dk7" podUID="c9c657b9-b508-4bb7-99f3-aed2502cfba4"
Sep 20 01:52:40.086572 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:40.086535    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-llv6h"
Sep 20 01:52:40.086572 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:40.086565    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-47994"
Sep 20 01:52:40.087179 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:40.086643    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-llv6h" podUID="05a98348-758f-4a4a-b150-35bbdeb5b359"
Sep 20 01:52:40.087179 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:40.086787    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-47994" podUID="9b376ae6-8f6a-4139-ab2d-96dc32e1201f"
Sep 20 01:52:41.088294 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:41.088265    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n4dk7"
Sep 20 01:52:41.088662 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:41.088363    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n4dk7" podUID="c9c657b9-b508-4bb7-99f3-aed2502cfba4"
Sep 20 01:52:41.753668 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:41.753631    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c9c657b9-b508-4bb7-99f3-aed2502cfba4-metrics-certs\") pod \"network-metrics-daemon-n4dk7\" (UID: \"c9c657b9-b508-4bb7-99f3-aed2502cfba4\") " pod="openshift-multus/network-metrics-daemon-n4dk7"
Sep 20 01:52:41.753873 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:41.753784    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/05a98348-758f-4a4a-b150-35bbdeb5b359-original-pull-secret\") pod \"global-pull-secret-syncer-llv6h\" (UID: \"05a98348-758f-4a4a-b150-35bbdeb5b359\") " pod="kube-system/global-pull-secret-syncer-llv6h"
Sep 20 01:52:41.753873 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:41.753789    2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 20 01:52:41.753873 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:41.753863    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c9c657b9-b508-4bb7-99f3-aed2502cfba4-metrics-certs podName:c9c657b9-b508-4bb7-99f3-aed2502cfba4 nodeName:}" failed. No retries permitted until 2026-09-20 01:52:57.753841701 +0000 UTC m=+34.496520900 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c9c657b9-b508-4bb7-99f3-aed2502cfba4-metrics-certs") pod "network-metrics-daemon-n4dk7" (UID: "c9c657b9-b508-4bb7-99f3-aed2502cfba4") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 20 01:52:41.754046 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:41.753920    2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 20 01:52:41.754046 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:41.753968    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/05a98348-758f-4a4a-b150-35bbdeb5b359-original-pull-secret podName:05a98348-758f-4a4a-b150-35bbdeb5b359 nodeName:}" failed. No retries permitted until 2026-09-20 01:52:57.753954373 +0000 UTC m=+34.496633569 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/05a98348-758f-4a4a-b150-35bbdeb5b359-original-pull-secret") pod "global-pull-secret-syncer-llv6h" (UID: "05a98348-758f-4a4a-b150-35bbdeb5b359") : object "kube-system"/"original-pull-secret" not registered
Sep 20 01:52:41.855061 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:41.855030    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mrb2z\" (UniqueName: \"kubernetes.io/projected/9b376ae6-8f6a-4139-ab2d-96dc32e1201f-kube-api-access-mrb2z\") pod \"network-check-target-47994\" (UID: \"9b376ae6-8f6a-4139-ab2d-96dc32e1201f\") " pod="openshift-network-diagnostics/network-check-target-47994"
Sep 20 01:52:41.855230 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:41.855197    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 20 01:52:41.855230 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:41.855222    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 20 01:52:41.855344 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:41.855233    2584 projected.go:196] Error preparing data for projected volume kube-api-access-mrb2z for pod openshift-network-diagnostics/network-check-target-47994: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 20 01:52:41.855344 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:41.855297    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9b376ae6-8f6a-4139-ab2d-96dc32e1201f-kube-api-access-mrb2z podName:9b376ae6-8f6a-4139-ab2d-96dc32e1201f nodeName:}" failed. No retries permitted until 2026-09-20 01:52:57.855278372 +0000 UTC m=+34.597957569 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-mrb2z" (UniqueName: "kubernetes.io/projected/9b376ae6-8f6a-4139-ab2d-96dc32e1201f-kube-api-access-mrb2z") pod "network-check-target-47994" (UID: "9b376ae6-8f6a-4139-ab2d-96dc32e1201f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 20 01:52:42.086654 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:42.086587    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-47994"
Sep 20 01:52:42.086812 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:42.086600    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-llv6h"
Sep 20 01:52:42.086812 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:42.086692    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-47994" podUID="9b376ae6-8f6a-4139-ab2d-96dc32e1201f"
Sep 20 01:52:42.086812 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:42.086770    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-llv6h" podUID="05a98348-758f-4a4a-b150-35bbdeb5b359"
Sep 20 01:52:43.086950 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:43.086910    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n4dk7"
Sep 20 01:52:43.087662 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:43.087053    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n4dk7" podUID="c9c657b9-b508-4bb7-99f3-aed2502cfba4"
Sep 20 01:52:44.086497 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:44.086464    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-llv6h"
Sep 20 01:52:44.086696 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:44.086464    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-47994"
Sep 20 01:52:44.086696 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:44.086599    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-llv6h" podUID="05a98348-758f-4a4a-b150-35bbdeb5b359"
Sep 20 01:52:44.086696 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:44.086651    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-47994" podUID="9b376ae6-8f6a-4139-ab2d-96dc32e1201f"
Sep 20 01:52:45.087390 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:45.087357    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n4dk7"
Sep 20 01:52:45.087860 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:45.087447    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n4dk7" podUID="c9c657b9-b508-4bb7-99f3-aed2502cfba4"
Sep 20 01:52:46.087627 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:46.087128    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-llv6h"
Sep 20 01:52:46.088244 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:46.087156    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-47994"
Sep 20 01:52:46.088244 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:46.087699    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-llv6h" podUID="05a98348-758f-4a4a-b150-35bbdeb5b359"
Sep 20 01:52:46.088244 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:46.087742    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-47994" podUID="9b376ae6-8f6a-4139-ab2d-96dc32e1201f"
Sep 20 01:52:46.282598 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:46.282533    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-v6d2k" event={"ID":"4fa3e7e2-7940-4d7d-af23-63ff07edba43","Type":"ContainerStarted","Data":"204a3137dea2fdb1b4ca2493beded9fdb9cd9fd345c2f21cd1b6e8d870220943"}
Sep 20 01:52:46.284143 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:46.284114    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-ftlkq" event={"ID":"d08af64d-ba39-4d61-9b95-c6febd32cbbb","Type":"ContainerStarted","Data":"667b9d29d8eac75752034a69e086b6609f53c99fa83dafb27efa15dd50be7c1f"}
Sep 20 01:52:46.285989 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:46.285967    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-k8rn8" event={"ID":"f9800646-e98f-45d8-aa4e-5eb53056f5e8","Type":"ContainerStarted","Data":"1426cc1dbcffa27b56aed727512d45ff138a77a0f3f6e1ba4620f6082333b1dd"}
Sep 20 01:52:46.287666 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:46.287644    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-m4zgq" event={"ID":"92615b78-f30d-4687-a822-6261a2d9ae55","Type":"ContainerStarted","Data":"058608003fee7e9ebf71c0d8c2215be2882cd0c8a3751d461477594a607eea3b"}
Sep 20 01:52:46.289335 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:46.289314    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-27wj7" event={"ID":"d0aee250-9aee-4473-ae29-83ffdbaeb920","Type":"ContainerStarted","Data":"030f82e51113e92d58c6d3988693ef2b520efb1df1f5cc93ba569f1cf4344f8f"}
Sep 20 01:52:46.292827 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:46.292796    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-p2wkn" event={"ID":"147f9cc3-d8b1-4c83-9a97-4b29246cd454","Type":"ContainerStarted","Data":"ca0f0d1c0a57792008b513e4cd15aa57a4ce69e2b3bb9f576a9b24ae3a9f936b"}
Sep 20 01:52:46.292953 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:46.292868    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-p2wkn" event={"ID":"147f9cc3-d8b1-4c83-9a97-4b29246cd454","Type":"ContainerStarted","Data":"d492c06ec766e333f58be599a3a3d703292771b792022ca157f29e210fa4931f"}
Sep 20 01:52:46.292953 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:46.292942    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-p2wkn" event={"ID":"147f9cc3-d8b1-4c83-9a97-4b29246cd454","Type":"ContainerStarted","Data":"b8cb3a16b8a509eca202595a351e9dc5501252d313d340167b09d3997449f9f1"}
Sep 20 01:52:46.293060 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:46.292986    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-p2wkn" event={"ID":"147f9cc3-d8b1-4c83-9a97-4b29246cd454","Type":"ContainerStarted","Data":"9a88d37afd06851f18ef56161b8bb11c84e6390b06b005a5a25d647fdeb9ec9a"}
Sep 20 01:52:46.293060 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:46.293025    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-p2wkn" event={"ID":"147f9cc3-d8b1-4c83-9a97-4b29246cd454","Type":"ContainerStarted","Data":"a91416675f6d361fa183d393e9f960833bd51b77f18fac50fa129483cbd8d1ea"}
Sep 20 01:52:46.293156 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:46.293060    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-p2wkn" event={"ID":"147f9cc3-d8b1-4c83-9a97-4b29246cd454","Type":"ContainerStarted","Data":"4e161d6e4d52bca9b5a7a26692793f9fc9a0925f96167420344264872d75e3c7"}
Sep 20 01:52:46.294377 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:46.294348    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-l9ssk" event={"ID":"43dd43c0-ff48-4f9a-a1c5-9020836702aa","Type":"ContainerStarted","Data":"00ca4747122a4c3d5e14c9ea578368bcad37ad7200310ba836300286fb5a544c"}
Sep 20 01:52:46.295910 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:46.295885    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-159.ec2.internal" event={"ID":"4f739e7563e31c6ca101623c66d03faf","Type":"ContainerStarted","Data":"c416c94f8fe2ba46aa3273706b40deb6c986e6d28a69450640d145fdab85063e"}
Sep 20 01:52:46.297591 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:46.297560    2584 generic.go:356] "Generic (PLEG): container finished" podID="55f1df651e44bec56d534fcb469f62e8" containerID="b3ee66d041a3ee692e3e2a5d1a6d0057803edc430072700d65c33d2a6a432c29" exitCode=0
Sep 20 01:52:46.297673 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:46.297638    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-159.ec2.internal" event={"ID":"55f1df651e44bec56d534fcb469f62e8","Type":"ContainerDied","Data":"b3ee66d041a3ee692e3e2a5d1a6d0057803edc430072700d65c33d2a6a432c29"}
Sep 20 01:52:46.299650 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:46.299628    2584 generic.go:356] "Generic (PLEG): container finished" podID="96a97560-011d-4806-ae10-4d53880eb9ff" containerID="effe0abf10b2b1a0a28996498310890fa8a9fdcc1919622b97c26c29199b4f91" exitCode=0
Sep 20 01:52:46.299729 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:46.299662    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-f94fv" event={"ID":"96a97560-011d-4806-ae10-4d53880eb9ff","Type":"ContainerDied","Data":"effe0abf10b2b1a0a28996498310890fa8a9fdcc1919622b97c26c29199b4f91"}
Sep 20 01:52:46.309611 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:46.309552    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-v6d2k" podStartSLOduration=2.544809907 podStartE2EDuration="21.309537527s" totalImagesPullingTime="18.76472762s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-20 01:52:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-20 01:52:46.296265087 +0000 UTC m=+23.038944292" watchObservedRunningTime="2026-09-20 01:52:46.309537527 +0000 UTC m=+23.052216737"
Sep 20 01:52:46.324132 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:46.324086    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-m4zgq" podStartSLOduration=2.485394883 podStartE2EDuration="21.324071036s" totalImagesPullingTime="18.838676153s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-20 01:52:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-20 01:52:46.323939944 +0000 UTC m=+23.066619151" watchObservedRunningTime="2026-09-20 01:52:46.324071036 +0000 UTC m=+23.066750240"
Sep 20 01:52:46.324437 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:46.324403    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-l9ssk" podStartSLOduration=2.429073331 podStartE2EDuration="21.324390678s" totalImagesPullingTime="18.895317347s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-20 01:52:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-20 01:52:46.309274853 +0000 UTC m=+23.051954056" watchObservedRunningTime="2026-09-20 01:52:46.324390678 +0000 UTC m=+23.067069884"
Sep 20 01:52:46.341681 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:46.341353    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-159.ec2.internal" podStartSLOduration=21.341333963 podStartE2EDuration="21.341333963s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-20 01:52:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-20 01:52:46.341181419 +0000 UTC m=+23.083860626" watchObservedRunningTime="2026-09-20 01:52:46.341333963 +0000 UTC m=+23.084013169"
Sep 20 01:52:46.385551 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:46.385486    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-k8rn8" podStartSLOduration=2.374058692 podStartE2EDuration="21.385463891s" totalImagesPullingTime="19.011405199s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-20 01:52:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-20 01:52:46.365064088 +0000 UTC m=+23.107743295" watchObservedRunningTime="2026-09-20 01:52:46.385463891 +0000 UTC m=+23.128143098"
Sep 20 01:52:46.400018 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:46.399978    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-ftlkq" podStartSLOduration=2.555574385 podStartE2EDuration="21.399965468s" totalImagesPullingTime="18.844391083s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-20 01:52:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-20 01:52:46.385062965 +0000 UTC m=+23.127742170" watchObservedRunningTime="2026-09-20 01:52:46.399965468 +0000 UTC m=+23.142644673"
Sep 20 01:52:46.893553 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:46.893387    2584 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 20 01:52:47.087137 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:47.087022    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n4dk7"
Sep 20 01:52:47.087286 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:47.087156    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n4dk7" podUID="c9c657b9-b508-4bb7-99f3-aed2502cfba4"
Sep 20 01:52:47.100406 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:47.100318    2584 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-20T01:52:46.893549328Z","UUID":"74594c8c-8726-437c-bb8a-17ead4057030","Handler":null,"Name":"","Endpoint":""}
Sep 20 01:52:47.102185 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:47.102146    2584 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 20 01:52:47.102185 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:47.102183    2584 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 20 01:52:47.303936 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:47.303899    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-159.ec2.internal" event={"ID":"55f1df651e44bec56d534fcb469f62e8","Type":"ContainerStarted","Data":"97c111b5dbba13aac1e4e4688b7aef7c9331801a00037c1735cf75b91ef8f754"}
Sep 20 01:52:47.305954 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:47.305918    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-27wj7" event={"ID":"d0aee250-9aee-4473-ae29-83ffdbaeb920","Type":"ContainerStarted","Data":"0b7818e342ec9cb2fb11c697eaa0f57650c32798e97062c982a020e67378a8cf"}
Sep 20 01:52:47.307641 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:47.307557    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-chcxk" event={"ID":"d94badda-e0c0-4b65-a95f-64f524896542","Type":"ContainerStarted","Data":"2bcf01f6b06142d9314c16609c3eb0a34cf79298d41273cab4a1e2fa2dcdd0c0"}
Sep 20 01:52:47.318186 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:47.318136    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-159.ec2.internal" podStartSLOduration=22.318123135 podStartE2EDuration="22.318123135s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-20 01:52:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-20 01:52:47.317821556 +0000 UTC m=+24.060500762" watchObservedRunningTime="2026-09-20 01:52:47.318123135 +0000 UTC m=+24.060802339"
Sep 20 01:52:47.330836 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:47.330789    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-chcxk" podStartSLOduration=3.492441811 podStartE2EDuration="22.330774962s" totalImagesPullingTime="18.838333151s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-20 01:52:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-20 01:52:47.330381313 +0000 UTC m=+24.073060531" watchObservedRunningTime="2026-09-20 01:52:47.330774962 +0000 UTC m=+24.073454200"
Sep 20 01:52:48.017810 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:48.017774    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-ftlkq"
Sep 20 01:52:48.086550 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:48.086509    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-llv6h"
Sep 20 01:52:48.086677 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:48.086510    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-47994"
Sep 20 01:52:48.086754 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:48.086717    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-47994" podUID="9b376ae6-8f6a-4139-ab2d-96dc32e1201f"
Sep 20 01:52:48.086754 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:48.086655    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-llv6h" podUID="05a98348-758f-4a4a-b150-35bbdeb5b359"
Sep 20 01:52:48.140889 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:48.140859    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-ftlkq"
Sep 20 01:52:48.141563 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:48.141543    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-ftlkq"
Sep 20 01:52:48.312954 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:48.312911    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-27wj7" event={"ID":"d0aee250-9aee-4473-ae29-83ffdbaeb920","Type":"ContainerStarted","Data":"e9cdafc8fe79ee2d1c8dd6ba2e570f616c931615d689a5f65ab17ba2457adcb0"}
Sep 20 01:52:48.316940 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:48.316909    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-p2wkn" event={"ID":"147f9cc3-d8b1-4c83-9a97-4b29246cd454","Type":"ContainerStarted","Data":"831a35fba85323f5972625e6871773b7be0dcf077a280d93466898375f6b8010"}
Sep 20 01:52:48.317796 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:48.317775    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-ftlkq"
Sep 20 01:52:48.328988 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:48.328944    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-27wj7" podStartSLOduration=2.37615671 podStartE2EDuration="23.328928411s" totalImagesPullingTime="20.952771701s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-20 01:52:25 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-20 01:52:48.328785795 +0000 UTC m=+25.071465001" watchObservedRunningTime="2026-09-20 01:52:48.328928411 +0000 UTC m=+25.071607617"
Sep 20 01:52:49.086800 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:49.086770    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n4dk7"
Sep 20 01:52:49.086980 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:49.086893    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n4dk7" podUID="c9c657b9-b508-4bb7-99f3-aed2502cfba4"
Sep 20 01:52:50.087177 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:50.087134    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-47994"
Sep 20 01:52:50.087668 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:50.087182    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-llv6h"
Sep 20 01:52:50.087668 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:50.087275    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-47994" podUID="9b376ae6-8f6a-4139-ab2d-96dc32e1201f"
Sep 20 01:52:50.087668 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:50.087401    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-llv6h" podUID="05a98348-758f-4a4a-b150-35bbdeb5b359"
Sep 20 01:52:51.089850 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:51.089680    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n4dk7"
Sep 20 01:52:51.090171 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:51.089946    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n4dk7" podUID="c9c657b9-b508-4bb7-99f3-aed2502cfba4"
Sep 20 01:52:51.324877 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:51.324795    2584 generic.go:356] "Generic (PLEG): container finished" podID="96a97560-011d-4806-ae10-4d53880eb9ff" containerID="0c67e30205e31786aa6aa4348e15d7fddbc3fd31c18246e4c548df939abb9000" exitCode=0
Sep 20 01:52:51.325064 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:51.324885    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-f94fv" event={"ID":"96a97560-011d-4806-ae10-4d53880eb9ff","Type":"ContainerDied","Data":"0c67e30205e31786aa6aa4348e15d7fddbc3fd31c18246e4c548df939abb9000"}
Sep 20 01:52:51.328950 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:51.328928    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-p2wkn" event={"ID":"147f9cc3-d8b1-4c83-9a97-4b29246cd454","Type":"ContainerStarted","Data":"f5514c19fdb18175ee02814081da295f71a54e159e2d5582e20735de0d022b9a"}
Sep 20 01:52:51.329158 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:51.329145    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-p2wkn"
Sep 20 01:52:51.329212 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:51.329196    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-p2wkn"
Sep 20 01:52:51.343883 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:51.343865    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-p2wkn"
Sep 20 01:52:51.368490 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:51.368442    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-p2wkn" podStartSLOduration=7.473872789 podStartE2EDuration="26.368428327s" totalImagesPullingTime="18.894555538s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-20 01:52:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-20 01:52:51.368159089 +0000 UTC m=+28.110838294" watchObservedRunningTime="2026-09-20 01:52:51.368428327 +0000 UTC m=+28.111107532"
Sep 20 01:52:52.086646 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:52.086602    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-llv6h"
Sep 20 01:52:52.086646 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:52.086630    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-47994"
Sep 20 01:52:52.086947 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:52.086738    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-llv6h" podUID="05a98348-758f-4a4a-b150-35bbdeb5b359"
Sep 20 01:52:52.086947 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:52.086896    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-47994" podUID="9b376ae6-8f6a-4139-ab2d-96dc32e1201f"
Sep 20 01:52:52.331588 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:52.331557    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-p2wkn"
Sep 20 01:52:52.346883 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:52.346815    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-p2wkn"
Sep 20 01:52:52.773788 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:52.773753    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-47994"]
Sep 20 01:52:52.773950 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:52.773862    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-47994"
Sep 20 01:52:52.773998 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:52.773958    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-47994" podUID="9b376ae6-8f6a-4139-ab2d-96dc32e1201f"
Sep 20 01:52:52.777442 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:52.777414    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-llv6h"]
Sep 20 01:52:52.777602 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:52.777512    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-llv6h"
Sep 20 01:52:52.777661 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:52.777621    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-llv6h" podUID="05a98348-758f-4a4a-b150-35bbdeb5b359"
Sep 20 01:52:52.778018 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:52.777995    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-n4dk7"]
Sep 20 01:52:52.778143 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:52.778113    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n4dk7"
Sep 20 01:52:52.778213 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:52.778190    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n4dk7" podUID="c9c657b9-b508-4bb7-99f3-aed2502cfba4"
Sep 20 01:52:53.338231 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:53.338199    2584 generic.go:356] "Generic (PLEG): container finished" podID="96a97560-011d-4806-ae10-4d53880eb9ff" containerID="68f7c8ad3422d0ad51325c2e751f1014a67e764c36a032bb3216d4d98066c778" exitCode=0
Sep 20 01:52:53.338764 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:53.338279    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-f94fv" event={"ID":"96a97560-011d-4806-ae10-4d53880eb9ff","Type":"ContainerDied","Data":"68f7c8ad3422d0ad51325c2e751f1014a67e764c36a032bb3216d4d98066c778"}
Sep 20 01:52:54.086475 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:54.086430    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n4dk7"
Sep 20 01:52:54.086475 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:54.086463    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-47994"
Sep 20 01:52:54.086723 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:54.086463    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-llv6h"
Sep 20 01:52:54.086723 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:54.086613    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n4dk7" podUID="c9c657b9-b508-4bb7-99f3-aed2502cfba4"
Sep 20 01:52:54.086723 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:54.086684    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-47994" podUID="9b376ae6-8f6a-4139-ab2d-96dc32e1201f"
Sep 20 01:52:54.086875 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:54.086779    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-llv6h" podUID="05a98348-758f-4a4a-b150-35bbdeb5b359"
Sep 20 01:52:54.343191 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:54.343109    2584 generic.go:356] "Generic (PLEG): container finished" podID="96a97560-011d-4806-ae10-4d53880eb9ff" containerID="a16a9f86d9e46ad955d165cae7db5ec6d379e5656f6d86953331fb0e94e725ba" exitCode=0
Sep 20 01:52:54.343655 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:54.343182    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-f94fv" event={"ID":"96a97560-011d-4806-ae10-4d53880eb9ff","Type":"ContainerDied","Data":"a16a9f86d9e46ad955d165cae7db5ec6d379e5656f6d86953331fb0e94e725ba"}
Sep 20 01:52:56.086786 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:56.086620    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-llv6h"
Sep 20 01:52:56.087146 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:56.086639    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n4dk7"
Sep 20 01:52:56.087146 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:56.086862    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-llv6h" podUID="05a98348-758f-4a4a-b150-35bbdeb5b359"
Sep 20 01:52:56.087146 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:56.086683    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-47994"
Sep 20 01:52:56.087146 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:56.086951    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n4dk7" podUID="c9c657b9-b508-4bb7-99f3-aed2502cfba4"
Sep 20 01:52:56.087146 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:56.087012    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-47994" podUID="9b376ae6-8f6a-4139-ab2d-96dc32e1201f"
Sep 20 01:52:57.761240 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:57.761206    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/05a98348-758f-4a4a-b150-35bbdeb5b359-original-pull-secret\") pod \"global-pull-secret-syncer-llv6h\" (UID: \"05a98348-758f-4a4a-b150-35bbdeb5b359\") " pod="kube-system/global-pull-secret-syncer-llv6h"
Sep 20 01:52:57.761689 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:57.761253    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c9c657b9-b508-4bb7-99f3-aed2502cfba4-metrics-certs\") pod \"network-metrics-daemon-n4dk7\" (UID: \"c9c657b9-b508-4bb7-99f3-aed2502cfba4\") " pod="openshift-multus/network-metrics-daemon-n4dk7"
Sep 20 01:52:57.761689 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:57.761380    2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 20 01:52:57.761689 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:57.761448    2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 20 01:52:57.761689 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:57.761462    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/05a98348-758f-4a4a-b150-35bbdeb5b359-original-pull-secret podName:05a98348-758f-4a4a-b150-35bbdeb5b359 nodeName:}" failed. No retries permitted until 2026-09-20 01:53:29.761439829 +0000 UTC m=+66.504119026 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/05a98348-758f-4a4a-b150-35bbdeb5b359-original-pull-secret") pod "global-pull-secret-syncer-llv6h" (UID: "05a98348-758f-4a4a-b150-35bbdeb5b359") : object "kube-system"/"original-pull-secret" not registered
Sep 20 01:52:57.761689 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:57.761507    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c9c657b9-b508-4bb7-99f3-aed2502cfba4-metrics-certs podName:c9c657b9-b508-4bb7-99f3-aed2502cfba4 nodeName:}" failed. No retries permitted until 2026-09-20 01:53:29.761494383 +0000 UTC m=+66.504173585 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c9c657b9-b508-4bb7-99f3-aed2502cfba4-metrics-certs") pod "network-metrics-daemon-n4dk7" (UID: "c9c657b9-b508-4bb7-99f3-aed2502cfba4") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 20 01:52:57.862558 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:57.862507    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mrb2z\" (UniqueName: \"kubernetes.io/projected/9b376ae6-8f6a-4139-ab2d-96dc32e1201f-kube-api-access-mrb2z\") pod \"network-check-target-47994\" (UID: \"9b376ae6-8f6a-4139-ab2d-96dc32e1201f\") " pod="openshift-network-diagnostics/network-check-target-47994"
Sep 20 01:52:57.862715 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:57.862658    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 20 01:52:57.862715 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:57.862685    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 20 01:52:57.862715 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:57.862695    2584 projected.go:196] Error preparing data for projected volume kube-api-access-mrb2z for pod openshift-network-diagnostics/network-check-target-47994: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 20 01:52:57.862859 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:57.862758    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9b376ae6-8f6a-4139-ab2d-96dc32e1201f-kube-api-access-mrb2z podName:9b376ae6-8f6a-4139-ab2d-96dc32e1201f nodeName:}" failed. No retries permitted until 2026-09-20 01:53:29.862738394 +0000 UTC m=+66.605417590 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-mrb2z" (UniqueName: "kubernetes.io/projected/9b376ae6-8f6a-4139-ab2d-96dc32e1201f-kube-api-access-mrb2z") pod "network-check-target-47994" (UID: "9b376ae6-8f6a-4139-ab2d-96dc32e1201f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 20 01:52:58.086213 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.086133    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-47994"
Sep 20 01:52:58.086213 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.086155    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n4dk7"
Sep 20 01:52:58.086213 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.086155    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-llv6h"
Sep 20 01:52:58.086492 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:58.086257    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-47994" podUID="9b376ae6-8f6a-4139-ab2d-96dc32e1201f"
Sep 20 01:52:58.086492 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:58.086329    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n4dk7" podUID="c9c657b9-b508-4bb7-99f3-aed2502cfba4"
Sep 20 01:52:58.086492 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:58.086398    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-llv6h" podUID="05a98348-758f-4a4a-b150-35bbdeb5b359"
Sep 20 01:52:58.328298 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.328268    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-159.ec2.internal" event="NodeReady"
Sep 20 01:52:58.328470 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.328406    2584 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 20 01:52:58.361386 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.361306    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5659b6b8c9-hzb2w"]
Sep 20 01:52:58.393711 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.393433    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-7mld6"]
Sep 20 01:52:58.393711 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.393590    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5659b6b8c9-hzb2w"
Sep 20 01:52:58.397749 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.397723    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-k9d2x\""
Sep 20 01:52:58.400307 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.398343    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 20 01:52:58.405088 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.405045    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-hlwl5"]
Sep 20 01:52:58.405237 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.405220    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 20 01:52:58.406613 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.406093    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 20 01:52:58.407205 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.406932    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-7mld6"
Sep 20 01:52:58.407971 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.407783    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 20 01:52:58.410135 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.409945    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Sep 20 01:52:58.410135 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.409968    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Sep 20 01:52:58.410135 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.410021    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-h7d26\""
Sep 20 01:52:58.410135 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.410029    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Sep 20 01:52:58.415266 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.415245    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-6f9c644dd9-gm67p"]
Sep 20 01:52:58.415406 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.415390    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-hlwl5"
Sep 20 01:52:58.417458 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.417438    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Sep 20 01:52:58.417751 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.417731    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-8vlbp\""
Sep 20 01:52:58.417886 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.417843    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Sep 20 01:52:58.430438 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.430413    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-fd4bb46c-ztdt2"]
Sep 20 01:52:58.430645 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.430615    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-6f9c644dd9-gm67p"
Sep 20 01:52:58.434431 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.434410    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Sep 20 01:52:58.434633 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.434614    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-4zqxf\""
Sep 20 01:52:58.434809 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.434794    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Sep 20 01:52:58.435069 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.435048    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 20 01:52:58.435499 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.435479    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 20 01:52:58.442855 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.442762    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Sep 20 01:52:58.444440 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.444394    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-57cdc4b768-pxh5z"]
Sep 20 01:52:58.444592 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.444547    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-fd4bb46c-ztdt2"
Sep 20 01:52:58.446962 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.446939    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Sep 20 01:52:58.447489 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.447465    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Sep 20 01:52:58.447830 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.447475    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-kc68v\""
Sep 20 01:52:58.447830 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.447666    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Sep 20 01:52:58.447830 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.447493    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Sep 20 01:52:58.454900 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.454872    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Sep 20 01:52:58.459020 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.458998    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-2xts8"]
Sep 20 01:52:58.459138 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.459101    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-pxh5z"
Sep 20 01:52:58.461440 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.461421    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Sep 20 01:52:58.461683 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.461453    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Sep 20 01:52:58.461818 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.461457    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Sep 20 01:52:58.462060 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.462041    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-t9jxf\""
Sep 20 01:52:58.462141 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.462113    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Sep 20 01:52:58.466756 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.466734    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/492ce9d4-19e5-4e0c-8b86-fd63a3b456a1-registry-tls\") pod \"image-registry-5659b6b8c9-hzb2w\" (UID: \"492ce9d4-19e5-4e0c-8b86-fd63a3b456a1\") " pod="openshift-image-registry/image-registry-5659b6b8c9-hzb2w"
Sep 20 01:52:58.466843 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.466774    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/492ce9d4-19e5-4e0c-8b86-fd63a3b456a1-bound-sa-token\") pod \"image-registry-5659b6b8c9-hzb2w\" (UID: \"492ce9d4-19e5-4e0c-8b86-fd63a3b456a1\") " pod="openshift-image-registry/image-registry-5659b6b8c9-hzb2w"
Sep 20 01:52:58.466843 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.466810    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/492ce9d4-19e5-4e0c-8b86-fd63a3b456a1-image-registry-private-configuration\") pod \"image-registry-5659b6b8c9-hzb2w\" (UID: \"492ce9d4-19e5-4e0c-8b86-fd63a3b456a1\") " pod="openshift-image-registry/image-registry-5659b6b8c9-hzb2w"
Sep 20 01:52:58.466843 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.466829    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/492ce9d4-19e5-4e0c-8b86-fd63a3b456a1-registry-certificates\") pod \"image-registry-5659b6b8c9-hzb2w\" (UID: \"492ce9d4-19e5-4e0c-8b86-fd63a3b456a1\") " pod="openshift-image-registry/image-registry-5659b6b8c9-hzb2w"
Sep 20 01:52:58.466956 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.466929    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/492ce9d4-19e5-4e0c-8b86-fd63a3b456a1-ca-trust-extracted\") pod \"image-registry-5659b6b8c9-hzb2w\" (UID: \"492ce9d4-19e5-4e0c-8b86-fd63a3b456a1\") " pod="openshift-image-registry/image-registry-5659b6b8c9-hzb2w"
Sep 20 01:52:58.467014 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.466989    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8s62g\" (UniqueName: \"kubernetes.io/projected/492ce9d4-19e5-4e0c-8b86-fd63a3b456a1-kube-api-access-8s62g\") pod \"image-registry-5659b6b8c9-hzb2w\" (UID: \"492ce9d4-19e5-4e0c-8b86-fd63a3b456a1\") " pod="openshift-image-registry/image-registry-5659b6b8c9-hzb2w"
Sep 20 01:52:58.467051 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.467031    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5d22d853-7ae7-48b3-8549-5e2bf2be770d-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-7mld6\" (UID: \"5d22d853-7ae7-48b3-8549-5e2bf2be770d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-7mld6"
Sep 20 01:52:58.467085 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.467048    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/492ce9d4-19e5-4e0c-8b86-fd63a3b456a1-installation-pull-secrets\") pod \"image-registry-5659b6b8c9-hzb2w\" (UID: \"492ce9d4-19e5-4e0c-8b86-fd63a3b456a1\") " pod="openshift-image-registry/image-registry-5659b6b8c9-hzb2w"
Sep 20 01:52:58.467085 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.467079    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/492ce9d4-19e5-4e0c-8b86-fd63a3b456a1-trusted-ca\") pod \"image-registry-5659b6b8c9-hzb2w\" (UID: \"492ce9d4-19e5-4e0c-8b86-fd63a3b456a1\") " pod="openshift-image-registry/image-registry-5659b6b8c9-hzb2w"
Sep 20 01:52:58.467206 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.467186    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mp5v5\" (UniqueName: \"kubernetes.io/projected/5d22d853-7ae7-48b3-8549-5e2bf2be770d-kube-api-access-mp5v5\") pod \"cluster-samples-operator-685d7d68b5-7mld6\" (UID: \"5d22d853-7ae7-48b3-8549-5e2bf2be770d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-7mld6"
Sep 20 01:52:58.470179 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.470141    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-bp5fk"]
Sep 20 01:52:58.470288 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.470272    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-2xts8"
Sep 20 01:52:58.472623 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.472604    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Sep 20 01:52:58.472737 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.472714    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-cw64p\""
Sep 20 01:52:58.472894 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.472869    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Sep 20 01:52:58.472998 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.472963    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Sep 20 01:52:58.473062 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.473007    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Sep 20 01:52:58.484258 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.484236    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-95688946b-7rdfv"]
Sep 20 01:52:58.484391 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.484361    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-bp5fk"
Sep 20 01:52:58.488417 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.488399    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 20 01:52:58.488558 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.488424    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-fm975\""
Sep 20 01:52:58.488558 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.488486    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Sep 20 01:52:58.488558 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.488547    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Sep 20 01:52:58.488729 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.488406    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 20 01:52:58.497848 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.497825    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5659b6b8c9-hzb2w"]
Sep 20 01:52:58.497976 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.497863    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-7mld6"]
Sep 20 01:52:58.497976 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.497890    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-57cdc4b768-pxh5z"]
Sep 20 01:52:58.497976 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.497911    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-hlwl5"]
Sep 20 01:52:58.497976 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.497932    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-2xts8"]
Sep 20 01:52:58.497976 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.497949    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-bp5fk"]
Sep 20 01:52:58.497976 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.497966    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-fd4bb46c-ztdt2"]
Sep 20 01:52:58.498250 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.497984    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-6f9c644dd9-gm67p"]
Sep 20 01:52:58.498250 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.498006    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-4dsc5"]
Sep 20 01:52:58.498250 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.498060    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-95688946b-7rdfv"
Sep 20 01:52:58.500077 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.500046    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Sep 20 01:52:58.500259 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.500080    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Sep 20 01:52:58.500259 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.500205    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Sep 20 01:52:58.500623 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.500543    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Sep 20 01:52:58.500623 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.500555    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 20 01:52:58.500623 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.500565    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 20 01:52:58.500867 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.500847    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-9j749\""
Sep 20 01:52:58.510278 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.510253    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6446c8d75d-fktjk"]
Sep 20 01:52:58.510419 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.510382    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-4dsc5"
Sep 20 01:52:58.513338 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.513290    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 20 01:52:58.513338 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.513308    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 20 01:52:58.513576 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.513391    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-w9zwc\""
Sep 20 01:52:58.513576 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.513418    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 20 01:52:58.524303 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.524266    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7cbc45fddc-q2gls"]
Sep 20 01:52:58.524431 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.524398    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6446c8d75d-fktjk"
Sep 20 01:52:58.526939 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.526760    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Sep 20 01:52:58.526939 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.526778    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Sep 20 01:52:58.526939 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.526912    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-x2q4j\""
Sep 20 01:52:58.537761 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.537739    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-c9zbv"]
Sep 20 01:52:58.537919 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.537874    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-q2gls"
Sep 20 01:52:58.540703 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.540679    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 20 01:52:58.540703 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.540696    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-s7q4n\""
Sep 20 01:52:58.540896 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.540766    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 20 01:52:58.552054 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.552030    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-848dd94bd6-xfg9p"]
Sep 20 01:52:58.552199 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.552161    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-c9zbv"
Sep 20 01:52:58.554703 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.554681    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-fp8gx\""
Sep 20 01:52:58.555955 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.555757    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 20 01:52:58.555955 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.555906    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 20 01:52:58.568038 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.568008    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a6fdf198-e4cd-4267-92fd-0ebee2f97522-serving-cert\") pod \"insights-operator-6f9c644dd9-gm67p\" (UID: \"a6fdf198-e4cd-4267-92fd-0ebee2f97522\") " pod="openshift-insights/insights-operator-6f9c644dd9-gm67p"
Sep 20 01:52:58.568223 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.568053    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b46ng\" (UniqueName: \"kubernetes.io/projected/2dbc9737-865d-4dbf-85bf-8d2b9781a8d5-kube-api-access-b46ng\") pod \"router-default-95688946b-7rdfv\" (UID: \"2dbc9737-865d-4dbf-85bf-8d2b9781a8d5\") " pod="openshift-ingress/router-default-95688946b-7rdfv"
Sep 20 01:52:58.568223 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.568077    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2dbc9737-865d-4dbf-85bf-8d2b9781a8d5-service-ca-bundle\") pod \"router-default-95688946b-7rdfv\" (UID: \"2dbc9737-865d-4dbf-85bf-8d2b9781a8d5\") " pod="openshift-ingress/router-default-95688946b-7rdfv"
Sep 20 01:52:58.568223 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.568137    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/492ce9d4-19e5-4e0c-8b86-fd63a3b456a1-image-registry-private-configuration\") pod \"image-registry-5659b6b8c9-hzb2w\" (UID: \"492ce9d4-19e5-4e0c-8b86-fd63a3b456a1\") " pod="openshift-image-registry/image-registry-5659b6b8c9-hzb2w"
Sep 20 01:52:58.568223 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.568183    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ftjb6\" (UniqueName: \"kubernetes.io/projected/261429fc-d268-4632-9767-2de6fe66a3a4-kube-api-access-ftjb6\") pod \"service-ca-operator-57cdc4b768-pxh5z\" (UID: \"261429fc-d268-4632-9767-2de6fe66a3a4\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-pxh5z"
Sep 20 01:52:58.568223 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.568223    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/492ce9d4-19e5-4e0c-8b86-fd63a3b456a1-ca-trust-extracted\") pod \"image-registry-5659b6b8c9-hzb2w\" (UID: \"492ce9d4-19e5-4e0c-8b86-fd63a3b456a1\") " pod="openshift-image-registry/image-registry-5659b6b8c9-hzb2w"
Sep 20 01:52:58.568501 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.568247    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a6fdf198-e4cd-4267-92fd-0ebee2f97522-tmp\") pod \"insights-operator-6f9c644dd9-gm67p\" (UID: \"a6fdf198-e4cd-4267-92fd-0ebee2f97522\") " pod="openshift-insights/insights-operator-6f9c644dd9-gm67p"
Sep 20 01:52:58.568501 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.568269    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5d22d853-7ae7-48b3-8549-5e2bf2be770d-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-7mld6\" (UID: \"5d22d853-7ae7-48b3-8549-5e2bf2be770d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-7mld6"
Sep 20 01:52:58.568501 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.568288    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/492ce9d4-19e5-4e0c-8b86-fd63a3b456a1-installation-pull-secrets\") pod \"image-registry-5659b6b8c9-hzb2w\" (UID: \"492ce9d4-19e5-4e0c-8b86-fd63a3b456a1\") " pod="openshift-image-registry/image-registry-5659b6b8c9-hzb2w"
Sep 20 01:52:58.568501 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.568307    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/492ce9d4-19e5-4e0c-8b86-fd63a3b456a1-trusted-ca\") pod \"image-registry-5659b6b8c9-hzb2w\" (UID: \"492ce9d4-19e5-4e0c-8b86-fd63a3b456a1\") " pod="openshift-image-registry/image-registry-5659b6b8c9-hzb2w"
Sep 20 01:52:58.568501 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.568327    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ece82f19-68b5-4810-be50-d8678cfdee02-serving-cert\") pod \"console-operator-fd4bb46c-ztdt2\" (UID: \"ece82f19-68b5-4810-be50-d8678cfdee02\") " pod="openshift-console-operator/console-operator-fd4bb46c-ztdt2"
Sep 20 01:52:58.568501 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.568355    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/492ce9d4-19e5-4e0c-8b86-fd63a3b456a1-registry-tls\") pod \"image-registry-5659b6b8c9-hzb2w\" (UID: \"492ce9d4-19e5-4e0c-8b86-fd63a3b456a1\") " pod="openshift-image-registry/image-registry-5659b6b8c9-hzb2w"
Sep 20 01:52:58.568501 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.568379    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/006f93ba-2bd5-48cb-b52d-2e288401cd0f-serving-cert\") pod \"kube-storage-version-migrator-operator-55889f7b47-2xts8\" (UID: \"006f93ba-2bd5-48cb-b52d-2e288401cd0f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-2xts8"
Sep 20 01:52:58.568501 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.568409    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kb8sw\" (UniqueName: \"kubernetes.io/projected/ece82f19-68b5-4810-be50-d8678cfdee02-kube-api-access-kb8sw\") pod \"console-operator-fd4bb46c-ztdt2\" (UID: \"ece82f19-68b5-4810-be50-d8678cfdee02\") " pod="openshift-console-operator/console-operator-fd4bb46c-ztdt2"
Sep 20 01:52:58.568501 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:58.568442    2584 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 20 01:52:58.568501 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.568455    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k8xrc\" (UniqueName: \"kubernetes.io/projected/092d6fa5-0734-4777-b6e0-2e1ef430076d-kube-api-access-k8xrc\") pod \"cluster-monitoring-operator-847b8bd8d7-bp5fk\" (UID: \"092d6fa5-0734-4777-b6e0-2e1ef430076d\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-bp5fk"
Sep 20 01:52:58.569033 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:58.568509    2584 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 20 01:52:58.569033 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:58.568550    2584 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5659b6b8c9-hzb2w: secret "image-registry-tls" not found
Sep 20 01:52:58.569033 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:58.568568    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5d22d853-7ae7-48b3-8549-5e2bf2be770d-samples-operator-tls podName:5d22d853-7ae7-48b3-8549-5e2bf2be770d nodeName:}" failed. No retries permitted until 2026-09-20 01:52:59.06851017 +0000 UTC m=+35.811189379 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/5d22d853-7ae7-48b3-8549-5e2bf2be770d-samples-operator-tls") pod "cluster-samples-operator-685d7d68b5-7mld6" (UID: "5d22d853-7ae7-48b3-8549-5e2bf2be770d") : secret "samples-operator-tls" not found
Sep 20 01:52:58.569033 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:58.568630    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/492ce9d4-19e5-4e0c-8b86-fd63a3b456a1-registry-tls podName:492ce9d4-19e5-4e0c-8b86-fd63a3b456a1 nodeName:}" failed. No retries permitted until 2026-09-20 01:52:59.068615311 +0000 UTC m=+35.811294501 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/492ce9d4-19e5-4e0c-8b86-fd63a3b456a1-registry-tls") pod "image-registry-5659b6b8c9-hzb2w" (UID: "492ce9d4-19e5-4e0c-8b86-fd63a3b456a1") : secret "image-registry-tls" not found
Sep 20 01:52:58.569033 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.568656    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/492ce9d4-19e5-4e0c-8b86-fd63a3b456a1-bound-sa-token\") pod \"image-registry-5659b6b8c9-hzb2w\" (UID: \"492ce9d4-19e5-4e0c-8b86-fd63a3b456a1\") " pod="openshift-image-registry/image-registry-5659b6b8c9-hzb2w"
Sep 20 01:52:58.569033 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.568683    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5dn9m\" (UniqueName: \"kubernetes.io/projected/a2305e14-b41d-45f8-86ce-c99c92938ba3-kube-api-access-5dn9m\") pod \"volume-data-source-validator-5c4db8bdf9-hlwl5\" (UID: \"a2305e14-b41d-45f8-86ce-c99c92938ba3\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-hlwl5"
Sep 20 01:52:58.569033 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.568704    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a6fdf198-e4cd-4267-92fd-0ebee2f97522-service-ca-bundle\") pod \"insights-operator-6f9c644dd9-gm67p\" (UID: \"a6fdf198-e4cd-4267-92fd-0ebee2f97522\") " pod="openshift-insights/insights-operator-6f9c644dd9-gm67p"
Sep 20 01:52:58.569033 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.568724    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/006f93ba-2bd5-48cb-b52d-2e288401cd0f-config\") pod \"kube-storage-version-migrator-operator-55889f7b47-2xts8\" (UID: \"006f93ba-2bd5-48cb-b52d-2e288401cd0f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-2xts8"
Sep 20 01:52:58.569033 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.568747    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nc4fw\" (UniqueName: \"kubernetes.io/projected/006f93ba-2bd5-48cb-b52d-2e288401cd0f-kube-api-access-nc4fw\") pod \"kube-storage-version-migrator-operator-55889f7b47-2xts8\" (UID: \"006f93ba-2bd5-48cb-b52d-2e288401cd0f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-2xts8"
Sep 20 01:52:58.569033 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.568780    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jnd46\" (UniqueName: \"kubernetes.io/projected/a6fdf198-e4cd-4267-92fd-0ebee2f97522-kube-api-access-jnd46\") pod \"insights-operator-6f9c644dd9-gm67p\" (UID: \"a6fdf198-e4cd-4267-92fd-0ebee2f97522\") " pod="openshift-insights/insights-operator-6f9c644dd9-gm67p"
Sep 20 01:52:58.569033 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.568804    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/261429fc-d268-4632-9767-2de6fe66a3a4-serving-cert\") pod \"service-ca-operator-57cdc4b768-pxh5z\" (UID: \"261429fc-d268-4632-9767-2de6fe66a3a4\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-pxh5z"
Sep 20 01:52:58.569033 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.568838    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/492ce9d4-19e5-4e0c-8b86-fd63a3b456a1-registry-certificates\") pod \"image-registry-5659b6b8c9-hzb2w\" (UID: \"492ce9d4-19e5-4e0c-8b86-fd63a3b456a1\") " pod="openshift-image-registry/image-registry-5659b6b8c9-hzb2w"
Sep 20 01:52:58.569033 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.568931    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/a6fdf198-e4cd-4267-92fd-0ebee2f97522-snapshots\") pod \"insights-operator-6f9c644dd9-gm67p\" (UID: \"a6fdf198-e4cd-4267-92fd-0ebee2f97522\") " pod="openshift-insights/insights-operator-6f9c644dd9-gm67p"
Sep 20 01:52:58.569033 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.568964    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/092d6fa5-0734-4777-b6e0-2e1ef430076d-telemetry-config\") pod \"cluster-monitoring-operator-847b8bd8d7-bp5fk\" (UID: \"092d6fa5-0734-4777-b6e0-2e1ef430076d\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-bp5fk"
Sep 20 01:52:58.569033 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.568988    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2dbc9737-865d-4dbf-85bf-8d2b9781a8d5-metrics-certs\") pod \"router-default-95688946b-7rdfv\" (UID: \"2dbc9737-865d-4dbf-85bf-8d2b9781a8d5\") " pod="openshift-ingress/router-default-95688946b-7rdfv"
Sep 20 01:52:58.569668 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.569016    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a6fdf198-e4cd-4267-92fd-0ebee2f97522-trusted-ca-bundle\") pod \"insights-operator-6f9c644dd9-gm67p\" (UID: \"a6fdf198-e4cd-4267-92fd-0ebee2f97522\") " pod="openshift-insights/insights-operator-6f9c644dd9-gm67p"
Sep 20 01:52:58.569668 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.569038    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/092d6fa5-0734-4777-b6e0-2e1ef430076d-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-847b8bd8d7-bp5fk\" (UID: \"092d6fa5-0734-4777-b6e0-2e1ef430076d\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-bp5fk"
Sep 20 01:52:58.569668 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.569058    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/2dbc9737-865d-4dbf-85bf-8d2b9781a8d5-default-certificate\") pod \"router-default-95688946b-7rdfv\" (UID: \"2dbc9737-865d-4dbf-85bf-8d2b9781a8d5\") " pod="openshift-ingress/router-default-95688946b-7rdfv"
Sep 20 01:52:58.569668 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.569099    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8s62g\" (UniqueName: \"kubernetes.io/projected/492ce9d4-19e5-4e0c-8b86-fd63a3b456a1-kube-api-access-8s62g\") pod \"image-registry-5659b6b8c9-hzb2w\" (UID: \"492ce9d4-19e5-4e0c-8b86-fd63a3b456a1\") " pod="openshift-image-registry/image-registry-5659b6b8c9-hzb2w"
Sep 20 01:52:58.569668 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.569121    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/261429fc-d268-4632-9767-2de6fe66a3a4-config\") pod \"service-ca-operator-57cdc4b768-pxh5z\" (UID: \"261429fc-d268-4632-9767-2de6fe66a3a4\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-pxh5z"
Sep 20 01:52:58.569668 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.569149    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ece82f19-68b5-4810-be50-d8678cfdee02-config\") pod \"console-operator-fd4bb46c-ztdt2\" (UID: \"ece82f19-68b5-4810-be50-d8678cfdee02\") " pod="openshift-console-operator/console-operator-fd4bb46c-ztdt2"
Sep 20 01:52:58.569668 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.569174    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/2dbc9737-865d-4dbf-85bf-8d2b9781a8d5-stats-auth\") pod \"router-default-95688946b-7rdfv\" (UID: \"2dbc9737-865d-4dbf-85bf-8d2b9781a8d5\") " pod="openshift-ingress/router-default-95688946b-7rdfv"
Sep 20 01:52:58.569668 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.569232    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mp5v5\" (UniqueName: \"kubernetes.io/projected/5d22d853-7ae7-48b3-8549-5e2bf2be770d-kube-api-access-mp5v5\") pod \"cluster-samples-operator-685d7d68b5-7mld6\" (UID: \"5d22d853-7ae7-48b3-8549-5e2bf2be770d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-7mld6"
Sep 20 01:52:58.569668 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.569294    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ece82f19-68b5-4810-be50-d8678cfdee02-trusted-ca\") pod \"console-operator-fd4bb46c-ztdt2\" (UID: \"ece82f19-68b5-4810-be50-d8678cfdee02\") " pod="openshift-console-operator/console-operator-fd4bb46c-ztdt2"
Sep 20 01:52:58.569668 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.569446    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/492ce9d4-19e5-4e0c-8b86-fd63a3b456a1-ca-trust-extracted\") pod \"image-registry-5659b6b8c9-hzb2w\" (UID: \"492ce9d4-19e5-4e0c-8b86-fd63a3b456a1\") " pod="openshift-image-registry/image-registry-5659b6b8c9-hzb2w"
Sep 20 01:52:58.569668 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.569474    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/492ce9d4-19e5-4e0c-8b86-fd63a3b456a1-registry-certificates\") pod \"image-registry-5659b6b8c9-hzb2w\" (UID: \"492ce9d4-19e5-4e0c-8b86-fd63a3b456a1\") " pod="openshift-image-registry/image-registry-5659b6b8c9-hzb2w"
Sep 20 01:52:58.570166 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.570145    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/492ce9d4-19e5-4e0c-8b86-fd63a3b456a1-trusted-ca\") pod \"image-registry-5659b6b8c9-hzb2w\" (UID: \"492ce9d4-19e5-4e0c-8b86-fd63a3b456a1\") " pod="openshift-image-registry/image-registry-5659b6b8c9-hzb2w"
Sep 20 01:52:58.570305 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.570261    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77cf88f65f-9qp8b"]
Sep 20 01:52:58.570470 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.570353    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-848dd94bd6-xfg9p"
Sep 20 01:52:58.573545 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.573076    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Sep 20 01:52:58.573545 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.573174    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Sep 20 01:52:58.573545 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.573193    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-cwh5q\""
Sep 20 01:52:58.573545 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.573345    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/492ce9d4-19e5-4e0c-8b86-fd63a3b456a1-image-registry-private-configuration\") pod \"image-registry-5659b6b8c9-hzb2w\" (UID: \"492ce9d4-19e5-4e0c-8b86-fd63a3b456a1\") " pod="openshift-image-registry/image-registry-5659b6b8c9-hzb2w"
Sep 20 01:52:58.573545 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.573382    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Sep 20 01:52:58.573545 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.573539    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Sep 20 01:52:58.574549 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.574512    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/492ce9d4-19e5-4e0c-8b86-fd63a3b456a1-installation-pull-secrets\") pod \"image-registry-5659b6b8c9-hzb2w\" (UID: \"492ce9d4-19e5-4e0c-8b86-fd63a3b456a1\") " pod="openshift-image-registry/image-registry-5659b6b8c9-hzb2w"
Sep 20 01:52:58.582716 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.582664    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/492ce9d4-19e5-4e0c-8b86-fd63a3b456a1-bound-sa-token\") pod \"image-registry-5659b6b8c9-hzb2w\" (UID: \"492ce9d4-19e5-4e0c-8b86-fd63a3b456a1\") " pod="openshift-image-registry/image-registry-5659b6b8c9-hzb2w"
Sep 20 01:52:58.582853 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.582824    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mp5v5\" (UniqueName: \"kubernetes.io/projected/5d22d853-7ae7-48b3-8549-5e2bf2be770d-kube-api-access-mp5v5\") pod \"cluster-samples-operator-685d7d68b5-7mld6\" (UID: \"5d22d853-7ae7-48b3-8549-5e2bf2be770d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-7mld6"
Sep 20 01:52:58.582853 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.582840    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8s62g\" (UniqueName: \"kubernetes.io/projected/492ce9d4-19e5-4e0c-8b86-fd63a3b456a1-kube-api-access-8s62g\") pod \"image-registry-5659b6b8c9-hzb2w\" (UID: \"492ce9d4-19e5-4e0c-8b86-fd63a3b456a1\") " pod="openshift-image-registry/image-registry-5659b6b8c9-hzb2w"
Sep 20 01:52:58.591765 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.591713    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-858b56ff6c-5f29j"]
Sep 20 01:52:58.592164 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.591879    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77cf88f65f-9qp8b"
Sep 20 01:52:58.594181 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.594157    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Sep 20 01:52:58.594720 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.594427    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Sep 20 01:52:58.594720 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.594492    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Sep 20 01:52:58.594720 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.594564    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Sep 20 01:52:58.602893 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.602858    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-4dsc5"]
Sep 20 01:52:58.603018 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.602900    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6446c8d75d-fktjk"]
Sep 20 01:52:58.603018 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.602923    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7cbc45fddc-q2gls"]
Sep 20 01:52:58.603018 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.602940    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-858b56ff6c-5f29j"]
Sep 20 01:52:58.603018 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.602956    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-c9zbv"]
Sep 20 01:52:58.603018 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.602978    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-858b56ff6c-5f29j"
Sep 20 01:52:58.603246 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.602984    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-95688946b-7rdfv"]
Sep 20 01:52:58.603246 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.603124    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-848dd94bd6-xfg9p"]
Sep 20 01:52:58.603246 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.603144    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77cf88f65f-9qp8b"]
Sep 20 01:52:58.605352 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.605328    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Sep 20 01:52:58.611218 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.611195    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:52:58.670020 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.669989    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a6fdf198-e4cd-4267-92fd-0ebee2f97522-serving-cert\") pod \"insights-operator-6f9c644dd9-gm67p\" (UID: \"a6fdf198-e4cd-4267-92fd-0ebee2f97522\") " pod="openshift-insights/insights-operator-6f9c644dd9-gm67p"
Sep 20 01:52:58.670213 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.670031    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/536c3d86-41b0-4e73-9eed-7acbd8b86920-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-77cf88f65f-9qp8b\" (UID: \"536c3d86-41b0-4e73-9eed-7acbd8b86920\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77cf88f65f-9qp8b"
Sep 20 01:52:58.670213 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.670060    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lbhph\" (UniqueName: \"kubernetes.io/projected/536c3d86-41b0-4e73-9eed-7acbd8b86920-kube-api-access-lbhph\") pod \"cluster-proxy-proxy-agent-77cf88f65f-9qp8b\" (UID: \"536c3d86-41b0-4e73-9eed-7acbd8b86920\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77cf88f65f-9qp8b"
Sep 20 01:52:58.670213 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.670082    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2075981b-d064-43eb-ab90-373b95ad49f4-cert\") pod \"ingress-canary-4dsc5\" (UID: \"2075981b-d064-43eb-ab90-373b95ad49f4\") " pod="openshift-ingress-canary/ingress-canary-4dsc5"
Sep 20 01:52:58.670213 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.670111    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b46ng\" (UniqueName: \"kubernetes.io/projected/2dbc9737-865d-4dbf-85bf-8d2b9781a8d5-kube-api-access-b46ng\") pod \"router-default-95688946b-7rdfv\" (UID: \"2dbc9737-865d-4dbf-85bf-8d2b9781a8d5\") " pod="openshift-ingress/router-default-95688946b-7rdfv"
Sep 20 01:52:58.670213 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.670177    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2dbc9737-865d-4dbf-85bf-8d2b9781a8d5-service-ca-bundle\") pod \"router-default-95688946b-7rdfv\" (UID: \"2dbc9737-865d-4dbf-85bf-8d2b9781a8d5\") " pod="openshift-ingress/router-default-95688946b-7rdfv"
Sep 20 01:52:58.670462 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.670258    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/a3c5d94e-85f4-41f9-9743-dd7fd1c38b98-klusterlet-config\") pod \"klusterlet-addon-workmgr-858b56ff6c-5f29j\" (UID: \"a3c5d94e-85f4-41f9-9743-dd7fd1c38b98\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-858b56ff6c-5f29j"
Sep 20 01:52:58.670462 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:58.670312    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2dbc9737-865d-4dbf-85bf-8d2b9781a8d5-service-ca-bundle podName:2dbc9737-865d-4dbf-85bf-8d2b9781a8d5 nodeName:}" failed. No retries permitted until 2026-09-20 01:52:59.170294107 +0000 UTC m=+35.912973291 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/2dbc9737-865d-4dbf-85bf-8d2b9781a8d5-service-ca-bundle") pod "router-default-95688946b-7rdfv" (UID: "2dbc9737-865d-4dbf-85bf-8d2b9781a8d5") : configmap references non-existent config key: service-ca.crt
Sep 20 01:52:58.670462 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.670335    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ftjb6\" (UniqueName: \"kubernetes.io/projected/261429fc-d268-4632-9767-2de6fe66a3a4-kube-api-access-ftjb6\") pod \"service-ca-operator-57cdc4b768-pxh5z\" (UID: \"261429fc-d268-4632-9767-2de6fe66a3a4\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-pxh5z"
Sep 20 01:52:58.670462 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.670368    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/536c3d86-41b0-4e73-9eed-7acbd8b86920-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-77cf88f65f-9qp8b\" (UID: \"536c3d86-41b0-4e73-9eed-7acbd8b86920\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77cf88f65f-9qp8b"
Sep 20 01:52:58.670462 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.670393    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g494p\" (UniqueName: \"kubernetes.io/projected/a3c5d94e-85f4-41f9-9743-dd7fd1c38b98-kube-api-access-g494p\") pod \"klusterlet-addon-workmgr-858b56ff6c-5f29j\" (UID: \"a3c5d94e-85f4-41f9-9743-dd7fd1c38b98\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-858b56ff6c-5f29j"
Sep 20 01:52:58.670462 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.670423    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a6fdf198-e4cd-4267-92fd-0ebee2f97522-tmp\") pod \"insights-operator-6f9c644dd9-gm67p\" (UID: \"a6fdf198-e4cd-4267-92fd-0ebee2f97522\") " pod="openshift-insights/insights-operator-6f9c644dd9-gm67p"
Sep 20 01:52:58.670462 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.670451    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/54d123d3-fc8b-49ee-9e26-d4a3ee494a1d-networking-console-plugin-cert\") pod \"networking-console-plugin-6446c8d75d-fktjk\" (UID: \"54d123d3-fc8b-49ee-9e26-d4a3ee494a1d\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-fktjk"
Sep 20 01:52:58.670854 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.670476    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8lwx4\" (UniqueName: \"kubernetes.io/projected/4de13b31-81d2-45d8-977f-2a59314b499d-kube-api-access-8lwx4\") pod \"managed-serviceaccount-addon-agent-848dd94bd6-xfg9p\" (UID: \"4de13b31-81d2-45d8-977f-2a59314b499d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-848dd94bd6-xfg9p"
Sep 20 01:52:58.670854 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.670502    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-htczw\" (UniqueName: \"kubernetes.io/projected/79e39187-4356-422e-b5d3-9c9f8d3a9a58-kube-api-access-htczw\") pod \"dns-default-c9zbv\" (UID: \"79e39187-4356-422e-b5d3-9c9f8d3a9a58\") " pod="openshift-dns/dns-default-c9zbv"
Sep 20 01:52:58.670854 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.670538    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a3c5d94e-85f4-41f9-9743-dd7fd1c38b98-tmp\") pod \"klusterlet-addon-workmgr-858b56ff6c-5f29j\" (UID: \"a3c5d94e-85f4-41f9-9743-dd7fd1c38b98\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-858b56ff6c-5f29j"
Sep 20 01:52:58.670854 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.670582    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/536c3d86-41b0-4e73-9eed-7acbd8b86920-hub\") pod \"cluster-proxy-proxy-agent-77cf88f65f-9qp8b\" (UID: \"536c3d86-41b0-4e73-9eed-7acbd8b86920\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77cf88f65f-9qp8b"
Sep 20 01:52:58.670854 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.670606    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ece82f19-68b5-4810-be50-d8678cfdee02-serving-cert\") pod \"console-operator-fd4bb46c-ztdt2\" (UID: \"ece82f19-68b5-4810-be50-d8678cfdee02\") " pod="openshift-console-operator/console-operator-fd4bb46c-ztdt2"
Sep 20 01:52:58.670854 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.670680    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/006f93ba-2bd5-48cb-b52d-2e288401cd0f-serving-cert\") pod \"kube-storage-version-migrator-operator-55889f7b47-2xts8\" (UID: \"006f93ba-2bd5-48cb-b52d-2e288401cd0f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-2xts8"
Sep 20 01:52:58.670854 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.670719    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kb8sw\" (UniqueName: \"kubernetes.io/projected/ece82f19-68b5-4810-be50-d8678cfdee02-kube-api-access-kb8sw\") pod \"console-operator-fd4bb46c-ztdt2\" (UID: \"ece82f19-68b5-4810-be50-d8678cfdee02\") " pod="openshift-console-operator/console-operator-fd4bb46c-ztdt2"
Sep 20 01:52:58.670854 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.670750    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k8xrc\" (UniqueName: \"kubernetes.io/projected/092d6fa5-0734-4777-b6e0-2e1ef430076d-kube-api-access-k8xrc\") pod \"cluster-monitoring-operator-847b8bd8d7-bp5fk\" (UID: \"092d6fa5-0734-4777-b6e0-2e1ef430076d\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-bp5fk"
Sep 20 01:52:58.670854 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.670776    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/79e39187-4356-422e-b5d3-9c9f8d3a9a58-config-volume\") pod \"dns-default-c9zbv\" (UID: \"79e39187-4356-422e-b5d3-9c9f8d3a9a58\") " pod="openshift-dns/dns-default-c9zbv"
Sep 20 01:52:58.670854 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.670796    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5dn9m\" (UniqueName: \"kubernetes.io/projected/a2305e14-b41d-45f8-86ce-c99c92938ba3-kube-api-access-5dn9m\") pod \"volume-data-source-validator-5c4db8bdf9-hlwl5\" (UID: \"a2305e14-b41d-45f8-86ce-c99c92938ba3\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-hlwl5"
Sep 20 01:52:58.670854 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.670815    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a6fdf198-e4cd-4267-92fd-0ebee2f97522-service-ca-bundle\") pod \"insights-operator-6f9c644dd9-gm67p\" (UID: \"a6fdf198-e4cd-4267-92fd-0ebee2f97522\") " pod="openshift-insights/insights-operator-6f9c644dd9-gm67p"
Sep 20 01:52:58.670854 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.670833    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/006f93ba-2bd5-48cb-b52d-2e288401cd0f-config\") pod \"kube-storage-version-migrator-operator-55889f7b47-2xts8\" (UID: \"006f93ba-2bd5-48cb-b52d-2e288401cd0f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-2xts8"
Sep 20 01:52:58.670854 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.670850    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nc4fw\" (UniqueName: \"kubernetes.io/projected/006f93ba-2bd5-48cb-b52d-2e288401cd0f-kube-api-access-nc4fw\") pod \"kube-storage-version-migrator-operator-55889f7b47-2xts8\" (UID: \"006f93ba-2bd5-48cb-b52d-2e288401cd0f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-2xts8"
Sep 20 01:52:58.671463 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.670875    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jnd46\" (UniqueName: \"kubernetes.io/projected/a6fdf198-e4cd-4267-92fd-0ebee2f97522-kube-api-access-jnd46\") pod \"insights-operator-6f9c644dd9-gm67p\" (UID: \"a6fdf198-e4cd-4267-92fd-0ebee2f97522\") " pod="openshift-insights/insights-operator-6f9c644dd9-gm67p"
Sep 20 01:52:58.671463 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.670880    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a6fdf198-e4cd-4267-92fd-0ebee2f97522-tmp\") pod \"insights-operator-6f9c644dd9-gm67p\" (UID: \"a6fdf198-e4cd-4267-92fd-0ebee2f97522\") " pod="openshift-insights/insights-operator-6f9c644dd9-gm67p"
Sep 20 01:52:58.671463 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.670891    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/261429fc-d268-4632-9767-2de6fe66a3a4-serving-cert\") pod \"service-ca-operator-57cdc4b768-pxh5z\" (UID: \"261429fc-d268-4632-9767-2de6fe66a3a4\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-pxh5z"
Sep 20 01:52:58.671463 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.671183    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/a6fdf198-e4cd-4267-92fd-0ebee2f97522-snapshots\") pod \"insights-operator-6f9c644dd9-gm67p\" (UID: \"a6fdf198-e4cd-4267-92fd-0ebee2f97522\") " pod="openshift-insights/insights-operator-6f9c644dd9-gm67p"
Sep 20 01:52:58.671463 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.671220    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/092d6fa5-0734-4777-b6e0-2e1ef430076d-telemetry-config\") pod \"cluster-monitoring-operator-847b8bd8d7-bp5fk\" (UID: \"092d6fa5-0734-4777-b6e0-2e1ef430076d\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-bp5fk"
Sep 20 01:52:58.671463 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.671246    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2dbc9737-865d-4dbf-85bf-8d2b9781a8d5-metrics-certs\") pod \"router-default-95688946b-7rdfv\" (UID: \"2dbc9737-865d-4dbf-85bf-8d2b9781a8d5\") " pod="openshift-ingress/router-default-95688946b-7rdfv"
Sep 20 01:52:58.671463 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.671285    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a6fdf198-e4cd-4267-92fd-0ebee2f97522-trusted-ca-bundle\") pod \"insights-operator-6f9c644dd9-gm67p\" (UID: \"a6fdf198-e4cd-4267-92fd-0ebee2f97522\") " pod="openshift-insights/insights-operator-6f9c644dd9-gm67p"
Sep 20 01:52:58.671463 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.671313    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/092d6fa5-0734-4777-b6e0-2e1ef430076d-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-847b8bd8d7-bp5fk\" (UID: \"092d6fa5-0734-4777-b6e0-2e1ef430076d\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-bp5fk"
Sep 20 01:52:58.671463 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.671342    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/2dbc9737-865d-4dbf-85bf-8d2b9781a8d5-default-certificate\") pod \"router-default-95688946b-7rdfv\" (UID: \"2dbc9737-865d-4dbf-85bf-8d2b9781a8d5\") " pod="openshift-ingress/router-default-95688946b-7rdfv"
Sep 20 01:52:58.671463 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.671370    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/79e39187-4356-422e-b5d3-9c9f8d3a9a58-tmp-dir\") pod \"dns-default-c9zbv\" (UID: \"79e39187-4356-422e-b5d3-9c9f8d3a9a58\") " pod="openshift-dns/dns-default-c9zbv"
Sep 20 01:52:58.671463 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.671397    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/261429fc-d268-4632-9767-2de6fe66a3a4-config\") pod \"service-ca-operator-57cdc4b768-pxh5z\" (UID: \"261429fc-d268-4632-9767-2de6fe66a3a4\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-pxh5z"
Sep 20 01:52:58.671463 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.671409    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a6fdf198-e4cd-4267-92fd-0ebee2f97522-service-ca-bundle\") pod \"insights-operator-6f9c644dd9-gm67p\" (UID: \"a6fdf198-e4cd-4267-92fd-0ebee2f97522\") " pod="openshift-insights/insights-operator-6f9c644dd9-gm67p"
Sep 20 01:52:58.671463 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.671423    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/54d123d3-fc8b-49ee-9e26-d4a3ee494a1d-nginx-conf\") pod \"networking-console-plugin-6446c8d75d-fktjk\" (UID: \"54d123d3-fc8b-49ee-9e26-d4a3ee494a1d\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-fktjk"
Sep 20 01:52:58.671463 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.671448    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/536c3d86-41b0-4e73-9eed-7acbd8b86920-ca\") pod \"cluster-proxy-proxy-agent-77cf88f65f-9qp8b\" (UID: \"536c3d86-41b0-4e73-9eed-7acbd8b86920\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77cf88f65f-9qp8b"
Sep 20 01:52:58.672164 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.671479    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/006f93ba-2bd5-48cb-b52d-2e288401cd0f-config\") pod \"kube-storage-version-migrator-operator-55889f7b47-2xts8\" (UID: \"006f93ba-2bd5-48cb-b52d-2e288401cd0f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-2xts8"
Sep 20 01:52:58.672164 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.671487    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-42n6j\" (UniqueName: \"kubernetes.io/projected/04d0364e-4801-47e7-942e-34b9377ccc5d-kube-api-access-42n6j\") pod \"network-check-source-7cbc45fddc-q2gls\" (UID: \"04d0364e-4801-47e7-942e-34b9377ccc5d\") " pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-q2gls"
Sep 20 01:52:58.672164 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.671565    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/536c3d86-41b0-4e73-9eed-7acbd8b86920-additional-service-ca\") pod \"cluster-proxy-proxy-agent-77cf88f65f-9qp8b\" (UID: \"536c3d86-41b0-4e73-9eed-7acbd8b86920\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77cf88f65f-9qp8b"
Sep 20 01:52:58.672164 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.671595    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ece82f19-68b5-4810-be50-d8678cfdee02-config\") pod \"console-operator-fd4bb46c-ztdt2\" (UID: \"ece82f19-68b5-4810-be50-d8678cfdee02\") " pod="openshift-console-operator/console-operator-fd4bb46c-ztdt2"
Sep 20 01:52:58.672164 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.671622    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4de13b31-81d2-45d8-977f-2a59314b499d-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-848dd94bd6-xfg9p\" (UID: \"4de13b31-81d2-45d8-977f-2a59314b499d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-848dd94bd6-xfg9p"
Sep 20 01:52:58.672164 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.671646    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/79e39187-4356-422e-b5d3-9c9f8d3a9a58-metrics-tls\") pod \"dns-default-c9zbv\" (UID: \"79e39187-4356-422e-b5d3-9c9f8d3a9a58\") " pod="openshift-dns/dns-default-c9zbv"
Sep 20 01:52:58.672164 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.671675    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/2dbc9737-865d-4dbf-85bf-8d2b9781a8d5-stats-auth\") pod \"router-default-95688946b-7rdfv\" (UID: \"2dbc9737-865d-4dbf-85bf-8d2b9781a8d5\") " pod="openshift-ingress/router-default-95688946b-7rdfv"
Sep 20 01:52:58.672164 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.671722    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ece82f19-68b5-4810-be50-d8678cfdee02-trusted-ca\") pod \"console-operator-fd4bb46c-ztdt2\" (UID: \"ece82f19-68b5-4810-be50-d8678cfdee02\") " pod="openshift-console-operator/console-operator-fd4bb46c-ztdt2"
Sep 20 01:52:58.672164 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.671749    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7pwl8\" (UniqueName: \"kubernetes.io/projected/2075981b-d064-43eb-ab90-373b95ad49f4-kube-api-access-7pwl8\") pod \"ingress-canary-4dsc5\" (UID: \"2075981b-d064-43eb-ab90-373b95ad49f4\") " pod="openshift-ingress-canary/ingress-canary-4dsc5"
Sep 20 01:52:58.672164 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.671960    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/a6fdf198-e4cd-4267-92fd-0ebee2f97522-snapshots\") pod \"insights-operator-6f9c644dd9-gm67p\" (UID: \"a6fdf198-e4cd-4267-92fd-0ebee2f97522\") " pod="openshift-insights/insights-operator-6f9c644dd9-gm67p"
Sep 20 01:52:58.672164 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.672037    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/092d6fa5-0734-4777-b6e0-2e1ef430076d-telemetry-config\") pod \"cluster-monitoring-operator-847b8bd8d7-bp5fk\" (UID: \"092d6fa5-0734-4777-b6e0-2e1ef430076d\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-bp5fk"
Sep 20 01:52:58.672707 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:58.672246    2584 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 20 01:52:58.672707 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:58.672262    2584 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 20 01:52:58.672707 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:58.672335    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2dbc9737-865d-4dbf-85bf-8d2b9781a8d5-metrics-certs podName:2dbc9737-865d-4dbf-85bf-8d2b9781a8d5 nodeName:}" failed. No retries permitted until 2026-09-20 01:52:59.172317597 +0000 UTC m=+35.914996795 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2dbc9737-865d-4dbf-85bf-8d2b9781a8d5-metrics-certs") pod "router-default-95688946b-7rdfv" (UID: "2dbc9737-865d-4dbf-85bf-8d2b9781a8d5") : secret "router-metrics-certs-default" not found
Sep 20 01:52:58.672707 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:58.672377    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/092d6fa5-0734-4777-b6e0-2e1ef430076d-cluster-monitoring-operator-tls podName:092d6fa5-0734-4777-b6e0-2e1ef430076d nodeName:}" failed. No retries permitted until 2026-09-20 01:52:59.172366341 +0000 UTC m=+35.915045538 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/092d6fa5-0734-4777-b6e0-2e1ef430076d-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-847b8bd8d7-bp5fk" (UID: "092d6fa5-0734-4777-b6e0-2e1ef430076d") : secret "cluster-monitoring-operator-tls" not found
Sep 20 01:52:58.672707 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.672387    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/261429fc-d268-4632-9767-2de6fe66a3a4-config\") pod \"service-ca-operator-57cdc4b768-pxh5z\" (UID: \"261429fc-d268-4632-9767-2de6fe66a3a4\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-pxh5z"
Sep 20 01:52:58.672707 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.672648    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ece82f19-68b5-4810-be50-d8678cfdee02-config\") pod \"console-operator-fd4bb46c-ztdt2\" (UID: \"ece82f19-68b5-4810-be50-d8678cfdee02\") " pod="openshift-console-operator/console-operator-fd4bb46c-ztdt2"
Sep 20 01:52:58.673010 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.672954    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a6fdf198-e4cd-4267-92fd-0ebee2f97522-trusted-ca-bundle\") pod \"insights-operator-6f9c644dd9-gm67p\" (UID: \"a6fdf198-e4cd-4267-92fd-0ebee2f97522\") " pod="openshift-insights/insights-operator-6f9c644dd9-gm67p"
Sep 20 01:52:58.673070 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.673037    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ece82f19-68b5-4810-be50-d8678cfdee02-trusted-ca\") pod \"console-operator-fd4bb46c-ztdt2\" (UID: \"ece82f19-68b5-4810-be50-d8678cfdee02\") " pod="openshift-console-operator/console-operator-fd4bb46c-ztdt2"
Sep 20 01:52:58.674015 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.673971    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a6fdf198-e4cd-4267-92fd-0ebee2f97522-serving-cert\") pod \"insights-operator-6f9c644dd9-gm67p\" (UID: \"a6fdf198-e4cd-4267-92fd-0ebee2f97522\") " pod="openshift-insights/insights-operator-6f9c644dd9-gm67p"
Sep 20 01:52:58.674663 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.674619    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/006f93ba-2bd5-48cb-b52d-2e288401cd0f-serving-cert\") pod \"kube-storage-version-migrator-operator-55889f7b47-2xts8\" (UID: \"006f93ba-2bd5-48cb-b52d-2e288401cd0f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-2xts8"
Sep 20 01:52:58.675018 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.674992    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/261429fc-d268-4632-9767-2de6fe66a3a4-serving-cert\") pod \"service-ca-operator-57cdc4b768-pxh5z\" (UID: \"261429fc-d268-4632-9767-2de6fe66a3a4\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-pxh5z"
Sep 20 01:52:58.675297 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.675273    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ece82f19-68b5-4810-be50-d8678cfdee02-serving-cert\") pod \"console-operator-fd4bb46c-ztdt2\" (UID: \"ece82f19-68b5-4810-be50-d8678cfdee02\") " pod="openshift-console-operator/console-operator-fd4bb46c-ztdt2"
Sep 20 01:52:58.675380 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.675322    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/2dbc9737-865d-4dbf-85bf-8d2b9781a8d5-default-certificate\") pod \"router-default-95688946b-7rdfv\" (UID: \"2dbc9737-865d-4dbf-85bf-8d2b9781a8d5\") " pod="openshift-ingress/router-default-95688946b-7rdfv"
Sep 20 01:52:58.675861 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.675838    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/2dbc9737-865d-4dbf-85bf-8d2b9781a8d5-stats-auth\") pod \"router-default-95688946b-7rdfv\" (UID: \"2dbc9737-865d-4dbf-85bf-8d2b9781a8d5\") " pod="openshift-ingress/router-default-95688946b-7rdfv"
Sep 20 01:52:58.678925 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.678896    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ftjb6\" (UniqueName: \"kubernetes.io/projected/261429fc-d268-4632-9767-2de6fe66a3a4-kube-api-access-ftjb6\") pod \"service-ca-operator-57cdc4b768-pxh5z\" (UID: \"261429fc-d268-4632-9767-2de6fe66a3a4\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-pxh5z"
Sep 20 01:52:58.679623 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.679597    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b46ng\" (UniqueName: \"kubernetes.io/projected/2dbc9737-865d-4dbf-85bf-8d2b9781a8d5-kube-api-access-b46ng\") pod \"router-default-95688946b-7rdfv\" (UID: \"2dbc9737-865d-4dbf-85bf-8d2b9781a8d5\") " pod="openshift-ingress/router-default-95688946b-7rdfv"
Sep 20 01:52:58.680153 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.680108    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jnd46\" (UniqueName: \"kubernetes.io/projected/a6fdf198-e4cd-4267-92fd-0ebee2f97522-kube-api-access-jnd46\") pod \"insights-operator-6f9c644dd9-gm67p\" (UID: \"a6fdf198-e4cd-4267-92fd-0ebee2f97522\") " pod="openshift-insights/insights-operator-6f9c644dd9-gm67p"
Sep 20 01:52:58.681056 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.681031    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nc4fw\" (UniqueName: \"kubernetes.io/projected/006f93ba-2bd5-48cb-b52d-2e288401cd0f-kube-api-access-nc4fw\") pod \"kube-storage-version-migrator-operator-55889f7b47-2xts8\" (UID: \"006f93ba-2bd5-48cb-b52d-2e288401cd0f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-2xts8"
Sep 20 01:52:58.681811 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.681766    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5dn9m\" (UniqueName: \"kubernetes.io/projected/a2305e14-b41d-45f8-86ce-c99c92938ba3-kube-api-access-5dn9m\") pod \"volume-data-source-validator-5c4db8bdf9-hlwl5\" (UID: \"a2305e14-b41d-45f8-86ce-c99c92938ba3\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-hlwl5"
Sep 20 01:52:58.681996 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.681956    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kb8sw\" (UniqueName: \"kubernetes.io/projected/ece82f19-68b5-4810-be50-d8678cfdee02-kube-api-access-kb8sw\") pod \"console-operator-fd4bb46c-ztdt2\" (UID: \"ece82f19-68b5-4810-be50-d8678cfdee02\") " pod="openshift-console-operator/console-operator-fd4bb46c-ztdt2"
Sep 20 01:52:58.682458 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.682140    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k8xrc\" (UniqueName: \"kubernetes.io/projected/092d6fa5-0734-4777-b6e0-2e1ef430076d-kube-api-access-k8xrc\") pod \"cluster-monitoring-operator-847b8bd8d7-bp5fk\" (UID: \"092d6fa5-0734-4777-b6e0-2e1ef430076d\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-bp5fk"
Sep 20 01:52:58.703917 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.703883    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:52:58.737036 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.736995    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-hlwl5"
Sep 20 01:52:58.746258 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.746218    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-6f9c644dd9-gm67p"
Sep 20 01:52:58.755984 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.755964    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-fd4bb46c-ztdt2"
Sep 20 01:52:58.769337 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.769309    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-pxh5z"
Sep 20 01:52:58.772810 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.772787    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/79e39187-4356-422e-b5d3-9c9f8d3a9a58-config-volume\") pod \"dns-default-c9zbv\" (UID: \"79e39187-4356-422e-b5d3-9c9f8d3a9a58\") " pod="openshift-dns/dns-default-c9zbv"
Sep 20 01:52:58.772928 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.772882    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/79e39187-4356-422e-b5d3-9c9f8d3a9a58-tmp-dir\") pod \"dns-default-c9zbv\" (UID: \"79e39187-4356-422e-b5d3-9c9f8d3a9a58\") " pod="openshift-dns/dns-default-c9zbv"
Sep 20 01:52:58.772928 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.772908    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/54d123d3-fc8b-49ee-9e26-d4a3ee494a1d-nginx-conf\") pod \"networking-console-plugin-6446c8d75d-fktjk\" (UID: \"54d123d3-fc8b-49ee-9e26-d4a3ee494a1d\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-fktjk"
Sep 20 01:52:58.773038 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.772932    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/536c3d86-41b0-4e73-9eed-7acbd8b86920-ca\") pod \"cluster-proxy-proxy-agent-77cf88f65f-9qp8b\" (UID: \"536c3d86-41b0-4e73-9eed-7acbd8b86920\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77cf88f65f-9qp8b"
Sep 20 01:52:58.773038 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.772960    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-42n6j\" (UniqueName: \"kubernetes.io/projected/04d0364e-4801-47e7-942e-34b9377ccc5d-kube-api-access-42n6j\") pod \"network-check-source-7cbc45fddc-q2gls\" (UID: \"04d0364e-4801-47e7-942e-34b9377ccc5d\") " pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-q2gls"
Sep 20 01:52:58.773038 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.772989    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/536c3d86-41b0-4e73-9eed-7acbd8b86920-additional-service-ca\") pod \"cluster-proxy-proxy-agent-77cf88f65f-9qp8b\" (UID: \"536c3d86-41b0-4e73-9eed-7acbd8b86920\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77cf88f65f-9qp8b"
Sep 20 01:52:58.773038 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.773027    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4de13b31-81d2-45d8-977f-2a59314b499d-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-848dd94bd6-xfg9p\" (UID: \"4de13b31-81d2-45d8-977f-2a59314b499d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-848dd94bd6-xfg9p"
Sep 20 01:52:58.773248 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.773050    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/79e39187-4356-422e-b5d3-9c9f8d3a9a58-metrics-tls\") pod \"dns-default-c9zbv\" (UID: \"79e39187-4356-422e-b5d3-9c9f8d3a9a58\") " pod="openshift-dns/dns-default-c9zbv"
Sep 20 01:52:58.773248 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.773086    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7pwl8\" (UniqueName: \"kubernetes.io/projected/2075981b-d064-43eb-ab90-373b95ad49f4-kube-api-access-7pwl8\") pod \"ingress-canary-4dsc5\" (UID: \"2075981b-d064-43eb-ab90-373b95ad49f4\") " pod="openshift-ingress-canary/ingress-canary-4dsc5"
Sep 20 01:52:58.773248 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.773144    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/536c3d86-41b0-4e73-9eed-7acbd8b86920-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-77cf88f65f-9qp8b\" (UID: \"536c3d86-41b0-4e73-9eed-7acbd8b86920\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77cf88f65f-9qp8b"
Sep 20 01:52:58.773248 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.773186    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lbhph\" (UniqueName: \"kubernetes.io/projected/536c3d86-41b0-4e73-9eed-7acbd8b86920-kube-api-access-lbhph\") pod \"cluster-proxy-proxy-agent-77cf88f65f-9qp8b\" (UID: \"536c3d86-41b0-4e73-9eed-7acbd8b86920\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77cf88f65f-9qp8b"
Sep 20 01:52:58.773248 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.773211    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2075981b-d064-43eb-ab90-373b95ad49f4-cert\") pod \"ingress-canary-4dsc5\" (UID: \"2075981b-d064-43eb-ab90-373b95ad49f4\") " pod="openshift-ingress-canary/ingress-canary-4dsc5"
Sep 20 01:52:58.773482 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.773254    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/a3c5d94e-85f4-41f9-9743-dd7fd1c38b98-klusterlet-config\") pod \"klusterlet-addon-workmgr-858b56ff6c-5f29j\" (UID: \"a3c5d94e-85f4-41f9-9743-dd7fd1c38b98\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-858b56ff6c-5f29j"
Sep 20 01:52:58.773482 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.773288    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/536c3d86-41b0-4e73-9eed-7acbd8b86920-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-77cf88f65f-9qp8b\" (UID: \"536c3d86-41b0-4e73-9eed-7acbd8b86920\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77cf88f65f-9qp8b"
Sep 20 01:52:58.773482 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.773291    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/79e39187-4356-422e-b5d3-9c9f8d3a9a58-tmp-dir\") pod \"dns-default-c9zbv\" (UID: \"79e39187-4356-422e-b5d3-9c9f8d3a9a58\") " pod="openshift-dns/dns-default-c9zbv"
Sep 20 01:52:58.773482 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.773315    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g494p\" (UniqueName: \"kubernetes.io/projected/a3c5d94e-85f4-41f9-9743-dd7fd1c38b98-kube-api-access-g494p\") pod \"klusterlet-addon-workmgr-858b56ff6c-5f29j\" (UID: \"a3c5d94e-85f4-41f9-9743-dd7fd1c38b98\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-858b56ff6c-5f29j"
Sep 20 01:52:58.773482 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.773347    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/54d123d3-fc8b-49ee-9e26-d4a3ee494a1d-networking-console-plugin-cert\") pod \"networking-console-plugin-6446c8d75d-fktjk\" (UID: \"54d123d3-fc8b-49ee-9e26-d4a3ee494a1d\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-fktjk"
Sep 20 01:52:58.773482 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:58.773350    2584 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 20 01:52:58.773482 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.773375    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8lwx4\" (UniqueName: \"kubernetes.io/projected/4de13b31-81d2-45d8-977f-2a59314b499d-kube-api-access-8lwx4\") pod \"managed-serviceaccount-addon-agent-848dd94bd6-xfg9p\" (UID: \"4de13b31-81d2-45d8-977f-2a59314b499d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-848dd94bd6-xfg9p"
Sep 20 01:52:58.773482 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.773404    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-htczw\" (UniqueName: \"kubernetes.io/projected/79e39187-4356-422e-b5d3-9c9f8d3a9a58-kube-api-access-htczw\") pod \"dns-default-c9zbv\" (UID: \"79e39187-4356-422e-b5d3-9c9f8d3a9a58\") " pod="openshift-dns/dns-default-c9zbv"
Sep 20 01:52:58.773482 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:58.773420    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79e39187-4356-422e-b5d3-9c9f8d3a9a58-metrics-tls podName:79e39187-4356-422e-b5d3-9c9f8d3a9a58 nodeName:}" failed. No retries permitted until 2026-09-20 01:52:59.273399383 +0000 UTC m=+36.016078569 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/79e39187-4356-422e-b5d3-9c9f8d3a9a58-metrics-tls") pod "dns-default-c9zbv" (UID: "79e39187-4356-422e-b5d3-9c9f8d3a9a58") : secret "dns-default-metrics-tls" not found
Sep 20 01:52:58.773482 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.773454    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a3c5d94e-85f4-41f9-9743-dd7fd1c38b98-tmp\") pod \"klusterlet-addon-workmgr-858b56ff6c-5f29j\" (UID: \"a3c5d94e-85f4-41f9-9743-dd7fd1c38b98\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-858b56ff6c-5f29j"
Sep 20 01:52:58.773482 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.773472    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/79e39187-4356-422e-b5d3-9c9f8d3a9a58-config-volume\") pod \"dns-default-c9zbv\" (UID: \"79e39187-4356-422e-b5d3-9c9f8d3a9a58\") " pod="openshift-dns/dns-default-c9zbv"
Sep 20 01:52:58.773997 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.773501    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/536c3d86-41b0-4e73-9eed-7acbd8b86920-hub\") pod \"cluster-proxy-proxy-agent-77cf88f65f-9qp8b\" (UID: \"536c3d86-41b0-4e73-9eed-7acbd8b86920\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77cf88f65f-9qp8b"
Sep 20 01:52:58.773997 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.773777    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/54d123d3-fc8b-49ee-9e26-d4a3ee494a1d-nginx-conf\") pod \"networking-console-plugin-6446c8d75d-fktjk\" (UID: \"54d123d3-fc8b-49ee-9e26-d4a3ee494a1d\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-fktjk"
Sep 20 01:52:58.773997 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.773910    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/536c3d86-41b0-4e73-9eed-7acbd8b86920-additional-service-ca\") pod \"cluster-proxy-proxy-agent-77cf88f65f-9qp8b\" (UID: \"536c3d86-41b0-4e73-9eed-7acbd8b86920\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77cf88f65f-9qp8b"
Sep 20 01:52:58.773997 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:58.773936    2584 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 20 01:52:58.773997 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:58.773989    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/54d123d3-fc8b-49ee-9e26-d4a3ee494a1d-networking-console-plugin-cert podName:54d123d3-fc8b-49ee-9e26-d4a3ee494a1d nodeName:}" failed. No retries permitted until 2026-09-20 01:52:59.2739737 +0000 UTC m=+36.016652903 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/54d123d3-fc8b-49ee-9e26-d4a3ee494a1d-networking-console-plugin-cert") pod "networking-console-plugin-6446c8d75d-fktjk" (UID: "54d123d3-fc8b-49ee-9e26-d4a3ee494a1d") : secret "networking-console-plugin-cert" not found
Sep 20 01:52:58.774245 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.774226    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a3c5d94e-85f4-41f9-9743-dd7fd1c38b98-tmp\") pod \"klusterlet-addon-workmgr-858b56ff6c-5f29j\" (UID: \"a3c5d94e-85f4-41f9-9743-dd7fd1c38b98\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-858b56ff6c-5f29j"
Sep 20 01:52:58.775967 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.775924    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/536c3d86-41b0-4e73-9eed-7acbd8b86920-ca\") pod \"cluster-proxy-proxy-agent-77cf88f65f-9qp8b\" (UID: \"536c3d86-41b0-4e73-9eed-7acbd8b86920\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77cf88f65f-9qp8b"
Sep 20 01:52:58.776383 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.776274    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4de13b31-81d2-45d8-977f-2a59314b499d-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-848dd94bd6-xfg9p\" (UID: \"4de13b31-81d2-45d8-977f-2a59314b499d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-848dd94bd6-xfg9p"
Sep 20 01:52:58.776383 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.776336    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/536c3d86-41b0-4e73-9eed-7acbd8b86920-hub\") pod \"cluster-proxy-proxy-agent-77cf88f65f-9qp8b\" (UID: \"536c3d86-41b0-4e73-9eed-7acbd8b86920\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77cf88f65f-9qp8b"
Sep 20 01:52:58.777160 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.777135    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/536c3d86-41b0-4e73-9eed-7acbd8b86920-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-77cf88f65f-9qp8b\" (UID: \"536c3d86-41b0-4e73-9eed-7acbd8b86920\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77cf88f65f-9qp8b"
Sep 20 01:52:58.777380 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.777341    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2075981b-d064-43eb-ab90-373b95ad49f4-cert\") pod \"ingress-canary-4dsc5\" (UID: \"2075981b-d064-43eb-ab90-373b95ad49f4\") " pod="openshift-ingress-canary/ingress-canary-4dsc5"
Sep 20 01:52:58.777794 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.777772    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/536c3d86-41b0-4e73-9eed-7acbd8b86920-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-77cf88f65f-9qp8b\" (UID: \"536c3d86-41b0-4e73-9eed-7acbd8b86920\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77cf88f65f-9qp8b"
Sep 20 01:52:58.777941 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.777921    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/a3c5d94e-85f4-41f9-9743-dd7fd1c38b98-klusterlet-config\") pod \"klusterlet-addon-workmgr-858b56ff6c-5f29j\" (UID: \"a3c5d94e-85f4-41f9-9743-dd7fd1c38b98\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-858b56ff6c-5f29j"
Sep 20 01:52:58.779812 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.779792    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-2xts8"
Sep 20 01:52:58.783197 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.783173    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7pwl8\" (UniqueName: \"kubernetes.io/projected/2075981b-d064-43eb-ab90-373b95ad49f4-kube-api-access-7pwl8\") pod \"ingress-canary-4dsc5\" (UID: \"2075981b-d064-43eb-ab90-373b95ad49f4\") " pod="openshift-ingress-canary/ingress-canary-4dsc5"
Sep 20 01:52:58.783197 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.783188    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-42n6j\" (UniqueName: \"kubernetes.io/projected/04d0364e-4801-47e7-942e-34b9377ccc5d-kube-api-access-42n6j\") pod \"network-check-source-7cbc45fddc-q2gls\" (UID: \"04d0364e-4801-47e7-942e-34b9377ccc5d\") " pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-q2gls"
Sep 20 01:52:58.783734 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.783693    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8lwx4\" (UniqueName: \"kubernetes.io/projected/4de13b31-81d2-45d8-977f-2a59314b499d-kube-api-access-8lwx4\") pod \"managed-serviceaccount-addon-agent-848dd94bd6-xfg9p\" (UID: \"4de13b31-81d2-45d8-977f-2a59314b499d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-848dd94bd6-xfg9p"
Sep 20 01:52:58.784045 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.784023    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-htczw\" (UniqueName: \"kubernetes.io/projected/79e39187-4356-422e-b5d3-9c9f8d3a9a58-kube-api-access-htczw\") pod \"dns-default-c9zbv\" (UID: \"79e39187-4356-422e-b5d3-9c9f8d3a9a58\") " pod="openshift-dns/dns-default-c9zbv"
Sep 20 01:52:58.784146 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.784072    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g494p\" (UniqueName: \"kubernetes.io/projected/a3c5d94e-85f4-41f9-9743-dd7fd1c38b98-kube-api-access-g494p\") pod \"klusterlet-addon-workmgr-858b56ff6c-5f29j\" (UID: \"a3c5d94e-85f4-41f9-9743-dd7fd1c38b98\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-858b56ff6c-5f29j"
Sep 20 01:52:58.785280 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.785246    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lbhph\" (UniqueName: \"kubernetes.io/projected/536c3d86-41b0-4e73-9eed-7acbd8b86920-kube-api-access-lbhph\") pod \"cluster-proxy-proxy-agent-77cf88f65f-9qp8b\" (UID: \"536c3d86-41b0-4e73-9eed-7acbd8b86920\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77cf88f65f-9qp8b"
Sep 20 01:52:58.821042 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.821011    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-4dsc5"
Sep 20 01:52:58.847374 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.847337    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-q2gls"
Sep 20 01:52:58.902279 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.902196    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-848dd94bd6-xfg9p"
Sep 20 01:52:58.912535 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.912492    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77cf88f65f-9qp8b"
Sep 20 01:52:58.928806 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:58.928785    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-858b56ff6c-5f29j"
Sep 20 01:52:59.077006 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:59.076961    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5d22d853-7ae7-48b3-8549-5e2bf2be770d-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-7mld6\" (UID: \"5d22d853-7ae7-48b3-8549-5e2bf2be770d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-7mld6"
Sep 20 01:52:59.077209 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:59.077031    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/492ce9d4-19e5-4e0c-8b86-fd63a3b456a1-registry-tls\") pod \"image-registry-5659b6b8c9-hzb2w\" (UID: \"492ce9d4-19e5-4e0c-8b86-fd63a3b456a1\") " pod="openshift-image-registry/image-registry-5659b6b8c9-hzb2w"
Sep 20 01:52:59.077209 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:59.077125    2584 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 20 01:52:59.077209 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:59.077205    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5d22d853-7ae7-48b3-8549-5e2bf2be770d-samples-operator-tls podName:5d22d853-7ae7-48b3-8549-5e2bf2be770d nodeName:}" failed. No retries permitted until 2026-09-20 01:53:00.077184051 +0000 UTC m=+36.819863241 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/5d22d853-7ae7-48b3-8549-5e2bf2be770d-samples-operator-tls") pod "cluster-samples-operator-685d7d68b5-7mld6" (UID: "5d22d853-7ae7-48b3-8549-5e2bf2be770d") : secret "samples-operator-tls" not found
Sep 20 01:52:59.077383 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:59.077212    2584 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 20 01:52:59.077383 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:59.077235    2584 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5659b6b8c9-hzb2w: secret "image-registry-tls" not found
Sep 20 01:52:59.077383 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:59.077292    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/492ce9d4-19e5-4e0c-8b86-fd63a3b456a1-registry-tls podName:492ce9d4-19e5-4e0c-8b86-fd63a3b456a1 nodeName:}" failed. No retries permitted until 2026-09-20 01:53:00.077274321 +0000 UTC m=+36.819953520 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/492ce9d4-19e5-4e0c-8b86-fd63a3b456a1-registry-tls") pod "image-registry-5659b6b8c9-hzb2w" (UID: "492ce9d4-19e5-4e0c-8b86-fd63a3b456a1") : secret "image-registry-tls" not found
Sep 20 01:52:59.180136 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:59.178982    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2dbc9737-865d-4dbf-85bf-8d2b9781a8d5-service-ca-bundle\") pod \"router-default-95688946b-7rdfv\" (UID: \"2dbc9737-865d-4dbf-85bf-8d2b9781a8d5\") " pod="openshift-ingress/router-default-95688946b-7rdfv"
Sep 20 01:52:59.180136 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:59.179163    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2dbc9737-865d-4dbf-85bf-8d2b9781a8d5-metrics-certs\") pod \"router-default-95688946b-7rdfv\" (UID: \"2dbc9737-865d-4dbf-85bf-8d2b9781a8d5\") " pod="openshift-ingress/router-default-95688946b-7rdfv"
Sep 20 01:52:59.180136 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:59.179204    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/092d6fa5-0734-4777-b6e0-2e1ef430076d-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-847b8bd8d7-bp5fk\" (UID: \"092d6fa5-0734-4777-b6e0-2e1ef430076d\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-bp5fk"
Sep 20 01:52:59.180136 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:59.179390    2584 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 20 01:52:59.180136 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:59.179460    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/092d6fa5-0734-4777-b6e0-2e1ef430076d-cluster-monitoring-operator-tls podName:092d6fa5-0734-4777-b6e0-2e1ef430076d nodeName:}" failed. No retries permitted until 2026-09-20 01:53:00.179442507 +0000 UTC m=+36.922121695 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/092d6fa5-0734-4777-b6e0-2e1ef430076d-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-847b8bd8d7-bp5fk" (UID: "092d6fa5-0734-4777-b6e0-2e1ef430076d") : secret "cluster-monitoring-operator-tls" not found
Sep 20 01:52:59.180136 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:59.179924    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2dbc9737-865d-4dbf-85bf-8d2b9781a8d5-service-ca-bundle podName:2dbc9737-865d-4dbf-85bf-8d2b9781a8d5 nodeName:}" failed. No retries permitted until 2026-09-20 01:53:00.179907146 +0000 UTC m=+36.922586344 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/2dbc9737-865d-4dbf-85bf-8d2b9781a8d5-service-ca-bundle") pod "router-default-95688946b-7rdfv" (UID: "2dbc9737-865d-4dbf-85bf-8d2b9781a8d5") : configmap references non-existent config key: service-ca.crt
Sep 20 01:52:59.180136 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:59.180000    2584 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 20 01:52:59.180136 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:59.180034    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2dbc9737-865d-4dbf-85bf-8d2b9781a8d5-metrics-certs podName:2dbc9737-865d-4dbf-85bf-8d2b9781a8d5 nodeName:}" failed. No retries permitted until 2026-09-20 01:53:00.180021931 +0000 UTC m=+36.922701116 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2dbc9737-865d-4dbf-85bf-8d2b9781a8d5-metrics-certs") pod "router-default-95688946b-7rdfv" (UID: "2dbc9737-865d-4dbf-85bf-8d2b9781a8d5") : secret "router-metrics-certs-default" not found
Sep 20 01:52:59.280149 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:59.279902    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/54d123d3-fc8b-49ee-9e26-d4a3ee494a1d-networking-console-plugin-cert\") pod \"networking-console-plugin-6446c8d75d-fktjk\" (UID: \"54d123d3-fc8b-49ee-9e26-d4a3ee494a1d\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-fktjk"
Sep 20 01:52:59.280311 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:59.280045    2584 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 20 01:52:59.280311 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:59.280266    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/54d123d3-fc8b-49ee-9e26-d4a3ee494a1d-networking-console-plugin-cert podName:54d123d3-fc8b-49ee-9e26-d4a3ee494a1d nodeName:}" failed. No retries permitted until 2026-09-20 01:53:00.280251672 +0000 UTC m=+37.022930855 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/54d123d3-fc8b-49ee-9e26-d4a3ee494a1d-networking-console-plugin-cert") pod "networking-console-plugin-6446c8d75d-fktjk" (UID: "54d123d3-fc8b-49ee-9e26-d4a3ee494a1d") : secret "networking-console-plugin-cert" not found
Sep 20 01:52:59.280404 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:59.280316    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/79e39187-4356-422e-b5d3-9c9f8d3a9a58-metrics-tls\") pod \"dns-default-c9zbv\" (UID: \"79e39187-4356-422e-b5d3-9c9f8d3a9a58\") " pod="openshift-dns/dns-default-c9zbv"
Sep 20 01:52:59.280482 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:59.280466    2584 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 20 01:52:59.280549 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:52:59.280539    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79e39187-4356-422e-b5d3-9c9f8d3a9a58-metrics-tls podName:79e39187-4356-422e-b5d3-9c9f8d3a9a58 nodeName:}" failed. No retries permitted until 2026-09-20 01:53:00.280508553 +0000 UTC m=+37.023187744 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/79e39187-4356-422e-b5d3-9c9f8d3a9a58-metrics-tls") pod "dns-default-c9zbv" (UID: "79e39187-4356-422e-b5d3-9c9f8d3a9a58") : secret "dns-default-metrics-tls" not found
Sep 20 01:52:59.425565 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:59.425535    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-848dd94bd6-xfg9p"]
Sep 20 01:52:59.429886 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:59.429838    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-858b56ff6c-5f29j"]
Sep 20 01:52:59.430589 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:59.430540    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4de13b31_81d2_45d8_977f_2a59314b499d.slice/crio-7a04d18b87a5b353350355f83889d6443c2196700d69f8d3c25bc9a0cbf4b36d WatchSource:0}: Error finding container 7a04d18b87a5b353350355f83889d6443c2196700d69f8d3c25bc9a0cbf4b36d: Status 404 returned error can't find the container with id 7a04d18b87a5b353350355f83889d6443c2196700d69f8d3c25bc9a0cbf4b36d
Sep 20 01:52:59.432911 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:59.432884    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3c5d94e_85f4_41f9_9743_dd7fd1c38b98.slice/crio-24f0f4ff649dc5cdf0850cdd4cebcd72e75674877fe2c15baefc5504562c2e1f WatchSource:0}: Error finding container 24f0f4ff649dc5cdf0850cdd4cebcd72e75674877fe2c15baefc5504562c2e1f: Status 404 returned error can't find the container with id 24f0f4ff649dc5cdf0850cdd4cebcd72e75674877fe2c15baefc5504562c2e1f
Sep 20 01:52:59.433088 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:59.433065    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-fd4bb46c-ztdt2"]
Sep 20 01:52:59.434364 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:59.434340    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-6f9c644dd9-gm67p"]
Sep 20 01:52:59.436171 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:59.436147    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-2xts8"]
Sep 20 01:52:59.437144 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:59.437118    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podece82f19_68b5_4810_be50_d8678cfdee02.slice/crio-40052f7244db9ce3b3d9bc58569475ff185fd41d22a37da297d402a3f5d9ba21 WatchSource:0}: Error finding container 40052f7244db9ce3b3d9bc58569475ff185fd41d22a37da297d402a3f5d9ba21: Status 404 returned error can't find the container with id 40052f7244db9ce3b3d9bc58569475ff185fd41d22a37da297d402a3f5d9ba21
Sep 20 01:52:59.439466 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:59.439395    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7cbc45fddc-q2gls"]
Sep 20 01:52:59.440770 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:59.440715    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-4dsc5"]
Sep 20 01:52:59.450546 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:59.450218    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04d0364e_4801_47e7_942e_34b9377ccc5d.slice/crio-daaa3625364d56ffa3925040de66616ebed961e1f9fb1a7eb8a8fccf2ad92cc2 WatchSource:0}: Error finding container daaa3625364d56ffa3925040de66616ebed961e1f9fb1a7eb8a8fccf2ad92cc2: Status 404 returned error can't find the container with id daaa3625364d56ffa3925040de66616ebed961e1f9fb1a7eb8a8fccf2ad92cc2
Sep 20 01:52:59.451294 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:59.451247    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2075981b_d064_43eb_ab90_373b95ad49f4.slice/crio-1ab1f5fb274bb1b29d361e15a18e0811142954a7e64ac2b0978ba865a0394519 WatchSource:0}: Error finding container 1ab1f5fb274bb1b29d361e15a18e0811142954a7e64ac2b0978ba865a0394519: Status 404 returned error can't find the container with id 1ab1f5fb274bb1b29d361e15a18e0811142954a7e64ac2b0978ba865a0394519
Sep 20 01:52:59.469570 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:59.469430    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-hlwl5"]
Sep 20 01:52:59.470689 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:59.470484    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-57cdc4b768-pxh5z"]
Sep 20 01:52:59.472400 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:59.472371    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod261429fc_d268_4632_9767_2de6fe66a3a4.slice/crio-52f87e4398e0a32bcd5996f311b049216cad39082f6a7ecf0332c158b6bd8194 WatchSource:0}: Error finding container 52f87e4398e0a32bcd5996f311b049216cad39082f6a7ecf0332c158b6bd8194: Status 404 returned error can't find the container with id 52f87e4398e0a32bcd5996f311b049216cad39082f6a7ecf0332c158b6bd8194
Sep 20 01:52:59.472566 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:52:59.472547    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77cf88f65f-9qp8b"]
Sep 20 01:52:59.473201 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:59.473132    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2305e14_b41d_45f8_86ce_c99c92938ba3.slice/crio-eb3d38a3a4613db39f871a92cd46b7f10b4f44bc125a846adb54622a4c3fc533 WatchSource:0}: Error finding container eb3d38a3a4613db39f871a92cd46b7f10b4f44bc125a846adb54622a4c3fc533: Status 404 returned error can't find the container with id eb3d38a3a4613db39f871a92cd46b7f10b4f44bc125a846adb54622a4c3fc533
Sep 20 01:52:59.476724 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:52:59.476694    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod536c3d86_41b0_4e73_9eed_7acbd8b86920.slice/crio-3414b78e32b9874780cbd21e7122018ede0fbb930a230788213c0f832cb44968 WatchSource:0}: Error finding container 3414b78e32b9874780cbd21e7122018ede0fbb930a230788213c0f832cb44968: Status 404 returned error can't find the container with id 3414b78e32b9874780cbd21e7122018ede0fbb930a230788213c0f832cb44968
Sep 20 01:53:00.087508 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:00.086660    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-llv6h"
Sep 20 01:53:00.087508 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:00.087127    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-47994"
Sep 20 01:53:00.087508 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:00.087447    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n4dk7"
Sep 20 01:53:00.092226 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:00.090796    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5d22d853-7ae7-48b3-8549-5e2bf2be770d-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-7mld6\" (UID: \"5d22d853-7ae7-48b3-8549-5e2bf2be770d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-7mld6"
Sep 20 01:53:00.092226 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:00.090844    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/492ce9d4-19e5-4e0c-8b86-fd63a3b456a1-registry-tls\") pod \"image-registry-5659b6b8c9-hzb2w\" (UID: \"492ce9d4-19e5-4e0c-8b86-fd63a3b456a1\") " pod="openshift-image-registry/image-registry-5659b6b8c9-hzb2w"
Sep 20 01:53:00.092226 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:53:00.091047    2584 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 20 01:53:00.092226 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:53:00.091065    2584 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5659b6b8c9-hzb2w: secret "image-registry-tls" not found
Sep 20 01:53:00.092226 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:53:00.091120    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/492ce9d4-19e5-4e0c-8b86-fd63a3b456a1-registry-tls podName:492ce9d4-19e5-4e0c-8b86-fd63a3b456a1 nodeName:}" failed. No retries permitted until 2026-09-20 01:53:02.091101675 +0000 UTC m=+38.833780864 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/492ce9d4-19e5-4e0c-8b86-fd63a3b456a1-registry-tls") pod "image-registry-5659b6b8c9-hzb2w" (UID: "492ce9d4-19e5-4e0c-8b86-fd63a3b456a1") : secret "image-registry-tls" not found
Sep 20 01:53:00.092226 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:53:00.091682    2584 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 20 01:53:00.092226 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:53:00.091735    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5d22d853-7ae7-48b3-8549-5e2bf2be770d-samples-operator-tls podName:5d22d853-7ae7-48b3-8549-5e2bf2be770d nodeName:}" failed. No retries permitted until 2026-09-20 01:53:02.091720241 +0000 UTC m=+38.834399447 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/5d22d853-7ae7-48b3-8549-5e2bf2be770d-samples-operator-tls") pod "cluster-samples-operator-685d7d68b5-7mld6" (UID: "5d22d853-7ae7-48b3-8549-5e2bf2be770d") : secret "samples-operator-tls" not found
Sep 20 01:53:00.092226 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:00.091984    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-cvrtw\""
Sep 20 01:53:00.099143 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:00.098571    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 20 01:53:00.099143 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:00.098822    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 20 01:53:00.099143 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:00.098895    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-jjlfs\""
Sep 20 01:53:00.191758 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:00.191722    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2dbc9737-865d-4dbf-85bf-8d2b9781a8d5-service-ca-bundle\") pod \"router-default-95688946b-7rdfv\" (UID: \"2dbc9737-865d-4dbf-85bf-8d2b9781a8d5\") " pod="openshift-ingress/router-default-95688946b-7rdfv"
Sep 20 01:53:00.191951 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:00.191870    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2dbc9737-865d-4dbf-85bf-8d2b9781a8d5-metrics-certs\") pod \"router-default-95688946b-7rdfv\" (UID: \"2dbc9737-865d-4dbf-85bf-8d2b9781a8d5\") " pod="openshift-ingress/router-default-95688946b-7rdfv"
Sep 20 01:53:00.191951 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:00.191919    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/092d6fa5-0734-4777-b6e0-2e1ef430076d-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-847b8bd8d7-bp5fk\" (UID: \"092d6fa5-0734-4777-b6e0-2e1ef430076d\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-bp5fk"
Sep 20 01:53:00.192838 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:53:00.192123    2584 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 20 01:53:00.192838 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:53:00.192203    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/092d6fa5-0734-4777-b6e0-2e1ef430076d-cluster-monitoring-operator-tls podName:092d6fa5-0734-4777-b6e0-2e1ef430076d nodeName:}" failed. No retries permitted until 2026-09-20 01:53:02.192182407 +0000 UTC m=+38.934861592 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/092d6fa5-0734-4777-b6e0-2e1ef430076d-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-847b8bd8d7-bp5fk" (UID: "092d6fa5-0734-4777-b6e0-2e1ef430076d") : secret "cluster-monitoring-operator-tls" not found
Sep 20 01:53:00.192838 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:53:00.192290    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2dbc9737-865d-4dbf-85bf-8d2b9781a8d5-service-ca-bundle podName:2dbc9737-865d-4dbf-85bf-8d2b9781a8d5 nodeName:}" failed. No retries permitted until 2026-09-20 01:53:02.192276489 +0000 UTC m=+38.934955672 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/2dbc9737-865d-4dbf-85bf-8d2b9781a8d5-service-ca-bundle") pod "router-default-95688946b-7rdfv" (UID: "2dbc9737-865d-4dbf-85bf-8d2b9781a8d5") : configmap references non-existent config key: service-ca.crt
Sep 20 01:53:00.192838 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:53:00.192379    2584 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 20 01:53:00.192838 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:53:00.192413    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2dbc9737-865d-4dbf-85bf-8d2b9781a8d5-metrics-certs podName:2dbc9737-865d-4dbf-85bf-8d2b9781a8d5 nodeName:}" failed. No retries permitted until 2026-09-20 01:53:02.192402599 +0000 UTC m=+38.935081788 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2dbc9737-865d-4dbf-85bf-8d2b9781a8d5-metrics-certs") pod "router-default-95688946b-7rdfv" (UID: "2dbc9737-865d-4dbf-85bf-8d2b9781a8d5") : secret "router-metrics-certs-default" not found
Sep 20 01:53:00.293872 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:00.292843    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/79e39187-4356-422e-b5d3-9c9f8d3a9a58-metrics-tls\") pod \"dns-default-c9zbv\" (UID: \"79e39187-4356-422e-b5d3-9c9f8d3a9a58\") " pod="openshift-dns/dns-default-c9zbv"
Sep 20 01:53:00.293872 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:00.292952    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/54d123d3-fc8b-49ee-9e26-d4a3ee494a1d-networking-console-plugin-cert\") pod \"networking-console-plugin-6446c8d75d-fktjk\" (UID: \"54d123d3-fc8b-49ee-9e26-d4a3ee494a1d\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-fktjk"
Sep 20 01:53:00.293872 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:53:00.293066    2584 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 20 01:53:00.293872 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:53:00.293129    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/54d123d3-fc8b-49ee-9e26-d4a3ee494a1d-networking-console-plugin-cert podName:54d123d3-fc8b-49ee-9e26-d4a3ee494a1d nodeName:}" failed. No retries permitted until 2026-09-20 01:53:02.293110181 +0000 UTC m=+39.035789371 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/54d123d3-fc8b-49ee-9e26-d4a3ee494a1d-networking-console-plugin-cert") pod "networking-console-plugin-6446c8d75d-fktjk" (UID: "54d123d3-fc8b-49ee-9e26-d4a3ee494a1d") : secret "networking-console-plugin-cert" not found
Sep 20 01:53:00.293872 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:53:00.293464    2584 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 20 01:53:00.293872 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:53:00.293515    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79e39187-4356-422e-b5d3-9c9f8d3a9a58-metrics-tls podName:79e39187-4356-422e-b5d3-9c9f8d3a9a58 nodeName:}" failed. No retries permitted until 2026-09-20 01:53:02.293498405 +0000 UTC m=+39.036177601 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/79e39187-4356-422e-b5d3-9c9f8d3a9a58-metrics-tls") pod "dns-default-c9zbv" (UID: "79e39187-4356-422e-b5d3-9c9f8d3a9a58") : secret "dns-default-metrics-tls" not found
Sep 20 01:53:00.363856 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:00.363768    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-2xts8" event={"ID":"006f93ba-2bd5-48cb-b52d-2e288401cd0f","Type":"ContainerStarted","Data":"4efb38176799302d13c5aec3717b2bcd6dd4d476ea9565be1a1074181f7336c7"}
Sep 20 01:53:00.365456 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:00.365382    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-858b56ff6c-5f29j" event={"ID":"a3c5d94e-85f4-41f9-9743-dd7fd1c38b98","Type":"ContainerStarted","Data":"24f0f4ff649dc5cdf0850cdd4cebcd72e75674877fe2c15baefc5504562c2e1f"}
Sep 20 01:53:00.368117 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:00.368039    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-4dsc5" event={"ID":"2075981b-d064-43eb-ab90-373b95ad49f4","Type":"ContainerStarted","Data":"1ab1f5fb274bb1b29d361e15a18e0811142954a7e64ac2b0978ba865a0394519"}
Sep 20 01:53:00.370275 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:00.370202    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-hlwl5" event={"ID":"a2305e14-b41d-45f8-86ce-c99c92938ba3","Type":"ContainerStarted","Data":"eb3d38a3a4613db39f871a92cd46b7f10b4f44bc125a846adb54622a4c3fc533"}
Sep 20 01:53:00.372578 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:00.372453    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-6f9c644dd9-gm67p" event={"ID":"a6fdf198-e4cd-4267-92fd-0ebee2f97522","Type":"ContainerStarted","Data":"18caefe24fed04d50a3744595ec7ea69d50d6c545889064e4d11fda6ded9542a"}
Sep 20 01:53:00.375242 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:00.375171    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-pxh5z" event={"ID":"261429fc-d268-4632-9767-2de6fe66a3a4","Type":"ContainerStarted","Data":"52f87e4398e0a32bcd5996f311b049216cad39082f6a7ecf0332c158b6bd8194"}
Sep 20 01:53:00.376748 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:00.376678    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-fd4bb46c-ztdt2" event={"ID":"ece82f19-68b5-4810-be50-d8678cfdee02","Type":"ContainerStarted","Data":"40052f7244db9ce3b3d9bc58569475ff185fd41d22a37da297d402a3f5d9ba21"}
Sep 20 01:53:00.378641 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:00.378572    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-q2gls" event={"ID":"04d0364e-4801-47e7-942e-34b9377ccc5d","Type":"ContainerStarted","Data":"daaa3625364d56ffa3925040de66616ebed961e1f9fb1a7eb8a8fccf2ad92cc2"}
Sep 20 01:53:00.385416 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:00.385379    2584 generic.go:356] "Generic (PLEG): container finished" podID="96a97560-011d-4806-ae10-4d53880eb9ff" containerID="b3a28775c0733a78577ce60108a58bcf148092dfa3c9f7f155b2b824bd509b36" exitCode=0
Sep 20 01:53:00.386548 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:00.385636    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-f94fv" event={"ID":"96a97560-011d-4806-ae10-4d53880eb9ff","Type":"ContainerDied","Data":"b3a28775c0733a78577ce60108a58bcf148092dfa3c9f7f155b2b824bd509b36"}
Sep 20 01:53:00.389237 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:00.389094    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77cf88f65f-9qp8b" event={"ID":"536c3d86-41b0-4e73-9eed-7acbd8b86920","Type":"ContainerStarted","Data":"3414b78e32b9874780cbd21e7122018ede0fbb930a230788213c0f832cb44968"}
Sep 20 01:53:00.401892 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:00.401825    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-848dd94bd6-xfg9p" event={"ID":"4de13b31-81d2-45d8-977f-2a59314b499d","Type":"ContainerStarted","Data":"7a04d18b87a5b353350355f83889d6443c2196700d69f8d3c25bc9a0cbf4b36d"}
Sep 20 01:53:00.535153 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:00.534995    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:53:01.449549 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:01.449251    2584 generic.go:356] "Generic (PLEG): container finished" podID="96a97560-011d-4806-ae10-4d53880eb9ff" containerID="9f963d300e4ca043905aabeedea36e1394fdc6cbeeb89fa211669ecd818924fd" exitCode=0
Sep 20 01:53:01.449549 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:01.449343    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-f94fv" event={"ID":"96a97560-011d-4806-ae10-4d53880eb9ff","Type":"ContainerDied","Data":"9f963d300e4ca043905aabeedea36e1394fdc6cbeeb89fa211669ecd818924fd"}
Sep 20 01:53:02.114422 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:02.113439    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/492ce9d4-19e5-4e0c-8b86-fd63a3b456a1-registry-tls\") pod \"image-registry-5659b6b8c9-hzb2w\" (UID: \"492ce9d4-19e5-4e0c-8b86-fd63a3b456a1\") " pod="openshift-image-registry/image-registry-5659b6b8c9-hzb2w"
Sep 20 01:53:02.114422 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:02.113691    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5d22d853-7ae7-48b3-8549-5e2bf2be770d-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-7mld6\" (UID: \"5d22d853-7ae7-48b3-8549-5e2bf2be770d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-7mld6"
Sep 20 01:53:02.114422 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:53:02.113832    2584 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 20 01:53:02.114422 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:53:02.113898    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5d22d853-7ae7-48b3-8549-5e2bf2be770d-samples-operator-tls podName:5d22d853-7ae7-48b3-8549-5e2bf2be770d nodeName:}" failed. No retries permitted until 2026-09-20 01:53:06.113878015 +0000 UTC m=+42.856557268 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/5d22d853-7ae7-48b3-8549-5e2bf2be770d-samples-operator-tls") pod "cluster-samples-operator-685d7d68b5-7mld6" (UID: "5d22d853-7ae7-48b3-8549-5e2bf2be770d") : secret "samples-operator-tls" not found
Sep 20 01:53:02.114422 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:53:02.114303    2584 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 20 01:53:02.114422 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:53:02.114319    2584 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5659b6b8c9-hzb2w: secret "image-registry-tls" not found
Sep 20 01:53:02.114422 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:53:02.114366    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/492ce9d4-19e5-4e0c-8b86-fd63a3b456a1-registry-tls podName:492ce9d4-19e5-4e0c-8b86-fd63a3b456a1 nodeName:}" failed. No retries permitted until 2026-09-20 01:53:06.114349791 +0000 UTC m=+42.857028977 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/492ce9d4-19e5-4e0c-8b86-fd63a3b456a1-registry-tls") pod "image-registry-5659b6b8c9-hzb2w" (UID: "492ce9d4-19e5-4e0c-8b86-fd63a3b456a1") : secret "image-registry-tls" not found
Sep 20 01:53:02.216120 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:02.215107    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2dbc9737-865d-4dbf-85bf-8d2b9781a8d5-service-ca-bundle\") pod \"router-default-95688946b-7rdfv\" (UID: \"2dbc9737-865d-4dbf-85bf-8d2b9781a8d5\") " pod="openshift-ingress/router-default-95688946b-7rdfv"
Sep 20 01:53:02.216120 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:02.215258    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2dbc9737-865d-4dbf-85bf-8d2b9781a8d5-metrics-certs\") pod \"router-default-95688946b-7rdfv\" (UID: \"2dbc9737-865d-4dbf-85bf-8d2b9781a8d5\") " pod="openshift-ingress/router-default-95688946b-7rdfv"
Sep 20 01:53:02.216120 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:02.215296    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/092d6fa5-0734-4777-b6e0-2e1ef430076d-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-847b8bd8d7-bp5fk\" (UID: \"092d6fa5-0734-4777-b6e0-2e1ef430076d\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-bp5fk"
Sep 20 01:53:02.216120 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:53:02.215535    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2dbc9737-865d-4dbf-85bf-8d2b9781a8d5-service-ca-bundle podName:2dbc9737-865d-4dbf-85bf-8d2b9781a8d5 nodeName:}" failed. No retries permitted until 2026-09-20 01:53:06.215501271 +0000 UTC m=+42.958180475 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/2dbc9737-865d-4dbf-85bf-8d2b9781a8d5-service-ca-bundle") pod "router-default-95688946b-7rdfv" (UID: "2dbc9737-865d-4dbf-85bf-8d2b9781a8d5") : configmap references non-existent config key: service-ca.crt
Sep 20 01:53:02.216120 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:53:02.215944    2584 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 20 01:53:02.216120 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:53:02.215993    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2dbc9737-865d-4dbf-85bf-8d2b9781a8d5-metrics-certs podName:2dbc9737-865d-4dbf-85bf-8d2b9781a8d5 nodeName:}" failed. No retries permitted until 2026-09-20 01:53:06.215979274 +0000 UTC m=+42.958658460 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2dbc9737-865d-4dbf-85bf-8d2b9781a8d5-metrics-certs") pod "router-default-95688946b-7rdfv" (UID: "2dbc9737-865d-4dbf-85bf-8d2b9781a8d5") : secret "router-metrics-certs-default" not found
Sep 20 01:53:02.216120 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:53:02.216047    2584 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 20 01:53:02.216120 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:53:02.216082    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/092d6fa5-0734-4777-b6e0-2e1ef430076d-cluster-monitoring-operator-tls podName:092d6fa5-0734-4777-b6e0-2e1ef430076d nodeName:}" failed. No retries permitted until 2026-09-20 01:53:06.216071193 +0000 UTC m=+42.958750376 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/092d6fa5-0734-4777-b6e0-2e1ef430076d-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-847b8bd8d7-bp5fk" (UID: "092d6fa5-0734-4777-b6e0-2e1ef430076d") : secret "cluster-monitoring-operator-tls" not found
Sep 20 01:53:02.317785 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:02.316847    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/54d123d3-fc8b-49ee-9e26-d4a3ee494a1d-networking-console-plugin-cert\") pod \"networking-console-plugin-6446c8d75d-fktjk\" (UID: \"54d123d3-fc8b-49ee-9e26-d4a3ee494a1d\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-fktjk"
Sep 20 01:53:02.317785 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:02.317018    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/79e39187-4356-422e-b5d3-9c9f8d3a9a58-metrics-tls\") pod \"dns-default-c9zbv\" (UID: \"79e39187-4356-422e-b5d3-9c9f8d3a9a58\") " pod="openshift-dns/dns-default-c9zbv"
Sep 20 01:53:02.317785 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:53:02.317160    2584 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 20 01:53:02.317785 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:53:02.317222    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79e39187-4356-422e-b5d3-9c9f8d3a9a58-metrics-tls podName:79e39187-4356-422e-b5d3-9c9f8d3a9a58 nodeName:}" failed. No retries permitted until 2026-09-20 01:53:06.317202405 +0000 UTC m=+43.059881593 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/79e39187-4356-422e-b5d3-9c9f8d3a9a58-metrics-tls") pod "dns-default-c9zbv" (UID: "79e39187-4356-422e-b5d3-9c9f8d3a9a58") : secret "dns-default-metrics-tls" not found
Sep 20 01:53:02.317785 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:53:02.317664    2584 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 20 01:53:02.317785 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:53:02.317719    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/54d123d3-fc8b-49ee-9e26-d4a3ee494a1d-networking-console-plugin-cert podName:54d123d3-fc8b-49ee-9e26-d4a3ee494a1d nodeName:}" failed. No retries permitted until 2026-09-20 01:53:06.317702665 +0000 UTC m=+43.060381862 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/54d123d3-fc8b-49ee-9e26-d4a3ee494a1d-networking-console-plugin-cert") pod "networking-console-plugin-6446c8d75d-fktjk" (UID: "54d123d3-fc8b-49ee-9e26-d4a3ee494a1d") : secret "networking-console-plugin-cert" not found
Sep 20 01:53:03.360220 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:03.359797    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:53:03.458599 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:03.458540    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:53:03.470348 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:03.470307    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-f94fv" event={"ID":"96a97560-011d-4806-ae10-4d53880eb9ff","Type":"ContainerStarted","Data":"7ca021584d401036c63ec709fc9cca941b16c6f535a7ca96b772d7a23ec34f9c"}
Sep 20 01:53:03.491633 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:03.491556    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-f94fv" podStartSLOduration=8.117734176 podStartE2EDuration="38.491510314s" totalImagesPullingTime="30.373776138s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-20 01:52:25 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-20 01:53:03.491106511 +0000 UTC m=+40.233785716" watchObservedRunningTime="2026-09-20 01:53:03.491510314 +0000 UTC m=+40.234189523"
Sep 20 01:53:03.599915 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:03.599868    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:53:03.707782 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:03.707713    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:53:03.805508 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:03.805471    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:53:04.499241 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:04.499205    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:53:04.554596 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:04.554560    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:53:06.158648 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:06.158612    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5d22d853-7ae7-48b3-8549-5e2bf2be770d-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-7mld6\" (UID: \"5d22d853-7ae7-48b3-8549-5e2bf2be770d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-7mld6"
Sep 20 01:53:06.159212 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:06.158660    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/492ce9d4-19e5-4e0c-8b86-fd63a3b456a1-registry-tls\") pod \"image-registry-5659b6b8c9-hzb2w\" (UID: \"492ce9d4-19e5-4e0c-8b86-fd63a3b456a1\") " pod="openshift-image-registry/image-registry-5659b6b8c9-hzb2w"
Sep 20 01:53:06.159212 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:53:06.158792    2584 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 20 01:53:06.159212 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:53:06.158831    2584 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 20 01:53:06.159212 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:53:06.158845    2584 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5659b6b8c9-hzb2w: secret "image-registry-tls" not found
Sep 20 01:53:06.159212 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:53:06.158876    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5d22d853-7ae7-48b3-8549-5e2bf2be770d-samples-operator-tls podName:5d22d853-7ae7-48b3-8549-5e2bf2be770d nodeName:}" failed. No retries permitted until 2026-09-20 01:53:14.158853785 +0000 UTC m=+50.901532971 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/5d22d853-7ae7-48b3-8549-5e2bf2be770d-samples-operator-tls") pod "cluster-samples-operator-685d7d68b5-7mld6" (UID: "5d22d853-7ae7-48b3-8549-5e2bf2be770d") : secret "samples-operator-tls" not found
Sep 20 01:53:06.159212 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:53:06.158898    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/492ce9d4-19e5-4e0c-8b86-fd63a3b456a1-registry-tls podName:492ce9d4-19e5-4e0c-8b86-fd63a3b456a1 nodeName:}" failed. No retries permitted until 2026-09-20 01:53:14.158888592 +0000 UTC m=+50.901567777 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/492ce9d4-19e5-4e0c-8b86-fd63a3b456a1-registry-tls") pod "image-registry-5659b6b8c9-hzb2w" (UID: "492ce9d4-19e5-4e0c-8b86-fd63a3b456a1") : secret "image-registry-tls" not found
Sep 20 01:53:06.260095 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:06.260058    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2dbc9737-865d-4dbf-85bf-8d2b9781a8d5-service-ca-bundle\") pod \"router-default-95688946b-7rdfv\" (UID: \"2dbc9737-865d-4dbf-85bf-8d2b9781a8d5\") " pod="openshift-ingress/router-default-95688946b-7rdfv"
Sep 20 01:53:06.260364 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:06.260208    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2dbc9737-865d-4dbf-85bf-8d2b9781a8d5-metrics-certs\") pod \"router-default-95688946b-7rdfv\" (UID: \"2dbc9737-865d-4dbf-85bf-8d2b9781a8d5\") " pod="openshift-ingress/router-default-95688946b-7rdfv"
Sep 20 01:53:06.260364 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:06.260248    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/092d6fa5-0734-4777-b6e0-2e1ef430076d-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-847b8bd8d7-bp5fk\" (UID: \"092d6fa5-0734-4777-b6e0-2e1ef430076d\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-bp5fk"
Sep 20 01:53:06.260364 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:53:06.260315    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2dbc9737-865d-4dbf-85bf-8d2b9781a8d5-service-ca-bundle podName:2dbc9737-865d-4dbf-85bf-8d2b9781a8d5 nodeName:}" failed. No retries permitted until 2026-09-20 01:53:14.260283777 +0000 UTC m=+51.002962974 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/2dbc9737-865d-4dbf-85bf-8d2b9781a8d5-service-ca-bundle") pod "router-default-95688946b-7rdfv" (UID: "2dbc9737-865d-4dbf-85bf-8d2b9781a8d5") : configmap references non-existent config key: service-ca.crt
Sep 20 01:53:06.260364 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:53:06.260357    2584 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 20 01:53:06.260599 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:53:06.260410    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2dbc9737-865d-4dbf-85bf-8d2b9781a8d5-metrics-certs podName:2dbc9737-865d-4dbf-85bf-8d2b9781a8d5 nodeName:}" failed. No retries permitted until 2026-09-20 01:53:14.260392532 +0000 UTC m=+51.003071733 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2dbc9737-865d-4dbf-85bf-8d2b9781a8d5-metrics-certs") pod "router-default-95688946b-7rdfv" (UID: "2dbc9737-865d-4dbf-85bf-8d2b9781a8d5") : secret "router-metrics-certs-default" not found
Sep 20 01:53:06.260599 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:53:06.260362    2584 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 20 01:53:06.260599 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:53:06.260472    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/092d6fa5-0734-4777-b6e0-2e1ef430076d-cluster-monitoring-operator-tls podName:092d6fa5-0734-4777-b6e0-2e1ef430076d nodeName:}" failed. No retries permitted until 2026-09-20 01:53:14.260460194 +0000 UTC m=+51.003139381 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/092d6fa5-0734-4777-b6e0-2e1ef430076d-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-847b8bd8d7-bp5fk" (UID: "092d6fa5-0734-4777-b6e0-2e1ef430076d") : secret "cluster-monitoring-operator-tls" not found
Sep 20 01:53:06.361849 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:06.361815    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/54d123d3-fc8b-49ee-9e26-d4a3ee494a1d-networking-console-plugin-cert\") pod \"networking-console-plugin-6446c8d75d-fktjk\" (UID: \"54d123d3-fc8b-49ee-9e26-d4a3ee494a1d\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-fktjk"
Sep 20 01:53:06.362046 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:06.361950    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/79e39187-4356-422e-b5d3-9c9f8d3a9a58-metrics-tls\") pod \"dns-default-c9zbv\" (UID: \"79e39187-4356-422e-b5d3-9c9f8d3a9a58\") " pod="openshift-dns/dns-default-c9zbv"
Sep 20 01:53:06.362046 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:53:06.361969    2584 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 20 01:53:06.362142 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:53:06.362048    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/54d123d3-fc8b-49ee-9e26-d4a3ee494a1d-networking-console-plugin-cert podName:54d123d3-fc8b-49ee-9e26-d4a3ee494a1d nodeName:}" failed. No retries permitted until 2026-09-20 01:53:14.362029541 +0000 UTC m=+51.104708727 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/54d123d3-fc8b-49ee-9e26-d4a3ee494a1d-networking-console-plugin-cert") pod "networking-console-plugin-6446c8d75d-fktjk" (UID: "54d123d3-fc8b-49ee-9e26-d4a3ee494a1d") : secret "networking-console-plugin-cert" not found
Sep 20 01:53:06.362142 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:53:06.362069    2584 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 20 01:53:06.362142 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:53:06.362116    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79e39187-4356-422e-b5d3-9c9f8d3a9a58-metrics-tls podName:79e39187-4356-422e-b5d3-9c9f8d3a9a58 nodeName:}" failed. No retries permitted until 2026-09-20 01:53:14.362101018 +0000 UTC m=+51.104780216 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/79e39187-4356-422e-b5d3-9c9f8d3a9a58-metrics-tls") pod "dns-default-c9zbv" (UID: "79e39187-4356-422e-b5d3-9c9f8d3a9a58") : secret "dns-default-metrics-tls" not found
Sep 20 01:53:07.220851 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:07.220811    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:53:11.219202 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:11.219157    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:53:11.270699 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:11.270657    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:53:13.666836 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:13.666801    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:53:13.721683 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:13.721649    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:53:14.231713 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:14.231681    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5d22d853-7ae7-48b3-8549-5e2bf2be770d-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-7mld6\" (UID: \"5d22d853-7ae7-48b3-8549-5e2bf2be770d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-7mld6"
Sep 20 01:53:14.231823 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:14.231737    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/492ce9d4-19e5-4e0c-8b86-fd63a3b456a1-registry-tls\") pod \"image-registry-5659b6b8c9-hzb2w\" (UID: \"492ce9d4-19e5-4e0c-8b86-fd63a3b456a1\") " pod="openshift-image-registry/image-registry-5659b6b8c9-hzb2w"
Sep 20 01:53:14.231823 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:53:14.231803    2584 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 20 01:53:14.231931 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:53:14.231861    2584 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 20 01:53:14.231931 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:53:14.231876    2584 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5659b6b8c9-hzb2w: secret "image-registry-tls" not found
Sep 20 01:53:14.231931 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:53:14.231878    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5d22d853-7ae7-48b3-8549-5e2bf2be770d-samples-operator-tls podName:5d22d853-7ae7-48b3-8549-5e2bf2be770d nodeName:}" failed. No retries permitted until 2026-09-20 01:53:30.231856579 +0000 UTC m=+66.974535776 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/5d22d853-7ae7-48b3-8549-5e2bf2be770d-samples-operator-tls") pod "cluster-samples-operator-685d7d68b5-7mld6" (UID: "5d22d853-7ae7-48b3-8549-5e2bf2be770d") : secret "samples-operator-tls" not found
Sep 20 01:53:14.232078 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:53:14.231937    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/492ce9d4-19e5-4e0c-8b86-fd63a3b456a1-registry-tls podName:492ce9d4-19e5-4e0c-8b86-fd63a3b456a1 nodeName:}" failed. No retries permitted until 2026-09-20 01:53:30.231907669 +0000 UTC m=+66.974586858 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/492ce9d4-19e5-4e0c-8b86-fd63a3b456a1-registry-tls") pod "image-registry-5659b6b8c9-hzb2w" (UID: "492ce9d4-19e5-4e0c-8b86-fd63a3b456a1") : secret "image-registry-tls" not found
Sep 20 01:53:14.332589 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:14.332184    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2dbc9737-865d-4dbf-85bf-8d2b9781a8d5-service-ca-bundle\") pod \"router-default-95688946b-7rdfv\" (UID: \"2dbc9737-865d-4dbf-85bf-8d2b9781a8d5\") " pod="openshift-ingress/router-default-95688946b-7rdfv"
Sep 20 01:53:14.332589 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:14.332281    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2dbc9737-865d-4dbf-85bf-8d2b9781a8d5-metrics-certs\") pod \"router-default-95688946b-7rdfv\" (UID: \"2dbc9737-865d-4dbf-85bf-8d2b9781a8d5\") " pod="openshift-ingress/router-default-95688946b-7rdfv"
Sep 20 01:53:14.332589 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:14.332308    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/092d6fa5-0734-4777-b6e0-2e1ef430076d-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-847b8bd8d7-bp5fk\" (UID: \"092d6fa5-0734-4777-b6e0-2e1ef430076d\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-bp5fk"
Sep 20 01:53:14.332589 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:53:14.332436    2584 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 20 01:53:14.332589 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:53:14.332484    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/092d6fa5-0734-4777-b6e0-2e1ef430076d-cluster-monitoring-operator-tls podName:092d6fa5-0734-4777-b6e0-2e1ef430076d nodeName:}" failed. No retries permitted until 2026-09-20 01:53:30.332471001 +0000 UTC m=+67.075150183 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/092d6fa5-0734-4777-b6e0-2e1ef430076d-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-847b8bd8d7-bp5fk" (UID: "092d6fa5-0734-4777-b6e0-2e1ef430076d") : secret "cluster-monitoring-operator-tls" not found
Sep 20 01:53:14.332589 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:53:14.332567    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2dbc9737-865d-4dbf-85bf-8d2b9781a8d5-service-ca-bundle podName:2dbc9737-865d-4dbf-85bf-8d2b9781a8d5 nodeName:}" failed. No retries permitted until 2026-09-20 01:53:30.332556871 +0000 UTC m=+67.075236070 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/2dbc9737-865d-4dbf-85bf-8d2b9781a8d5-service-ca-bundle") pod "router-default-95688946b-7rdfv" (UID: "2dbc9737-865d-4dbf-85bf-8d2b9781a8d5") : configmap references non-existent config key: service-ca.crt
Sep 20 01:53:14.332989 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:53:14.332619    2584 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 20 01:53:14.332989 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:53:14.332659    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2dbc9737-865d-4dbf-85bf-8d2b9781a8d5-metrics-certs podName:2dbc9737-865d-4dbf-85bf-8d2b9781a8d5 nodeName:}" failed. No retries permitted until 2026-09-20 01:53:30.332646748 +0000 UTC m=+67.075325935 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2dbc9737-865d-4dbf-85bf-8d2b9781a8d5-metrics-certs") pod "router-default-95688946b-7rdfv" (UID: "2dbc9737-865d-4dbf-85bf-8d2b9781a8d5") : secret "router-metrics-certs-default" not found
Sep 20 01:53:14.433560 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:14.433383    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/79e39187-4356-422e-b5d3-9c9f8d3a9a58-metrics-tls\") pod \"dns-default-c9zbv\" (UID: \"79e39187-4356-422e-b5d3-9c9f8d3a9a58\") " pod="openshift-dns/dns-default-c9zbv"
Sep 20 01:53:14.433560 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:14.433481    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/54d123d3-fc8b-49ee-9e26-d4a3ee494a1d-networking-console-plugin-cert\") pod \"networking-console-plugin-6446c8d75d-fktjk\" (UID: \"54d123d3-fc8b-49ee-9e26-d4a3ee494a1d\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-fktjk"
Sep 20 01:53:14.433812 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:53:14.433619    2584 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 20 01:53:14.433812 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:53:14.433687    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/54d123d3-fc8b-49ee-9e26-d4a3ee494a1d-networking-console-plugin-cert podName:54d123d3-fc8b-49ee-9e26-d4a3ee494a1d nodeName:}" failed. No retries permitted until 2026-09-20 01:53:30.433666285 +0000 UTC m=+67.176345488 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/54d123d3-fc8b-49ee-9e26-d4a3ee494a1d-networking-console-plugin-cert") pod "networking-console-plugin-6446c8d75d-fktjk" (UID: "54d123d3-fc8b-49ee-9e26-d4a3ee494a1d") : secret "networking-console-plugin-cert" not found
Sep 20 01:53:14.433812 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:53:14.433689    2584 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 20 01:53:14.433812 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:53:14.433723    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79e39187-4356-422e-b5d3-9c9f8d3a9a58-metrics-tls podName:79e39187-4356-422e-b5d3-9c9f8d3a9a58 nodeName:}" failed. No retries permitted until 2026-09-20 01:53:30.433713635 +0000 UTC m=+67.176392817 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/79e39187-4356-422e-b5d3-9c9f8d3a9a58-metrics-tls") pod "dns-default-c9zbv" (UID: "79e39187-4356-422e-b5d3-9c9f8d3a9a58") : secret "dns-default-metrics-tls" not found
Sep 20 01:53:14.477241 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:14.477211    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:53:14.506650 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:14.506618    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-2xts8" event={"ID":"006f93ba-2bd5-48cb-b52d-2e288401cd0f","Type":"ContainerStarted","Data":"20fa7567a45751431d7e920b98db5528e3034c979543beec80de861649bbd313"}
Sep 20 01:53:14.509123 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:14.509096    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-858b56ff6c-5f29j" event={"ID":"a3c5d94e-85f4-41f9-9743-dd7fd1c38b98","Type":"ContainerStarted","Data":"78ef0c0e19849d7dd6b7ab85e36f584700df4b1073d0bfcbba14fe6cf45cc08e"}
Sep 20 01:53:14.510061 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:14.509722    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-858b56ff6c-5f29j"
Sep 20 01:53:14.512164 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:14.511836    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-4dsc5" event={"ID":"2075981b-d064-43eb-ab90-373b95ad49f4","Type":"ContainerStarted","Data":"945ff774286d65e0e198d728e8b8bd82453d1640f4648a13030a4eb045494353"}
Sep 20 01:53:14.513646 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:14.513556    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-hlwl5" event={"ID":"a2305e14-b41d-45f8-86ce-c99c92938ba3","Type":"ContainerStarted","Data":"43c622671cbbf76c8c0bb9053acf96bcfe856d08ae95d7aab013108282a5549e"}
Sep 20 01:53:14.515260 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:14.515237    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-6f9c644dd9-gm67p" event={"ID":"a6fdf198-e4cd-4267-92fd-0ebee2f97522","Type":"ContainerStarted","Data":"2cb9fcb228f51f16e659703c59d8c66af85e8c3073166bd806e2c27976ae0190"}
Sep 20 01:53:14.517136 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:14.516685    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-858b56ff6c-5f29j"
Sep 20 01:53:14.517609 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:14.517291    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-pxh5z" event={"ID":"261429fc-d268-4632-9767-2de6fe66a3a4","Type":"ContainerStarted","Data":"0641c73c708427c920259b253c3ffa773af2f3002552782c0a6160be4c660287"}
Sep 20 01:53:14.519213 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:14.519195    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-ztdt2_ece82f19-68b5-4810-be50-d8678cfdee02/console-operator/0.log"
Sep 20 01:53:14.519303 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:14.519232    2584 generic.go:356] "Generic (PLEG): container finished" podID="ece82f19-68b5-4810-be50-d8678cfdee02" containerID="603beccdeb373ab18389d312eedaa091903af40e6fe96b69e8e1d5bd40d911e8" exitCode=255
Sep 20 01:53:14.519366 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:14.519310    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-fd4bb46c-ztdt2" event={"ID":"ece82f19-68b5-4810-be50-d8678cfdee02","Type":"ContainerDied","Data":"603beccdeb373ab18389d312eedaa091903af40e6fe96b69e8e1d5bd40d911e8"}
Sep 20 01:53:14.519606 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:14.519451    2584 scope.go:117] "RemoveContainer" containerID="603beccdeb373ab18389d312eedaa091903af40e6fe96b69e8e1d5bd40d911e8"
Sep 20 01:53:14.521336 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:14.521011    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-q2gls" event={"ID":"04d0364e-4801-47e7-942e-34b9377ccc5d","Type":"ContainerStarted","Data":"fa601265238cc7fd8ad4128a539d2d54e1c95446abddac45818bed6b0af3e4f6"}
Sep 20 01:53:14.522923 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:14.522883    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-2xts8" podStartSLOduration=10.998512884 podStartE2EDuration="25.522870366s" totalImagesPullingTime="14.524357482s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-20 01:52:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-20 01:53:14.522026974 +0000 UTC m=+51.264706180" watchObservedRunningTime="2026-09-20 01:53:14.522870366 +0000 UTC m=+51.265549570"
Sep 20 01:53:14.525025 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:14.524994    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77cf88f65f-9qp8b" event={"ID":"536c3d86-41b0-4e73-9eed-7acbd8b86920","Type":"ContainerStarted","Data":"23f0175077874c0589e638b9f97ebf56c590bf3860eaa493b116971a9fe3ef32"}
Sep 20 01:53:14.525515 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:14.525399    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77cf88f65f-9qp8b" event={"ID":"536c3d86-41b0-4e73-9eed-7acbd8b86920","Type":"ContainerStarted","Data":"470801ddd51424ae99bb434d66a0487c0e1fc33fa1b69a9d35af59d2e3a45f51"}
Sep 20 01:53:14.527035 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:14.526629    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-848dd94bd6-xfg9p" event={"ID":"4de13b31-81d2-45d8-977f-2a59314b499d","Type":"ContainerStarted","Data":"9c7aac4c94f1d569a044e629594a0d045bdb4fd88597b68a7ec1662d7e1a7710"}
Sep 20 01:53:14.540543 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:14.538067    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-hlwl5" podStartSLOduration=17.269058316 podStartE2EDuration="25.538048576s" totalImagesPullingTime="8.26899026s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-20 01:52:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-20 01:53:14.537580089 +0000 UTC m=+51.280259297" watchObservedRunningTime="2026-09-20 01:53:14.538048576 +0000 UTC m=+51.280727786"
Sep 20 01:53:14.547007 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:14.546574    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:53:14.555195 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:14.555144    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-pxh5z" podStartSLOduration=11.201397139000001 podStartE2EDuration="25.555120317s" totalImagesPullingTime="14.353723178s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-20 01:52:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-20 01:53:14.553636813 +0000 UTC m=+51.296316012" watchObservedRunningTime="2026-09-20 01:53:14.555120317 +0000 UTC m=+51.297799523"
Sep 20 01:53:14.580261 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:14.580168    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-6f9c644dd9-gm67p" podStartSLOduration=11.200105846 podStartE2EDuration="25.580139283s" totalImagesPullingTime="14.380033437s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-20 01:52:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-20 01:53:14.578735469 +0000 UTC m=+51.321414675" watchObservedRunningTime="2026-09-20 01:53:14.580139283 +0000 UTC m=+51.322818485"
Sep 20 01:53:14.621096 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:14.618969    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-q2gls" podStartSLOduration=11.095278394 podStartE2EDuration="25.618947403s" totalImagesPullingTime="14.523669009s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-20 01:52:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-20 01:53:14.618025343 +0000 UTC m=+51.360704554" watchObservedRunningTime="2026-09-20 01:53:14.618947403 +0000 UTC m=+51.361626611"
Sep 20 01:53:14.641160 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:14.641023    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-858b56ff6c-5f29j" podStartSLOduration=25.078327066 podStartE2EDuration="39.641000866s" totalImagesPullingTime="14.5626738s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-20 01:52:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-20 01:53:14.64074011 +0000 UTC m=+51.383419316" watchObservedRunningTime="2026-09-20 01:53:14.641000866 +0000 UTC m=+51.383680067"
Sep 20 01:53:14.650342 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:14.650287    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:53:14.688043 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:14.687721    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-4dsc5" podStartSLOduration=2.166067852 podStartE2EDuration="16.687697372s" totalImagesPullingTime="14.52162952s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-20 01:52:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-20 01:53:14.661101958 +0000 UTC m=+51.403781162" watchObservedRunningTime="2026-09-20 01:53:14.687697372 +0000 UTC m=+51.430376577"
Sep 20 01:53:14.688557 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:14.688304    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-848dd94bd6-xfg9p" podStartSLOduration=25.148487372 podStartE2EDuration="39.68829182s" totalImagesPullingTime="14.539804448s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-20 01:52:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-20 01:53:14.686507373 +0000 UTC m=+51.429186579" watchObservedRunningTime="2026-09-20 01:53:14.68829182 +0000 UTC m=+51.430971027"
Sep 20 01:53:14.719426 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:14.719386    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:53:15.410612 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:15.410570    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:53:15.510866 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:15.510823    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:53:15.532575 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:15.532547    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-ztdt2_ece82f19-68b5-4810-be50-d8678cfdee02/console-operator/1.log"
Sep 20 01:53:15.533093 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:15.533072    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-ztdt2_ece82f19-68b5-4810-be50-d8678cfdee02/console-operator/0.log"
Sep 20 01:53:15.533213 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:15.533118    2584 generic.go:356] "Generic (PLEG): container finished" podID="ece82f19-68b5-4810-be50-d8678cfdee02" containerID="b1c16fb2c81564e97565a45d1177c35ecfba60c40d1ce2c27f6c3d17b6cd6bed" exitCode=255
Sep 20 01:53:15.533213 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:15.533198    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-fd4bb46c-ztdt2" event={"ID":"ece82f19-68b5-4810-be50-d8678cfdee02","Type":"ContainerDied","Data":"b1c16fb2c81564e97565a45d1177c35ecfba60c40d1ce2c27f6c3d17b6cd6bed"}
Sep 20 01:53:15.533319 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:15.533274    2584 scope.go:117] "RemoveContainer" containerID="603beccdeb373ab18389d312eedaa091903af40e6fe96b69e8e1d5bd40d911e8"
Sep 20 01:53:15.533947 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:15.533931    2584 scope.go:117] "RemoveContainer" containerID="b1c16fb2c81564e97565a45d1177c35ecfba60c40d1ce2c27f6c3d17b6cd6bed"
Sep 20 01:53:15.534148 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:53:15.534128    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-fd4bb46c-ztdt2_openshift-console-operator(ece82f19-68b5-4810-be50-d8678cfdee02)\"" pod="openshift-console-operator/console-operator-fd4bb46c-ztdt2" podUID="ece82f19-68b5-4810-be50-d8678cfdee02"
Sep 20 01:53:15.537002 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:15.536970    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77cf88f65f-9qp8b" event={"ID":"536c3d86-41b0-4e73-9eed-7acbd8b86920","Type":"ContainerStarted","Data":"8ad2e1bed8e9d0b5a03e880416a9833016817a8873cf31c241cf5e671bfeb55a"}
Sep 20 01:53:15.568040 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:15.567988    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77cf88f65f-9qp8b" podStartSLOduration=26.073155144 podStartE2EDuration="40.567971891s" totalImagesPullingTime="14.494816747s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-20 01:52:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-20 01:53:15.567171457 +0000 UTC m=+52.309850681" watchObservedRunningTime="2026-09-20 01:53:15.567971891 +0000 UTC m=+52.310651110"
Sep 20 01:53:15.865775 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:15.865736    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:53:15.942656 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:15.942621    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:53:15.947081 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:15.947055    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-bjhc4"]
Sep 20 01:53:15.950572 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:15.950550    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-bjhc4"
Sep 20 01:53:15.952820 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:15.952802    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 20 01:53:15.952820 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:15.952814    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 20 01:53:15.952968 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:15.952892    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-fcgnp\""
Sep 20 01:53:15.960772 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:15.960739    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-bjhc4"]
Sep 20 01:53:16.020226 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:16.020192    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:53:16.051409 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:16.051375    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/62740de5-15dc-4e29-aa81-731ab68b2b3a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bjhc4\" (UID: \"62740de5-15dc-4e29-aa81-731ab68b2b3a\") " pod="openshift-insights/insights-runtime-extractor-bjhc4"
Sep 20 01:53:16.051595 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:16.051425    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-847h8\" (UniqueName: \"kubernetes.io/projected/62740de5-15dc-4e29-aa81-731ab68b2b3a-kube-api-access-847h8\") pod \"insights-runtime-extractor-bjhc4\" (UID: \"62740de5-15dc-4e29-aa81-731ab68b2b3a\") " pod="openshift-insights/insights-runtime-extractor-bjhc4"
Sep 20 01:53:16.051595 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:16.051448    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/62740de5-15dc-4e29-aa81-731ab68b2b3a-crio-socket\") pod \"insights-runtime-extractor-bjhc4\" (UID: \"62740de5-15dc-4e29-aa81-731ab68b2b3a\") " pod="openshift-insights/insights-runtime-extractor-bjhc4"
Sep 20 01:53:16.051595 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:16.051582    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/62740de5-15dc-4e29-aa81-731ab68b2b3a-data-volume\") pod \"insights-runtime-extractor-bjhc4\" (UID: \"62740de5-15dc-4e29-aa81-731ab68b2b3a\") " pod="openshift-insights/insights-runtime-extractor-bjhc4"
Sep 20 01:53:16.051744 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:16.051625    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/62740de5-15dc-4e29-aa81-731ab68b2b3a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bjhc4\" (UID: \"62740de5-15dc-4e29-aa81-731ab68b2b3a\") " pod="openshift-insights/insights-runtime-extractor-bjhc4"
Sep 20 01:53:16.124188 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:16.124091    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:53:16.152385 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:16.152345    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/62740de5-15dc-4e29-aa81-731ab68b2b3a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bjhc4\" (UID: \"62740de5-15dc-4e29-aa81-731ab68b2b3a\") " pod="openshift-insights/insights-runtime-extractor-bjhc4"
Sep 20 01:53:16.152597 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:16.152401    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-847h8\" (UniqueName: \"kubernetes.io/projected/62740de5-15dc-4e29-aa81-731ab68b2b3a-kube-api-access-847h8\") pod \"insights-runtime-extractor-bjhc4\" (UID: \"62740de5-15dc-4e29-aa81-731ab68b2b3a\") " pod="openshift-insights/insights-runtime-extractor-bjhc4"
Sep 20 01:53:16.152597 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:53:16.152473    2584 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 20 01:53:16.152597 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:16.152512    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/62740de5-15dc-4e29-aa81-731ab68b2b3a-crio-socket\") pod \"insights-runtime-extractor-bjhc4\" (UID: \"62740de5-15dc-4e29-aa81-731ab68b2b3a\") " pod="openshift-insights/insights-runtime-extractor-bjhc4"
Sep 20 01:53:16.152597 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:53:16.152553    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/62740de5-15dc-4e29-aa81-731ab68b2b3a-kube-rbac-proxy-cm podName:62740de5-15dc-4e29-aa81-731ab68b2b3a nodeName:}" failed. No retries permitted until 2026-09-20 01:53:16.652535975 +0000 UTC m=+53.395215171 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/62740de5-15dc-4e29-aa81-731ab68b2b3a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-bjhc4" (UID: "62740de5-15dc-4e29-aa81-731ab68b2b3a") : configmap "kube-rbac-proxy" not found
Sep 20 01:53:16.152814 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:16.152608    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/62740de5-15dc-4e29-aa81-731ab68b2b3a-crio-socket\") pod \"insights-runtime-extractor-bjhc4\" (UID: \"62740de5-15dc-4e29-aa81-731ab68b2b3a\") " pod="openshift-insights/insights-runtime-extractor-bjhc4"
Sep 20 01:53:16.152814 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:16.152622    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/62740de5-15dc-4e29-aa81-731ab68b2b3a-data-volume\") pod \"insights-runtime-extractor-bjhc4\" (UID: \"62740de5-15dc-4e29-aa81-731ab68b2b3a\") " pod="openshift-insights/insights-runtime-extractor-bjhc4"
Sep 20 01:53:16.152814 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:16.152676    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/62740de5-15dc-4e29-aa81-731ab68b2b3a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bjhc4\" (UID: \"62740de5-15dc-4e29-aa81-731ab68b2b3a\") " pod="openshift-insights/insights-runtime-extractor-bjhc4"
Sep 20 01:53:16.152960 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:53:16.152867    2584 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 20 01:53:16.152960 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:16.152896    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/62740de5-15dc-4e29-aa81-731ab68b2b3a-data-volume\") pod \"insights-runtime-extractor-bjhc4\" (UID: \"62740de5-15dc-4e29-aa81-731ab68b2b3a\") " pod="openshift-insights/insights-runtime-extractor-bjhc4"
Sep 20 01:53:16.152960 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:53:16.152938    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/62740de5-15dc-4e29-aa81-731ab68b2b3a-insights-runtime-extractor-tls podName:62740de5-15dc-4e29-aa81-731ab68b2b3a nodeName:}" failed. No retries permitted until 2026-09-20 01:53:16.652921891 +0000 UTC m=+53.395601076 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/62740de5-15dc-4e29-aa81-731ab68b2b3a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bjhc4" (UID: "62740de5-15dc-4e29-aa81-731ab68b2b3a") : secret "insights-runtime-extractor-tls" not found
Sep 20 01:53:16.166342 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:16.166316    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-847h8\" (UniqueName: \"kubernetes.io/projected/62740de5-15dc-4e29-aa81-731ab68b2b3a-kube-api-access-847h8\") pod \"insights-runtime-extractor-bjhc4\" (UID: \"62740de5-15dc-4e29-aa81-731ab68b2b3a\") " pod="openshift-insights/insights-runtime-extractor-bjhc4"
Sep 20 01:53:16.220780 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:16.220748    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:53:16.542117 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:16.542089    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-ztdt2_ece82f19-68b5-4810-be50-d8678cfdee02/console-operator/1.log"
Sep 20 01:53:16.542467 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:16.542448    2584 scope.go:117] "RemoveContainer" containerID="b1c16fb2c81564e97565a45d1177c35ecfba60c40d1ce2c27f6c3d17b6cd6bed"
Sep 20 01:53:16.542663 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:53:16.542642    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-fd4bb46c-ztdt2_openshift-console-operator(ece82f19-68b5-4810-be50-d8678cfdee02)\"" pod="openshift-console-operator/console-operator-fd4bb46c-ztdt2" podUID="ece82f19-68b5-4810-be50-d8678cfdee02"
Sep 20 01:53:16.658836 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:16.658794    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/62740de5-15dc-4e29-aa81-731ab68b2b3a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bjhc4\" (UID: \"62740de5-15dc-4e29-aa81-731ab68b2b3a\") " pod="openshift-insights/insights-runtime-extractor-bjhc4"
Sep 20 01:53:16.659031 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:53:16.658917    2584 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 20 01:53:16.659031 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:16.658937    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/62740de5-15dc-4e29-aa81-731ab68b2b3a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bjhc4\" (UID: \"62740de5-15dc-4e29-aa81-731ab68b2b3a\") " pod="openshift-insights/insights-runtime-extractor-bjhc4"
Sep 20 01:53:16.659137 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:53:16.659034    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/62740de5-15dc-4e29-aa81-731ab68b2b3a-kube-rbac-proxy-cm podName:62740de5-15dc-4e29-aa81-731ab68b2b3a nodeName:}" failed. No retries permitted until 2026-09-20 01:53:17.659014141 +0000 UTC m=+54.401693325 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/62740de5-15dc-4e29-aa81-731ab68b2b3a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-bjhc4" (UID: "62740de5-15dc-4e29-aa81-731ab68b2b3a") : configmap "kube-rbac-proxy" not found
Sep 20 01:53:16.659137 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:53:16.659107    2584 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 20 01:53:16.659236 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:53:16.659176    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/62740de5-15dc-4e29-aa81-731ab68b2b3a-insights-runtime-extractor-tls podName:62740de5-15dc-4e29-aa81-731ab68b2b3a nodeName:}" failed. No retries permitted until 2026-09-20 01:53:17.659158396 +0000 UTC m=+54.401837599 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/62740de5-15dc-4e29-aa81-731ab68b2b3a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bjhc4" (UID: "62740de5-15dc-4e29-aa81-731ab68b2b3a") : secret "insights-runtime-extractor-tls" not found
Sep 20 01:53:17.044741 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:17.044709    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:53:17.139450 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:17.139409    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:53:17.667705 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:17.667673    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/62740de5-15dc-4e29-aa81-731ab68b2b3a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bjhc4\" (UID: \"62740de5-15dc-4e29-aa81-731ab68b2b3a\") " pod="openshift-insights/insights-runtime-extractor-bjhc4"
Sep 20 01:53:17.667880 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:53:17.667807    2584 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 20 01:53:17.667921 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:17.667870    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/62740de5-15dc-4e29-aa81-731ab68b2b3a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bjhc4\" (UID: \"62740de5-15dc-4e29-aa81-731ab68b2b3a\") " pod="openshift-insights/insights-runtime-extractor-bjhc4"
Sep 20 01:53:17.667961 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:53:17.667952    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/62740de5-15dc-4e29-aa81-731ab68b2b3a-kube-rbac-proxy-cm podName:62740de5-15dc-4e29-aa81-731ab68b2b3a nodeName:}" failed. No retries permitted until 2026-09-20 01:53:19.667938272 +0000 UTC m=+56.410617455 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/62740de5-15dc-4e29-aa81-731ab68b2b3a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-bjhc4" (UID: "62740de5-15dc-4e29-aa81-731ab68b2b3a") : configmap "kube-rbac-proxy" not found
Sep 20 01:53:17.668001 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:53:17.667984    2584 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 20 01:53:17.668046 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:53:17.668037    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/62740de5-15dc-4e29-aa81-731ab68b2b3a-insights-runtime-extractor-tls podName:62740de5-15dc-4e29-aa81-731ab68b2b3a nodeName:}" failed. No retries permitted until 2026-09-20 01:53:19.66802606 +0000 UTC m=+56.410705243 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/62740de5-15dc-4e29-aa81-731ab68b2b3a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bjhc4" (UID: "62740de5-15dc-4e29-aa81-731ab68b2b3a") : secret "insights-runtime-extractor-tls" not found
Sep 20 01:53:17.771575 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:17.771541    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:53:17.824944 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:17.824910    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:53:17.897468 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:17.897438    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-l9ssk_43dd43c0-ff48-4f9a-a1c5-9020836702aa/dns-node-resolver/0.log"
Sep 20 01:53:18.645182 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:18.645153    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:53:18.756815 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:18.756781    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-fd4bb46c-ztdt2"
Sep 20 01:53:18.756994 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:18.756845    2584 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-fd4bb46c-ztdt2"
Sep 20 01:53:18.757199 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:18.757187    2584 scope.go:117] "RemoveContainer" containerID="b1c16fb2c81564e97565a45d1177c35ecfba60c40d1ce2c27f6c3d17b6cd6bed"
Sep 20 01:53:18.757354 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:53:18.757337    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-fd4bb46c-ztdt2_openshift-console-operator(ece82f19-68b5-4810-be50-d8678cfdee02)\"" pod="openshift-console-operator/console-operator-fd4bb46c-ztdt2" podUID="ece82f19-68b5-4810-be50-d8678cfdee02"
Sep 20 01:53:19.096904 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:19.096880    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-v6d2k_4fa3e7e2-7940-4d7d-af23-63ff07edba43/node-ca/0.log"
Sep 20 01:53:19.497377 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:19.497347    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-4dsc5_2075981b-d064-43eb-ab90-373b95ad49f4/serve-healthcheck-canary/0.log"
Sep 20 01:53:19.688827 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:19.688791    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/62740de5-15dc-4e29-aa81-731ab68b2b3a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bjhc4\" (UID: \"62740de5-15dc-4e29-aa81-731ab68b2b3a\") " pod="openshift-insights/insights-runtime-extractor-bjhc4"
Sep 20 01:53:19.689202 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:19.688898    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/62740de5-15dc-4e29-aa81-731ab68b2b3a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bjhc4\" (UID: \"62740de5-15dc-4e29-aa81-731ab68b2b3a\") " pod="openshift-insights/insights-runtime-extractor-bjhc4"
Sep 20 01:53:19.689202 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:53:19.688948    2584 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 20 01:53:19.689202 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:53:19.688973    2584 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 20 01:53:19.689202 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:53:19.689011    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/62740de5-15dc-4e29-aa81-731ab68b2b3a-insights-runtime-extractor-tls podName:62740de5-15dc-4e29-aa81-731ab68b2b3a nodeName:}" failed. No retries permitted until 2026-09-20 01:53:23.688995597 +0000 UTC m=+60.431674779 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/62740de5-15dc-4e29-aa81-731ab68b2b3a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bjhc4" (UID: "62740de5-15dc-4e29-aa81-731ab68b2b3a") : secret "insights-runtime-extractor-tls" not found
Sep 20 01:53:19.689202 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:53:19.689027    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/62740de5-15dc-4e29-aa81-731ab68b2b3a-kube-rbac-proxy-cm podName:62740de5-15dc-4e29-aa81-731ab68b2b3a nodeName:}" failed. No retries permitted until 2026-09-20 01:53:23.689019462 +0000 UTC m=+60.431698645 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/62740de5-15dc-4e29-aa81-731ab68b2b3a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-bjhc4" (UID: "62740de5-15dc-4e29-aa81-731ab68b2b3a") : configmap "kube-rbac-proxy" not found
Sep 20 01:53:21.230400 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:21.230364    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:53:21.417806 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:21.417775    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:53:21.519543 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:21.519453    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:53:21.646205 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:21.646175    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:53:21.730612 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:21.730579    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:53:22.099360 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:22.099331    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:53:22.709872 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:22.709836    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:53:23.096652 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:23.096569    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:53:23.496561 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:23.496506    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:53:23.702090 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:23.702049    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:53:23.729303 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:23.729273    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/62740de5-15dc-4e29-aa81-731ab68b2b3a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bjhc4\" (UID: \"62740de5-15dc-4e29-aa81-731ab68b2b3a\") " pod="openshift-insights/insights-runtime-extractor-bjhc4"
Sep 20 01:53:23.729712 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:23.729374    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/62740de5-15dc-4e29-aa81-731ab68b2b3a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bjhc4\" (UID: \"62740de5-15dc-4e29-aa81-731ab68b2b3a\") " pod="openshift-insights/insights-runtime-extractor-bjhc4"
Sep 20 01:53:23.729712 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:53:23.729428    2584 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 20 01:53:23.729712 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:53:23.729457    2584 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 20 01:53:23.729712 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:53:23.729505    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/62740de5-15dc-4e29-aa81-731ab68b2b3a-kube-rbac-proxy-cm podName:62740de5-15dc-4e29-aa81-731ab68b2b3a nodeName:}" failed. No retries permitted until 2026-09-20 01:53:31.729485547 +0000 UTC m=+68.472164734 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/62740de5-15dc-4e29-aa81-731ab68b2b3a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-bjhc4" (UID: "62740de5-15dc-4e29-aa81-731ab68b2b3a") : configmap "kube-rbac-proxy" not found
Sep 20 01:53:23.729712 ip-10-0-130-159 kubenswrapper[2584]: E0920 01:53:23.729546    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/62740de5-15dc-4e29-aa81-731ab68b2b3a-insights-runtime-extractor-tls podName:62740de5-15dc-4e29-aa81-731ab68b2b3a nodeName:}" failed. No retries permitted until 2026-09-20 01:53:31.729513091 +0000 UTC m=+68.472192280 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/62740de5-15dc-4e29-aa81-731ab68b2b3a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bjhc4" (UID: "62740de5-15dc-4e29-aa81-731ab68b2b3a") : secret "insights-runtime-extractor-tls" not found
Sep 20 01:53:24.103301 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:24.103271    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:53:24.354668 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:24.354595    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-p2wkn"
Sep 20 01:53:24.698341 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:24.698306    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:53:24.991217 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:24.991122    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:53:25.091242 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:25.091200    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:53:25.141333 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:25.141297    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:53:25.496727 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:25.496692    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:53:25.698068 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:25.698033    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:53:25.754322 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:25.754234    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:53:26.296724 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:26.296693    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:53:26.898585 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:26.898544    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:53:27.000752 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:27.000716    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:53:27.100319 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:27.100282    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:53:27.151900 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:27.151800    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:53:27.298946 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:27.298909    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:53:27.598000 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:27.597970    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:53:27.698051 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:27.698023    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:53:28.500822 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:28.500772    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:53:28.897053 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:28.896956    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:53:29.096389 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:29.096337    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:53:29.302675 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:29.302624    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:53:29.707495 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:29.707455    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:53:29.785850 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:29.785807    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/05a98348-758f-4a4a-b150-35bbdeb5b359-original-pull-secret\") pod \"global-pull-secret-syncer-llv6h\" (UID: \"05a98348-758f-4a4a-b150-35bbdeb5b359\") " pod="kube-system/global-pull-secret-syncer-llv6h"
Sep 20 01:53:29.786025 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:29.785880    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c9c657b9-b508-4bb7-99f3-aed2502cfba4-metrics-certs\") pod \"network-metrics-daemon-n4dk7\" (UID: \"c9c657b9-b508-4bb7-99f3-aed2502cfba4\") " pod="openshift-multus/network-metrics-daemon-n4dk7"
Sep 20 01:53:29.788210 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:29.788183    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 20 01:53:29.788330 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:29.788193    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 20 01:53:29.799186 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:29.799160    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/05a98348-758f-4a4a-b150-35bbdeb5b359-original-pull-secret\") pod \"global-pull-secret-syncer-llv6h\" (UID: \"05a98348-758f-4a4a-b150-35bbdeb5b359\") " pod="kube-system/global-pull-secret-syncer-llv6h"
Sep 20 01:53:29.799293 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:29.799161    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c9c657b9-b508-4bb7-99f3-aed2502cfba4-metrics-certs\") pod \"network-metrics-daemon-n4dk7\" (UID: \"c9c657b9-b508-4bb7-99f3-aed2502cfba4\") " pod="openshift-multus/network-metrics-daemon-n4dk7"
Sep 20 01:53:29.804118 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:29.804098    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-llv6h"
Sep 20 01:53:29.825850 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:29.825827    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-jjlfs\""
Sep 20 01:53:29.833819 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:29.833778    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n4dk7"
Sep 20 01:53:29.887202 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:29.887166    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mrb2z\" (UniqueName: \"kubernetes.io/projected/9b376ae6-8f6a-4139-ab2d-96dc32e1201f-kube-api-access-mrb2z\") pod \"network-check-target-47994\" (UID: \"9b376ae6-8f6a-4139-ab2d-96dc32e1201f\") " pod="openshift-network-diagnostics/network-check-target-47994"
Sep 20 01:53:29.889943 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:29.889926    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mrb2z\" (UniqueName: \"kubernetes.io/projected/9b376ae6-8f6a-4139-ab2d-96dc32e1201f-kube-api-access-mrb2z\") pod \"network-check-target-47994\" (UID: \"9b376ae6-8f6a-4139-ab2d-96dc32e1201f\") " pod="openshift-network-diagnostics/network-check-target-47994"
Sep 20 01:53:30.076302 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:30.076159    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-llv6h"]
Sep 20 01:53:30.078873 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:53:30.078840    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05a98348_758f_4a4a_b150_35bbdeb5b359.slice/crio-9960ca9e4b98c6f1cbd34a15b142c297d7f32cac724ddda86d245a98d828cb99 WatchSource:0}: Error finding container 9960ca9e4b98c6f1cbd34a15b142c297d7f32cac724ddda86d245a98d828cb99: Status 404 returned error can't find the container with id 9960ca9e4b98c6f1cbd34a15b142c297d7f32cac724ddda86d245a98d828cb99
Sep 20 01:53:30.088124 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:30.088096    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:53:30.115571 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:30.115546    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-n4dk7"]
Sep 20 01:53:30.117097 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:30.117077    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-cvrtw\""
Sep 20 01:53:30.117808 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:53:30.117782    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9c657b9_b508_4bb7_99f3_aed2502cfba4.slice/crio-83527f103e5139aebfb1695066b40c2ae88b56e34e332f9385cbbdb382ad5e18 WatchSource:0}: Error finding container 83527f103e5139aebfb1695066b40c2ae88b56e34e332f9385cbbdb382ad5e18: Status 404 returned error can't find the container with id 83527f103e5139aebfb1695066b40c2ae88b56e34e332f9385cbbdb382ad5e18
Sep 20 01:53:30.125854 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:30.125835    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-47994"
Sep 20 01:53:30.246435 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:30.246393    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:53:30.290559 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:30.290503    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5d22d853-7ae7-48b3-8549-5e2bf2be770d-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-7mld6\" (UID: \"5d22d853-7ae7-48b3-8549-5e2bf2be770d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-7mld6"
Sep 20 01:53:30.290735 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:30.290580    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/492ce9d4-19e5-4e0c-8b86-fd63a3b456a1-registry-tls\") pod \"image-registry-5659b6b8c9-hzb2w\" (UID: \"492ce9d4-19e5-4e0c-8b86-fd63a3b456a1\") " pod="openshift-image-registry/image-registry-5659b6b8c9-hzb2w"
Sep 20 01:53:30.293049 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:30.293023    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5d22d853-7ae7-48b3-8549-5e2bf2be770d-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-7mld6\" (UID: \"5d22d853-7ae7-48b3-8549-5e2bf2be770d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-7mld6"
Sep 20 01:53:30.293049 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:30.293041    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/492ce9d4-19e5-4e0c-8b86-fd63a3b456a1-registry-tls\") pod \"image-registry-5659b6b8c9-hzb2w\" (UID: \"492ce9d4-19e5-4e0c-8b86-fd63a3b456a1\") " pod="openshift-image-registry/image-registry-5659b6b8c9-hzb2w"
Sep 20 01:53:30.347596 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:30.347493    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:53:30.391923 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:30.391890    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2dbc9737-865d-4dbf-85bf-8d2b9781a8d5-metrics-certs\") pod \"router-default-95688946b-7rdfv\" (UID: \"2dbc9737-865d-4dbf-85bf-8d2b9781a8d5\") " pod="openshift-ingress/router-default-95688946b-7rdfv"
Sep 20 01:53:30.391923 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:30.391926    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/092d6fa5-0734-4777-b6e0-2e1ef430076d-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-847b8bd8d7-bp5fk\" (UID: \"092d6fa5-0734-4777-b6e0-2e1ef430076d\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-bp5fk"
Sep 20 01:53:30.392131 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:30.391989    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2dbc9737-865d-4dbf-85bf-8d2b9781a8d5-service-ca-bundle\") pod \"router-default-95688946b-7rdfv\" (UID: \"2dbc9737-865d-4dbf-85bf-8d2b9781a8d5\") " pod="openshift-ingress/router-default-95688946b-7rdfv"
Sep 20 01:53:30.392686 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:30.392666    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2dbc9737-865d-4dbf-85bf-8d2b9781a8d5-service-ca-bundle\") pod \"router-default-95688946b-7rdfv\" (UID: \"2dbc9737-865d-4dbf-85bf-8d2b9781a8d5\") " pod="openshift-ingress/router-default-95688946b-7rdfv"
Sep 20 01:53:30.394304 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:30.394284    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/092d6fa5-0734-4777-b6e0-2e1ef430076d-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-847b8bd8d7-bp5fk\" (UID: \"092d6fa5-0734-4777-b6e0-2e1ef430076d\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-bp5fk"
Sep 20 01:53:30.394386 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:30.394316    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2dbc9737-865d-4dbf-85bf-8d2b9781a8d5-metrics-certs\") pod \"router-default-95688946b-7rdfv\" (UID: \"2dbc9737-865d-4dbf-85bf-8d2b9781a8d5\") " pod="openshift-ingress/router-default-95688946b-7rdfv"
Sep 20 01:53:30.396420 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:30.396400    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-47994"]
Sep 20 01:53:30.398754 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:53:30.398726    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b376ae6_8f6a_4139_ab2d_96dc32e1201f.slice/crio-de24d1993f3e64213ab616586f7095dcc971230cc745d330c7dcc84f79ceeb4a WatchSource:0}: Error finding container de24d1993f3e64213ab616586f7095dcc971230cc745d330c7dcc84f79ceeb4a: Status 404 returned error can't find the container with id de24d1993f3e64213ab616586f7095dcc971230cc745d330c7dcc84f79ceeb4a
Sep 20 01:53:30.492769 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:30.492731    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/54d123d3-fc8b-49ee-9e26-d4a3ee494a1d-networking-console-plugin-cert\") pod \"networking-console-plugin-6446c8d75d-fktjk\" (UID: \"54d123d3-fc8b-49ee-9e26-d4a3ee494a1d\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-fktjk"
Sep 20 01:53:30.492964 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:30.492840    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/79e39187-4356-422e-b5d3-9c9f8d3a9a58-metrics-tls\") pod \"dns-default-c9zbv\" (UID: \"79e39187-4356-422e-b5d3-9c9f8d3a9a58\") " pod="openshift-dns/dns-default-c9zbv"
Sep 20 01:53:30.495118 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:30.495094    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/79e39187-4356-422e-b5d3-9c9f8d3a9a58-metrics-tls\") pod \"dns-default-c9zbv\" (UID: \"79e39187-4356-422e-b5d3-9c9f8d3a9a58\") " pod="openshift-dns/dns-default-c9zbv"
Sep 20 01:53:30.495233 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:30.495122    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/54d123d3-fc8b-49ee-9e26-d4a3ee494a1d-networking-console-plugin-cert\") pod \"networking-console-plugin-6446c8d75d-fktjk\" (UID: \"54d123d3-fc8b-49ee-9e26-d4a3ee494a1d\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-fktjk"
Sep 20 01:53:30.518420 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:30.518393    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-k9d2x\""
Sep 20 01:53:30.526390 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:30.526364    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5659b6b8c9-hzb2w"
Sep 20 01:53:30.528571 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:30.528551    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-h7d26\""
Sep 20 01:53:30.537497 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:30.537473    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-7mld6"
Sep 20 01:53:30.590121 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:30.590064    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-llv6h" event={"ID":"05a98348-758f-4a4a-b150-35bbdeb5b359","Type":"ContainerStarted","Data":"9960ca9e4b98c6f1cbd34a15b142c297d7f32cac724ddda86d245a98d828cb99"}
Sep 20 01:53:30.591509 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:30.591485    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-47994" event={"ID":"9b376ae6-8f6a-4139-ab2d-96dc32e1201f","Type":"ContainerStarted","Data":"740bfc83499134c0ea585dd9528379d9cebfb4245e31d1eeaab5b964337ac2fc"}
Sep 20 01:53:30.591638 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:30.591546    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-47994" event={"ID":"9b376ae6-8f6a-4139-ab2d-96dc32e1201f","Type":"ContainerStarted","Data":"de24d1993f3e64213ab616586f7095dcc971230cc745d330c7dcc84f79ceeb4a"}
Sep 20 01:53:30.591911 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:30.591849    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-47994"
Sep 20 01:53:30.593036 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:30.593013    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-n4dk7" event={"ID":"c9c657b9-b508-4bb7-99f3-aed2502cfba4","Type":"ContainerStarted","Data":"83527f103e5139aebfb1695066b40c2ae88b56e34e332f9385cbbdb382ad5e18"}
Sep 20 01:53:30.596071 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:30.596018    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-fm975\""
Sep 20 01:53:30.604849 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:30.604822    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-bp5fk"
Sep 20 01:53:30.611808 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:30.611752    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-9j749\""
Sep 20 01:53:30.619685 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:30.619659    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-95688946b-7rdfv"
Sep 20 01:53:30.636465 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:30.636418    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-x2q4j\""
Sep 20 01:53:30.644802 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:30.644776    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6446c8d75d-fktjk"
Sep 20 01:53:30.665707 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:30.665651    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-fp8gx\""
Sep 20 01:53:30.674362 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:30.674043    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-c9zbv"
Sep 20 01:53:30.700353 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:30.700184    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:53:30.801644 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:30.800966    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-47994" podStartSLOduration=65.800944253 podStartE2EDuration="1m5.800944253s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-20 01:52:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-20 01:53:30.608305368 +0000 UTC m=+67.350984587" watchObservedRunningTime="2026-09-20 01:53:30.800944253 +0000 UTC m=+67.543623459"
Sep 20 01:53:30.803653 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:30.803578    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5659b6b8c9-hzb2w"]
Sep 20 01:53:30.806951 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:53:30.806917    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod492ce9d4_19e5_4e0c_8b86_fd63a3b456a1.slice/crio-1dff9b31fac2393db745430ecb925e7db872ba51e1d3ac7cfd89f953d18a2180 WatchSource:0}: Error finding container 1dff9b31fac2393db745430ecb925e7db872ba51e1d3ac7cfd89f953d18a2180: Status 404 returned error can't find the container with id 1dff9b31fac2393db745430ecb925e7db872ba51e1d3ac7cfd89f953d18a2180
Sep 20 01:53:30.823461 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:30.823433    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-7mld6"]
Sep 20 01:53:30.889676 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:30.889644    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-bp5fk"]
Sep 20 01:53:30.894206 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:53:30.894152    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod092d6fa5_0734_4777_b6e0_2e1ef430076d.slice/crio-afd7651f60e60a859366db0eb22050e1f470da7c3592a369c11aaa9e064f9209 WatchSource:0}: Error finding container afd7651f60e60a859366db0eb22050e1f470da7c3592a369c11aaa9e064f9209: Status 404 returned error can't find the container with id afd7651f60e60a859366db0eb22050e1f470da7c3592a369c11aaa9e064f9209
Sep 20 01:53:30.896570 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:30.896543    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:53:30.919318 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:30.919258    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-95688946b-7rdfv"]
Sep 20 01:53:30.922327 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:53:30.922295    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2dbc9737_865d_4dbf_85bf_8d2b9781a8d5.slice/crio-cc4a69cc500b64516915fc4632d02c794386a1efd33784c9c01b6e80b318b593 WatchSource:0}: Error finding container cc4a69cc500b64516915fc4632d02c794386a1efd33784c9c01b6e80b318b593: Status 404 returned error can't find the container with id cc4a69cc500b64516915fc4632d02c794386a1efd33784c9c01b6e80b318b593
Sep 20 01:53:30.943889 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:30.943767    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6446c8d75d-fktjk"]
Sep 20 01:53:30.947598 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:53:30.947565    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54d123d3_fc8b_49ee_9e26_d4a3ee494a1d.slice/crio-dcd58fc8b8cbb1e3f562b62d4c10fd6b5361cf745c3b8a5377c0281c96a75923 WatchSource:0}: Error finding container dcd58fc8b8cbb1e3f562b62d4c10fd6b5361cf745c3b8a5377c0281c96a75923: Status 404 returned error can't find the container with id dcd58fc8b8cbb1e3f562b62d4c10fd6b5361cf745c3b8a5377c0281c96a75923
Sep 20 01:53:30.989271 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:30.989166    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-c9zbv"]
Sep 20 01:53:30.993090 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:53:30.993055    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79e39187_4356_422e_b5d3_9c9f8d3a9a58.slice/crio-225d929306a3554de9f698431a1669260e9e2d85c95d590fa2c2104a0e6e27c5 WatchSource:0}: Error finding container 225d929306a3554de9f698431a1669260e9e2d85c95d590fa2c2104a0e6e27c5: Status 404 returned error can't find the container with id 225d929306a3554de9f698431a1669260e9e2d85c95d590fa2c2104a0e6e27c5
Sep 20 01:53:31.086916 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:31.086883    2584 scope.go:117] "RemoveContainer" containerID="b1c16fb2c81564e97565a45d1177c35ecfba60c40d1ce2c27f6c3d17b6cd6bed"
Sep 20 01:53:31.303677 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:31.303621    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:53:31.617737 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:31.615447    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-ztdt2_ece82f19-68b5-4810-be50-d8678cfdee02/console-operator/1.log"
Sep 20 01:53:31.617737 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:31.615565    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-fd4bb46c-ztdt2" event={"ID":"ece82f19-68b5-4810-be50-d8678cfdee02","Type":"ContainerStarted","Data":"23ab3061b73104928434c090bafef59d9c29654c982adeee9be24ef99abb0f1f"}
Sep 20 01:53:31.617737 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:31.617138    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-fd4bb46c-ztdt2"
Sep 20 01:53:31.621206 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:31.621171    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6446c8d75d-fktjk" event={"ID":"54d123d3-fc8b-49ee-9e26-d4a3ee494a1d","Type":"ContainerStarted","Data":"dcd58fc8b8cbb1e3f562b62d4c10fd6b5361cf745c3b8a5377c0281c96a75923"}
Sep 20 01:53:31.629798 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:31.629753    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-n4dk7" event={"ID":"c9c657b9-b508-4bb7-99f3-aed2502cfba4","Type":"ContainerStarted","Data":"fd27dea6f5105e6f93965e976550c1b9a246d8381e223652ceb440b2d45107e1"}
Sep 20 01:53:31.634862 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:31.634058    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-fd4bb46c-ztdt2" podStartSLOduration=28.096959258 podStartE2EDuration="42.634038281s" totalImagesPullingTime="14.537079023s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-20 01:52:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-20 01:53:31.63251452 +0000 UTC m=+68.375193726" watchObservedRunningTime="2026-09-20 01:53:31.634038281 +0000 UTC m=+68.376717487"
Sep 20 01:53:31.638348 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:31.638247    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5659b6b8c9-hzb2w" event={"ID":"492ce9d4-19e5-4e0c-8b86-fd63a3b456a1","Type":"ContainerStarted","Data":"72cf887ab33b8b302652857f17ce99191725ac32defc252653e4f29e0b7dcc86"}
Sep 20 01:53:31.638348 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:31.638301    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5659b6b8c9-hzb2w" event={"ID":"492ce9d4-19e5-4e0c-8b86-fd63a3b456a1","Type":"ContainerStarted","Data":"1dff9b31fac2393db745430ecb925e7db872ba51e1d3ac7cfd89f953d18a2180"}
Sep 20 01:53:31.645534 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:31.645486    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-c9zbv" event={"ID":"79e39187-4356-422e-b5d3-9c9f8d3a9a58","Type":"ContainerStarted","Data":"225d929306a3554de9f698431a1669260e9e2d85c95d590fa2c2104a0e6e27c5"}
Sep 20 01:53:31.648384 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:31.648353    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-95688946b-7rdfv" event={"ID":"2dbc9737-865d-4dbf-85bf-8d2b9781a8d5","Type":"ContainerStarted","Data":"a8bab122c83ae0205774992104a9e6a4c901bc44fe6163ef64a68048def05af7"}
Sep 20 01:53:31.648511 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:31.648408    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-95688946b-7rdfv" event={"ID":"2dbc9737-865d-4dbf-85bf-8d2b9781a8d5","Type":"ContainerStarted","Data":"cc4a69cc500b64516915fc4632d02c794386a1efd33784c9c01b6e80b318b593"}
Sep 20 01:53:31.650569 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:31.650542    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-7mld6" event={"ID":"5d22d853-7ae7-48b3-8549-5e2bf2be770d","Type":"ContainerStarted","Data":"2188e16dca25108c02ecef1296179ee2e9b361d157f3bbdb6fd6897c6592aca8"}
Sep 20 01:53:31.653407 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:31.653357    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-bp5fk" event={"ID":"092d6fa5-0734-4777-b6e0-2e1ef430076d","Type":"ContainerStarted","Data":"afd7651f60e60a859366db0eb22050e1f470da7c3592a369c11aaa9e064f9209"}
Sep 20 01:53:31.686424 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:31.685795    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-5659b6b8c9-hzb2w" podStartSLOduration=66.685770518 podStartE2EDuration="1m6.685770518s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-20 01:52:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-20 01:53:31.664565096 +0000 UTC m=+68.407244304" watchObservedRunningTime="2026-09-20 01:53:31.685770518 +0000 UTC m=+68.428449724"
Sep 20 01:53:31.709994 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:31.709843    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:53:31.772737 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:31.772707    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-fd4bb46c-ztdt2"
Sep 20 01:53:31.789720 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:31.789657    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-95688946b-7rdfv" podStartSLOduration=42.789631162 podStartE2EDuration="42.789631162s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-20 01:52:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-20 01:53:31.686983959 +0000 UTC m=+68.429663169" watchObservedRunningTime="2026-09-20 01:53:31.789631162 +0000 UTC m=+68.532310369"
Sep 20 01:53:31.819810 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:31.819719    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/62740de5-15dc-4e29-aa81-731ab68b2b3a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bjhc4\" (UID: \"62740de5-15dc-4e29-aa81-731ab68b2b3a\") " pod="openshift-insights/insights-runtime-extractor-bjhc4"
Sep 20 01:53:31.821214 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:31.821181    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/62740de5-15dc-4e29-aa81-731ab68b2b3a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bjhc4\" (UID: \"62740de5-15dc-4e29-aa81-731ab68b2b3a\") " pod="openshift-insights/insights-runtime-extractor-bjhc4"
Sep 20 01:53:31.821346 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:31.821248    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/62740de5-15dc-4e29-aa81-731ab68b2b3a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bjhc4\" (UID: \"62740de5-15dc-4e29-aa81-731ab68b2b3a\") " pod="openshift-insights/insights-runtime-extractor-bjhc4"
Sep 20 01:53:31.834897 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:31.830729    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/62740de5-15dc-4e29-aa81-731ab68b2b3a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bjhc4\" (UID: \"62740de5-15dc-4e29-aa81-731ab68b2b3a\") " pod="openshift-insights/insights-runtime-extractor-bjhc4"
Sep 20 01:53:31.863443 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:31.863222    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-fcgnp\""
Sep 20 01:53:31.871331 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:31.871009    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-bjhc4"
Sep 20 01:53:31.916932 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:31.916835    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:53:32.006872 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:32.006419    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:53:32.103850 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:32.103673    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:53:32.200149 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:32.200054    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-bjhc4"]
Sep 20 01:53:32.250158 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:32.250105    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:53:32.425843 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:32.425759    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:53:32.525813 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:32.525671    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:53:32.620804 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:32.620771    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-95688946b-7rdfv"
Sep 20 01:53:32.624068 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:32.623982    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-95688946b-7rdfv"
Sep 20 01:53:32.664061 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:32.664024    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-n4dk7" event={"ID":"c9c657b9-b508-4bb7-99f3-aed2502cfba4","Type":"ContainerStarted","Data":"187c7c9129c6e377042b760434520dbb2e0511e6ab78a24b5874954c956e9ef8"}
Sep 20 01:53:32.664587 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:32.664480    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-95688946b-7rdfv"
Sep 20 01:53:32.665337 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:32.665031    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-5659b6b8c9-hzb2w"
Sep 20 01:53:32.666185 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:32.666166    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-95688946b-7rdfv"
Sep 20 01:53:32.682328 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:32.682206    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-n4dk7" podStartSLOduration=66.38546881 podStartE2EDuration="1m7.682184916s" totalImagesPullingTime="1.296716106s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-20 01:52:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-20 01:53:32.680606503 +0000 UTC m=+69.423285708" watchObservedRunningTime="2026-09-20 01:53:32.682184916 +0000 UTC m=+69.424864121"
Sep 20 01:53:33.039065 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:33.038946    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:53:33.139897 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:33.139834    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:53:33.263787 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:33.263687    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:53:33.310688 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:53:33.310601    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62740de5_15dc_4e29_aa81_731ab68b2b3a.slice/crio-2be31b5084b5d0ac433014d6849a8472cafb5fddb1ee7e09119802e32c973396 WatchSource:0}: Error finding container 2be31b5084b5d0ac433014d6849a8472cafb5fddb1ee7e09119802e32c973396: Status 404 returned error can't find the container with id 2be31b5084b5d0ac433014d6849a8472cafb5fddb1ee7e09119802e32c973396
Sep 20 01:53:33.315600 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:33.315562    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:53:33.636190 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:33.636089    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:53:33.669933 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:33.669882    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bjhc4" event={"ID":"62740de5-15dc-4e29-aa81-731ab68b2b3a","Type":"ContainerStarted","Data":"2be31b5084b5d0ac433014d6849a8472cafb5fddb1ee7e09119802e32c973396"}
Sep 20 01:53:33.835875 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:33.835842    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:53:33.936089 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:33.936011    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:53:35.257751 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:35.257710    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:53:35.310914 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:35.310867    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:53:36.689713 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:36.689622    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-c9zbv" event={"ID":"79e39187-4356-422e-b5d3-9c9f8d3a9a58","Type":"ContainerStarted","Data":"35345e5c99abb49c71110fa14eb75c2673dcc0dfb9eda85fe236a6de127f3a98"}
Sep 20 01:53:36.697174 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:36.696993    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-7mld6" event={"ID":"5d22d853-7ae7-48b3-8549-5e2bf2be770d","Type":"ContainerStarted","Data":"8678ffa79d3f0759961120a3f151c3c81d5988366bf1c391642dffcc3bfac365"}
Sep 20 01:53:36.699507 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:36.699430    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bjhc4" event={"ID":"62740de5-15dc-4e29-aa81-731ab68b2b3a","Type":"ContainerStarted","Data":"63245094e676b52b55dea556a5f63c09687ec594fc5ab622c43c06d280165131"}
Sep 20 01:53:36.705376 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:36.704600    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-bp5fk" event={"ID":"092d6fa5-0734-4777-b6e0-2e1ef430076d","Type":"ContainerStarted","Data":"5ed573fbdbf068da05166a3356b51412061b0c6c35cc6f10c47d58aaa8ff66e5"}
Sep 20 01:53:36.710382 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:36.710332    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-llv6h" event={"ID":"05a98348-758f-4a4a-b150-35bbdeb5b359","Type":"ContainerStarted","Data":"13e776fc84ed29abe90d086276fbdf999b02d75ab8f077b7b4955632eab774d3"}
Sep 20 01:53:36.716609 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:36.715836    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6446c8d75d-fktjk" event={"ID":"54d123d3-fc8b-49ee-9e26-d4a3ee494a1d","Type":"ContainerStarted","Data":"b093e17d9519ac11208ddd19acd5b22916936ca16c33e69f02a0d490a185fd78"}
Sep 20 01:53:36.739150 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:36.739086    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-bp5fk" podStartSLOduration=42.212906403 podStartE2EDuration="47.739059044s" totalImagesPullingTime="5.526152641s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-20 01:52:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-20 01:53:36.723825272 +0000 UTC m=+73.466504479" watchObservedRunningTime="2026-09-20 01:53:36.739059044 +0000 UTC m=+73.481738250"
Sep 20 01:53:36.739309 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:36.739205    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6446c8d75d-fktjk" podStartSLOduration=37.268306347 podStartE2EDuration="42.739195909s" totalImagesPullingTime="5.470889562s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-20 01:52:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-20 01:53:36.737971836 +0000 UTC m=+73.480651039" watchObservedRunningTime="2026-09-20 01:53:36.739195909 +0000 UTC m=+73.481875116"
Sep 20 01:53:36.754132 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:36.754039    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-llv6h" podStartSLOduration=65.402264496 podStartE2EDuration="1m11.754017808s" totalImagesPullingTime="6.351753312s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-20 01:52:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-20 01:53:36.75324686 +0000 UTC m=+73.495926064" watchObservedRunningTime="2026-09-20 01:53:36.754017808 +0000 UTC m=+73.496697014"
Sep 20 01:53:36.842263 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:36.842220    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:53:36.994971 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:36.994933    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:53:37.107798 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:37.107311    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:53:37.722110 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:37.722071    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-c9zbv" event={"ID":"79e39187-4356-422e-b5d3-9c9f8d3a9a58","Type":"ContainerStarted","Data":"dbdb0cb2bd656c8a43a5139a74b418f4fa9486c87f81ed0da7dceaf9a60cdddf"}
Sep 20 01:53:37.722601 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:37.722228    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-c9zbv"
Sep 20 01:53:37.723917 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:37.723894    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-7mld6" event={"ID":"5d22d853-7ae7-48b3-8549-5e2bf2be770d","Type":"ContainerStarted","Data":"d7389447c00625039b45633376b9292070bebfbfc2b13951717fc99d53a28dcc"}
Sep 20 01:53:37.730058 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:37.730029    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bjhc4" event={"ID":"62740de5-15dc-4e29-aa81-731ab68b2b3a","Type":"ContainerStarted","Data":"f04c2543172bfd6caf691a7c3ba41b61982f3885807db1d1a108cfe7c5937995"}
Sep 20 01:53:37.738847 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:37.738803    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-c9zbv" podStartSLOduration=34.305002934 podStartE2EDuration="39.738788592s" totalImagesPullingTime="5.433785658s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-20 01:52:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-20 01:53:37.737722649 +0000 UTC m=+74.480401855" watchObservedRunningTime="2026-09-20 01:53:37.738788592 +0000 UTC m=+74.481467796"
Sep 20 01:53:37.754371 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:37.754278    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-7mld6" podStartSLOduration=43.224984473 podStartE2EDuration="48.754261494s" totalImagesPullingTime="5.529277021s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-20 01:52:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-20 01:53:37.753147064 +0000 UTC m=+74.495826294" watchObservedRunningTime="2026-09-20 01:53:37.754261494 +0000 UTC m=+74.496940699"
Sep 20 01:53:38.567585 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:38.567441    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:53:38.626862 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:38.626832    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:53:39.740980 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:39.740949    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bjhc4" event={"ID":"62740de5-15dc-4e29-aa81-731ab68b2b3a","Type":"ContainerStarted","Data":"f0e9f56d1a59c03c72e6a425fd16ce6bdb6c3d7d854e12a8d249ff3d51b7820d"}
Sep 20 01:53:39.758292 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:39.758240    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-bjhc4" podStartSLOduration=22.01236828 podStartE2EDuration="24.758220466s" totalImagesPullingTime="2.745852186s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-20 01:53:15 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-20 01:53:39.756901306 +0000 UTC m=+76.499580511" watchObservedRunningTime="2026-09-20 01:53:39.758220466 +0000 UTC m=+76.500899672"
Sep 20 01:53:41.117361 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:41.117321    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:53:41.204191 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:41.204153    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:53:41.302652 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:41.302611    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:53:41.402663 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:41.402587    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:53:41.561801 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:41.561765    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:53:41.667667 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:41.667573    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:53:44.762599 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:44.762560    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:53:44.865798 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:44.865756    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:53:45.028632 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:45.028541    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:53:45.129369 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:45.129331    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:53:45.355422 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:45.355339    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:53:46.659827 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:46.659776    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:53:46.760586 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:46.760550    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:53:47.647748 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:47.647716    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:53:47.737548 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:47.737497    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-c9zbv"
Sep 20 01:53:47.840654 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:47.840616    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:53:47.894281 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:47.894245    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:53:49.186897 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:49.186859    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:53:49.307650 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:49.307612    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:53:49.411485 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:49.411452    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:53:49.420272 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:49.420241    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-sdbvz"]
Sep 20 01:53:49.424405 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:49.424380    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-sdbvz"
Sep 20 01:53:49.426767 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:49.426734    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 20 01:53:49.426933 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:49.426913    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 20 01:53:49.427576 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:49.427558    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 20 01:53:49.427681 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:49.427616    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-bclt5\""
Sep 20 01:53:49.427681 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:49.427615    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 20 01:53:49.475017 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:49.474981    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d088ddb2-d975-4caa-807e-5dad7943101a-node-exporter-accelerators-collector-config\") pod \"node-exporter-sdbvz\" (UID: \"d088ddb2-d975-4caa-807e-5dad7943101a\") " pod="openshift-monitoring/node-exporter-sdbvz"
Sep 20 01:53:49.475210 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:49.475026    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d088ddb2-d975-4caa-807e-5dad7943101a-node-exporter-wtmp\") pod \"node-exporter-sdbvz\" (UID: \"d088ddb2-d975-4caa-807e-5dad7943101a\") " pod="openshift-monitoring/node-exporter-sdbvz"
Sep 20 01:53:49.475210 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:49.475081    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d088ddb2-d975-4caa-807e-5dad7943101a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-sdbvz\" (UID: \"d088ddb2-d975-4caa-807e-5dad7943101a\") " pod="openshift-monitoring/node-exporter-sdbvz"
Sep 20 01:53:49.475210 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:49.475114    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d088ddb2-d975-4caa-807e-5dad7943101a-metrics-client-ca\") pod \"node-exporter-sdbvz\" (UID: \"d088ddb2-d975-4caa-807e-5dad7943101a\") " pod="openshift-monitoring/node-exporter-sdbvz"
Sep 20 01:53:49.475210 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:49.475136    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d088ddb2-d975-4caa-807e-5dad7943101a-sys\") pod \"node-exporter-sdbvz\" (UID: \"d088ddb2-d975-4caa-807e-5dad7943101a\") " pod="openshift-monitoring/node-exporter-sdbvz"
Sep 20 01:53:49.475210 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:49.475159    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d088ddb2-d975-4caa-807e-5dad7943101a-node-exporter-tls\") pod \"node-exporter-sdbvz\" (UID: \"d088ddb2-d975-4caa-807e-5dad7943101a\") " pod="openshift-monitoring/node-exporter-sdbvz"
Sep 20 01:53:49.475210 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:49.475193    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k58rv\" (UniqueName: \"kubernetes.io/projected/d088ddb2-d975-4caa-807e-5dad7943101a-kube-api-access-k58rv\") pod \"node-exporter-sdbvz\" (UID: \"d088ddb2-d975-4caa-807e-5dad7943101a\") " pod="openshift-monitoring/node-exporter-sdbvz"
Sep 20 01:53:49.475455 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:49.475241    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d088ddb2-d975-4caa-807e-5dad7943101a-node-exporter-textfile\") pod \"node-exporter-sdbvz\" (UID: \"d088ddb2-d975-4caa-807e-5dad7943101a\") " pod="openshift-monitoring/node-exporter-sdbvz"
Sep 20 01:53:49.475455 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:49.475274    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d088ddb2-d975-4caa-807e-5dad7943101a-root\") pod \"node-exporter-sdbvz\" (UID: \"d088ddb2-d975-4caa-807e-5dad7943101a\") " pod="openshift-monitoring/node-exporter-sdbvz"
Sep 20 01:53:49.488729 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:49.488692    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:53:49.575720 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:49.575676    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d088ddb2-d975-4caa-807e-5dad7943101a-node-exporter-textfile\") pod \"node-exporter-sdbvz\" (UID: \"d088ddb2-d975-4caa-807e-5dad7943101a\") " pod="openshift-monitoring/node-exporter-sdbvz"
Sep 20 01:53:49.575720 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:49.575731    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d088ddb2-d975-4caa-807e-5dad7943101a-root\") pod \"node-exporter-sdbvz\" (UID: \"d088ddb2-d975-4caa-807e-5dad7943101a\") " pod="openshift-monitoring/node-exporter-sdbvz"
Sep 20 01:53:49.575988 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:49.575767    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d088ddb2-d975-4caa-807e-5dad7943101a-node-exporter-accelerators-collector-config\") pod \"node-exporter-sdbvz\" (UID: \"d088ddb2-d975-4caa-807e-5dad7943101a\") " pod="openshift-monitoring/node-exporter-sdbvz"
Sep 20 01:53:49.575988 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:49.575791    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d088ddb2-d975-4caa-807e-5dad7943101a-root\") pod \"node-exporter-sdbvz\" (UID: \"d088ddb2-d975-4caa-807e-5dad7943101a\") " pod="openshift-monitoring/node-exporter-sdbvz"
Sep 20 01:53:49.575988 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:49.575794    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d088ddb2-d975-4caa-807e-5dad7943101a-node-exporter-wtmp\") pod \"node-exporter-sdbvz\" (UID: \"d088ddb2-d975-4caa-807e-5dad7943101a\") " pod="openshift-monitoring/node-exporter-sdbvz"
Sep 20 01:53:49.575988 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:49.575875    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d088ddb2-d975-4caa-807e-5dad7943101a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-sdbvz\" (UID: \"d088ddb2-d975-4caa-807e-5dad7943101a\") " pod="openshift-monitoring/node-exporter-sdbvz"
Sep 20 01:53:49.575988 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:49.575898    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d088ddb2-d975-4caa-807e-5dad7943101a-node-exporter-wtmp\") pod \"node-exporter-sdbvz\" (UID: \"d088ddb2-d975-4caa-807e-5dad7943101a\") " pod="openshift-monitoring/node-exporter-sdbvz"
Sep 20 01:53:49.575988 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:49.575904    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d088ddb2-d975-4caa-807e-5dad7943101a-metrics-client-ca\") pod \"node-exporter-sdbvz\" (UID: \"d088ddb2-d975-4caa-807e-5dad7943101a\") " pod="openshift-monitoring/node-exporter-sdbvz"
Sep 20 01:53:49.575988 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:49.575931    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d088ddb2-d975-4caa-807e-5dad7943101a-sys\") pod \"node-exporter-sdbvz\" (UID: \"d088ddb2-d975-4caa-807e-5dad7943101a\") " pod="openshift-monitoring/node-exporter-sdbvz"
Sep 20 01:53:49.575988 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:49.575953    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d088ddb2-d975-4caa-807e-5dad7943101a-node-exporter-tls\") pod \"node-exporter-sdbvz\" (UID: \"d088ddb2-d975-4caa-807e-5dad7943101a\") " pod="openshift-monitoring/node-exporter-sdbvz"
Sep 20 01:53:49.576310 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:49.575994    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k58rv\" (UniqueName: \"kubernetes.io/projected/d088ddb2-d975-4caa-807e-5dad7943101a-kube-api-access-k58rv\") pod \"node-exporter-sdbvz\" (UID: \"d088ddb2-d975-4caa-807e-5dad7943101a\") " pod="openshift-monitoring/node-exporter-sdbvz"
Sep 20 01:53:49.576310 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:49.575994    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d088ddb2-d975-4caa-807e-5dad7943101a-sys\") pod \"node-exporter-sdbvz\" (UID: \"d088ddb2-d975-4caa-807e-5dad7943101a\") " pod="openshift-monitoring/node-exporter-sdbvz"
Sep 20 01:53:49.576310 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:49.576141    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d088ddb2-d975-4caa-807e-5dad7943101a-node-exporter-textfile\") pod \"node-exporter-sdbvz\" (UID: \"d088ddb2-d975-4caa-807e-5dad7943101a\") " pod="openshift-monitoring/node-exporter-sdbvz"
Sep 20 01:53:49.576853 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:49.576443    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d088ddb2-d975-4caa-807e-5dad7943101a-node-exporter-accelerators-collector-config\") pod \"node-exporter-sdbvz\" (UID: \"d088ddb2-d975-4caa-807e-5dad7943101a\") " pod="openshift-monitoring/node-exporter-sdbvz"
Sep 20 01:53:49.576853 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:49.576466    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d088ddb2-d975-4caa-807e-5dad7943101a-metrics-client-ca\") pod \"node-exporter-sdbvz\" (UID: \"d088ddb2-d975-4caa-807e-5dad7943101a\") " pod="openshift-monitoring/node-exporter-sdbvz"
Sep 20 01:53:49.578374 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:49.578352    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d088ddb2-d975-4caa-807e-5dad7943101a-node-exporter-tls\") pod \"node-exporter-sdbvz\" (UID: \"d088ddb2-d975-4caa-807e-5dad7943101a\") " pod="openshift-monitoring/node-exporter-sdbvz"
Sep 20 01:53:49.578477 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:49.578398    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d088ddb2-d975-4caa-807e-5dad7943101a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-sdbvz\" (UID: \"d088ddb2-d975-4caa-807e-5dad7943101a\") " pod="openshift-monitoring/node-exporter-sdbvz"
Sep 20 01:53:49.583932 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:49.583908    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k58rv\" (UniqueName: \"kubernetes.io/projected/d088ddb2-d975-4caa-807e-5dad7943101a-kube-api-access-k58rv\") pod \"node-exporter-sdbvz\" (UID: \"d088ddb2-d975-4caa-807e-5dad7943101a\") " pod="openshift-monitoring/node-exporter-sdbvz"
Sep 20 01:53:49.587231 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:49.587189    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:53:49.703107 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:49.703069    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:53:49.733655 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:49.733580    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-sdbvz"
Sep 20 01:53:49.745222 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:53:49.745189    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd088ddb2_d975_4caa_807e_5dad7943101a.slice/crio-1c196924516147b3762075fdb768377d720904f198f80426f8ec0fc7afc7622c WatchSource:0}: Error finding container 1c196924516147b3762075fdb768377d720904f198f80426f8ec0fc7afc7622c: Status 404 returned error can't find the container with id 1c196924516147b3762075fdb768377d720904f198f80426f8ec0fc7afc7622c
Sep 20 01:53:49.785278 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:49.785240    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-sdbvz" event={"ID":"d088ddb2-d975-4caa-807e-5dad7943101a","Type":"ContainerStarted","Data":"1c196924516147b3762075fdb768377d720904f198f80426f8ec0fc7afc7622c"}
Sep 20 01:53:49.962124 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:49.962081    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:53:50.066217 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:50.066126    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:53:50.115885 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:50.115838    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:53:50.407421 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:50.407329    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:53:50.491539 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:50.491490    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:53:50.550121 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:50.550053    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:53:51.794140 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:51.794096    2584 generic.go:356] "Generic (PLEG): container finished" podID="d088ddb2-d975-4caa-807e-5dad7943101a" containerID="c245a60a5ada67f6a8268482902b841ab3567d8060298fc1471fa6e0fab9ce21" exitCode=0
Sep 20 01:53:51.794552 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:51.794170    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-sdbvz" event={"ID":"d088ddb2-d975-4caa-807e-5dad7943101a","Type":"ContainerDied","Data":"c245a60a5ada67f6a8268482902b841ab3567d8060298fc1471fa6e0fab9ce21"}
Sep 20 01:53:51.869501 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:51.869470    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:53:51.970091 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:51.970054    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:53:52.335617 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:52.335587    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:53:52.389017 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:52.388979    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:53:52.709205 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:52.709159    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:53:52.800855 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:52.800809    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-sdbvz" event={"ID":"d088ddb2-d975-4caa-807e-5dad7943101a","Type":"ContainerStarted","Data":"95032d444f6e3c49918bdaf8584a9900e15820e4a6423d1a8e9f069f32a72b98"}
Sep 20 01:53:52.801240 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:52.800900    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-sdbvz" event={"ID":"d088ddb2-d975-4caa-807e-5dad7943101a","Type":"ContainerStarted","Data":"5d2523b9fb96ab197b90d6832bac709146269ccd4abf6e77e9010782a8e8ec69"}
Sep 20 01:53:52.818001 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:52.817948    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-sdbvz" podStartSLOduration=2.7641816710000002 podStartE2EDuration="3.817932022s" totalImagesPullingTime="990.426891ms" totalInitContainerRuntime="63.32346ms" isStatefulPod=true podCreationTimestamp="2026-09-20 01:53:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-20 01:53:52.817632997 +0000 UTC m=+89.560312202" watchObservedRunningTime="2026-09-20 01:53:52.817932022 +0000 UTC m=+89.560611226"
Sep 20 01:53:52.952957 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:52.952917    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:53:53.674891 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:53.674861    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-5659b6b8c9-hzb2w"
Sep 20 01:53:53.833480 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:53.833437    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:53:53.934044 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:53.933953    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:53:54.216007 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:54.215973    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:53:54.283122 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:54.283087    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:53:54.730066 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:54.730027    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:53:55.607847 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:55.607810    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:53:59.086823 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:59.086781    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:53:59.934144 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:59.934106    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:53:59.988533 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:53:59.988494    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:54:00.687773 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:54:00.687735    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:54:00.788721 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:54:00.788685    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:54:01.655941 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:54:01.655912    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-47994"
Sep 20 01:54:02.110634 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:54:02.110597    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:54:02.210973 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:54:02.210928    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:54:04.752021 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:54:04.751983    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5659b6b8c9-hzb2w"]
Sep 20 01:54:04.849368 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:54:04.849332    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:54:04.949965 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:54:04.949933    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:54:05.050050 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:54:05.049973    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:54:05.150494 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:54:05.150457    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:54:09.000793 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:54:09.000749    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:54:09.102065 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:54:09.102033    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:54:11.221488 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:54:11.221453    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:54:11.276782 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:54:11.276749    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:54:26.220885 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:54:26.220843    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:54:26.320902 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:54:26.320865    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:54:27.707908 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:54:27.707869    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:54:27.809266 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:54:27.809222    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:54:28.914327 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:54:28.914279    2584 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77cf88f65f-9qp8b" podUID="536c3d86-41b0-4e73-9eed-7acbd8b86920" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Sep 20 01:54:29.773207 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:54:29.773167    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-5659b6b8c9-hzb2w" podUID="492ce9d4-19e5-4e0c-8b86-fd63a3b456a1" containerName="registry" containerID="cri-o://72cf887ab33b8b302652857f17ce99191725ac32defc252653e4f29e0b7dcc86" gracePeriod=30
Sep 20 01:54:29.938138 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:54:29.938108    2584 generic.go:356] "Generic (PLEG): container finished" podID="492ce9d4-19e5-4e0c-8b86-fd63a3b456a1" containerID="72cf887ab33b8b302652857f17ce99191725ac32defc252653e4f29e0b7dcc86" exitCode=0
Sep 20 01:54:29.938507 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:54:29.938185    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5659b6b8c9-hzb2w" event={"ID":"492ce9d4-19e5-4e0c-8b86-fd63a3b456a1","Type":"ContainerDied","Data":"72cf887ab33b8b302652857f17ce99191725ac32defc252653e4f29e0b7dcc86"}
Sep 20 01:54:30.018137 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:54:30.018108    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5659b6b8c9-hzb2w"
Sep 20 01:54:30.125732 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:54:30.125635    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/492ce9d4-19e5-4e0c-8b86-fd63a3b456a1-registry-tls\") pod \"492ce9d4-19e5-4e0c-8b86-fd63a3b456a1\" (UID: \"492ce9d4-19e5-4e0c-8b86-fd63a3b456a1\") "
Sep 20 01:54:30.125732 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:54:30.125696    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/492ce9d4-19e5-4e0c-8b86-fd63a3b456a1-trusted-ca\") pod \"492ce9d4-19e5-4e0c-8b86-fd63a3b456a1\" (UID: \"492ce9d4-19e5-4e0c-8b86-fd63a3b456a1\") "
Sep 20 01:54:30.125732 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:54:30.125727    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/492ce9d4-19e5-4e0c-8b86-fd63a3b456a1-bound-sa-token\") pod \"492ce9d4-19e5-4e0c-8b86-fd63a3b456a1\" (UID: \"492ce9d4-19e5-4e0c-8b86-fd63a3b456a1\") "
Sep 20 01:54:30.125993 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:54:30.125743    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/492ce9d4-19e5-4e0c-8b86-fd63a3b456a1-registry-certificates\") pod \"492ce9d4-19e5-4e0c-8b86-fd63a3b456a1\" (UID: \"492ce9d4-19e5-4e0c-8b86-fd63a3b456a1\") "
Sep 20 01:54:30.125993 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:54:30.125778    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/492ce9d4-19e5-4e0c-8b86-fd63a3b456a1-installation-pull-secrets\") pod \"492ce9d4-19e5-4e0c-8b86-fd63a3b456a1\" (UID: \"492ce9d4-19e5-4e0c-8b86-fd63a3b456a1\") "
Sep 20 01:54:30.125993 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:54:30.125843    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/492ce9d4-19e5-4e0c-8b86-fd63a3b456a1-image-registry-private-configuration\") pod \"492ce9d4-19e5-4e0c-8b86-fd63a3b456a1\" (UID: \"492ce9d4-19e5-4e0c-8b86-fd63a3b456a1\") "
Sep 20 01:54:30.125993 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:54:30.125876    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8s62g\" (UniqueName: \"kubernetes.io/projected/492ce9d4-19e5-4e0c-8b86-fd63a3b456a1-kube-api-access-8s62g\") pod \"492ce9d4-19e5-4e0c-8b86-fd63a3b456a1\" (UID: \"492ce9d4-19e5-4e0c-8b86-fd63a3b456a1\") "
Sep 20 01:54:30.125993 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:54:30.125901    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/492ce9d4-19e5-4e0c-8b86-fd63a3b456a1-ca-trust-extracted\") pod \"492ce9d4-19e5-4e0c-8b86-fd63a3b456a1\" (UID: \"492ce9d4-19e5-4e0c-8b86-fd63a3b456a1\") "
Sep 20 01:54:30.126211 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:54:30.126155    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/492ce9d4-19e5-4e0c-8b86-fd63a3b456a1-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "492ce9d4-19e5-4e0c-8b86-fd63a3b456a1" (UID: "492ce9d4-19e5-4e0c-8b86-fd63a3b456a1"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 20 01:54:30.126589 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:54:30.126555    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/492ce9d4-19e5-4e0c-8b86-fd63a3b456a1-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "492ce9d4-19e5-4e0c-8b86-fd63a3b456a1" (UID: "492ce9d4-19e5-4e0c-8b86-fd63a3b456a1"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 20 01:54:30.128293 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:54:30.128263    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/492ce9d4-19e5-4e0c-8b86-fd63a3b456a1-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "492ce9d4-19e5-4e0c-8b86-fd63a3b456a1" (UID: "492ce9d4-19e5-4e0c-8b86-fd63a3b456a1"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 20 01:54:30.128400 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:54:30.128348    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/492ce9d4-19e5-4e0c-8b86-fd63a3b456a1-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "492ce9d4-19e5-4e0c-8b86-fd63a3b456a1" (UID: "492ce9d4-19e5-4e0c-8b86-fd63a3b456a1"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 20 01:54:30.128640 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:54:30.128610    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/492ce9d4-19e5-4e0c-8b86-fd63a3b456a1-kube-api-access-8s62g" (OuterVolumeSpecName: "kube-api-access-8s62g") pod "492ce9d4-19e5-4e0c-8b86-fd63a3b456a1" (UID: "492ce9d4-19e5-4e0c-8b86-fd63a3b456a1"). InnerVolumeSpecName "kube-api-access-8s62g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 20 01:54:30.128734 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:54:30.128708    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/492ce9d4-19e5-4e0c-8b86-fd63a3b456a1-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "492ce9d4-19e5-4e0c-8b86-fd63a3b456a1" (UID: "492ce9d4-19e5-4e0c-8b86-fd63a3b456a1"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 20 01:54:30.128734 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:54:30.128710    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/492ce9d4-19e5-4e0c-8b86-fd63a3b456a1-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "492ce9d4-19e5-4e0c-8b86-fd63a3b456a1" (UID: "492ce9d4-19e5-4e0c-8b86-fd63a3b456a1"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 20 01:54:30.134146 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:54:30.134124    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/492ce9d4-19e5-4e0c-8b86-fd63a3b456a1-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "492ce9d4-19e5-4e0c-8b86-fd63a3b456a1" (UID: "492ce9d4-19e5-4e0c-8b86-fd63a3b456a1"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 20 01:54:30.227066 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:54:30.227029    2584 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/492ce9d4-19e5-4e0c-8b86-fd63a3b456a1-registry-tls\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 01:54:30.227066 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:54:30.227059    2584 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/492ce9d4-19e5-4e0c-8b86-fd63a3b456a1-trusted-ca\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 01:54:30.227066 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:54:30.227069    2584 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/492ce9d4-19e5-4e0c-8b86-fd63a3b456a1-bound-sa-token\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 01:54:30.227066 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:54:30.227077    2584 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/492ce9d4-19e5-4e0c-8b86-fd63a3b456a1-registry-certificates\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 01:54:30.227329 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:54:30.227087    2584 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/492ce9d4-19e5-4e0c-8b86-fd63a3b456a1-installation-pull-secrets\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 01:54:30.227329 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:54:30.227097    2584 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/492ce9d4-19e5-4e0c-8b86-fd63a3b456a1-image-registry-private-configuration\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 01:54:30.227329 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:54:30.227105    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8s62g\" (UniqueName: \"kubernetes.io/projected/492ce9d4-19e5-4e0c-8b86-fd63a3b456a1-kube-api-access-8s62g\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 01:54:30.227329 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:54:30.227114    2584 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/492ce9d4-19e5-4e0c-8b86-fd63a3b456a1-ca-trust-extracted\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 01:54:30.378172 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:54:30.378083    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:54:30.943323 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:54:30.943294    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5659b6b8c9-hzb2w"
Sep 20 01:54:30.943837 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:54:30.943292    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5659b6b8c9-hzb2w" event={"ID":"492ce9d4-19e5-4e0c-8b86-fd63a3b456a1","Type":"ContainerDied","Data":"1dff9b31fac2393db745430ecb925e7db872ba51e1d3ac7cfd89f953d18a2180"}
Sep 20 01:54:30.943837 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:54:30.943447    2584 scope.go:117] "RemoveContainer" containerID="72cf887ab33b8b302652857f17ce99191725ac32defc252653e4f29e0b7dcc86"
Sep 20 01:54:30.963121 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:54:30.963092    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5659b6b8c9-hzb2w"]
Sep 20 01:54:30.966839 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:54:30.966815    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-5659b6b8c9-hzb2w"]
Sep 20 01:54:31.091273 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:54:31.091240    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="492ce9d4-19e5-4e0c-8b86-fd63a3b456a1" path="/var/lib/kubelet/pods/492ce9d4-19e5-4e0c-8b86-fd63a3b456a1/volumes"
Sep 20 01:54:33.301085 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:54:33.301057    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:54:33.398664 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:54:33.398630    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:54:33.500506 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:54:33.500467    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:54:35.963936 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:54:35.963899    2584 generic.go:356] "Generic (PLEG): container finished" podID="261429fc-d268-4632-9767-2de6fe66a3a4" containerID="0641c73c708427c920259b253c3ffa773af2f3002552782c0a6160be4c660287" exitCode=0
Sep 20 01:54:35.964338 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:54:35.963972    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-pxh5z" event={"ID":"261429fc-d268-4632-9767-2de6fe66a3a4","Type":"ContainerDied","Data":"0641c73c708427c920259b253c3ffa773af2f3002552782c0a6160be4c660287"}
Sep 20 01:54:35.964413 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:54:35.964398    2584 scope.go:117] "RemoveContainer" containerID="0641c73c708427c920259b253c3ffa773af2f3002552782c0a6160be4c660287"
Sep 20 01:54:35.965628 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:54:35.965605    2584 generic.go:356] "Generic (PLEG): container finished" podID="a6fdf198-e4cd-4267-92fd-0ebee2f97522" containerID="2cb9fcb228f51f16e659703c59d8c66af85e8c3073166bd806e2c27976ae0190" exitCode=0
Sep 20 01:54:35.965744 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:54:35.965673    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-6f9c644dd9-gm67p" event={"ID":"a6fdf198-e4cd-4267-92fd-0ebee2f97522","Type":"ContainerDied","Data":"2cb9fcb228f51f16e659703c59d8c66af85e8c3073166bd806e2c27976ae0190"}
Sep 20 01:54:35.965948 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:54:35.965935    2584 scope.go:117] "RemoveContainer" containerID="2cb9fcb228f51f16e659703c59d8c66af85e8c3073166bd806e2c27976ae0190"
Sep 20 01:54:36.971004 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:54:36.970968    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-pxh5z" event={"ID":"261429fc-d268-4632-9767-2de6fe66a3a4","Type":"ContainerStarted","Data":"1b1b36134482faa33f7e897a7b658b4fbb44228b73058a594b271369a7aee678"}
Sep 20 01:54:36.972800 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:54:36.972774    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-6f9c644dd9-gm67p" event={"ID":"a6fdf198-e4cd-4267-92fd-0ebee2f97522","Type":"ContainerStarted","Data":"77f3136149c5ff2c14155d1986545596dc34c9a129f97c0e96bda2d0209f00d8"}
Sep 20 01:54:38.913578 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:54:38.913539    2584 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77cf88f65f-9qp8b" podUID="536c3d86-41b0-4e73-9eed-7acbd8b86920" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Sep 20 01:54:46.007159 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:54:46.007122    2584 generic.go:356] "Generic (PLEG): container finished" podID="006f93ba-2bd5-48cb-b52d-2e288401cd0f" containerID="20fa7567a45751431d7e920b98db5528e3034c979543beec80de861649bbd313" exitCode=0
Sep 20 01:54:46.007723 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:54:46.007171    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-2xts8" event={"ID":"006f93ba-2bd5-48cb-b52d-2e288401cd0f","Type":"ContainerDied","Data":"20fa7567a45751431d7e920b98db5528e3034c979543beec80de861649bbd313"}
Sep 20 01:54:46.007723 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:54:46.007626    2584 scope.go:117] "RemoveContainer" containerID="20fa7567a45751431d7e920b98db5528e3034c979543beec80de861649bbd313"
Sep 20 01:54:47.012139 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:54:47.012103    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-2xts8" event={"ID":"006f93ba-2bd5-48cb-b52d-2e288401cd0f","Type":"ContainerStarted","Data":"18a62b396d5f82b324c6ed798471e1118d38a5267ca8c246e6c3ce5077611ed5"}
Sep 20 01:54:48.913461 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:54:48.913418    2584 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77cf88f65f-9qp8b" podUID="536c3d86-41b0-4e73-9eed-7acbd8b86920" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Sep 20 01:54:48.913920 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:54:48.913500    2584 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77cf88f65f-9qp8b"
Sep 20 01:54:48.914220 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:54:48.914177    2584 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="service-proxy" containerStatusID={"Type":"cri-o","ID":"8ad2e1bed8e9d0b5a03e880416a9833016817a8873cf31c241cf5e671bfeb55a"} pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77cf88f65f-9qp8b" containerMessage="Container service-proxy failed liveness probe, will be restarted"
Sep 20 01:54:48.914267 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:54:48.914250    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77cf88f65f-9qp8b" podUID="536c3d86-41b0-4e73-9eed-7acbd8b86920" containerName="service-proxy" containerID="cri-o://8ad2e1bed8e9d0b5a03e880416a9833016817a8873cf31c241cf5e671bfeb55a" gracePeriod=30
Sep 20 01:54:50.030240 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:54:50.029070    2584 generic.go:356] "Generic (PLEG): container finished" podID="536c3d86-41b0-4e73-9eed-7acbd8b86920" containerID="8ad2e1bed8e9d0b5a03e880416a9833016817a8873cf31c241cf5e671bfeb55a" exitCode=2
Sep 20 01:54:50.030240 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:54:50.029174    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77cf88f65f-9qp8b" event={"ID":"536c3d86-41b0-4e73-9eed-7acbd8b86920","Type":"ContainerDied","Data":"8ad2e1bed8e9d0b5a03e880416a9833016817a8873cf31c241cf5e671bfeb55a"}
Sep 20 01:54:50.030240 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:54:50.029219    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77cf88f65f-9qp8b" event={"ID":"536c3d86-41b0-4e73-9eed-7acbd8b86920","Type":"ContainerStarted","Data":"28996031fa091dbae8cff17602c048e8cf3bc2e0985ec7519d5abf09c4145cd1"}
Sep 20 01:55:08.613650 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:55:08.613567    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:55:09.003722 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:55:09.003686    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:55:17.076580 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:55:17.076543    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:55:17.218233 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:55:17.218197    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:55:17.271963 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:55:17.271932    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:55:17.828379 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:55:17.828335    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:55:21.940555 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:55:21.940501    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:55:22.000090 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:55:22.000045    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:55:28.105941 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:55:28.105897    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:55:28.161379 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:55:28.161327    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:55:28.216347 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:55:28.216313    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:56:07.036713 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:56:07.036682    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:56:07.092213 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:56:07.092181    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:56:23.845746 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:56:23.845694    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:56:23.910799 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:56:23.910767    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:56:32.058950 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:56:32.058910    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:56:32.110593 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:56:32.110562    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:56:32.164784 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:56:32.164745    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:56:32.974553 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:56:32.974500    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:56:35.789224 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:56:35.789179    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:56:35.889829 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:56:35.889790    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:56:41.226547 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:56:41.226442    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:56:41.327365 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:56:41.327331    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:56:43.384115 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:56:43.384082    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:56:43.484654 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:56:43.484619    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:56:43.543570 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:56:43.543502    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:57:25.032179 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:57:25.032151    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-ztdt2_ece82f19-68b5-4810-be50-d8678cfdee02/console-operator/1.log"
Sep 20 01:57:25.037616 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:57:25.037594    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-ztdt2_ece82f19-68b5-4810-be50-d8678cfdee02/console-operator/1.log"
Sep 20 01:57:25.042273 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:57:25.042252    2584 kubelet.go:1648] "Image garbage collection succeeded"
Sep 20 01:58:15.483976 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:58:15.483877    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:58:16.582806 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:58:16.582774    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:58:21.534759 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:58:21.534687    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:58:21.587448 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:58:21.587401    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:58:37.107222 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:58:37.107182    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:58:37.910250 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:58:37.910191    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:58:40.718226 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:58:40.718196    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:58:40.818841 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:58:40.818811    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:58:42.990087 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:58:42.990045    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:58:43.094051 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:58:43.094007    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:58:43.263276 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:58:43.263187    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:58:43.470221 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:58:43.470188    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:58:45.236895 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:58:45.236853    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:58:45.797476 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:58:45.797420    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:58:47.512301 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:58:47.512248    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:58:47.562676 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:58:47.562620    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:58:53.789303 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:58:53.789265    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:58:59.125702 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:58:59.125640    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:58:59.226446 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:58:59.226392    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:59:05.564577 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:05.564539    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:59:05.663755 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:05.663706    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:59:07.343768 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:07.343725    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:59:07.444362 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:07.444321    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:59:07.785711 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:07.785667    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:59:14.166620 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:14.166566    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:59:14.266961 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:14.266914    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:59:18.077941 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:18.077908    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:59:18.174753 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:18.174724    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:59:18.275070 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:18.275018    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:59:20.049171 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:20.049141    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:59:20.174141 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:20.174093    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:59:20.656747 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:20.656707    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:59:20.852624 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:20.852590    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:59:20.954339 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:20.954310    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:59:20.996825 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:20.996790    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-59d4cf4f4c-tqwb6"]
Sep 20 01:59:20.997083 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:20.997071    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="492ce9d4-19e5-4e0c-8b86-fd63a3b456a1" containerName="registry"
Sep 20 01:59:20.997124 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:20.997085    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="492ce9d4-19e5-4e0c-8b86-fd63a3b456a1" containerName="registry"
Sep 20 01:59:20.997157 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:20.997137    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="492ce9d4-19e5-4e0c-8b86-fd63a3b456a1" containerName="registry"
Sep 20 01:59:21.000054 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:21.000037    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-59d4cf4f4c-tqwb6"
Sep 20 01:59:21.002783 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:21.002757    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-l7wm4\""
Sep 20 01:59:21.003471 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:21.003455    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 20 01:59:21.003750 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:21.003733    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 20 01:59:21.003851 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:21.003771    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 20 01:59:21.004232 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:21.004217    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 20 01:59:21.031039 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:21.031007    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-59d4cf4f4c-tqwb6"]
Sep 20 01:59:21.084857 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:21.084815    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c7f8158a-9f79-4227-8623-1b767c7e29a3-apiservice-cert\") pod \"opendatahub-operator-controller-manager-59d4cf4f4c-tqwb6\" (UID: \"c7f8158a-9f79-4227-8623-1b767c7e29a3\") " pod="opendatahub/opendatahub-operator-controller-manager-59d4cf4f4c-tqwb6"
Sep 20 01:59:21.085283 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:21.084864    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-stzwj\" (UniqueName: \"kubernetes.io/projected/c7f8158a-9f79-4227-8623-1b767c7e29a3-kube-api-access-stzwj\") pod \"opendatahub-operator-controller-manager-59d4cf4f4c-tqwb6\" (UID: \"c7f8158a-9f79-4227-8623-1b767c7e29a3\") " pod="opendatahub/opendatahub-operator-controller-manager-59d4cf4f4c-tqwb6"
Sep 20 01:59:21.085283 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:21.084912    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c7f8158a-9f79-4227-8623-1b767c7e29a3-tmp\") pod \"opendatahub-operator-controller-manager-59d4cf4f4c-tqwb6\" (UID: \"c7f8158a-9f79-4227-8623-1b767c7e29a3\") " pod="opendatahub/opendatahub-operator-controller-manager-59d4cf4f4c-tqwb6"
Sep 20 01:59:21.085283 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:21.084946    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c7f8158a-9f79-4227-8623-1b767c7e29a3-webhook-cert\") pod \"opendatahub-operator-controller-manager-59d4cf4f4c-tqwb6\" (UID: \"c7f8158a-9f79-4227-8623-1b767c7e29a3\") " pod="opendatahub/opendatahub-operator-controller-manager-59d4cf4f4c-tqwb6"
Sep 20 01:59:21.085283 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:21.085070    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c7f8158a-9f79-4227-8623-1b767c7e29a3-manifests\") pod \"opendatahub-operator-controller-manager-59d4cf4f4c-tqwb6\" (UID: \"c7f8158a-9f79-4227-8623-1b767c7e29a3\") " pod="opendatahub/opendatahub-operator-controller-manager-59d4cf4f4c-tqwb6"
Sep 20 01:59:21.185734 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:21.185697    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c7f8158a-9f79-4227-8623-1b767c7e29a3-apiservice-cert\") pod \"opendatahub-operator-controller-manager-59d4cf4f4c-tqwb6\" (UID: \"c7f8158a-9f79-4227-8623-1b767c7e29a3\") " pod="opendatahub/opendatahub-operator-controller-manager-59d4cf4f4c-tqwb6"
Sep 20 01:59:21.185734 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:21.185736    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-stzwj\" (UniqueName: \"kubernetes.io/projected/c7f8158a-9f79-4227-8623-1b767c7e29a3-kube-api-access-stzwj\") pod \"opendatahub-operator-controller-manager-59d4cf4f4c-tqwb6\" (UID: \"c7f8158a-9f79-4227-8623-1b767c7e29a3\") " pod="opendatahub/opendatahub-operator-controller-manager-59d4cf4f4c-tqwb6"
Sep 20 01:59:21.186003 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:21.185764    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c7f8158a-9f79-4227-8623-1b767c7e29a3-tmp\") pod \"opendatahub-operator-controller-manager-59d4cf4f4c-tqwb6\" (UID: \"c7f8158a-9f79-4227-8623-1b767c7e29a3\") " pod="opendatahub/opendatahub-operator-controller-manager-59d4cf4f4c-tqwb6"
Sep 20 01:59:21.186003 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:21.185790    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c7f8158a-9f79-4227-8623-1b767c7e29a3-webhook-cert\") pod \"opendatahub-operator-controller-manager-59d4cf4f4c-tqwb6\" (UID: \"c7f8158a-9f79-4227-8623-1b767c7e29a3\") " pod="opendatahub/opendatahub-operator-controller-manager-59d4cf4f4c-tqwb6"
Sep 20 01:59:21.186003 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:21.185858    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c7f8158a-9f79-4227-8623-1b767c7e29a3-manifests\") pod \"opendatahub-operator-controller-manager-59d4cf4f4c-tqwb6\" (UID: \"c7f8158a-9f79-4227-8623-1b767c7e29a3\") " pod="opendatahub/opendatahub-operator-controller-manager-59d4cf4f4c-tqwb6"
Sep 20 01:59:21.186285 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:21.186262    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c7f8158a-9f79-4227-8623-1b767c7e29a3-manifests\") pod \"opendatahub-operator-controller-manager-59d4cf4f4c-tqwb6\" (UID: \"c7f8158a-9f79-4227-8623-1b767c7e29a3\") " pod="opendatahub/opendatahub-operator-controller-manager-59d4cf4f4c-tqwb6"
Sep 20 01:59:21.186685 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:21.186661    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:59:21.188337 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:21.188314    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c7f8158a-9f79-4227-8623-1b767c7e29a3-webhook-cert\") pod \"opendatahub-operator-controller-manager-59d4cf4f4c-tqwb6\" (UID: \"c7f8158a-9f79-4227-8623-1b767c7e29a3\") " pod="opendatahub/opendatahub-operator-controller-manager-59d4cf4f4c-tqwb6"
Sep 20 01:59:21.188565 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:21.188539    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c7f8158a-9f79-4227-8623-1b767c7e29a3-tmp\") pod \"opendatahub-operator-controller-manager-59d4cf4f4c-tqwb6\" (UID: \"c7f8158a-9f79-4227-8623-1b767c7e29a3\") " pod="opendatahub/opendatahub-operator-controller-manager-59d4cf4f4c-tqwb6"
Sep 20 01:59:21.188877 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:21.188858    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c7f8158a-9f79-4227-8623-1b767c7e29a3-apiservice-cert\") pod \"opendatahub-operator-controller-manager-59d4cf4f4c-tqwb6\" (UID: \"c7f8158a-9f79-4227-8623-1b767c7e29a3\") " pod="opendatahub/opendatahub-operator-controller-manager-59d4cf4f4c-tqwb6"
Sep 20 01:59:21.194596 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:21.194566    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-stzwj\" (UniqueName: \"kubernetes.io/projected/c7f8158a-9f79-4227-8623-1b767c7e29a3-kube-api-access-stzwj\") pod \"opendatahub-operator-controller-manager-59d4cf4f4c-tqwb6\" (UID: \"c7f8158a-9f79-4227-8623-1b767c7e29a3\") " pod="opendatahub/opendatahub-operator-controller-manager-59d4cf4f4c-tqwb6"
Sep 20 01:59:21.282936 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:21.282849    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:59:21.310173 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:21.310130    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-59d4cf4f4c-tqwb6"
Sep 20 01:59:21.383621 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:21.383593    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:59:21.484277 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:21.484249    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:59:21.615900 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:21.615875    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-59d4cf4f4c-tqwb6"]
Sep 20 01:59:21.619129 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:59:21.619097    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7f8158a_9f79_4227_8623_1b767c7e29a3.slice/crio-0142088639a4c9ccfc96c61e61fa2d2a13ea0b9dd840c409b8ad8de17930d3fa WatchSource:0}: Error finding container 0142088639a4c9ccfc96c61e61fa2d2a13ea0b9dd840c409b8ad8de17930d3fa: Status 404 returned error can't find the container with id 0142088639a4c9ccfc96c61e61fa2d2a13ea0b9dd840c409b8ad8de17930d3fa
Sep 20 01:59:21.621730 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:21.621705    2584 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 20 01:59:21.623258 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:21.623233    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:59:21.729158 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:21.729124    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:59:22.053896 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:22.053866    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-59d4cf4f4c-tqwb6" event={"ID":"c7f8158a-9f79-4227-8623-1b767c7e29a3","Type":"ContainerStarted","Data":"0142088639a4c9ccfc96c61e61fa2d2a13ea0b9dd840c409b8ad8de17930d3fa"}
Sep 20 01:59:25.068882 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:25.068848    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-59d4cf4f4c-tqwb6" event={"ID":"c7f8158a-9f79-4227-8623-1b767c7e29a3","Type":"ContainerStarted","Data":"9f9acc7a30218a93e180ce8023aba462e3a4d3785b58bb58bdb140d9a1dbc0ce"}
Sep 20 01:59:26.074798 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:26.074759    2584 generic.go:356] "Generic (PLEG): container finished" podID="c7f8158a-9f79-4227-8623-1b767c7e29a3" containerID="9f9acc7a30218a93e180ce8023aba462e3a4d3785b58bb58bdb140d9a1dbc0ce" exitCode=0
Sep 20 01:59:26.075240 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:26.074820    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-59d4cf4f4c-tqwb6" event={"ID":"c7f8158a-9f79-4227-8623-1b767c7e29a3","Type":"ContainerDied","Data":"9f9acc7a30218a93e180ce8023aba462e3a4d3785b58bb58bdb140d9a1dbc0ce"}
Sep 20 01:59:27.085913 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:27.085875    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-59d4cf4f4c-tqwb6" event={"ID":"c7f8158a-9f79-4227-8623-1b767c7e29a3","Type":"ContainerStarted","Data":"5c30ac20425477073d4918ee9c356c245bc8793d41e87c0490e0a7760dff98ac"}
Sep 20 01:59:27.090880 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:27.090857    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-59d4cf4f4c-tqwb6"
Sep 20 01:59:27.106066 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:27.106024    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-59d4cf4f4c-tqwb6" podStartSLOduration=2.739036064 podStartE2EDuration="7.106006048s" totalImagesPullingTime="3.334140989s" totalInitContainerRuntime="1.032828995s" isStatefulPod=false podCreationTimestamp="2026-09-20 01:59:20 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-20 01:59:27.105182464 +0000 UTC m=+423.847861667" watchObservedRunningTime="2026-09-20 01:59:27.106006048 +0000 UTC m=+423.848685235"
Sep 20 01:59:31.773478 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:31.773447    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:59:31.877536 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:31.877477    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:59:39.095849 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:39.095768    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-59d4cf4f4c-tqwb6"
Sep 20 01:59:40.645512 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:40.645476    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:59:40.759918 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:40.759883    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:59:41.052438 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:41.052401    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:59:41.118755 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:41.118719    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-86bbc588c6-zlhv2"]
Sep 20 01:59:41.121807 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:41.121789    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-86bbc588c6-zlhv2"
Sep 20 01:59:41.124102 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:41.124082    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 20 01:59:41.126391 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:41.126372    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Sep 20 01:59:41.140869 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:41.140842    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-86bbc588c6-zlhv2"]
Sep 20 01:59:41.185008 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:41.184974    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:59:41.261795 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:41.261756    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gdt69\" (UniqueName: \"kubernetes.io/projected/004ecde9-2ce2-41cf-b558-ae17143cd051-kube-api-access-gdt69\") pod \"kube-auth-proxy-86bbc588c6-zlhv2\" (UID: \"004ecde9-2ce2-41cf-b558-ae17143cd051\") " pod="openshift-ingress/kube-auth-proxy-86bbc588c6-zlhv2"
Sep 20 01:59:41.261999 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:41.261815    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/004ecde9-2ce2-41cf-b558-ae17143cd051-tls-certs\") pod \"kube-auth-proxy-86bbc588c6-zlhv2\" (UID: \"004ecde9-2ce2-41cf-b558-ae17143cd051\") " pod="openshift-ingress/kube-auth-proxy-86bbc588c6-zlhv2"
Sep 20 01:59:41.261999 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:41.261885    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/004ecde9-2ce2-41cf-b558-ae17143cd051-tmp\") pod \"kube-auth-proxy-86bbc588c6-zlhv2\" (UID: \"004ecde9-2ce2-41cf-b558-ae17143cd051\") " pod="openshift-ingress/kube-auth-proxy-86bbc588c6-zlhv2"
Sep 20 01:59:41.362696 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:41.362598    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/004ecde9-2ce2-41cf-b558-ae17143cd051-tls-certs\") pod \"kube-auth-proxy-86bbc588c6-zlhv2\" (UID: \"004ecde9-2ce2-41cf-b558-ae17143cd051\") " pod="openshift-ingress/kube-auth-proxy-86bbc588c6-zlhv2"
Sep 20 01:59:41.362696 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:41.362666    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/004ecde9-2ce2-41cf-b558-ae17143cd051-tmp\") pod \"kube-auth-proxy-86bbc588c6-zlhv2\" (UID: \"004ecde9-2ce2-41cf-b558-ae17143cd051\") " pod="openshift-ingress/kube-auth-proxy-86bbc588c6-zlhv2"
Sep 20 01:59:41.362885 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:41.362705    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gdt69\" (UniqueName: \"kubernetes.io/projected/004ecde9-2ce2-41cf-b558-ae17143cd051-kube-api-access-gdt69\") pod \"kube-auth-proxy-86bbc588c6-zlhv2\" (UID: \"004ecde9-2ce2-41cf-b558-ae17143cd051\") " pod="openshift-ingress/kube-auth-proxy-86bbc588c6-zlhv2"
Sep 20 01:59:41.365027 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:41.365001    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/004ecde9-2ce2-41cf-b558-ae17143cd051-tmp\") pod \"kube-auth-proxy-86bbc588c6-zlhv2\" (UID: \"004ecde9-2ce2-41cf-b558-ae17143cd051\") " pod="openshift-ingress/kube-auth-proxy-86bbc588c6-zlhv2"
Sep 20 01:59:41.365164 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:41.365144    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/004ecde9-2ce2-41cf-b558-ae17143cd051-tls-certs\") pod \"kube-auth-proxy-86bbc588c6-zlhv2\" (UID: \"004ecde9-2ce2-41cf-b558-ae17143cd051\") " pod="openshift-ingress/kube-auth-proxy-86bbc588c6-zlhv2"
Sep 20 01:59:41.373100 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:41.373075    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gdt69\" (UniqueName: \"kubernetes.io/projected/004ecde9-2ce2-41cf-b558-ae17143cd051-kube-api-access-gdt69\") pod \"kube-auth-proxy-86bbc588c6-zlhv2\" (UID: \"004ecde9-2ce2-41cf-b558-ae17143cd051\") " pod="openshift-ingress/kube-auth-proxy-86bbc588c6-zlhv2"
Sep 20 01:59:41.430705 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:41.430667    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-86bbc588c6-zlhv2"
Sep 20 01:59:41.708019 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:41.707930    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-86bbc588c6-zlhv2"]
Sep 20 01:59:41.710272 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:59:41.710246    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod004ecde9_2ce2_41cf_b558_ae17143cd051.slice/crio-f86943c34d6904348bd2936a516901a088a663a03a359204a89882cb2826c3f0 WatchSource:0}: Error finding container f86943c34d6904348bd2936a516901a088a663a03a359204a89882cb2826c3f0: Status 404 returned error can't find the container with id f86943c34d6904348bd2936a516901a088a663a03a359204a89882cb2826c3f0
Sep 20 01:59:42.148115 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:42.148031    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-86bbc588c6-zlhv2" event={"ID":"004ecde9-2ce2-41cf-b558-ae17143cd051","Type":"ContainerStarted","Data":"f86943c34d6904348bd2936a516901a088a663a03a359204a89882cb2826c3f0"}
Sep 20 01:59:42.520557 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:42.520501    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-42hqf"]
Sep 20 01:59:42.523564 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:42.523506    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:59:42.524141 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:42.524120    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-42hqf"
Sep 20 01:59:42.526994 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:42.526753    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Sep 20 01:59:42.526994 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:42.526868    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Sep 20 01:59:42.527181 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:42.527083    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Sep 20 01:59:42.527181 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:42.527155    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 20 01:59:42.527403 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:42.527317    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-s579t\""
Sep 20 01:59:42.540474 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:42.540441    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-42hqf"]
Sep 20 01:59:42.663854 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:42.663821    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:59:42.674826 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:42.674710    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/7e91c38e-3104-4e7d-8876-aec191ce18dd-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-42hqf\" (UID: \"7e91c38e-3104-4e7d-8876-aec191ce18dd\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-42hqf"
Sep 20 01:59:42.674826 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:42.674760    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/7e91c38e-3104-4e7d-8876-aec191ce18dd-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-42hqf\" (UID: \"7e91c38e-3104-4e7d-8876-aec191ce18dd\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-42hqf"
Sep 20 01:59:42.674826 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:42.674787    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7e91c38e-3104-4e7d-8876-aec191ce18dd-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-42hqf\" (UID: \"7e91c38e-3104-4e7d-8876-aec191ce18dd\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-42hqf"
Sep 20 01:59:42.675133 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:42.674897    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/7e91c38e-3104-4e7d-8876-aec191ce18dd-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-42hqf\" (UID: \"7e91c38e-3104-4e7d-8876-aec191ce18dd\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-42hqf"
Sep 20 01:59:42.675133 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:42.674971    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2v4t5\" (UniqueName: \"kubernetes.io/projected/7e91c38e-3104-4e7d-8876-aec191ce18dd-kube-api-access-2v4t5\") pod \"istiod-openshift-gateway-7b7995d5cd-42hqf\" (UID: \"7e91c38e-3104-4e7d-8876-aec191ce18dd\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-42hqf"
Sep 20 01:59:42.675133 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:42.675035    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/7e91c38e-3104-4e7d-8876-aec191ce18dd-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-42hqf\" (UID: \"7e91c38e-3104-4e7d-8876-aec191ce18dd\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-42hqf"
Sep 20 01:59:42.675298 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:42.675158    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7e91c38e-3104-4e7d-8876-aec191ce18dd-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-42hqf\" (UID: \"7e91c38e-3104-4e7d-8876-aec191ce18dd\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-42hqf"
Sep 20 01:59:42.776515 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:42.776425    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/7e91c38e-3104-4e7d-8876-aec191ce18dd-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-42hqf\" (UID: \"7e91c38e-3104-4e7d-8876-aec191ce18dd\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-42hqf"
Sep 20 01:59:42.776515 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:42.776502    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2v4t5\" (UniqueName: \"kubernetes.io/projected/7e91c38e-3104-4e7d-8876-aec191ce18dd-kube-api-access-2v4t5\") pod \"istiod-openshift-gateway-7b7995d5cd-42hqf\" (UID: \"7e91c38e-3104-4e7d-8876-aec191ce18dd\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-42hqf"
Sep 20 01:59:42.777053 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:42.776564    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/7e91c38e-3104-4e7d-8876-aec191ce18dd-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-42hqf\" (UID: \"7e91c38e-3104-4e7d-8876-aec191ce18dd\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-42hqf"
Sep 20 01:59:42.777053 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:42.776592    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7e91c38e-3104-4e7d-8876-aec191ce18dd-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-42hqf\" (UID: \"7e91c38e-3104-4e7d-8876-aec191ce18dd\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-42hqf"
Sep 20 01:59:42.777053 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:42.776620    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/7e91c38e-3104-4e7d-8876-aec191ce18dd-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-42hqf\" (UID: \"7e91c38e-3104-4e7d-8876-aec191ce18dd\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-42hqf"
Sep 20 01:59:42.777053 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:42.776644    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/7e91c38e-3104-4e7d-8876-aec191ce18dd-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-42hqf\" (UID: \"7e91c38e-3104-4e7d-8876-aec191ce18dd\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-42hqf"
Sep 20 01:59:42.777053 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:42.776662    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7e91c38e-3104-4e7d-8876-aec191ce18dd-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-42hqf\" (UID: \"7e91c38e-3104-4e7d-8876-aec191ce18dd\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-42hqf"
Sep 20 01:59:42.777322 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:42.777181    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/7e91c38e-3104-4e7d-8876-aec191ce18dd-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-42hqf\" (UID: \"7e91c38e-3104-4e7d-8876-aec191ce18dd\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-42hqf"
Sep 20 01:59:42.780077 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:42.780031    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/7e91c38e-3104-4e7d-8876-aec191ce18dd-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-42hqf\" (UID: \"7e91c38e-3104-4e7d-8876-aec191ce18dd\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-42hqf"
Sep 20 01:59:42.780589 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:42.780542    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/7e91c38e-3104-4e7d-8876-aec191ce18dd-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-42hqf\" (UID: \"7e91c38e-3104-4e7d-8876-aec191ce18dd\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-42hqf"
Sep 20 01:59:42.780744 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:42.780720    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7e91c38e-3104-4e7d-8876-aec191ce18dd-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-42hqf\" (UID: \"7e91c38e-3104-4e7d-8876-aec191ce18dd\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-42hqf"
Sep 20 01:59:42.781497 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:42.781460    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/7e91c38e-3104-4e7d-8876-aec191ce18dd-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-42hqf\" (UID: \"7e91c38e-3104-4e7d-8876-aec191ce18dd\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-42hqf"
Sep 20 01:59:42.786398 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:42.786374    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2v4t5\" (UniqueName: \"kubernetes.io/projected/7e91c38e-3104-4e7d-8876-aec191ce18dd-kube-api-access-2v4t5\") pod \"istiod-openshift-gateway-7b7995d5cd-42hqf\" (UID: \"7e91c38e-3104-4e7d-8876-aec191ce18dd\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-42hqf"
Sep 20 01:59:42.787547 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:42.787503    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7e91c38e-3104-4e7d-8876-aec191ce18dd-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-42hqf\" (UID: \"7e91c38e-3104-4e7d-8876-aec191ce18dd\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-42hqf"
Sep 20 01:59:42.837448 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:42.837417    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-42hqf"
Sep 20 01:59:43.126987 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:43.126859    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-42hqf"]
Sep 20 01:59:43.130254 ip-10-0-130-159 kubenswrapper[2584]: W0920 01:59:43.130222    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e91c38e_3104_4e7d_8876_aec191ce18dd.slice/crio-56d2b90462563fe6289b2e176c82f9abd23d37b2421012910615bd2b2aaa0378 WatchSource:0}: Error finding container 56d2b90462563fe6289b2e176c82f9abd23d37b2421012910615bd2b2aaa0378: Status 404 returned error can't find the container with id 56d2b90462563fe6289b2e176c82f9abd23d37b2421012910615bd2b2aaa0378
Sep 20 01:59:43.155054 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:43.155015    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-42hqf" event={"ID":"7e91c38e-3104-4e7d-8876-aec191ce18dd","Type":"ContainerStarted","Data":"56d2b90462563fe6289b2e176c82f9abd23d37b2421012910615bd2b2aaa0378"}
Sep 20 01:59:43.546214 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:43.546169    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:59:45.167663 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:45.167619    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-86bbc588c6-zlhv2" event={"ID":"004ecde9-2ce2-41cf-b558-ae17143cd051","Type":"ContainerStarted","Data":"9fe12d42cf4a8c6c4970f12df6a4ee9dec46ccb62e9e4e1125c5a885af5aed90"}
Sep 20 01:59:45.198200 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:45.198121    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-86bbc588c6-zlhv2" podStartSLOduration=1.352372519 podStartE2EDuration="4.198099746s" totalImagesPullingTime="2.845727227s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-20 01:59:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-20 01:59:45.195229091 +0000 UTC m=+441.937908297" watchObservedRunningTime="2026-09-20 01:59:45.198099746 +0000 UTC m=+441.940778954"
Sep 20 01:59:46.465692 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:46.465643    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 20 01:59:46.466013 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:46.465721    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 20 01:59:47.179231 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:47.179195    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-42hqf" event={"ID":"7e91c38e-3104-4e7d-8876-aec191ce18dd","Type":"ContainerStarted","Data":"d2b555faf13b049d9632109cf17c1b459e7340c6806317f94fc0929910e5c48a"}
Sep 20 01:59:47.179494 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:47.179365    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-42hqf"
Sep 20 01:59:47.181314 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:47.181287    2584 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-42hqf container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 20 01:59:47.181429 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:47.181332    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-42hqf" podUID="7e91c38e-3104-4e7d-8876-aec191ce18dd" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 20 01:59:47.200104 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:47.200055    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-42hqf" podStartSLOduration=1.867405813 podStartE2EDuration="5.200038226s" totalImagesPullingTime="3.332632413s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-20 01:59:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-20 01:59:47.198475007 +0000 UTC m=+443.941154239" watchObservedRunningTime="2026-09-20 01:59:47.200038226 +0000 UTC m=+443.942717431"
Sep 20 01:59:47.751807 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:47.751713    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:59:47.808350 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:47.808277    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:59:48.044701 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:48.044602    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:59:48.143876 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:48.143831    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:59:48.184135 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:48.184107    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-42hqf"
Sep 20 01:59:48.208967 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:48.208931    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:59:51.733830 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:51.733787    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:59:51.796786 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:51.796749    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:59:51.898226 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:51.898195    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:59:51.997281 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:51.997202    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:59:52.100265 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:52.100228    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:59:52.232866 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:52.232836    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:59:52.366262 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:52.366185    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:59:58.090483 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:58.090449    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 01:59:58.166192 ip-10-0-130-159 kubenswrapper[2584]: I0920 01:59:58.166162    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:00:00.148543 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:00:00.148489    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:00:00.244338 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:00:00.244297    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:00:01.043036 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:00:01.043004    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:00:01.098064 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:00:01.098033    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-v8pxt"]
Sep 20 02:00:01.101676 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:00:01.101653    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-v8pxt"
Sep 20 02:00:01.103990 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:00:01.103971    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Sep 20 02:00:01.103990 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:00:01.103982    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-hp66n\""
Sep 20 02:00:01.107113 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:00:01.107092    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-6vgqp"]
Sep 20 02:00:01.110058 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:00:01.110042    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6vgqp"
Sep 20 02:00:01.112136 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:00:01.112120    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Sep 20 02:00:01.112714 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:00:01.112700    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-kf8qt\""
Sep 20 02:00:01.116923 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:00:01.116772    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-v8pxt"]
Sep 20 02:00:01.125651 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:00:01.125627    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-6vgqp"]
Sep 20 02:00:01.128252 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:00:01.128228    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-22n6f\" (UniqueName: \"kubernetes.io/projected/fe1d0a3b-8799-4617-8763-fa987349cf78-kube-api-access-22n6f\") pod \"kserve-controller-manager-d6f4b65b8-v8pxt\" (UID: \"fe1d0a3b-8799-4617-8763-fa987349cf78\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-v8pxt"
Sep 20 02:00:01.128409 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:00:01.128322    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fe1d0a3b-8799-4617-8763-fa987349cf78-cert\") pod \"kserve-controller-manager-d6f4b65b8-v8pxt\" (UID: \"fe1d0a3b-8799-4617-8763-fa987349cf78\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-v8pxt"
Sep 20 02:00:01.157060 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:00:01.157031    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:00:01.229199 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:00:01.229165    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-22n6f\" (UniqueName: \"kubernetes.io/projected/fe1d0a3b-8799-4617-8763-fa987349cf78-kube-api-access-22n6f\") pod \"kserve-controller-manager-d6f4b65b8-v8pxt\" (UID: \"fe1d0a3b-8799-4617-8763-fa987349cf78\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-v8pxt"
Sep 20 02:00:01.229380 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:00:01.229218    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8zf4j\" (UniqueName: \"kubernetes.io/projected/4f29b72e-019b-4f87-b9c2-9154fd7bb6ec-kube-api-access-8zf4j\") pod \"llmisvc-controller-manager-777c6bd7d5-6vgqp\" (UID: \"4f29b72e-019b-4f87-b9c2-9154fd7bb6ec\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6vgqp"
Sep 20 02:00:01.229380 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:00:01.229263    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4f29b72e-019b-4f87-b9c2-9154fd7bb6ec-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-6vgqp\" (UID: \"4f29b72e-019b-4f87-b9c2-9154fd7bb6ec\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6vgqp"
Sep 20 02:00:01.229380 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:00:01.229296    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fe1d0a3b-8799-4617-8763-fa987349cf78-cert\") pod \"kserve-controller-manager-d6f4b65b8-v8pxt\" (UID: \"fe1d0a3b-8799-4617-8763-fa987349cf78\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-v8pxt"
Sep 20 02:00:01.231979 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:00:01.231950    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fe1d0a3b-8799-4617-8763-fa987349cf78-cert\") pod \"kserve-controller-manager-d6f4b65b8-v8pxt\" (UID: \"fe1d0a3b-8799-4617-8763-fa987349cf78\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-v8pxt"
Sep 20 02:00:01.248793 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:00:01.248763    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-22n6f\" (UniqueName: \"kubernetes.io/projected/fe1d0a3b-8799-4617-8763-fa987349cf78-kube-api-access-22n6f\") pod \"kserve-controller-manager-d6f4b65b8-v8pxt\" (UID: \"fe1d0a3b-8799-4617-8763-fa987349cf78\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-v8pxt"
Sep 20 02:00:01.261168 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:00:01.261142    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:00:01.330269 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:00:01.330178    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8zf4j\" (UniqueName: \"kubernetes.io/projected/4f29b72e-019b-4f87-b9c2-9154fd7bb6ec-kube-api-access-8zf4j\") pod \"llmisvc-controller-manager-777c6bd7d5-6vgqp\" (UID: \"4f29b72e-019b-4f87-b9c2-9154fd7bb6ec\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6vgqp"
Sep 20 02:00:01.330269 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:00:01.330263    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4f29b72e-019b-4f87-b9c2-9154fd7bb6ec-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-6vgqp\" (UID: \"4f29b72e-019b-4f87-b9c2-9154fd7bb6ec\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6vgqp"
Sep 20 02:00:01.330463 ip-10-0-130-159 kubenswrapper[2584]: E0920 02:00:01.330390    2584 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found
Sep 20 02:00:01.330500 ip-10-0-130-159 kubenswrapper[2584]: E0920 02:00:01.330474    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f29b72e-019b-4f87-b9c2-9154fd7bb6ec-cert podName:4f29b72e-019b-4f87-b9c2-9154fd7bb6ec nodeName:}" failed. No retries permitted until 2026-09-20 02:00:01.830451358 +0000 UTC m=+458.573130562 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/4f29b72e-019b-4f87-b9c2-9154fd7bb6ec-cert") pod "llmisvc-controller-manager-777c6bd7d5-6vgqp" (UID: "4f29b72e-019b-4f87-b9c2-9154fd7bb6ec") : secret "llmisvc-webhook-server-cert" not found
Sep 20 02:00:01.348741 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:00:01.348704    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8zf4j\" (UniqueName: \"kubernetes.io/projected/4f29b72e-019b-4f87-b9c2-9154fd7bb6ec-kube-api-access-8zf4j\") pod \"llmisvc-controller-manager-777c6bd7d5-6vgqp\" (UID: \"4f29b72e-019b-4f87-b9c2-9154fd7bb6ec\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6vgqp"
Sep 20 02:00:01.403420 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:00:01.403381    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:00:01.412419 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:00:01.412390    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-v8pxt"
Sep 20 02:00:01.686784 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:00:01.686761    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-v8pxt"]
Sep 20 02:00:01.689331 ip-10-0-130-159 kubenswrapper[2584]: W0920 02:00:01.689303    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe1d0a3b_8799_4617_8763_fa987349cf78.slice/crio-382e9d097104c46328cacdbf02bb50774f8165ee920aae83fcf674a3f3d91127 WatchSource:0}: Error finding container 382e9d097104c46328cacdbf02bb50774f8165ee920aae83fcf674a3f3d91127: Status 404 returned error can't find the container with id 382e9d097104c46328cacdbf02bb50774f8165ee920aae83fcf674a3f3d91127
Sep 20 02:00:01.835664 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:00:01.835619    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4f29b72e-019b-4f87-b9c2-9154fd7bb6ec-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-6vgqp\" (UID: \"4f29b72e-019b-4f87-b9c2-9154fd7bb6ec\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6vgqp"
Sep 20 02:00:01.838274 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:00:01.838244    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4f29b72e-019b-4f87-b9c2-9154fd7bb6ec-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-6vgqp\" (UID: \"4f29b72e-019b-4f87-b9c2-9154fd7bb6ec\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6vgqp"
Sep 20 02:00:02.021245 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:00:02.021212    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6vgqp"
Sep 20 02:00:02.248598 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:00:02.248557    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-v8pxt" event={"ID":"fe1d0a3b-8799-4617-8763-fa987349cf78","Type":"ContainerStarted","Data":"382e9d097104c46328cacdbf02bb50774f8165ee920aae83fcf674a3f3d91127"}
Sep 20 02:00:02.293839 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:00:02.293812    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-6vgqp"]
Sep 20 02:00:02.296750 ip-10-0-130-159 kubenswrapper[2584]: W0920 02:00:02.296715    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod4f29b72e_019b_4f87_b9c2_9154fd7bb6ec.slice/crio-ebd8381403e5b762ceb94337bad636e42874b014c4f32c5c5a681101d9c66dd5 WatchSource:0}: Error finding container ebd8381403e5b762ceb94337bad636e42874b014c4f32c5c5a681101d9c66dd5: Status 404 returned error can't find the container with id ebd8381403e5b762ceb94337bad636e42874b014c4f32c5c5a681101d9c66dd5
Sep 20 02:00:02.345009 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:00:02.344977    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:00:02.518227 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:00:02.518195    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:00:03.226446 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:00:03.226415    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-42hqf"]
Sep 20 02:00:03.226699 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:00:03.226671    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-42hqf" podUID="7e91c38e-3104-4e7d-8876-aec191ce18dd" containerName="discovery" containerID="cri-o://d2b555faf13b049d9632109cf17c1b459e7340c6806317f94fc0929910e5c48a" gracePeriod=30
Sep 20 02:00:03.255277 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:00:03.255243    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6vgqp" event={"ID":"4f29b72e-019b-4f87-b9c2-9154fd7bb6ec","Type":"ContainerStarted","Data":"ebd8381403e5b762ceb94337bad636e42874b014c4f32c5c5a681101d9c66dd5"}
Sep 20 02:00:03.486123 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:00:03.486008    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-42hqf"
Sep 20 02:00:03.561274 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:00:03.561234    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7e91c38e-3104-4e7d-8876-aec191ce18dd-istio-kubeconfig\") pod \"7e91c38e-3104-4e7d-8876-aec191ce18dd\" (UID: \"7e91c38e-3104-4e7d-8876-aec191ce18dd\") "
Sep 20 02:00:03.561465 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:00:03.561304    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2v4t5\" (UniqueName: \"kubernetes.io/projected/7e91c38e-3104-4e7d-8876-aec191ce18dd-kube-api-access-2v4t5\") pod \"7e91c38e-3104-4e7d-8876-aec191ce18dd\" (UID: \"7e91c38e-3104-4e7d-8876-aec191ce18dd\") "
Sep 20 02:00:03.561465 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:00:03.561354    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/7e91c38e-3104-4e7d-8876-aec191ce18dd-istio-csr-ca-configmap\") pod \"7e91c38e-3104-4e7d-8876-aec191ce18dd\" (UID: \"7e91c38e-3104-4e7d-8876-aec191ce18dd\") "
Sep 20 02:00:03.561465 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:00:03.561382    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/7e91c38e-3104-4e7d-8876-aec191ce18dd-istio-csr-dns-cert\") pod \"7e91c38e-3104-4e7d-8876-aec191ce18dd\" (UID: \"7e91c38e-3104-4e7d-8876-aec191ce18dd\") "
Sep 20 02:00:03.561465 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:00:03.561419    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7e91c38e-3104-4e7d-8876-aec191ce18dd-istio-token\") pod \"7e91c38e-3104-4e7d-8876-aec191ce18dd\" (UID: \"7e91c38e-3104-4e7d-8876-aec191ce18dd\") "
Sep 20 02:00:03.561720 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:00:03.561480    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/7e91c38e-3104-4e7d-8876-aec191ce18dd-cacerts\") pod \"7e91c38e-3104-4e7d-8876-aec191ce18dd\" (UID: \"7e91c38e-3104-4e7d-8876-aec191ce18dd\") "
Sep 20 02:00:03.561720 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:00:03.561536    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/7e91c38e-3104-4e7d-8876-aec191ce18dd-local-certs\") pod \"7e91c38e-3104-4e7d-8876-aec191ce18dd\" (UID: \"7e91c38e-3104-4e7d-8876-aec191ce18dd\") "
Sep 20 02:00:03.562202 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:00:03.562119    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7e91c38e-3104-4e7d-8876-aec191ce18dd-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "7e91c38e-3104-4e7d-8876-aec191ce18dd" (UID: "7e91c38e-3104-4e7d-8876-aec191ce18dd"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 20 02:00:03.564289 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:00:03.564249    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7e91c38e-3104-4e7d-8876-aec191ce18dd-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "7e91c38e-3104-4e7d-8876-aec191ce18dd" (UID: "7e91c38e-3104-4e7d-8876-aec191ce18dd"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 20 02:00:03.564289 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:00:03.564278    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e91c38e-3104-4e7d-8876-aec191ce18dd-istio-token" (OuterVolumeSpecName: "istio-token") pod "7e91c38e-3104-4e7d-8876-aec191ce18dd" (UID: "7e91c38e-3104-4e7d-8876-aec191ce18dd"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 20 02:00:03.564480 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:00:03.564354    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7e91c38e-3104-4e7d-8876-aec191ce18dd-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "7e91c38e-3104-4e7d-8876-aec191ce18dd" (UID: "7e91c38e-3104-4e7d-8876-aec191ce18dd"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 20 02:00:03.564480 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:00:03.564377    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7e91c38e-3104-4e7d-8876-aec191ce18dd-cacerts" (OuterVolumeSpecName: "cacerts") pod "7e91c38e-3104-4e7d-8876-aec191ce18dd" (UID: "7e91c38e-3104-4e7d-8876-aec191ce18dd"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 20 02:00:03.564784 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:00:03.564761    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e91c38e-3104-4e7d-8876-aec191ce18dd-local-certs" (OuterVolumeSpecName: "local-certs") pod "7e91c38e-3104-4e7d-8876-aec191ce18dd" (UID: "7e91c38e-3104-4e7d-8876-aec191ce18dd"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 20 02:00:03.564784 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:00:03.564777    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e91c38e-3104-4e7d-8876-aec191ce18dd-kube-api-access-2v4t5" (OuterVolumeSpecName: "kube-api-access-2v4t5") pod "7e91c38e-3104-4e7d-8876-aec191ce18dd" (UID: "7e91c38e-3104-4e7d-8876-aec191ce18dd"). InnerVolumeSpecName "kube-api-access-2v4t5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 20 02:00:03.662904 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:00:03.662870    2584 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/7e91c38e-3104-4e7d-8876-aec191ce18dd-cacerts\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:00:03.662904 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:00:03.662902    2584 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/7e91c38e-3104-4e7d-8876-aec191ce18dd-local-certs\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:00:03.662904 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:00:03.662914    2584 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7e91c38e-3104-4e7d-8876-aec191ce18dd-istio-kubeconfig\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:00:03.663185 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:00:03.662922    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2v4t5\" (UniqueName: \"kubernetes.io/projected/7e91c38e-3104-4e7d-8876-aec191ce18dd-kube-api-access-2v4t5\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:00:03.663185 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:00:03.662934    2584 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/7e91c38e-3104-4e7d-8876-aec191ce18dd-istio-csr-ca-configmap\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:00:03.663185 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:00:03.662946    2584 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/7e91c38e-3104-4e7d-8876-aec191ce18dd-istio-csr-dns-cert\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:00:03.663185 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:00:03.662954    2584 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7e91c38e-3104-4e7d-8876-aec191ce18dd-istio-token\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:00:04.261732 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:00:04.261696    2584 generic.go:356] "Generic (PLEG): container finished" podID="7e91c38e-3104-4e7d-8876-aec191ce18dd" containerID="d2b555faf13b049d9632109cf17c1b459e7340c6806317f94fc0929910e5c48a" exitCode=0
Sep 20 02:00:04.261732 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:00:04.261742    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-42hqf"
Sep 20 02:00:04.262240 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:00:04.261746    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-42hqf" event={"ID":"7e91c38e-3104-4e7d-8876-aec191ce18dd","Type":"ContainerDied","Data":"d2b555faf13b049d9632109cf17c1b459e7340c6806317f94fc0929910e5c48a"}
Sep 20 02:00:04.262240 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:00:04.261789    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-42hqf" event={"ID":"7e91c38e-3104-4e7d-8876-aec191ce18dd","Type":"ContainerDied","Data":"56d2b90462563fe6289b2e176c82f9abd23d37b2421012910615bd2b2aaa0378"}
Sep 20 02:00:04.262240 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:00:04.261808    2584 scope.go:117] "RemoveContainer" containerID="d2b555faf13b049d9632109cf17c1b459e7340c6806317f94fc0929910e5c48a"
Sep 20 02:00:04.286762 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:00:04.286670    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-42hqf"]
Sep 20 02:00:04.289929 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:00:04.289890    2584 scope.go:117] "RemoveContainer" containerID="d2b555faf13b049d9632109cf17c1b459e7340c6806317f94fc0929910e5c48a"
Sep 20 02:00:04.290556 ip-10-0-130-159 kubenswrapper[2584]: E0920 02:00:04.290487    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d2b555faf13b049d9632109cf17c1b459e7340c6806317f94fc0929910e5c48a\": container with ID starting with d2b555faf13b049d9632109cf17c1b459e7340c6806317f94fc0929910e5c48a not found: ID does not exist" containerID="d2b555faf13b049d9632109cf17c1b459e7340c6806317f94fc0929910e5c48a"
Sep 20 02:00:04.290665 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:00:04.290552    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d2b555faf13b049d9632109cf17c1b459e7340c6806317f94fc0929910e5c48a"} err="failed to get container status \"d2b555faf13b049d9632109cf17c1b459e7340c6806317f94fc0929910e5c48a\": rpc error: code = NotFound desc = could not find container \"d2b555faf13b049d9632109cf17c1b459e7340c6806317f94fc0929910e5c48a\": container with ID starting with d2b555faf13b049d9632109cf17c1b459e7340c6806317f94fc0929910e5c48a not found: ID does not exist"
Sep 20 02:00:04.291912 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:00:04.291888    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-42hqf"]
Sep 20 02:00:05.095728 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:00:05.095697    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7e91c38e-3104-4e7d-8876-aec191ce18dd" path="/var/lib/kubelet/pods/7e91c38e-3104-4e7d-8876-aec191ce18dd/volumes"
Sep 20 02:00:05.268173 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:00:05.268142    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-v8pxt" event={"ID":"fe1d0a3b-8799-4617-8763-fa987349cf78","Type":"ContainerStarted","Data":"8f6bacfa2141aa18975e53b4d617ff379075903c5dedd8f92d15841df78a81be"}
Sep 20 02:00:05.268630 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:00:05.268206    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-v8pxt"
Sep 20 02:00:05.286247 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:00:05.286193    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-v8pxt" podStartSLOduration=1.598102819 podStartE2EDuration="4.286178048s" totalImagesPullingTime="2.688075229s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-20 02:00:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-20 02:00:05.284028622 +0000 UTC m=+462.026707824" watchObservedRunningTime="2026-09-20 02:00:05.286178048 +0000 UTC m=+462.028857252"
Sep 20 02:00:07.280943 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:00:07.280901    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6vgqp" event={"ID":"4f29b72e-019b-4f87-b9c2-9154fd7bb6ec","Type":"ContainerStarted","Data":"11c857b22ea995f58eec25204bc30c27e82975c9324c3c0f3eef65c3ed64225b"}
Sep 20 02:00:07.281403 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:00:07.280969    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6vgqp"
Sep 20 02:00:07.298410 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:00:07.298357    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6vgqp" podStartSLOduration=2.390491692 podStartE2EDuration="6.29834045s" totalImagesPullingTime="3.907848758s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-20 02:00:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-20 02:00:07.296724825 +0000 UTC m=+464.039404042" watchObservedRunningTime="2026-09-20 02:00:07.29834045 +0000 UTC m=+464.041019654"
Sep 20 02:00:36.280275 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:00:36.280236    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-v8pxt"
Sep 20 02:00:38.287873 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:00:38.287837    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6vgqp"
Sep 20 02:00:52.820625 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:00:52.820595    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:00:52.918297 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:00:52.918265    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:00:53.018420 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:00:53.018368    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:01:13.024746 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:13.024663    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:01:15.072790 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:15.072754    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-z4hxf"]
Sep 20 02:01:15.073201 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:15.073060    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e91c38e-3104-4e7d-8876-aec191ce18dd" containerName="discovery"
Sep 20 02:01:15.073201 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:15.073071    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e91c38e-3104-4e7d-8876-aec191ce18dd" containerName="discovery"
Sep 20 02:01:15.073201 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:15.073137    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="7e91c38e-3104-4e7d-8876-aec191ce18dd" containerName="discovery"
Sep 20 02:01:15.076010 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:15.075992    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-z4hxf"
Sep 20 02:01:15.078333 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:15.078304    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 20 02:01:15.079164 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:15.079142    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-wzqj8\""
Sep 20 02:01:15.079230 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:15.079142    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 20 02:01:15.095418 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:15.095385    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-z4hxf"]
Sep 20 02:01:15.172248 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:15.172208    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qbrpq\" (UniqueName: \"kubernetes.io/projected/c6732ac4-8d64-4bc8-a9d9-8a01acd0f611-kube-api-access-qbrpq\") pod \"limitador-operator-controller-manager-74495c69f8-z4hxf\" (UID: \"c6732ac4-8d64-4bc8-a9d9-8a01acd0f611\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-z4hxf"
Sep 20 02:01:15.273308 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:15.273267    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qbrpq\" (UniqueName: \"kubernetes.io/projected/c6732ac4-8d64-4bc8-a9d9-8a01acd0f611-kube-api-access-qbrpq\") pod \"limitador-operator-controller-manager-74495c69f8-z4hxf\" (UID: \"c6732ac4-8d64-4bc8-a9d9-8a01acd0f611\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-z4hxf"
Sep 20 02:01:15.287743 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:15.287710    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qbrpq\" (UniqueName: \"kubernetes.io/projected/c6732ac4-8d64-4bc8-a9d9-8a01acd0f611-kube-api-access-qbrpq\") pod \"limitador-operator-controller-manager-74495c69f8-z4hxf\" (UID: \"c6732ac4-8d64-4bc8-a9d9-8a01acd0f611\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-z4hxf"
Sep 20 02:01:15.392180 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:15.392072    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-z4hxf"
Sep 20 02:01:15.635384 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:15.635351    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:01:15.663005 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:15.662977    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-z4hxf"]
Sep 20 02:01:15.665862 ip-10-0-130-159 kubenswrapper[2584]: W0920 02:01:15.665828    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6732ac4_8d64_4bc8_a9d9_8a01acd0f611.slice/crio-2d03a35412d7dca29540602a1c200f73ca32f36f2989df0fa9c07e04de2c7eed WatchSource:0}: Error finding container 2d03a35412d7dca29540602a1c200f73ca32f36f2989df0fa9c07e04de2c7eed: Status 404 returned error can't find the container with id 2d03a35412d7dca29540602a1c200f73ca32f36f2989df0fa9c07e04de2c7eed
Sep 20 02:01:16.028793 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:16.028763    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:01:16.471141 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:16.471043    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:01:16.569842 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:16.569809    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-z4hxf" event={"ID":"c6732ac4-8d64-4bc8-a9d9-8a01acd0f611","Type":"ContainerStarted","Data":"2d03a35412d7dca29540602a1c200f73ca32f36f2989df0fa9c07e04de2c7eed"}
Sep 20 02:01:18.584641 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:18.584544    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-z4hxf" event={"ID":"c6732ac4-8d64-4bc8-a9d9-8a01acd0f611","Type":"ContainerStarted","Data":"da1f1eece9135fe5b8c30c386de4c5acc1c829bab204e8b3bb6e0e70712d8730"}
Sep 20 02:01:18.584641 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:18.584624    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-z4hxf"
Sep 20 02:01:18.601206 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:18.601157    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-z4hxf" podStartSLOduration=0.968362196 podStartE2EDuration="3.601143166s" totalImagesPullingTime="2.63278097s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-20 02:01:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-20 02:01:18.600119155 +0000 UTC m=+535.342798361" watchObservedRunningTime="2026-09-20 02:01:18.601143166 +0000 UTC m=+535.343822372"
Sep 20 02:01:19.274496 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:19.274463    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-b8g7k"]
Sep 20 02:01:19.277738 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:19.277722    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-b8g7k"
Sep 20 02:01:19.279827 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:19.279800    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:01:19.280094 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:19.280078    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-j9sqj\""
Sep 20 02:01:19.292085 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:19.292057    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-b8g7k"]
Sep 20 02:01:19.409242 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:19.409202    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5c0360f4-a2a5-43ff-af02-874c15a1f071-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-b8g7k\" (UID: \"5c0360f4-a2a5-43ff-af02-874c15a1f071\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-b8g7k"
Sep 20 02:01:19.409424 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:19.409270    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2xtcm\" (UniqueName: \"kubernetes.io/projected/5c0360f4-a2a5-43ff-af02-874c15a1f071-kube-api-access-2xtcm\") pod \"kuadrant-operator-controller-manager-76f8f686bf-b8g7k\" (UID: \"5c0360f4-a2a5-43ff-af02-874c15a1f071\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-b8g7k"
Sep 20 02:01:19.509841 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:19.509773    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5c0360f4-a2a5-43ff-af02-874c15a1f071-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-b8g7k\" (UID: \"5c0360f4-a2a5-43ff-af02-874c15a1f071\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-b8g7k"
Sep 20 02:01:19.510046 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:19.509895    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2xtcm\" (UniqueName: \"kubernetes.io/projected/5c0360f4-a2a5-43ff-af02-874c15a1f071-kube-api-access-2xtcm\") pod \"kuadrant-operator-controller-manager-76f8f686bf-b8g7k\" (UID: \"5c0360f4-a2a5-43ff-af02-874c15a1f071\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-b8g7k"
Sep 20 02:01:19.510252 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:19.510230    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5c0360f4-a2a5-43ff-af02-874c15a1f071-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-b8g7k\" (UID: \"5c0360f4-a2a5-43ff-af02-874c15a1f071\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-b8g7k"
Sep 20 02:01:19.518814 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:19.518784    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2xtcm\" (UniqueName: \"kubernetes.io/projected/5c0360f4-a2a5-43ff-af02-874c15a1f071-kube-api-access-2xtcm\") pod \"kuadrant-operator-controller-manager-76f8f686bf-b8g7k\" (UID: \"5c0360f4-a2a5-43ff-af02-874c15a1f071\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-b8g7k"
Sep 20 02:01:19.587852 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:19.587766    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-b8g7k"
Sep 20 02:01:19.863021 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:19.862898    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-b8g7k"]
Sep 20 02:01:19.866083 ip-10-0-130-159 kubenswrapper[2584]: W0920 02:01:19.866050    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c0360f4_a2a5_43ff_af02_874c15a1f071.slice/crio-021380080ef91044daad9048ee390d13cf6a3409cb6f2701fb77f25bf20655ee WatchSource:0}: Error finding container 021380080ef91044daad9048ee390d13cf6a3409cb6f2701fb77f25bf20655ee: Status 404 returned error can't find the container with id 021380080ef91044daad9048ee390d13cf6a3409cb6f2701fb77f25bf20655ee
Sep 20 02:01:20.127275 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:20.127177    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:01:20.323631 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:20.323594    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:01:20.423079 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:20.423000    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:01:20.597759 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:20.597725    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-b8g7k" event={"ID":"5c0360f4-a2a5-43ff-af02-874c15a1f071","Type":"ContainerStarted","Data":"021380080ef91044daad9048ee390d13cf6a3409cb6f2701fb77f25bf20655ee"}
Sep 20 02:01:21.401371 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:21.401335    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-mc75w"]
Sep 20 02:01:21.404787 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:21.404764    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-mc75w"
Sep 20 02:01:21.407169 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:21.407144    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Sep 20 02:01:21.407295 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:21.407190    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-tvvww\""
Sep 20 02:01:21.416737 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:21.416714    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-mc75w"]
Sep 20 02:01:21.527294 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:21.527254    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pt2fm\" (UniqueName: \"kubernetes.io/projected/9002335e-a543-4c6c-ba94-cd0812a7978d-kube-api-access-pt2fm\") pod \"dns-operator-controller-manager-7b84cd7c9c-mc75w\" (UID: \"9002335e-a543-4c6c-ba94-cd0812a7978d\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-mc75w"
Sep 20 02:01:21.628134 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:21.628098    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pt2fm\" (UniqueName: \"kubernetes.io/projected/9002335e-a543-4c6c-ba94-cd0812a7978d-kube-api-access-pt2fm\") pod \"dns-operator-controller-manager-7b84cd7c9c-mc75w\" (UID: \"9002335e-a543-4c6c-ba94-cd0812a7978d\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-mc75w"
Sep 20 02:01:21.638919 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:21.638895    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pt2fm\" (UniqueName: \"kubernetes.io/projected/9002335e-a543-4c6c-ba94-cd0812a7978d-kube-api-access-pt2fm\") pod \"dns-operator-controller-manager-7b84cd7c9c-mc75w\" (UID: \"9002335e-a543-4c6c-ba94-cd0812a7978d\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-mc75w"
Sep 20 02:01:21.643471 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:21.643441    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:01:21.715959 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:21.715918    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-mc75w"
Sep 20 02:01:22.013969 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:22.013936    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-mc75w"]
Sep 20 02:01:22.015832 ip-10-0-130-159 kubenswrapper[2584]: W0920 02:01:22.015803    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9002335e_a543_4c6c_ba94_cd0812a7978d.slice/crio-de2f159ac60be60d9a196987cac16f36bfb8085ed70da8b81072494195a7c1b6 WatchSource:0}: Error finding container de2f159ac60be60d9a196987cac16f36bfb8085ed70da8b81072494195a7c1b6: Status 404 returned error can't find the container with id de2f159ac60be60d9a196987cac16f36bfb8085ed70da8b81072494195a7c1b6
Sep 20 02:01:22.047013 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:22.046974    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:01:22.148878 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:22.148845    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:01:22.610180 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:22.610140    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-mc75w" event={"ID":"9002335e-a543-4c6c-ba94-cd0812a7978d","Type":"ContainerStarted","Data":"de2f159ac60be60d9a196987cac16f36bfb8085ed70da8b81072494195a7c1b6"}
Sep 20 02:01:23.268796 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:23.268744    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:01:24.277682 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:24.277640    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-m6b4s"]
Sep 20 02:01:24.281153 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:24.281130    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-m6b4s"
Sep 20 02:01:24.284368 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:24.284137    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-9xfcf\""
Sep 20 02:01:24.292883 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:24.292843    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-m6b4s"]
Sep 20 02:01:24.454848 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:24.454808    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v5fpg\" (UniqueName: \"kubernetes.io/projected/464f7342-77e6-4c63-808c-cc6c73f8df11-kube-api-access-v5fpg\") pod \"authorino-operator-76b478bd78-m6b4s\" (UID: \"464f7342-77e6-4c63-808c-cc6c73f8df11\") " pod="kuadrant-system/authorino-operator-76b478bd78-m6b4s"
Sep 20 02:01:24.556112 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:24.556018    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v5fpg\" (UniqueName: \"kubernetes.io/projected/464f7342-77e6-4c63-808c-cc6c73f8df11-kube-api-access-v5fpg\") pod \"authorino-operator-76b478bd78-m6b4s\" (UID: \"464f7342-77e6-4c63-808c-cc6c73f8df11\") " pod="kuadrant-system/authorino-operator-76b478bd78-m6b4s"
Sep 20 02:01:24.564423 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:24.564390    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v5fpg\" (UniqueName: \"kubernetes.io/projected/464f7342-77e6-4c63-808c-cc6c73f8df11-kube-api-access-v5fpg\") pod \"authorino-operator-76b478bd78-m6b4s\" (UID: \"464f7342-77e6-4c63-808c-cc6c73f8df11\") " pod="kuadrant-system/authorino-operator-76b478bd78-m6b4s"
Sep 20 02:01:24.600016 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:24.599963    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-m6b4s"
Sep 20 02:01:24.892347 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:24.892321    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-m6b4s"]
Sep 20 02:01:24.894331 ip-10-0-130-159 kubenswrapper[2584]: W0920 02:01:24.894309    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod464f7342_77e6_4c63_808c_cc6c73f8df11.slice/crio-8be43015c69bf20fecf22cc9279d9381ea70e22f3cbe80963ee19fcdccbafb2f WatchSource:0}: Error finding container 8be43015c69bf20fecf22cc9279d9381ea70e22f3cbe80963ee19fcdccbafb2f: Status 404 returned error can't find the container with id 8be43015c69bf20fecf22cc9279d9381ea70e22f3cbe80963ee19fcdccbafb2f
Sep 20 02:01:25.626215 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:25.626174    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-m6b4s" event={"ID":"464f7342-77e6-4c63-808c-cc6c73f8df11","Type":"ContainerStarted","Data":"8be43015c69bf20fecf22cc9279d9381ea70e22f3cbe80963ee19fcdccbafb2f"}
Sep 20 02:01:25.628052 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:25.628014    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-b8g7k" event={"ID":"5c0360f4-a2a5-43ff-af02-874c15a1f071","Type":"ContainerStarted","Data":"4a76460a4f8e265ed4bec5b6e653d1e24e0cec95f43b852b608bfd909e71460d"}
Sep 20 02:01:25.628185 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:25.628111    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-b8g7k"
Sep 20 02:01:25.630227 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:25.630201    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-mc75w" event={"ID":"9002335e-a543-4c6c-ba94-cd0812a7978d","Type":"ContainerStarted","Data":"1aa873dd678c671b2ee0557b790f34ea32ae7672e1a69e62cf1cbbb7dba5f457"}
Sep 20 02:01:25.630381 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:25.630282    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-mc75w"
Sep 20 02:01:25.646817 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:25.646762    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-b8g7k" podStartSLOduration=1.879360556 podStartE2EDuration="6.646744893s" totalImagesPullingTime="4.767384337s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-20 02:01:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-20 02:01:25.646613389 +0000 UTC m=+542.389292594" watchObservedRunningTime="2026-09-20 02:01:25.646744893 +0000 UTC m=+542.389424095"
Sep 20 02:01:25.649372 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:25.649337    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:01:25.666425 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:25.666368    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-mc75w" podStartSLOduration=2.002859972 podStartE2EDuration="4.666350859s" totalImagesPullingTime="2.663490887s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-20 02:01:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-20 02:01:25.665269888 +0000 UTC m=+542.407949096" watchObservedRunningTime="2026-09-20 02:01:25.666350859 +0000 UTC m=+542.409030064"
Sep 20 02:01:25.749120 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:25.749087    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:01:27.464176 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:27.464144    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:01:27.849899 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:27.849811    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:01:28.646887 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:28.646794    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-m6b4s" event={"ID":"464f7342-77e6-4c63-808c-cc6c73f8df11","Type":"ContainerStarted","Data":"7f2a855c2d47e643dd212db07b843b3129d343f1421fd73820c98a76adc4f882"}
Sep 20 02:01:28.646887 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:28.646865    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-76b478bd78-m6b4s"
Sep 20 02:01:28.667118 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:28.667067    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-76b478bd78-m6b4s" podStartSLOduration=1.264473076 podStartE2EDuration="4.667049384s" totalImagesPullingTime="3.402576308s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-20 02:01:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-20 02:01:28.666021822 +0000 UTC m=+545.408701030" watchObservedRunningTime="2026-09-20 02:01:28.667049384 +0000 UTC m=+545.409728588"
Sep 20 02:01:29.593702 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:29.593662    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-z4hxf"
Sep 20 02:01:29.632674 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:29.632638    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:01:29.732872 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:29.732841    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:01:30.035928 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:30.035895    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:01:30.139859 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:30.139826    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:01:30.280235 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:30.280195    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:01:30.399610 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:30.399534    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:01:31.963021 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:31.962986    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:01:32.063327 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:32.063296    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:01:35.722964 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:35.722931    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:01:35.776491 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:35.776456    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:01:36.637227 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:36.637201    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-mc75w"
Sep 20 02:01:36.637423 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:36.637271    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-b8g7k"
Sep 20 02:01:38.239654 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:38.239620    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-fp2r8"]
Sep 20 02:01:38.243034 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:38.243010    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-fp2r8"
Sep 20 02:01:38.258904 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:38.258873    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-fp2r8"]
Sep 20 02:01:38.268953 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:38.268924    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f05d74f7-a537-4ac8-ae97-52a388d0091e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f685c496f-fp2r8\" (UID: \"f05d74f7-a537-4ac8-ae97-52a388d0091e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-fp2r8"
Sep 20 02:01:38.269093 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:38.268960    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mr8nt\" (UniqueName: \"kubernetes.io/projected/f05d74f7-a537-4ac8-ae97-52a388d0091e-kube-api-access-mr8nt\") pod \"kuadrant-operator-controller-manager-5f685c496f-fp2r8\" (UID: \"f05d74f7-a537-4ac8-ae97-52a388d0091e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-fp2r8"
Sep 20 02:01:38.370173 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:38.370132    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f05d74f7-a537-4ac8-ae97-52a388d0091e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f685c496f-fp2r8\" (UID: \"f05d74f7-a537-4ac8-ae97-52a388d0091e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-fp2r8"
Sep 20 02:01:38.370173 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:38.370175    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mr8nt\" (UniqueName: \"kubernetes.io/projected/f05d74f7-a537-4ac8-ae97-52a388d0091e-kube-api-access-mr8nt\") pod \"kuadrant-operator-controller-manager-5f685c496f-fp2r8\" (UID: \"f05d74f7-a537-4ac8-ae97-52a388d0091e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-fp2r8"
Sep 20 02:01:38.370531 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:38.370497    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f05d74f7-a537-4ac8-ae97-52a388d0091e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f685c496f-fp2r8\" (UID: \"f05d74f7-a537-4ac8-ae97-52a388d0091e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-fp2r8"
Sep 20 02:01:38.379373 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:38.379340    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mr8nt\" (UniqueName: \"kubernetes.io/projected/f05d74f7-a537-4ac8-ae97-52a388d0091e-kube-api-access-mr8nt\") pod \"kuadrant-operator-controller-manager-5f685c496f-fp2r8\" (UID: \"f05d74f7-a537-4ac8-ae97-52a388d0091e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-fp2r8"
Sep 20 02:01:38.553247 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:38.553153    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-fp2r8"
Sep 20 02:01:38.827806 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:38.827772    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-fp2r8"]
Sep 20 02:01:38.829694 ip-10-0-130-159 kubenswrapper[2584]: W0920 02:01:38.829665    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf05d74f7_a537_4ac8_ae97_52a388d0091e.slice/crio-0d35203a9ce33206cfb6f6600fc68901468235d8d338b42b659092ff12c40d42 WatchSource:0}: Error finding container 0d35203a9ce33206cfb6f6600fc68901468235d8d338b42b659092ff12c40d42: Status 404 returned error can't find the container with id 0d35203a9ce33206cfb6f6600fc68901468235d8d338b42b659092ff12c40d42
Sep 20 02:01:38.911212 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:38.911179    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-fp2r8"]
Sep 20 02:01:38.917791 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:38.917760    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-fp2r8"]
Sep 20 02:01:38.929410 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:38.929377    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-b8g7k"]
Sep 20 02:01:38.929665 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:38.929618    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-b8g7k" podUID="5c0360f4-a2a5-43ff-af02-874c15a1f071" containerName="manager" containerID="cri-o://4a76460a4f8e265ed4bec5b6e653d1e24e0cec95f43b852b608bfd909e71460d" gracePeriod=2
Sep 20 02:01:38.933625 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:38.933596    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-rdvz8"]
Sep 20 02:01:38.933994 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:38.933978    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f05d74f7-a537-4ac8-ae97-52a388d0091e" containerName="manager"
Sep 20 02:01:38.934067 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:38.933997    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="f05d74f7-a537-4ac8-ae97-52a388d0091e" containerName="manager"
Sep 20 02:01:38.934104 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:38.934076    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="f05d74f7-a537-4ac8-ae97-52a388d0091e" containerName="manager"
Sep 20 02:01:38.937727 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:38.937700    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-rdvz8"
Sep 20 02:01:38.941693 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:38.941589    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-b8g7k"]
Sep 20 02:01:38.944640 ip-10-0-130-159 kubenswrapper[2584]: E0920 02:01:38.944608    2584 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"5c0360f4-a2a5-43ff-af02-874c15a1f071\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-76f8f686bf-b8g7k\": pods \"kuadrant-operator-controller-manager-76f8f686bf-b8g7k\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-b8g7k"
Sep 20 02:01:38.952945 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:38.952909    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-rdvz8"]
Sep 20 02:01:38.956787 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:38.956730    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-j8fd5"]
Sep 20 02:01:38.957274 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:38.957256    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c0360f4-a2a5-43ff-af02-874c15a1f071" containerName="manager"
Sep 20 02:01:38.957274 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:38.957274    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c0360f4-a2a5-43ff-af02-874c15a1f071" containerName="manager"
Sep 20 02:01:38.957434 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:38.957379    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="5c0360f4-a2a5-43ff-af02-874c15a1f071" containerName="manager"
Sep 20 02:01:38.960722 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:38.960699    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-z4hxf"]
Sep 20 02:01:38.960843 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:38.960803    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-j8fd5"
Sep 20 02:01:38.960915 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:38.960893    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-z4hxf" podUID="c6732ac4-8d64-4bc8-a9d9-8a01acd0f611" containerName="manager" containerID="cri-o://da1f1eece9135fe5b8c30c386de4c5acc1c829bab204e8b3bb6e0e70712d8730" gracePeriod=2
Sep 20 02:01:38.969361 ip-10-0-130-159 kubenswrapper[2584]: E0920 02:01:38.968777    2584 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-b8g7k\" is forbidden: User \"system:node:ip-10-0-130-159.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-159.ec2.internal' and this object" podUID="5c0360f4-a2a5-43ff-af02-874c15a1f071" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-b8g7k"
Sep 20 02:01:38.970009 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:38.969878    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-z4hxf"]
Sep 20 02:01:38.973431 ip-10-0-130-159 kubenswrapper[2584]: E0920 02:01:38.973074    2584 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-z4hxf\" is forbidden: User \"system:node:ip-10-0-130-159.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-159.ec2.internal' and this object" podUID="c6732ac4-8d64-4bc8-a9d9-8a01acd0f611" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-z4hxf"
Sep 20 02:01:38.974585 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:38.974560    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-j8fd5"]
Sep 20 02:01:38.975014 ip-10-0-130-159 kubenswrapper[2584]: E0920 02:01:38.974975    2584 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-b8g7k\" is forbidden: User \"system:node:ip-10-0-130-159.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-159.ec2.internal' and this object" podUID="5c0360f4-a2a5-43ff-af02-874c15a1f071" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-b8g7k"
Sep 20 02:01:38.975544 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:38.975494    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6p5lk\" (UniqueName: \"kubernetes.io/projected/35590bf4-b1ea-413a-a884-337626e3f883-kube-api-access-6p5lk\") pod \"kuadrant-operator-controller-manager-76f8f686bf-j8fd5\" (UID: \"35590bf4-b1ea-413a-a884-337626e3f883\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-j8fd5"
Sep 20 02:01:38.975646 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:38.975572    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6d4cf2a2-2da3-4682-b9dc-1b1ed9547a3c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f685c496f-rdvz8\" (UID: \"6d4cf2a2-2da3-4682-b9dc-1b1ed9547a3c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-rdvz8"
Sep 20 02:01:38.975646 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:38.975636    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6x264\" (UniqueName: \"kubernetes.io/projected/6d4cf2a2-2da3-4682-b9dc-1b1ed9547a3c-kube-api-access-6x264\") pod \"kuadrant-operator-controller-manager-5f685c496f-rdvz8\" (UID: \"6d4cf2a2-2da3-4682-b9dc-1b1ed9547a3c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-rdvz8"
Sep 20 02:01:38.975770 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:38.975659    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/35590bf4-b1ea-413a-a884-337626e3f883-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-j8fd5\" (UID: \"35590bf4-b1ea-413a-a884-337626e3f883\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-j8fd5"
Sep 20 02:01:38.989809 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:38.989771    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-krtwd"]
Sep 20 02:01:38.990126 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:38.990100    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c6732ac4-8d64-4bc8-a9d9-8a01acd0f611" containerName="manager"
Sep 20 02:01:38.990126 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:38.990114    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="c6732ac4-8d64-4bc8-a9d9-8a01acd0f611" containerName="manager"
Sep 20 02:01:38.990322 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:38.990181    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="c6732ac4-8d64-4bc8-a9d9-8a01acd0f611" containerName="manager"
Sep 20 02:01:38.993031 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:38.993006    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-krtwd"
Sep 20 02:01:38.997218 ip-10-0-130-159 kubenswrapper[2584]: E0920 02:01:38.997190    2584 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-z4hxf\" is forbidden: User \"system:node:ip-10-0-130-159.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-159.ec2.internal' and this object" podUID="c6732ac4-8d64-4bc8-a9d9-8a01acd0f611" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-z4hxf"
Sep 20 02:01:38.998982 ip-10-0-130-159 kubenswrapper[2584]: E0920 02:01:38.998949    2584 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-b8g7k\" is forbidden: User \"system:node:ip-10-0-130-159.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-159.ec2.internal' and this object" podUID="5c0360f4-a2a5-43ff-af02-874c15a1f071" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-b8g7k"
Sep 20 02:01:39.003854 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:39.003825    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-krtwd"]
Sep 20 02:01:39.076630 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:39.076554    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6x264\" (UniqueName: \"kubernetes.io/projected/6d4cf2a2-2da3-4682-b9dc-1b1ed9547a3c-kube-api-access-6x264\") pod \"kuadrant-operator-controller-manager-5f685c496f-rdvz8\" (UID: \"6d4cf2a2-2da3-4682-b9dc-1b1ed9547a3c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-rdvz8"
Sep 20 02:01:39.076630 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:39.076594    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/35590bf4-b1ea-413a-a884-337626e3f883-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-j8fd5\" (UID: \"35590bf4-b1ea-413a-a884-337626e3f883\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-j8fd5"
Sep 20 02:01:39.076859 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:39.076836    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rj4h4\" (UniqueName: \"kubernetes.io/projected/db69bff3-503c-495e-99a8-0d90f8f0a154-kube-api-access-rj4h4\") pod \"limitador-operator-controller-manager-74495c69f8-krtwd\" (UID: \"db69bff3-503c-495e-99a8-0d90f8f0a154\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-krtwd"
Sep 20 02:01:39.076926 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:39.076905    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6p5lk\" (UniqueName: \"kubernetes.io/projected/35590bf4-b1ea-413a-a884-337626e3f883-kube-api-access-6p5lk\") pod \"kuadrant-operator-controller-manager-76f8f686bf-j8fd5\" (UID: \"35590bf4-b1ea-413a-a884-337626e3f883\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-j8fd5"
Sep 20 02:01:39.076985 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:39.076969    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6d4cf2a2-2da3-4682-b9dc-1b1ed9547a3c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f685c496f-rdvz8\" (UID: \"6d4cf2a2-2da3-4682-b9dc-1b1ed9547a3c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-rdvz8"
Sep 20 02:01:39.077045 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:39.076986    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/35590bf4-b1ea-413a-a884-337626e3f883-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-j8fd5\" (UID: \"35590bf4-b1ea-413a-a884-337626e3f883\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-j8fd5"
Sep 20 02:01:39.077256 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:39.077241    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6d4cf2a2-2da3-4682-b9dc-1b1ed9547a3c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f685c496f-rdvz8\" (UID: \"6d4cf2a2-2da3-4682-b9dc-1b1ed9547a3c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-rdvz8"
Sep 20 02:01:39.085655 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:39.085621    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6x264\" (UniqueName: \"kubernetes.io/projected/6d4cf2a2-2da3-4682-b9dc-1b1ed9547a3c-kube-api-access-6x264\") pod \"kuadrant-operator-controller-manager-5f685c496f-rdvz8\" (UID: \"6d4cf2a2-2da3-4682-b9dc-1b1ed9547a3c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-rdvz8"
Sep 20 02:01:39.085814 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:39.085797    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6p5lk\" (UniqueName: \"kubernetes.io/projected/35590bf4-b1ea-413a-a884-337626e3f883-kube-api-access-6p5lk\") pod \"kuadrant-operator-controller-manager-76f8f686bf-j8fd5\" (UID: \"35590bf4-b1ea-413a-a884-337626e3f883\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-j8fd5"
Sep 20 02:01:39.177975 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:39.177941    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rj4h4\" (UniqueName: \"kubernetes.io/projected/db69bff3-503c-495e-99a8-0d90f8f0a154-kube-api-access-rj4h4\") pod \"limitador-operator-controller-manager-74495c69f8-krtwd\" (UID: \"db69bff3-503c-495e-99a8-0d90f8f0a154\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-krtwd"
Sep 20 02:01:39.186617 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:39.186579    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rj4h4\" (UniqueName: \"kubernetes.io/projected/db69bff3-503c-495e-99a8-0d90f8f0a154-kube-api-access-rj4h4\") pod \"limitador-operator-controller-manager-74495c69f8-krtwd\" (UID: \"db69bff3-503c-495e-99a8-0d90f8f0a154\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-krtwd"
Sep 20 02:01:39.193031 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:39.193003    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-b8g7k"
Sep 20 02:01:39.200703 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:39.200679    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-z4hxf"
Sep 20 02:01:39.279060 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:39.279023    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qbrpq\" (UniqueName: \"kubernetes.io/projected/c6732ac4-8d64-4bc8-a9d9-8a01acd0f611-kube-api-access-qbrpq\") pod \"c6732ac4-8d64-4bc8-a9d9-8a01acd0f611\" (UID: \"c6732ac4-8d64-4bc8-a9d9-8a01acd0f611\") "
Sep 20 02:01:39.279463 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:39.279130    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5c0360f4-a2a5-43ff-af02-874c15a1f071-extensions-socket-volume\") pod \"5c0360f4-a2a5-43ff-af02-874c15a1f071\" (UID: \"5c0360f4-a2a5-43ff-af02-874c15a1f071\") "
Sep 20 02:01:39.279463 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:39.279153    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2xtcm\" (UniqueName: \"kubernetes.io/projected/5c0360f4-a2a5-43ff-af02-874c15a1f071-kube-api-access-2xtcm\") pod \"5c0360f4-a2a5-43ff-af02-874c15a1f071\" (UID: \"5c0360f4-a2a5-43ff-af02-874c15a1f071\") "
Sep 20 02:01:39.279636 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:39.279608    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c0360f4-a2a5-43ff-af02-874c15a1f071-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "5c0360f4-a2a5-43ff-af02-874c15a1f071" (UID: "5c0360f4-a2a5-43ff-af02-874c15a1f071"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 20 02:01:39.281379 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:39.281342    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c0360f4-a2a5-43ff-af02-874c15a1f071-kube-api-access-2xtcm" (OuterVolumeSpecName: "kube-api-access-2xtcm") pod "5c0360f4-a2a5-43ff-af02-874c15a1f071" (UID: "5c0360f4-a2a5-43ff-af02-874c15a1f071"). InnerVolumeSpecName "kube-api-access-2xtcm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 20 02:01:39.281379 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:39.281359    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c6732ac4-8d64-4bc8-a9d9-8a01acd0f611-kube-api-access-qbrpq" (OuterVolumeSpecName: "kube-api-access-qbrpq") pod "c6732ac4-8d64-4bc8-a9d9-8a01acd0f611" (UID: "c6732ac4-8d64-4bc8-a9d9-8a01acd0f611"). InnerVolumeSpecName "kube-api-access-qbrpq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 20 02:01:39.345467 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:39.345373    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-rdvz8"
Sep 20 02:01:39.352582 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:39.352552    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-j8fd5"
Sep 20 02:01:39.357631 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:39.357604    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-krtwd"
Sep 20 02:01:39.379976 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:39.379938    2584 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5c0360f4-a2a5-43ff-af02-874c15a1f071-extensions-socket-volume\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:01:39.379976 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:39.379964    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2xtcm\" (UniqueName: \"kubernetes.io/projected/5c0360f4-a2a5-43ff-af02-874c15a1f071-kube-api-access-2xtcm\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:01:39.379976 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:39.379975    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qbrpq\" (UniqueName: \"kubernetes.io/projected/c6732ac4-8d64-4bc8-a9d9-8a01acd0f611-kube-api-access-qbrpq\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:01:39.641099 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:39.641071    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-rdvz8"]
Sep 20 02:01:39.643888 ip-10-0-130-159 kubenswrapper[2584]: W0920 02:01:39.643858    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d4cf2a2_2da3_4682_b9dc_1b1ed9547a3c.slice/crio-a84c0e3fa92705e89be75269319fd194dfe100f74698e7587dbb34f2a2b947e2 WatchSource:0}: Error finding container a84c0e3fa92705e89be75269319fd194dfe100f74698e7587dbb34f2a2b947e2: Status 404 returned error can't find the container with id a84c0e3fa92705e89be75269319fd194dfe100f74698e7587dbb34f2a2b947e2
Sep 20 02:01:39.655758 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:39.655736    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-76b478bd78-m6b4s"
Sep 20 02:01:39.656646 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:39.656625    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-j8fd5"]
Sep 20 02:01:39.659131 ip-10-0-130-159 kubenswrapper[2584]: W0920 02:01:39.659111    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35590bf4_b1ea_413a_a884_337626e3f883.slice/crio-f044f1d7a65c3b7207355faec5c99b702a071665c72c0593f6a021d3f9f0d208 WatchSource:0}: Error finding container f044f1d7a65c3b7207355faec5c99b702a071665c72c0593f6a021d3f9f0d208: Status 404 returned error can't find the container with id f044f1d7a65c3b7207355faec5c99b702a071665c72c0593f6a021d3f9f0d208
Sep 20 02:01:39.666941 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:39.666919    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-krtwd"]
Sep 20 02:01:39.669251 ip-10-0-130-159 kubenswrapper[2584]: W0920 02:01:39.669228    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb69bff3_503c_495e_99a8_0d90f8f0a154.slice/crio-7b6ac32d6f61dd7ae02ed37efd6858e1e88ecae0b8bba403662cbd127fabc14f WatchSource:0}: Error finding container 7b6ac32d6f61dd7ae02ed37efd6858e1e88ecae0b8bba403662cbd127fabc14f: Status 404 returned error can't find the container with id 7b6ac32d6f61dd7ae02ed37efd6858e1e88ecae0b8bba403662cbd127fabc14f
Sep 20 02:01:39.701554 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:39.701507    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-j8fd5" event={"ID":"35590bf4-b1ea-413a-a884-337626e3f883","Type":"ContainerStarted","Data":"f044f1d7a65c3b7207355faec5c99b702a071665c72c0593f6a021d3f9f0d208"}
Sep 20 02:01:39.703302 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:39.703274    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-rdvz8" event={"ID":"6d4cf2a2-2da3-4682-b9dc-1b1ed9547a3c","Type":"ContainerStarted","Data":"a84c0e3fa92705e89be75269319fd194dfe100f74698e7587dbb34f2a2b947e2"}
Sep 20 02:01:39.705638 ip-10-0-130-159 kubenswrapper[2584]: E0920 02:01:39.705616    2584 kuberuntime_manager.go:649] "Missing actuated resource record" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-fp2r8" container="manager"
Sep 20 02:01:39.707104 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:39.707010    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-krtwd" event={"ID":"db69bff3-503c-495e-99a8-0d90f8f0a154","Type":"ContainerStarted","Data":"7b6ac32d6f61dd7ae02ed37efd6858e1e88ecae0b8bba403662cbd127fabc14f"}
Sep 20 02:01:39.709479 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:39.709399    2584 generic.go:356] "Generic (PLEG): container finished" podID="5c0360f4-a2a5-43ff-af02-874c15a1f071" containerID="4a76460a4f8e265ed4bec5b6e653d1e24e0cec95f43b852b608bfd909e71460d" exitCode=0
Sep 20 02:01:39.709479 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:39.709470    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-b8g7k"
Sep 20 02:01:39.709651 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:39.709575    2584 scope.go:117] "RemoveContainer" containerID="4a76460a4f8e265ed4bec5b6e653d1e24e0cec95f43b852b608bfd909e71460d"
Sep 20 02:01:39.712243 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:39.712138    2584 generic.go:356] "Generic (PLEG): container finished" podID="c6732ac4-8d64-4bc8-a9d9-8a01acd0f611" containerID="da1f1eece9135fe5b8c30c386de4c5acc1c829bab204e8b3bb6e0e70712d8730" exitCode=0
Sep 20 02:01:39.712243 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:39.712179    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-z4hxf"
Sep 20 02:01:39.723694 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:39.723674    2584 scope.go:117] "RemoveContainer" containerID="4a76460a4f8e265ed4bec5b6e653d1e24e0cec95f43b852b608bfd909e71460d"
Sep 20 02:01:39.724005 ip-10-0-130-159 kubenswrapper[2584]: E0920 02:01:39.723987    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4a76460a4f8e265ed4bec5b6e653d1e24e0cec95f43b852b608bfd909e71460d\": container with ID starting with 4a76460a4f8e265ed4bec5b6e653d1e24e0cec95f43b852b608bfd909e71460d not found: ID does not exist" containerID="4a76460a4f8e265ed4bec5b6e653d1e24e0cec95f43b852b608bfd909e71460d"
Sep 20 02:01:39.724095 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:39.724012    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a76460a4f8e265ed4bec5b6e653d1e24e0cec95f43b852b608bfd909e71460d"} err="failed to get container status \"4a76460a4f8e265ed4bec5b6e653d1e24e0cec95f43b852b608bfd909e71460d\": rpc error: code = NotFound desc = could not find container \"4a76460a4f8e265ed4bec5b6e653d1e24e0cec95f43b852b608bfd909e71460d\": container with ID starting with 4a76460a4f8e265ed4bec5b6e653d1e24e0cec95f43b852b608bfd909e71460d not found: ID does not exist"
Sep 20 02:01:39.724095 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:39.724028    2584 scope.go:117] "RemoveContainer" containerID="da1f1eece9135fe5b8c30c386de4c5acc1c829bab204e8b3bb6e0e70712d8730"
Sep 20 02:01:39.742276 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:39.742249    2584 scope.go:117] "RemoveContainer" containerID="da1f1eece9135fe5b8c30c386de4c5acc1c829bab204e8b3bb6e0e70712d8730"
Sep 20 02:01:39.742661 ip-10-0-130-159 kubenswrapper[2584]: E0920 02:01:39.742633    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"da1f1eece9135fe5b8c30c386de4c5acc1c829bab204e8b3bb6e0e70712d8730\": container with ID starting with da1f1eece9135fe5b8c30c386de4c5acc1c829bab204e8b3bb6e0e70712d8730 not found: ID does not exist" containerID="da1f1eece9135fe5b8c30c386de4c5acc1c829bab204e8b3bb6e0e70712d8730"
Sep 20 02:01:39.742762 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:39.742673    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da1f1eece9135fe5b8c30c386de4c5acc1c829bab204e8b3bb6e0e70712d8730"} err="failed to get container status \"da1f1eece9135fe5b8c30c386de4c5acc1c829bab204e8b3bb6e0e70712d8730\": rpc error: code = NotFound desc = could not find container \"da1f1eece9135fe5b8c30c386de4c5acc1c829bab204e8b3bb6e0e70712d8730\": container with ID starting with da1f1eece9135fe5b8c30c386de4c5acc1c829bab204e8b3bb6e0e70712d8730 not found: ID does not exist"
Sep 20 02:01:40.727566 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:40.727533    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-j8fd5" event={"ID":"35590bf4-b1ea-413a-a884-337626e3f883","Type":"ContainerStarted","Data":"4fb559582e6e6c2b7e9bb2cd657099bde8fcf5c01aff6e226acbbbab60575298"}
Sep 20 02:01:40.728044 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:40.727704    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-j8fd5"
Sep 20 02:01:40.729252 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:40.729229    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-rdvz8" event={"ID":"6d4cf2a2-2da3-4682-b9dc-1b1ed9547a3c","Type":"ContainerStarted","Data":"a6ccb20b7a5ee873042570231896423a2fdc6c8b77c12bddc0c5649a1c301a7a"}
Sep 20 02:01:40.729377 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:40.729275    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-rdvz8"
Sep 20 02:01:40.731008 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:40.730981    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-fp2r8" podUID="f05d74f7-a537-4ac8-ae97-52a388d0091e" containerName="manager" containerID="cri-o://f6c9eef0b4575d868d7e303f479de8e27b8ff3a6a61ebdf31db3dd37da33f0e6" gracePeriod=2
Sep 20 02:01:40.732702 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:40.732673    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-krtwd" event={"ID":"db69bff3-503c-495e-99a8-0d90f8f0a154","Type":"ContainerStarted","Data":"b1420ba29ee1c04dbfffa3b090db104ff1926ae92ae0ca4fbf0860218d6dd15e"}
Sep 20 02:01:40.732789 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:40.732722    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-krtwd"
Sep 20 02:01:40.749250 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:40.749205    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-j8fd5" podStartSLOduration=2.749188262 podStartE2EDuration="2.749188262s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-20 02:01:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-20 02:01:40.747460212 +0000 UTC m=+557.490139419" watchObservedRunningTime="2026-09-20 02:01:40.749188262 +0000 UTC m=+557.491867467"
Sep 20 02:01:40.749362 ip-10-0-130-159 kubenswrapper[2584]: E0920 02:01:40.749323    2584 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-5f685c496f-fp2r8\" is forbidden: User \"system:node:ip-10-0-130-159.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-159.ec2.internal' and this object" podUID="f05d74f7-a537-4ac8-ae97-52a388d0091e" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-fp2r8"
Sep 20 02:01:40.775000 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:40.774945    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-rdvz8" podStartSLOduration=2.7749264350000002 podStartE2EDuration="2.774926435s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-20 02:01:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-20 02:01:40.774357933 +0000 UTC m=+557.517037138" watchObservedRunningTime="2026-09-20 02:01:40.774926435 +0000 UTC m=+557.517605701"
Sep 20 02:01:40.792645 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:40.792599    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-krtwd" podStartSLOduration=2.79258294 podStartE2EDuration="2.79258294s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-20 02:01:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-20 02:01:40.791768048 +0000 UTC m=+557.534447256" watchObservedRunningTime="2026-09-20 02:01:40.79258294 +0000 UTC m=+557.535262145"
Sep 20 02:01:40.986130 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:40.986065    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-fp2r8"
Sep 20 02:01:40.988159 ip-10-0-130-159 kubenswrapper[2584]: E0920 02:01:40.988129    2584 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-5f685c496f-fp2r8\" is forbidden: User \"system:node:ip-10-0-130-159.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-159.ec2.internal' and this object" podUID="f05d74f7-a537-4ac8-ae97-52a388d0091e" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-fp2r8"
Sep 20 02:01:41.095856 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:41.095827    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mr8nt\" (UniqueName: \"kubernetes.io/projected/f05d74f7-a537-4ac8-ae97-52a388d0091e-kube-api-access-mr8nt\") pod \"f05d74f7-a537-4ac8-ae97-52a388d0091e\" (UID: \"f05d74f7-a537-4ac8-ae97-52a388d0091e\") "
Sep 20 02:01:41.096054 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:41.095922    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f05d74f7-a537-4ac8-ae97-52a388d0091e-extensions-socket-volume\") pod \"f05d74f7-a537-4ac8-ae97-52a388d0091e\" (UID: \"f05d74f7-a537-4ac8-ae97-52a388d0091e\") "
Sep 20 02:01:41.096185 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:41.096159    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5c0360f4-a2a5-43ff-af02-874c15a1f071" path="/var/lib/kubelet/pods/5c0360f4-a2a5-43ff-af02-874c15a1f071/volumes"
Sep 20 02:01:41.096242 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:41.096189    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f05d74f7-a537-4ac8-ae97-52a388d0091e-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "f05d74f7-a537-4ac8-ae97-52a388d0091e" (UID: "f05d74f7-a537-4ac8-ae97-52a388d0091e"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 20 02:01:41.096670 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:41.096649    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c6732ac4-8d64-4bc8-a9d9-8a01acd0f611" path="/var/lib/kubelet/pods/c6732ac4-8d64-4bc8-a9d9-8a01acd0f611/volumes"
Sep 20 02:01:41.098205 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:41.098167    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f05d74f7-a537-4ac8-ae97-52a388d0091e-kube-api-access-mr8nt" (OuterVolumeSpecName: "kube-api-access-mr8nt") pod "f05d74f7-a537-4ac8-ae97-52a388d0091e" (UID: "f05d74f7-a537-4ac8-ae97-52a388d0091e"). InnerVolumeSpecName "kube-api-access-mr8nt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 20 02:01:41.197283 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:41.197233    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mr8nt\" (UniqueName: \"kubernetes.io/projected/f05d74f7-a537-4ac8-ae97-52a388d0091e-kube-api-access-mr8nt\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:01:41.197283 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:41.197269    2584 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f05d74f7-a537-4ac8-ae97-52a388d0091e-extensions-socket-volume\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:01:41.738488 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:41.738452    2584 generic.go:356] "Generic (PLEG): container finished" podID="f05d74f7-a537-4ac8-ae97-52a388d0091e" containerID="f6c9eef0b4575d868d7e303f479de8e27b8ff3a6a61ebdf31db3dd37da33f0e6" exitCode=2
Sep 20 02:01:41.739007 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:41.738501    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-fp2r8"
Sep 20 02:01:41.739007 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:41.738562    2584 scope.go:117] "RemoveContainer" containerID="f6c9eef0b4575d868d7e303f479de8e27b8ff3a6a61ebdf31db3dd37da33f0e6"
Sep 20 02:01:41.747508 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:41.747490    2584 scope.go:117] "RemoveContainer" containerID="f6c9eef0b4575d868d7e303f479de8e27b8ff3a6a61ebdf31db3dd37da33f0e6"
Sep 20 02:01:41.747822 ip-10-0-130-159 kubenswrapper[2584]: E0920 02:01:41.747802    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f6c9eef0b4575d868d7e303f479de8e27b8ff3a6a61ebdf31db3dd37da33f0e6\": container with ID starting with f6c9eef0b4575d868d7e303f479de8e27b8ff3a6a61ebdf31db3dd37da33f0e6 not found: ID does not exist" containerID="f6c9eef0b4575d868d7e303f479de8e27b8ff3a6a61ebdf31db3dd37da33f0e6"
Sep 20 02:01:41.747873 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:41.747831    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f6c9eef0b4575d868d7e303f479de8e27b8ff3a6a61ebdf31db3dd37da33f0e6"} err="failed to get container status \"f6c9eef0b4575d868d7e303f479de8e27b8ff3a6a61ebdf31db3dd37da33f0e6\": rpc error: code = NotFound desc = could not find container \"f6c9eef0b4575d868d7e303f479de8e27b8ff3a6a61ebdf31db3dd37da33f0e6\": container with ID starting with f6c9eef0b4575d868d7e303f479de8e27b8ff3a6a61ebdf31db3dd37da33f0e6 not found: ID does not exist"
Sep 20 02:01:41.748825 ip-10-0-130-159 kubenswrapper[2584]: E0920 02:01:41.748802    2584 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-5f685c496f-fp2r8\" is forbidden: User \"system:node:ip-10-0-130-159.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-159.ec2.internal' and this object" podUID="f05d74f7-a537-4ac8-ae97-52a388d0091e" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-fp2r8"
Sep 20 02:01:42.292186 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:42.292147    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:01:42.392746 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:42.392716    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:01:43.092529 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:43.092497    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f05d74f7-a537-4ac8-ae97-52a388d0091e" path="/var/lib/kubelet/pods/f05d74f7-a537-4ac8-ae97-52a388d0091e/volumes"
Sep 20 02:01:44.883969 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:44.883933    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:01:44.985197 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:44.985169    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:01:48.704254 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:48.704212    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:01:48.802086 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:48.802031    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:01:48.901843 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:48.901808    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:01:48.961405 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:48.961296    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:01:49.882977 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:49.882933    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:01:50.026451 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:50.026420    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:01:50.653306 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:50.653276    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:01:50.706014 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:50.705973    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:01:51.740922 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:51.740889    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-rdvz8"
Sep 20 02:01:51.741315 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:51.741098    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-j8fd5"
Sep 20 02:01:51.741315 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:51.741144    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-krtwd"
Sep 20 02:01:51.799485 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:51.799456    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-j8fd5"]
Sep 20 02:01:51.799704 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:51.799675    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-j8fd5" podUID="35590bf4-b1ea-413a-a884-337626e3f883" containerName="manager" containerID="cri-o://4fb559582e6e6c2b7e9bb2cd657099bde8fcf5c01aff6e226acbbbab60575298" gracePeriod=10
Sep 20 02:01:52.057861 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:52.057836    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-j8fd5"
Sep 20 02:01:52.092214 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:52.092176    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6p5lk\" (UniqueName: \"kubernetes.io/projected/35590bf4-b1ea-413a-a884-337626e3f883-kube-api-access-6p5lk\") pod \"35590bf4-b1ea-413a-a884-337626e3f883\" (UID: \"35590bf4-b1ea-413a-a884-337626e3f883\") "
Sep 20 02:01:52.092415 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:52.092272    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/35590bf4-b1ea-413a-a884-337626e3f883-extensions-socket-volume\") pod \"35590bf4-b1ea-413a-a884-337626e3f883\" (UID: \"35590bf4-b1ea-413a-a884-337626e3f883\") "
Sep 20 02:01:52.092678 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:52.092649    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/35590bf4-b1ea-413a-a884-337626e3f883-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "35590bf4-b1ea-413a-a884-337626e3f883" (UID: "35590bf4-b1ea-413a-a884-337626e3f883"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 20 02:01:52.094271 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:52.094242    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/35590bf4-b1ea-413a-a884-337626e3f883-kube-api-access-6p5lk" (OuterVolumeSpecName: "kube-api-access-6p5lk") pod "35590bf4-b1ea-413a-a884-337626e3f883" (UID: "35590bf4-b1ea-413a-a884-337626e3f883"). InnerVolumeSpecName "kube-api-access-6p5lk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 20 02:01:52.162557 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:52.162490    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-gxh8t"]
Sep 20 02:01:52.162922 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:52.162905    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="35590bf4-b1ea-413a-a884-337626e3f883" containerName="manager"
Sep 20 02:01:52.163011 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:52.162926    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="35590bf4-b1ea-413a-a884-337626e3f883" containerName="manager"
Sep 20 02:01:52.163064 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:52.163016    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="35590bf4-b1ea-413a-a884-337626e3f883" containerName="manager"
Sep 20 02:01:52.167198 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:52.167175    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-gxh8t"
Sep 20 02:01:52.183426 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:52.183399    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-gxh8t"]
Sep 20 02:01:52.193702 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:52.193678    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6p5lk\" (UniqueName: \"kubernetes.io/projected/35590bf4-b1ea-413a-a884-337626e3f883-kube-api-access-6p5lk\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:01:52.193702 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:52.193700    2584 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/35590bf4-b1ea-413a-a884-337626e3f883-extensions-socket-volume\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:01:52.295017 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:52.294923    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/af876e7a-2edc-43c3-a936-0bc828f859b9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-gxh8t\" (UID: \"af876e7a-2edc-43c3-a936-0bc828f859b9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-gxh8t"
Sep 20 02:01:52.295198 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:52.295025    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qs6nj\" (UniqueName: \"kubernetes.io/projected/af876e7a-2edc-43c3-a936-0bc828f859b9-kube-api-access-qs6nj\") pod \"kuadrant-operator-controller-manager-5769f99d9f-gxh8t\" (UID: \"af876e7a-2edc-43c3-a936-0bc828f859b9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-gxh8t"
Sep 20 02:01:52.395766 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:52.395729    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/af876e7a-2edc-43c3-a936-0bc828f859b9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-gxh8t\" (UID: \"af876e7a-2edc-43c3-a936-0bc828f859b9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-gxh8t"
Sep 20 02:01:52.395985 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:52.395796    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qs6nj\" (UniqueName: \"kubernetes.io/projected/af876e7a-2edc-43c3-a936-0bc828f859b9-kube-api-access-qs6nj\") pod \"kuadrant-operator-controller-manager-5769f99d9f-gxh8t\" (UID: \"af876e7a-2edc-43c3-a936-0bc828f859b9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-gxh8t"
Sep 20 02:01:52.396206 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:52.396180    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/af876e7a-2edc-43c3-a936-0bc828f859b9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-gxh8t\" (UID: \"af876e7a-2edc-43c3-a936-0bc828f859b9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-gxh8t"
Sep 20 02:01:52.407638 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:52.407605    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qs6nj\" (UniqueName: \"kubernetes.io/projected/af876e7a-2edc-43c3-a936-0bc828f859b9-kube-api-access-qs6nj\") pod \"kuadrant-operator-controller-manager-5769f99d9f-gxh8t\" (UID: \"af876e7a-2edc-43c3-a936-0bc828f859b9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-gxh8t"
Sep 20 02:01:52.476978 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:52.476935    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-gxh8t"
Sep 20 02:01:52.752923 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:52.752904    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-gxh8t"]
Sep 20 02:01:52.755156 ip-10-0-130-159 kubenswrapper[2584]: W0920 02:01:52.755125    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf876e7a_2edc_43c3_a936_0bc828f859b9.slice/crio-ca4859fe507227054416a74e2ecaf5045381db53b0c57a57c7488e9c5e3041e9 WatchSource:0}: Error finding container ca4859fe507227054416a74e2ecaf5045381db53b0c57a57c7488e9c5e3041e9: Status 404 returned error can't find the container with id ca4859fe507227054416a74e2ecaf5045381db53b0c57a57c7488e9c5e3041e9
Sep 20 02:01:52.797542 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:52.797443    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-gxh8t" event={"ID":"af876e7a-2edc-43c3-a936-0bc828f859b9","Type":"ContainerStarted","Data":"ca4859fe507227054416a74e2ecaf5045381db53b0c57a57c7488e9c5e3041e9"}
Sep 20 02:01:52.799884 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:52.799855    2584 generic.go:356] "Generic (PLEG): container finished" podID="35590bf4-b1ea-413a-a884-337626e3f883" containerID="4fb559582e6e6c2b7e9bb2cd657099bde8fcf5c01aff6e226acbbbab60575298" exitCode=0
Sep 20 02:01:52.799992 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:52.799911    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-j8fd5"
Sep 20 02:01:52.799992 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:52.799975    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-j8fd5" event={"ID":"35590bf4-b1ea-413a-a884-337626e3f883","Type":"ContainerDied","Data":"4fb559582e6e6c2b7e9bb2cd657099bde8fcf5c01aff6e226acbbbab60575298"}
Sep 20 02:01:52.800093 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:52.800026    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-j8fd5" event={"ID":"35590bf4-b1ea-413a-a884-337626e3f883","Type":"ContainerDied","Data":"f044f1d7a65c3b7207355faec5c99b702a071665c72c0593f6a021d3f9f0d208"}
Sep 20 02:01:52.800093 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:52.800060    2584 scope.go:117] "RemoveContainer" containerID="4fb559582e6e6c2b7e9bb2cd657099bde8fcf5c01aff6e226acbbbab60575298"
Sep 20 02:01:52.811926 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:52.811903    2584 scope.go:117] "RemoveContainer" containerID="4fb559582e6e6c2b7e9bb2cd657099bde8fcf5c01aff6e226acbbbab60575298"
Sep 20 02:01:52.812230 ip-10-0-130-159 kubenswrapper[2584]: E0920 02:01:52.812212    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4fb559582e6e6c2b7e9bb2cd657099bde8fcf5c01aff6e226acbbbab60575298\": container with ID starting with 4fb559582e6e6c2b7e9bb2cd657099bde8fcf5c01aff6e226acbbbab60575298 not found: ID does not exist" containerID="4fb559582e6e6c2b7e9bb2cd657099bde8fcf5c01aff6e226acbbbab60575298"
Sep 20 02:01:52.812271 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:52.812239    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4fb559582e6e6c2b7e9bb2cd657099bde8fcf5c01aff6e226acbbbab60575298"} err="failed to get container status \"4fb559582e6e6c2b7e9bb2cd657099bde8fcf5c01aff6e226acbbbab60575298\": rpc error: code = NotFound desc = could not find container \"4fb559582e6e6c2b7e9bb2cd657099bde8fcf5c01aff6e226acbbbab60575298\": container with ID starting with 4fb559582e6e6c2b7e9bb2cd657099bde8fcf5c01aff6e226acbbbab60575298 not found: ID does not exist"
Sep 20 02:01:52.826968 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:52.826940    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-j8fd5"]
Sep 20 02:01:52.832657 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:52.832631    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-j8fd5"]
Sep 20 02:01:53.092629 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:53.092550    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="35590bf4-b1ea-413a-a884-337626e3f883" path="/var/lib/kubelet/pods/35590bf4-b1ea-413a-a884-337626e3f883/volumes"
Sep 20 02:01:53.807327 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:53.807288    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-gxh8t" event={"ID":"af876e7a-2edc-43c3-a936-0bc828f859b9","Type":"ContainerStarted","Data":"ade5cebfb9803ec0ea1070d615995ba2f73e2cc6333818c99ff51daa60905785"}
Sep 20 02:01:53.807832 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:53.807419    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-gxh8t"
Sep 20 02:01:53.826569 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:53.826494    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-gxh8t" podStartSLOduration=1.826478453 podStartE2EDuration="1.826478453s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-20 02:01:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-20 02:01:53.825346062 +0000 UTC m=+570.568025267" watchObservedRunningTime="2026-09-20 02:01:53.826478453 +0000 UTC m=+570.569157659"
Sep 20 02:01:56.737636 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:56.737599    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:01:56.800371 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:01:56.800338    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:02:03.299739 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:02:03.299701    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:02:03.401421 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:02:03.401384    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:02:04.814831 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:02:04.814803    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-gxh8t"
Sep 20 02:02:04.864121 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:02:04.864086    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-rdvz8"]
Sep 20 02:02:04.864423 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:02:04.864390    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-rdvz8" podUID="6d4cf2a2-2da3-4682-b9dc-1b1ed9547a3c" containerName="manager" containerID="cri-o://a6ccb20b7a5ee873042570231896423a2fdc6c8b77c12bddc0c5649a1c301a7a" gracePeriod=10
Sep 20 02:02:05.121354 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:02:05.121328    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-rdvz8"
Sep 20 02:02:05.208453 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:02:05.208416    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6x264\" (UniqueName: \"kubernetes.io/projected/6d4cf2a2-2da3-4682-b9dc-1b1ed9547a3c-kube-api-access-6x264\") pod \"6d4cf2a2-2da3-4682-b9dc-1b1ed9547a3c\" (UID: \"6d4cf2a2-2da3-4682-b9dc-1b1ed9547a3c\") "
Sep 20 02:02:05.208453 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:02:05.208461    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6d4cf2a2-2da3-4682-b9dc-1b1ed9547a3c-extensions-socket-volume\") pod \"6d4cf2a2-2da3-4682-b9dc-1b1ed9547a3c\" (UID: \"6d4cf2a2-2da3-4682-b9dc-1b1ed9547a3c\") "
Sep 20 02:02:05.208953 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:02:05.208927    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d4cf2a2-2da3-4682-b9dc-1b1ed9547a3c-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "6d4cf2a2-2da3-4682-b9dc-1b1ed9547a3c" (UID: "6d4cf2a2-2da3-4682-b9dc-1b1ed9547a3c"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 20 02:02:05.210649 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:02:05.210629    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6d4cf2a2-2da3-4682-b9dc-1b1ed9547a3c-kube-api-access-6x264" (OuterVolumeSpecName: "kube-api-access-6x264") pod "6d4cf2a2-2da3-4682-b9dc-1b1ed9547a3c" (UID: "6d4cf2a2-2da3-4682-b9dc-1b1ed9547a3c"). InnerVolumeSpecName "kube-api-access-6x264". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 20 02:02:05.309403 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:02:05.309364    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6x264\" (UniqueName: \"kubernetes.io/projected/6d4cf2a2-2da3-4682-b9dc-1b1ed9547a3c-kube-api-access-6x264\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:02:05.309403 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:02:05.309397    2584 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6d4cf2a2-2da3-4682-b9dc-1b1ed9547a3c-extensions-socket-volume\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:02:05.872785 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:02:05.872749    2584 generic.go:356] "Generic (PLEG): container finished" podID="6d4cf2a2-2da3-4682-b9dc-1b1ed9547a3c" containerID="a6ccb20b7a5ee873042570231896423a2fdc6c8b77c12bddc0c5649a1c301a7a" exitCode=0
Sep 20 02:02:05.873183 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:02:05.872792    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-rdvz8"
Sep 20 02:02:05.873183 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:02:05.872835    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-rdvz8" event={"ID":"6d4cf2a2-2da3-4682-b9dc-1b1ed9547a3c","Type":"ContainerDied","Data":"a6ccb20b7a5ee873042570231896423a2fdc6c8b77c12bddc0c5649a1c301a7a"}
Sep 20 02:02:05.873183 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:02:05.872882    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-rdvz8" event={"ID":"6d4cf2a2-2da3-4682-b9dc-1b1ed9547a3c","Type":"ContainerDied","Data":"a84c0e3fa92705e89be75269319fd194dfe100f74698e7587dbb34f2a2b947e2"}
Sep 20 02:02:05.873183 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:02:05.872905    2584 scope.go:117] "RemoveContainer" containerID="a6ccb20b7a5ee873042570231896423a2fdc6c8b77c12bddc0c5649a1c301a7a"
Sep 20 02:02:05.881775 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:02:05.881754    2584 scope.go:117] "RemoveContainer" containerID="a6ccb20b7a5ee873042570231896423a2fdc6c8b77c12bddc0c5649a1c301a7a"
Sep 20 02:02:05.882091 ip-10-0-130-159 kubenswrapper[2584]: E0920 02:02:05.882073    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a6ccb20b7a5ee873042570231896423a2fdc6c8b77c12bddc0c5649a1c301a7a\": container with ID starting with a6ccb20b7a5ee873042570231896423a2fdc6c8b77c12bddc0c5649a1c301a7a not found: ID does not exist" containerID="a6ccb20b7a5ee873042570231896423a2fdc6c8b77c12bddc0c5649a1c301a7a"
Sep 20 02:02:05.882164 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:02:05.882102    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a6ccb20b7a5ee873042570231896423a2fdc6c8b77c12bddc0c5649a1c301a7a"} err="failed to get container status \"a6ccb20b7a5ee873042570231896423a2fdc6c8b77c12bddc0c5649a1c301a7a\": rpc error: code = NotFound desc = could not find container \"a6ccb20b7a5ee873042570231896423a2fdc6c8b77c12bddc0c5649a1c301a7a\": container with ID starting with a6ccb20b7a5ee873042570231896423a2fdc6c8b77c12bddc0c5649a1c301a7a not found: ID does not exist"
Sep 20 02:02:05.893874 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:02:05.893842    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-rdvz8"]
Sep 20 02:02:05.900871 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:02:05.900847    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-rdvz8"]
Sep 20 02:02:07.092058 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:02:07.092027    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6d4cf2a2-2da3-4682-b9dc-1b1ed9547a3c" path="/var/lib/kubelet/pods/6d4cf2a2-2da3-4682-b9dc-1b1ed9547a3c/volumes"
Sep 20 02:02:08.174847 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:02:08.174808    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:02:08.409664 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:02:08.409628    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:02:24.614349 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:02:24.614311    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:02:24.615384 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:02:24.615362    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-ghrz7"]
Sep 20 02:02:24.615732 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:02:24.615720    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6d4cf2a2-2da3-4682-b9dc-1b1ed9547a3c" containerName="manager"
Sep 20 02:02:24.615778 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:02:24.615734    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="6d4cf2a2-2da3-4682-b9dc-1b1ed9547a3c" containerName="manager"
Sep 20 02:02:24.615832 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:02:24.615821    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="6d4cf2a2-2da3-4682-b9dc-1b1ed9547a3c" containerName="manager"
Sep 20 02:02:24.618912 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:02:24.618898    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-ghrz7"
Sep 20 02:02:24.621212 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:02:24.621187    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Sep 20 02:02:24.621212 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:02:24.621210    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-dxnrh\""
Sep 20 02:02:24.626233 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:02:24.626185    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-ghrz7"]
Sep 20 02:02:24.706325 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:02:24.706291    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-ghrz7"]
Sep 20 02:02:24.709658 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:02:24.709622    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:02:24.767637 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:02:24.767606    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:02:24.774752 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:02:24.774732    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rbwdt\" (UniqueName: \"kubernetes.io/projected/d822af5d-9b83-4c99-baf7-3527ed2657da-kube-api-access-rbwdt\") pod \"limitador-limitador-8588749977-ghrz7\" (UID: \"d822af5d-9b83-4c99-baf7-3527ed2657da\") " pod="kuadrant-system/limitador-limitador-8588749977-ghrz7"
Sep 20 02:02:24.774835 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:02:24.774768    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/d822af5d-9b83-4c99-baf7-3527ed2657da-config-file\") pod \"limitador-limitador-8588749977-ghrz7\" (UID: \"d822af5d-9b83-4c99-baf7-3527ed2657da\") " pod="kuadrant-system/limitador-limitador-8588749977-ghrz7"
Sep 20 02:02:24.875779 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:02:24.875686    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/d822af5d-9b83-4c99-baf7-3527ed2657da-config-file\") pod \"limitador-limitador-8588749977-ghrz7\" (UID: \"d822af5d-9b83-4c99-baf7-3527ed2657da\") " pod="kuadrant-system/limitador-limitador-8588749977-ghrz7"
Sep 20 02:02:24.875962 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:02:24.875809    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rbwdt\" (UniqueName: \"kubernetes.io/projected/d822af5d-9b83-4c99-baf7-3527ed2657da-kube-api-access-rbwdt\") pod \"limitador-limitador-8588749977-ghrz7\" (UID: \"d822af5d-9b83-4c99-baf7-3527ed2657da\") " pod="kuadrant-system/limitador-limitador-8588749977-ghrz7"
Sep 20 02:02:24.876344 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:02:24.876324    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/d822af5d-9b83-4c99-baf7-3527ed2657da-config-file\") pod \"limitador-limitador-8588749977-ghrz7\" (UID: \"d822af5d-9b83-4c99-baf7-3527ed2657da\") " pod="kuadrant-system/limitador-limitador-8588749977-ghrz7"
Sep 20 02:02:24.884452 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:02:24.884430    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rbwdt\" (UniqueName: \"kubernetes.io/projected/d822af5d-9b83-4c99-baf7-3527ed2657da-kube-api-access-rbwdt\") pod \"limitador-limitador-8588749977-ghrz7\" (UID: \"d822af5d-9b83-4c99-baf7-3527ed2657da\") " pod="kuadrant-system/limitador-limitador-8588749977-ghrz7"
Sep 20 02:02:24.929237 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:02:24.929213    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-ghrz7"
Sep 20 02:02:25.067001 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:02:25.066967    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-ztdt2_ece82f19-68b5-4810-be50-d8678cfdee02/console-operator/1.log"
Sep 20 02:02:25.070937 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:02:25.070910    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-ztdt2_ece82f19-68b5-4810-be50-d8678cfdee02/console-operator/1.log"
Sep 20 02:02:25.202071 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:02:25.202020    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-ghrz7"]
Sep 20 02:02:25.204392 ip-10-0-130-159 kubenswrapper[2584]: W0920 02:02:25.204363    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd822af5d_9b83_4c99_baf7_3527ed2657da.slice/crio-bbd0853320e8a0625953113a08a9625e4ddbad883859900a3c82749dd06c9786 WatchSource:0}: Error finding container bbd0853320e8a0625953113a08a9625e4ddbad883859900a3c82749dd06c9786: Status 404 returned error can't find the container with id bbd0853320e8a0625953113a08a9625e4ddbad883859900a3c82749dd06c9786
Sep 20 02:02:25.448470 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:02:25.448434    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:02:25.549300 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:02:25.549221    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:02:25.622349 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:02:25.622315    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-59c974bd48-9lhb6"]
Sep 20 02:02:25.627466 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:02:25.627191    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-9lhb6"
Sep 20 02:02:25.629153 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:02:25.629133    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-9lhb6"]
Sep 20 02:02:25.629445 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:02:25.629419    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-gnzt4\""
Sep 20 02:02:25.681890 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:02:25.681860    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gbs29\" (UniqueName: \"kubernetes.io/projected/21f9b63f-9a02-4876-a87d-9f132c6d5397-kube-api-access-gbs29\") pod \"authorino-59c974bd48-9lhb6\" (UID: \"21f9b63f-9a02-4876-a87d-9f132c6d5397\") " pod="kuadrant-system/authorino-59c974bd48-9lhb6"
Sep 20 02:02:25.783096 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:02:25.783056    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gbs29\" (UniqueName: \"kubernetes.io/projected/21f9b63f-9a02-4876-a87d-9f132c6d5397-kube-api-access-gbs29\") pod \"authorino-59c974bd48-9lhb6\" (UID: \"21f9b63f-9a02-4876-a87d-9f132c6d5397\") " pod="kuadrant-system/authorino-59c974bd48-9lhb6"
Sep 20 02:02:25.791599 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:02:25.791565    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gbs29\" (UniqueName: \"kubernetes.io/projected/21f9b63f-9a02-4876-a87d-9f132c6d5397-kube-api-access-gbs29\") pod \"authorino-59c974bd48-9lhb6\" (UID: \"21f9b63f-9a02-4876-a87d-9f132c6d5397\") " pod="kuadrant-system/authorino-59c974bd48-9lhb6"
Sep 20 02:02:25.938906 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:02:25.938814    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-9lhb6"
Sep 20 02:02:25.974956 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:02:25.974924    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-ghrz7" event={"ID":"d822af5d-9b83-4c99-baf7-3527ed2657da","Type":"ContainerStarted","Data":"bbd0853320e8a0625953113a08a9625e4ddbad883859900a3c82749dd06c9786"}
Sep 20 02:02:26.217385 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:02:26.217325    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-9lhb6"]
Sep 20 02:02:26.219480 ip-10-0-130-159 kubenswrapper[2584]: W0920 02:02:26.219453    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod21f9b63f_9a02_4876_a87d_9f132c6d5397.slice/crio-3f3c30c80cf21f2dbaf4c981069a1b09245a7eb79c09dfb3e91762f501cd56be WatchSource:0}: Error finding container 3f3c30c80cf21f2dbaf4c981069a1b09245a7eb79c09dfb3e91762f501cd56be: Status 404 returned error can't find the container with id 3f3c30c80cf21f2dbaf4c981069a1b09245a7eb79c09dfb3e91762f501cd56be
Sep 20 02:02:26.981986 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:02:26.981948    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-ghrz7" event={"ID":"d822af5d-9b83-4c99-baf7-3527ed2657da","Type":"ContainerStarted","Data":"1a029df5f499ce23f4f76dce968b3afc5ecd01dceaeae11be731aafa2d0250c7"}
Sep 20 02:02:26.982464 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:02:26.982049    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-8588749977-ghrz7"
Sep 20 02:02:26.983168 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:02:26.983143    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-9lhb6" event={"ID":"21f9b63f-9a02-4876-a87d-9f132c6d5397","Type":"ContainerStarted","Data":"3f3c30c80cf21f2dbaf4c981069a1b09245a7eb79c09dfb3e91762f501cd56be"}
Sep 20 02:02:26.999179 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:02:26.999125    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-8588749977-ghrz7" podStartSLOduration=1.940061254 podStartE2EDuration="2.999109034s" totalImagesPullingTime="1.05904778s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-20 02:02:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-20 02:02:26.997191152 +0000 UTC m=+603.739870359" watchObservedRunningTime="2026-09-20 02:02:26.999109034 +0000 UTC m=+603.741788310"
Sep 20 02:02:30.001113 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:02:30.001067    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-9lhb6" event={"ID":"21f9b63f-9a02-4876-a87d-9f132c6d5397","Type":"ContainerStarted","Data":"efc4e6dfd8a9766f2d14487eaf7254e8ec9430a527145181e95d65e306f55e42"}
Sep 20 02:02:30.015950 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:02:30.015901    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-59c974bd48-9lhb6" podStartSLOduration=1.325820246 podStartE2EDuration="5.015883116s" totalImagesPullingTime="3.69006287s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-20 02:02:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-20 02:02:30.014435211 +0000 UTC m=+606.757114428" watchObservedRunningTime="2026-09-20 02:02:30.015883116 +0000 UTC m=+606.758562321"
Sep 20 02:02:37.989446 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:02:37.989412    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-8588749977-ghrz7"
Sep 20 02:02:40.002306 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:02:40.002271    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-ghrz7"]
Sep 20 02:02:40.002842 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:02:40.002508    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-8588749977-ghrz7" podUID="d822af5d-9b83-4c99-baf7-3527ed2657da" containerName="limitador" containerID="cri-o://1a029df5f499ce23f4f76dce968b3afc5ecd01dceaeae11be731aafa2d0250c7" gracePeriod=30
Sep 20 02:02:40.545439 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:02:40.545414    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-ghrz7"
Sep 20 02:02:40.607896 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:02:40.607815    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rbwdt\" (UniqueName: \"kubernetes.io/projected/d822af5d-9b83-4c99-baf7-3527ed2657da-kube-api-access-rbwdt\") pod \"d822af5d-9b83-4c99-baf7-3527ed2657da\" (UID: \"d822af5d-9b83-4c99-baf7-3527ed2657da\") "
Sep 20 02:02:40.608064 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:02:40.607900    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/d822af5d-9b83-4c99-baf7-3527ed2657da-config-file\") pod \"d822af5d-9b83-4c99-baf7-3527ed2657da\" (UID: \"d822af5d-9b83-4c99-baf7-3527ed2657da\") "
Sep 20 02:02:40.608296 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:02:40.608249    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d822af5d-9b83-4c99-baf7-3527ed2657da-config-file" (OuterVolumeSpecName: "config-file") pod "d822af5d-9b83-4c99-baf7-3527ed2657da" (UID: "d822af5d-9b83-4c99-baf7-3527ed2657da"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 20 02:02:40.609991 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:02:40.609968    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d822af5d-9b83-4c99-baf7-3527ed2657da-kube-api-access-rbwdt" (OuterVolumeSpecName: "kube-api-access-rbwdt") pod "d822af5d-9b83-4c99-baf7-3527ed2657da" (UID: "d822af5d-9b83-4c99-baf7-3527ed2657da"). InnerVolumeSpecName "kube-api-access-rbwdt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 20 02:02:40.708712 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:02:40.708663    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rbwdt\" (UniqueName: \"kubernetes.io/projected/d822af5d-9b83-4c99-baf7-3527ed2657da-kube-api-access-rbwdt\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:02:40.708712 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:02:40.708707    2584 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/d822af5d-9b83-4c99-baf7-3527ed2657da-config-file\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:02:41.054625 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:02:41.054592    2584 generic.go:356] "Generic (PLEG): container finished" podID="d822af5d-9b83-4c99-baf7-3527ed2657da" containerID="1a029df5f499ce23f4f76dce968b3afc5ecd01dceaeae11be731aafa2d0250c7" exitCode=0
Sep 20 02:02:41.054625 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:02:41.054631    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-ghrz7" event={"ID":"d822af5d-9b83-4c99-baf7-3527ed2657da","Type":"ContainerDied","Data":"1a029df5f499ce23f4f76dce968b3afc5ecd01dceaeae11be731aafa2d0250c7"}
Sep 20 02:02:41.055107 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:02:41.054639    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-ghrz7"
Sep 20 02:02:41.055107 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:02:41.054672    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-ghrz7" event={"ID":"d822af5d-9b83-4c99-baf7-3527ed2657da","Type":"ContainerDied","Data":"bbd0853320e8a0625953113a08a9625e4ddbad883859900a3c82749dd06c9786"}
Sep 20 02:02:41.055107 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:02:41.054694    2584 scope.go:117] "RemoveContainer" containerID="1a029df5f499ce23f4f76dce968b3afc5ecd01dceaeae11be731aafa2d0250c7"
Sep 20 02:02:41.064142 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:02:41.064122    2584 scope.go:117] "RemoveContainer" containerID="1a029df5f499ce23f4f76dce968b3afc5ecd01dceaeae11be731aafa2d0250c7"
Sep 20 02:02:41.064438 ip-10-0-130-159 kubenswrapper[2584]: E0920 02:02:41.064414    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1a029df5f499ce23f4f76dce968b3afc5ecd01dceaeae11be731aafa2d0250c7\": container with ID starting with 1a029df5f499ce23f4f76dce968b3afc5ecd01dceaeae11be731aafa2d0250c7 not found: ID does not exist" containerID="1a029df5f499ce23f4f76dce968b3afc5ecd01dceaeae11be731aafa2d0250c7"
Sep 20 02:02:41.064503 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:02:41.064444    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1a029df5f499ce23f4f76dce968b3afc5ecd01dceaeae11be731aafa2d0250c7"} err="failed to get container status \"1a029df5f499ce23f4f76dce968b3afc5ecd01dceaeae11be731aafa2d0250c7\": rpc error: code = NotFound desc = could not find container \"1a029df5f499ce23f4f76dce968b3afc5ecd01dceaeae11be731aafa2d0250c7\": container with ID starting with 1a029df5f499ce23f4f76dce968b3afc5ecd01dceaeae11be731aafa2d0250c7 not found: ID does not exist"
Sep 20 02:02:41.075828 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:02:41.075803    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-ghrz7"]
Sep 20 02:02:41.079208 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:02:41.079187    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-ghrz7"]
Sep 20 02:02:41.092617 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:02:41.092587    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d822af5d-9b83-4c99-baf7-3527ed2657da" path="/var/lib/kubelet/pods/d822af5d-9b83-4c99-baf7-3527ed2657da/volumes"
Sep 20 02:02:42.398340 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:02:42.398308    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:02:42.496253 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:02:42.496220    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:02:42.706461 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:02:42.706432    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:02:42.807352 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:02:42.807324    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:02:42.878320 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:02:42.878284    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-sph88"]
Sep 20 02:02:42.878712 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:02:42.878691    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d822af5d-9b83-4c99-baf7-3527ed2657da" containerName="limitador"
Sep 20 02:02:42.878712 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:02:42.878711    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="d822af5d-9b83-4c99-baf7-3527ed2657da" containerName="limitador"
Sep 20 02:02:42.878868 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:02:42.878788    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="d822af5d-9b83-4c99-baf7-3527ed2657da" containerName="limitador"
Sep 20 02:02:42.883303 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:02:42.883281    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-sph88"
Sep 20 02:02:42.885566 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:02:42.885539    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 20 02:02:42.885697 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:02:42.885620    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Sep 20 02:02:42.885892 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:02:42.885872    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 20 02:02:42.886461 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:02:42.886438    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-zmgtk\""
Sep 20 02:02:42.889313 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:02:42.889289    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-sph88"]
Sep 20 02:02:42.905995 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:02:42.905950    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:02:42.928646 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:02:42.928606    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/c9a02530-1010-4497-a848-8d1373a51d01-data\") pod \"postgres-69fb4594cf-sph88\" (UID: \"c9a02530-1010-4497-a848-8d1373a51d01\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-sph88"
Sep 20 02:02:42.928827 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:02:42.928671    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2n4ng\" (UniqueName: \"kubernetes.io/projected/c9a02530-1010-4497-a848-8d1373a51d01-kube-api-access-2n4ng\") pod \"postgres-69fb4594cf-sph88\" (UID: \"c9a02530-1010-4497-a848-8d1373a51d01\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-sph88"
Sep 20 02:02:43.029860 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:02:43.029754    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2n4ng\" (UniqueName: \"kubernetes.io/projected/c9a02530-1010-4497-a848-8d1373a51d01-kube-api-access-2n4ng\") pod \"postgres-69fb4594cf-sph88\" (UID: \"c9a02530-1010-4497-a848-8d1373a51d01\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-sph88"
Sep 20 02:02:43.029860 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:02:43.029859    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/c9a02530-1010-4497-a848-8d1373a51d01-data\") pod \"postgres-69fb4594cf-sph88\" (UID: \"c9a02530-1010-4497-a848-8d1373a51d01\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-sph88"
Sep 20 02:02:43.030301 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:02:43.030283    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/c9a02530-1010-4497-a848-8d1373a51d01-data\") pod \"postgres-69fb4594cf-sph88\" (UID: \"c9a02530-1010-4497-a848-8d1373a51d01\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-sph88"
Sep 20 02:02:43.037622 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:02:43.037592    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2n4ng\" (UniqueName: \"kubernetes.io/projected/c9a02530-1010-4497-a848-8d1373a51d01-kube-api-access-2n4ng\") pod \"postgres-69fb4594cf-sph88\" (UID: \"c9a02530-1010-4497-a848-8d1373a51d01\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-sph88"
Sep 20 02:02:43.196262 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:02:43.196216    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-sph88"
Sep 20 02:02:43.466585 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:02:43.466461    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-sph88"]
Sep 20 02:02:43.469018 ip-10-0-130-159 kubenswrapper[2584]: W0920 02:02:43.468991    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9a02530_1010_4497_a848_8d1373a51d01.slice/crio-94f6575945438aeb80e3fbfb9a48480505325425900374e3910a8ff0e72098e1 WatchSource:0}: Error finding container 94f6575945438aeb80e3fbfb9a48480505325425900374e3910a8ff0e72098e1: Status 404 returned error can't find the container with id 94f6575945438aeb80e3fbfb9a48480505325425900374e3910a8ff0e72098e1
Sep 20 02:02:44.076256 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:02:44.076221    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-sph88" event={"ID":"c9a02530-1010-4497-a848-8d1373a51d01","Type":"ContainerStarted","Data":"94f6575945438aeb80e3fbfb9a48480505325425900374e3910a8ff0e72098e1"}
Sep 20 02:02:47.184737 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:02:47.184703    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:02:51.115895 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:02:51.115861    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-sph88" event={"ID":"c9a02530-1010-4497-a848-8d1373a51d01","Type":"ContainerStarted","Data":"7935e990b7a97f56dc54b29a15420a3546a88cc54500e8565078cc72749edaca"}
Sep 20 02:02:51.116303 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:02:51.115924    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-sph88"
Sep 20 02:02:51.132149 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:02:51.132095    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-sph88" podStartSLOduration=2.185134631 podStartE2EDuration="9.132078636s" totalImagesPullingTime="6.946944005s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-20 02:02:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-20 02:02:51.130190918 +0000 UTC m=+627.872870126" watchObservedRunningTime="2026-09-20 02:02:51.132078636 +0000 UTC m=+627.874757878"
Sep 20 02:02:57.150282 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:02:57.150252    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-sph88"
Sep 20 02:02:57.471829 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:02:57.471792    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:02:57.622301 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:02:57.622267    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:03:00.152482 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:00.152446    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-9lhb6"]
Sep 20 02:03:00.152906 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:00.152661    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-59c974bd48-9lhb6" podUID="21f9b63f-9a02-4876-a87d-9f132c6d5397" containerName="authorino" containerID="cri-o://efc4e6dfd8a9766f2d14487eaf7254e8ec9430a527145181e95d65e306f55e42" gracePeriod=30
Sep 20 02:03:00.409706 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:00.409644    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-9lhb6"
Sep 20 02:03:00.480140 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:00.480096    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gbs29\" (UniqueName: \"kubernetes.io/projected/21f9b63f-9a02-4876-a87d-9f132c6d5397-kube-api-access-gbs29\") pod \"21f9b63f-9a02-4876-a87d-9f132c6d5397\" (UID: \"21f9b63f-9a02-4876-a87d-9f132c6d5397\") "
Sep 20 02:03:00.482145 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:00.482117    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/21f9b63f-9a02-4876-a87d-9f132c6d5397-kube-api-access-gbs29" (OuterVolumeSpecName: "kube-api-access-gbs29") pod "21f9b63f-9a02-4876-a87d-9f132c6d5397" (UID: "21f9b63f-9a02-4876-a87d-9f132c6d5397"). InnerVolumeSpecName "kube-api-access-gbs29". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 20 02:03:00.580983 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:00.580938    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gbs29\" (UniqueName: \"kubernetes.io/projected/21f9b63f-9a02-4876-a87d-9f132c6d5397-kube-api-access-gbs29\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:03:01.167180 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:01.165974    2584 generic.go:356] "Generic (PLEG): container finished" podID="21f9b63f-9a02-4876-a87d-9f132c6d5397" containerID="efc4e6dfd8a9766f2d14487eaf7254e8ec9430a527145181e95d65e306f55e42" exitCode=0
Sep 20 02:03:01.167180 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:01.166114    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-9lhb6" event={"ID":"21f9b63f-9a02-4876-a87d-9f132c6d5397","Type":"ContainerDied","Data":"efc4e6dfd8a9766f2d14487eaf7254e8ec9430a527145181e95d65e306f55e42"}
Sep 20 02:03:01.167180 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:01.166154    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-9lhb6" event={"ID":"21f9b63f-9a02-4876-a87d-9f132c6d5397","Type":"ContainerDied","Data":"3f3c30c80cf21f2dbaf4c981069a1b09245a7eb79c09dfb3e91762f501cd56be"}
Sep 20 02:03:01.167180 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:01.166193    2584 scope.go:117] "RemoveContainer" containerID="efc4e6dfd8a9766f2d14487eaf7254e8ec9430a527145181e95d65e306f55e42"
Sep 20 02:03:01.167180 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:01.166428    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-9lhb6"
Sep 20 02:03:01.176245 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:01.176227    2584 scope.go:117] "RemoveContainer" containerID="efc4e6dfd8a9766f2d14487eaf7254e8ec9430a527145181e95d65e306f55e42"
Sep 20 02:03:01.176546 ip-10-0-130-159 kubenswrapper[2584]: E0920 02:03:01.176511    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"efc4e6dfd8a9766f2d14487eaf7254e8ec9430a527145181e95d65e306f55e42\": container with ID starting with efc4e6dfd8a9766f2d14487eaf7254e8ec9430a527145181e95d65e306f55e42 not found: ID does not exist" containerID="efc4e6dfd8a9766f2d14487eaf7254e8ec9430a527145181e95d65e306f55e42"
Sep 20 02:03:01.176634 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:01.176558    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"efc4e6dfd8a9766f2d14487eaf7254e8ec9430a527145181e95d65e306f55e42"} err="failed to get container status \"efc4e6dfd8a9766f2d14487eaf7254e8ec9430a527145181e95d65e306f55e42\": rpc error: code = NotFound desc = could not find container \"efc4e6dfd8a9766f2d14487eaf7254e8ec9430a527145181e95d65e306f55e42\": container with ID starting with efc4e6dfd8a9766f2d14487eaf7254e8ec9430a527145181e95d65e306f55e42 not found: ID does not exist"
Sep 20 02:03:01.183136 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:01.183113    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-9lhb6"]
Sep 20 02:03:01.185595 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:01.185573    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-59c974bd48-9lhb6"]
Sep 20 02:03:03.092765 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:03.092732    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="21f9b63f-9a02-4876-a87d-9f132c6d5397" path="/var/lib/kubelet/pods/21f9b63f-9a02-4876-a87d-9f132c6d5397/volumes"
Sep 20 02:03:03.701978 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:03.701950    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:03:05.046639 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:05.046608    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:03:05.081537 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:05.081484    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-78847b7f6-4scd8"]
Sep 20 02:03:05.081874 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:05.081860    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="21f9b63f-9a02-4876-a87d-9f132c6d5397" containerName="authorino"
Sep 20 02:03:05.081929 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:05.081876    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="21f9b63f-9a02-4876-a87d-9f132c6d5397" containerName="authorino"
Sep 20 02:03:05.081972 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:05.081961    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="21f9b63f-9a02-4876-a87d-9f132c6d5397" containerName="authorino"
Sep 20 02:03:05.106307 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:05.106273    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-78847b7f6-4scd8"
Sep 20 02:03:05.108499 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:05.108473    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-mv62k\""
Sep 20 02:03:05.108671 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:05.108542    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 20 02:03:05.109205 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:05.109184    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 20 02:03:05.109404 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:05.109206    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 20 02:03:05.113761 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:05.113737    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-78847b7f6-4scd8"]
Sep 20 02:03:05.126798 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:05.126757    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:03:05.189363 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:05.189330    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:03:05.221462 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:05.221429    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3e99b7aa-98ee-4336-bc6d-5241a1ca03d7-metrics-certs\") pod \"maas-controller-78847b7f6-4scd8\" (UID: \"3e99b7aa-98ee-4336-bc6d-5241a1ca03d7\") " pod="opendatahub/maas-controller-78847b7f6-4scd8"
Sep 20 02:03:05.221609 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:05.221495    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m5znt\" (UniqueName: \"kubernetes.io/projected/3e99b7aa-98ee-4336-bc6d-5241a1ca03d7-kube-api-access-m5znt\") pod \"maas-controller-78847b7f6-4scd8\" (UID: \"3e99b7aa-98ee-4336-bc6d-5241a1ca03d7\") " pod="opendatahub/maas-controller-78847b7f6-4scd8"
Sep 20 02:03:05.221609 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:05.221582    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3e99b7aa-98ee-4336-bc6d-5241a1ca03d7-webhook-cert\") pod \"maas-controller-78847b7f6-4scd8\" (UID: \"3e99b7aa-98ee-4336-bc6d-5241a1ca03d7\") " pod="opendatahub/maas-controller-78847b7f6-4scd8"
Sep 20 02:03:05.266453 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:05.266418    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:03:05.270180 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:05.270155    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6f56f85f68-2bfth"]
Sep 20 02:03:05.273846 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:05.273829    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6f56f85f68-2bfth"
Sep 20 02:03:05.284163 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:05.284139    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6f56f85f68-2bfth"]
Sep 20 02:03:05.322664 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:05.322583    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m5znt\" (UniqueName: \"kubernetes.io/projected/3e99b7aa-98ee-4336-bc6d-5241a1ca03d7-kube-api-access-m5znt\") pod \"maas-controller-78847b7f6-4scd8\" (UID: \"3e99b7aa-98ee-4336-bc6d-5241a1ca03d7\") " pod="opendatahub/maas-controller-78847b7f6-4scd8"
Sep 20 02:03:05.322664 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:05.322644    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3e99b7aa-98ee-4336-bc6d-5241a1ca03d7-webhook-cert\") pod \"maas-controller-78847b7f6-4scd8\" (UID: \"3e99b7aa-98ee-4336-bc6d-5241a1ca03d7\") " pod="opendatahub/maas-controller-78847b7f6-4scd8"
Sep 20 02:03:05.322851 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:05.322682    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3e99b7aa-98ee-4336-bc6d-5241a1ca03d7-metrics-certs\") pod \"maas-controller-78847b7f6-4scd8\" (UID: \"3e99b7aa-98ee-4336-bc6d-5241a1ca03d7\") " pod="opendatahub/maas-controller-78847b7f6-4scd8"
Sep 20 02:03:05.325214 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:05.325179    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3e99b7aa-98ee-4336-bc6d-5241a1ca03d7-webhook-cert\") pod \"maas-controller-78847b7f6-4scd8\" (UID: \"3e99b7aa-98ee-4336-bc6d-5241a1ca03d7\") " pod="opendatahub/maas-controller-78847b7f6-4scd8"
Sep 20 02:03:05.325214 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:05.325207    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3e99b7aa-98ee-4336-bc6d-5241a1ca03d7-metrics-certs\") pod \"maas-controller-78847b7f6-4scd8\" (UID: \"3e99b7aa-98ee-4336-bc6d-5241a1ca03d7\") " pod="opendatahub/maas-controller-78847b7f6-4scd8"
Sep 20 02:03:05.330947 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:05.330923    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m5znt\" (UniqueName: \"kubernetes.io/projected/3e99b7aa-98ee-4336-bc6d-5241a1ca03d7-kube-api-access-m5znt\") pod \"maas-controller-78847b7f6-4scd8\" (UID: \"3e99b7aa-98ee-4336-bc6d-5241a1ca03d7\") " pod="opendatahub/maas-controller-78847b7f6-4scd8"
Sep 20 02:03:05.418542 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:05.418480    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-78847b7f6-4scd8"
Sep 20 02:03:05.423997 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:05.423972    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t477j\" (UniqueName: \"kubernetes.io/projected/ec515e4f-40b2-41bf-8cf8-063323865e36-kube-api-access-t477j\") pod \"maas-controller-6f56f85f68-2bfth\" (UID: \"ec515e4f-40b2-41bf-8cf8-063323865e36\") " pod="opendatahub/maas-controller-6f56f85f68-2bfth"
Sep 20 02:03:05.424074 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:05.424018    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ec515e4f-40b2-41bf-8cf8-063323865e36-metrics-certs\") pod \"maas-controller-6f56f85f68-2bfth\" (UID: \"ec515e4f-40b2-41bf-8cf8-063323865e36\") " pod="opendatahub/maas-controller-6f56f85f68-2bfth"
Sep 20 02:03:05.424074 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:05.424066    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ec515e4f-40b2-41bf-8cf8-063323865e36-webhook-cert\") pod \"maas-controller-6f56f85f68-2bfth\" (UID: \"ec515e4f-40b2-41bf-8cf8-063323865e36\") " pod="opendatahub/maas-controller-6f56f85f68-2bfth"
Sep 20 02:03:05.524949 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:05.524910    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ec515e4f-40b2-41bf-8cf8-063323865e36-metrics-certs\") pod \"maas-controller-6f56f85f68-2bfth\" (UID: \"ec515e4f-40b2-41bf-8cf8-063323865e36\") " pod="opendatahub/maas-controller-6f56f85f68-2bfth"
Sep 20 02:03:05.525117 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:05.524978    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ec515e4f-40b2-41bf-8cf8-063323865e36-webhook-cert\") pod \"maas-controller-6f56f85f68-2bfth\" (UID: \"ec515e4f-40b2-41bf-8cf8-063323865e36\") " pod="opendatahub/maas-controller-6f56f85f68-2bfth"
Sep 20 02:03:05.525117 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:05.525047    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t477j\" (UniqueName: \"kubernetes.io/projected/ec515e4f-40b2-41bf-8cf8-063323865e36-kube-api-access-t477j\") pod \"maas-controller-6f56f85f68-2bfth\" (UID: \"ec515e4f-40b2-41bf-8cf8-063323865e36\") " pod="opendatahub/maas-controller-6f56f85f68-2bfth"
Sep 20 02:03:05.527436 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:05.527411    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ec515e4f-40b2-41bf-8cf8-063323865e36-webhook-cert\") pod \"maas-controller-6f56f85f68-2bfth\" (UID: \"ec515e4f-40b2-41bf-8cf8-063323865e36\") " pod="opendatahub/maas-controller-6f56f85f68-2bfth"
Sep 20 02:03:05.527581 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:05.527445    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ec515e4f-40b2-41bf-8cf8-063323865e36-metrics-certs\") pod \"maas-controller-6f56f85f68-2bfth\" (UID: \"ec515e4f-40b2-41bf-8cf8-063323865e36\") " pod="opendatahub/maas-controller-6f56f85f68-2bfth"
Sep 20 02:03:05.532140 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:05.532115    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t477j\" (UniqueName: \"kubernetes.io/projected/ec515e4f-40b2-41bf-8cf8-063323865e36-kube-api-access-t477j\") pod \"maas-controller-6f56f85f68-2bfth\" (UID: \"ec515e4f-40b2-41bf-8cf8-063323865e36\") " pod="opendatahub/maas-controller-6f56f85f68-2bfth"
Sep 20 02:03:05.584557 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:05.584454    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6f56f85f68-2bfth"
Sep 20 02:03:05.689714 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:05.689685    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-78847b7f6-4scd8"]
Sep 20 02:03:05.691432 ip-10-0-130-159 kubenswrapper[2584]: W0920 02:03:05.691407    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e99b7aa_98ee_4336_bc6d_5241a1ca03d7.slice/crio-252ca52ce573364418f99cfaca038caddba2aa35111c548e42bfd449bdb6e3ad WatchSource:0}: Error finding container 252ca52ce573364418f99cfaca038caddba2aa35111c548e42bfd449bdb6e3ad: Status 404 returned error can't find the container with id 252ca52ce573364418f99cfaca038caddba2aa35111c548e42bfd449bdb6e3ad
Sep 20 02:03:05.858400 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:05.858373    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6f56f85f68-2bfth"]
Sep 20 02:03:05.860148 ip-10-0-130-159 kubenswrapper[2584]: W0920 02:03:05.860124    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec515e4f_40b2_41bf_8cf8_063323865e36.slice/crio-8c7ace170704b25e94d251e85260eb7ab5e8303e4af4f6d92d6d497ee080242e WatchSource:0}: Error finding container 8c7ace170704b25e94d251e85260eb7ab5e8303e4af4f6d92d6d497ee080242e: Status 404 returned error can't find the container with id 8c7ace170704b25e94d251e85260eb7ab5e8303e4af4f6d92d6d497ee080242e
Sep 20 02:03:06.191692 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:06.191665    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6f56f85f68-2bfth" event={"ID":"ec515e4f-40b2-41bf-8cf8-063323865e36","Type":"ContainerStarted","Data":"8c7ace170704b25e94d251e85260eb7ab5e8303e4af4f6d92d6d497ee080242e"}
Sep 20 02:03:06.192748 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:06.192727    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-78847b7f6-4scd8" event={"ID":"3e99b7aa-98ee-4336-bc6d-5241a1ca03d7","Type":"ContainerStarted","Data":"252ca52ce573364418f99cfaca038caddba2aa35111c548e42bfd449bdb6e3ad"}
Sep 20 02:03:08.642503 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:08.642465    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:03:08.747262 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:08.747229    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:03:09.213660 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:09.213620    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6f56f85f68-2bfth" event={"ID":"ec515e4f-40b2-41bf-8cf8-063323865e36","Type":"ContainerStarted","Data":"402a721c4867462656354f6301ac78148ca602e2f7b695993a1f7252503dde42"}
Sep 20 02:03:09.213870 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:09.213839    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6f56f85f68-2bfth"
Sep 20 02:03:09.215309 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:09.215266    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-78847b7f6-4scd8" event={"ID":"3e99b7aa-98ee-4336-bc6d-5241a1ca03d7","Type":"ContainerStarted","Data":"e9065580da6d2a6635d824c8681e009eabc79c75bedb46de6a6659c708980db8"}
Sep 20 02:03:09.215475 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:09.215355    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-78847b7f6-4scd8"
Sep 20 02:03:09.237306 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:09.237246    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6f56f85f68-2bfth" podStartSLOduration=1.6343186109999999 podStartE2EDuration="4.237229603s" totalImagesPullingTime="2.602910992s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-20 02:03:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-20 02:03:09.236274506 +0000 UTC m=+645.978953712" watchObservedRunningTime="2026-09-20 02:03:09.237229603 +0000 UTC m=+645.979908848"
Sep 20 02:03:09.255044 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:09.254981    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-78847b7f6-4scd8" podStartSLOduration=1.493439398 podStartE2EDuration="4.254958435s" totalImagesPullingTime="2.761519037s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-20 02:03:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-20 02:03:09.25320256 +0000 UTC m=+645.995881782" watchObservedRunningTime="2026-09-20 02:03:09.254958435 +0000 UTC m=+645.997637678"
Sep 20 02:03:20.223212 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:20.223178    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-78847b7f6-4scd8"
Sep 20 02:03:20.223643 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:20.223260    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6f56f85f68-2bfth"
Sep 20 02:03:20.289464 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:20.289425    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-78847b7f6-4scd8"]
Sep 20 02:03:20.289704 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:20.289671    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-78847b7f6-4scd8" podUID="3e99b7aa-98ee-4336-bc6d-5241a1ca03d7" containerName="manager" containerID="cri-o://e9065580da6d2a6635d824c8681e009eabc79c75bedb46de6a6659c708980db8" gracePeriod=30
Sep 20 02:03:20.635017 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:20.634992    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-78847b7f6-4scd8"
Sep 20 02:03:20.668817 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:20.668785    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3e99b7aa-98ee-4336-bc6d-5241a1ca03d7-webhook-cert\") pod \"3e99b7aa-98ee-4336-bc6d-5241a1ca03d7\" (UID: \"3e99b7aa-98ee-4336-bc6d-5241a1ca03d7\") "
Sep 20 02:03:20.668989 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:20.668833    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3e99b7aa-98ee-4336-bc6d-5241a1ca03d7-metrics-certs\") pod \"3e99b7aa-98ee-4336-bc6d-5241a1ca03d7\" (UID: \"3e99b7aa-98ee-4336-bc6d-5241a1ca03d7\") "
Sep 20 02:03:20.668989 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:20.668902    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m5znt\" (UniqueName: \"kubernetes.io/projected/3e99b7aa-98ee-4336-bc6d-5241a1ca03d7-kube-api-access-m5znt\") pod \"3e99b7aa-98ee-4336-bc6d-5241a1ca03d7\" (UID: \"3e99b7aa-98ee-4336-bc6d-5241a1ca03d7\") "
Sep 20 02:03:20.671156 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:20.671111    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3e99b7aa-98ee-4336-bc6d-5241a1ca03d7-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "3e99b7aa-98ee-4336-bc6d-5241a1ca03d7" (UID: "3e99b7aa-98ee-4336-bc6d-5241a1ca03d7"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 20 02:03:20.671156 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:20.671135    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3e99b7aa-98ee-4336-bc6d-5241a1ca03d7-kube-api-access-m5znt" (OuterVolumeSpecName: "kube-api-access-m5znt") pod "3e99b7aa-98ee-4336-bc6d-5241a1ca03d7" (UID: "3e99b7aa-98ee-4336-bc6d-5241a1ca03d7"). InnerVolumeSpecName "kube-api-access-m5znt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 20 02:03:20.671314 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:20.671122    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3e99b7aa-98ee-4336-bc6d-5241a1ca03d7-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "3e99b7aa-98ee-4336-bc6d-5241a1ca03d7" (UID: "3e99b7aa-98ee-4336-bc6d-5241a1ca03d7"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 20 02:03:20.769477 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:20.769376    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m5znt\" (UniqueName: \"kubernetes.io/projected/3e99b7aa-98ee-4336-bc6d-5241a1ca03d7-kube-api-access-m5znt\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:03:20.769477 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:20.769413    2584 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3e99b7aa-98ee-4336-bc6d-5241a1ca03d7-webhook-cert\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:03:20.769477 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:20.769425    2584 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3e99b7aa-98ee-4336-bc6d-5241a1ca03d7-metrics-certs\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:03:21.275631 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:21.275589    2584 generic.go:356] "Generic (PLEG): container finished" podID="3e99b7aa-98ee-4336-bc6d-5241a1ca03d7" containerID="e9065580da6d2a6635d824c8681e009eabc79c75bedb46de6a6659c708980db8" exitCode=0
Sep 20 02:03:21.276075 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:21.275649    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-78847b7f6-4scd8"
Sep 20 02:03:21.276075 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:21.275658    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-78847b7f6-4scd8" event={"ID":"3e99b7aa-98ee-4336-bc6d-5241a1ca03d7","Type":"ContainerDied","Data":"e9065580da6d2a6635d824c8681e009eabc79c75bedb46de6a6659c708980db8"}
Sep 20 02:03:21.276075 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:21.275722    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-78847b7f6-4scd8" event={"ID":"3e99b7aa-98ee-4336-bc6d-5241a1ca03d7","Type":"ContainerDied","Data":"252ca52ce573364418f99cfaca038caddba2aa35111c548e42bfd449bdb6e3ad"}
Sep 20 02:03:21.276075 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:21.275744    2584 scope.go:117] "RemoveContainer" containerID="e9065580da6d2a6635d824c8681e009eabc79c75bedb46de6a6659c708980db8"
Sep 20 02:03:21.284476 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:21.284457    2584 scope.go:117] "RemoveContainer" containerID="e9065580da6d2a6635d824c8681e009eabc79c75bedb46de6a6659c708980db8"
Sep 20 02:03:21.284798 ip-10-0-130-159 kubenswrapper[2584]: E0920 02:03:21.284776    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e9065580da6d2a6635d824c8681e009eabc79c75bedb46de6a6659c708980db8\": container with ID starting with e9065580da6d2a6635d824c8681e009eabc79c75bedb46de6a6659c708980db8 not found: ID does not exist" containerID="e9065580da6d2a6635d824c8681e009eabc79c75bedb46de6a6659c708980db8"
Sep 20 02:03:21.284884 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:21.284805    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e9065580da6d2a6635d824c8681e009eabc79c75bedb46de6a6659c708980db8"} err="failed to get container status \"e9065580da6d2a6635d824c8681e009eabc79c75bedb46de6a6659c708980db8\": rpc error: code = NotFound desc = could not find container \"e9065580da6d2a6635d824c8681e009eabc79c75bedb46de6a6659c708980db8\": container with ID starting with e9065580da6d2a6635d824c8681e009eabc79c75bedb46de6a6659c708980db8 not found: ID does not exist"
Sep 20 02:03:21.291287 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:21.291255    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-78847b7f6-4scd8"]
Sep 20 02:03:21.299470 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:21.299444    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-78847b7f6-4scd8"]
Sep 20 02:03:23.092297 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:23.092249    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3e99b7aa-98ee-4336-bc6d-5241a1ca03d7" path="/var/lib/kubelet/pods/3e99b7aa-98ee-4336-bc6d-5241a1ca03d7/volumes"
Sep 20 02:03:40.756394 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:40.756363    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:03:42.102266 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:42.102229    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:03:42.201063 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:42.201026    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:03:42.297009 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:42.296971    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-5747fd5c8c-zx8np"]
Sep 20 02:03:42.297383 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:42.297369    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e99b7aa-98ee-4336-bc6d-5241a1ca03d7" containerName="manager"
Sep 20 02:03:42.297434 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:42.297385    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e99b7aa-98ee-4336-bc6d-5241a1ca03d7" containerName="manager"
Sep 20 02:03:42.297474 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:42.297450    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="3e99b7aa-98ee-4336-bc6d-5241a1ca03d7" containerName="manager"
Sep 20 02:03:42.300093 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:42.300045    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:03:42.300620 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:42.300599    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5747fd5c8c-zx8np"
Sep 20 02:03:42.302837 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:42.302815    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 20 02:03:42.302953 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:42.302847    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 20 02:03:42.303467 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:42.303447    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-z4spz\""
Sep 20 02:03:42.311474 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:42.311442    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5747fd5c8c-zx8np"]
Sep 20 02:03:42.362823 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:42.362727    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b05ebed5-1002-4b61-8fac-d1227f949a39-maas-api-tls\") pod \"maas-api-5747fd5c8c-zx8np\" (UID: \"b05ebed5-1002-4b61-8fac-d1227f949a39\") " pod="odh-ai-gateway-infra/maas-api-5747fd5c8c-zx8np"
Sep 20 02:03:42.362823 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:42.362766    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b05ebed5-1002-4b61-8fac-d1227f949a39-openshift-service-ca\") pod \"maas-api-5747fd5c8c-zx8np\" (UID: \"b05ebed5-1002-4b61-8fac-d1227f949a39\") " pod="odh-ai-gateway-infra/maas-api-5747fd5c8c-zx8np"
Sep 20 02:03:42.362823 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:42.362786    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b05ebed5-1002-4b61-8fac-d1227f949a39-metrics-certs\") pod \"maas-api-5747fd5c8c-zx8np\" (UID: \"b05ebed5-1002-4b61-8fac-d1227f949a39\") " pod="odh-ai-gateway-infra/maas-api-5747fd5c8c-zx8np"
Sep 20 02:03:42.363057 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:42.362903    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v67kq\" (UniqueName: \"kubernetes.io/projected/b05ebed5-1002-4b61-8fac-d1227f949a39-kube-api-access-v67kq\") pod \"maas-api-5747fd5c8c-zx8np\" (UID: \"b05ebed5-1002-4b61-8fac-d1227f949a39\") " pod="odh-ai-gateway-infra/maas-api-5747fd5c8c-zx8np"
Sep 20 02:03:42.369249 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:42.369225    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:03:42.463985 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:42.463937    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b05ebed5-1002-4b61-8fac-d1227f949a39-maas-api-tls\") pod \"maas-api-5747fd5c8c-zx8np\" (UID: \"b05ebed5-1002-4b61-8fac-d1227f949a39\") " pod="odh-ai-gateway-infra/maas-api-5747fd5c8c-zx8np"
Sep 20 02:03:42.463985 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:42.463975    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b05ebed5-1002-4b61-8fac-d1227f949a39-openshift-service-ca\") pod \"maas-api-5747fd5c8c-zx8np\" (UID: \"b05ebed5-1002-4b61-8fac-d1227f949a39\") " pod="odh-ai-gateway-infra/maas-api-5747fd5c8c-zx8np"
Sep 20 02:03:42.464354 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:42.464003    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b05ebed5-1002-4b61-8fac-d1227f949a39-metrics-certs\") pod \"maas-api-5747fd5c8c-zx8np\" (UID: \"b05ebed5-1002-4b61-8fac-d1227f949a39\") " pod="odh-ai-gateway-infra/maas-api-5747fd5c8c-zx8np"
Sep 20 02:03:42.464354 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:42.464066    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v67kq\" (UniqueName: \"kubernetes.io/projected/b05ebed5-1002-4b61-8fac-d1227f949a39-kube-api-access-v67kq\") pod \"maas-api-5747fd5c8c-zx8np\" (UID: \"b05ebed5-1002-4b61-8fac-d1227f949a39\") " pod="odh-ai-gateway-infra/maas-api-5747fd5c8c-zx8np"
Sep 20 02:03:42.464865 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:42.464841    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b05ebed5-1002-4b61-8fac-d1227f949a39-openshift-service-ca\") pod \"maas-api-5747fd5c8c-zx8np\" (UID: \"b05ebed5-1002-4b61-8fac-d1227f949a39\") " pod="odh-ai-gateway-infra/maas-api-5747fd5c8c-zx8np"
Sep 20 02:03:42.467162 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:42.467137    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b05ebed5-1002-4b61-8fac-d1227f949a39-metrics-certs\") pod \"maas-api-5747fd5c8c-zx8np\" (UID: \"b05ebed5-1002-4b61-8fac-d1227f949a39\") " pod="odh-ai-gateway-infra/maas-api-5747fd5c8c-zx8np"
Sep 20 02:03:42.467300 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:42.467279    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b05ebed5-1002-4b61-8fac-d1227f949a39-maas-api-tls\") pod \"maas-api-5747fd5c8c-zx8np\" (UID: \"b05ebed5-1002-4b61-8fac-d1227f949a39\") " pod="odh-ai-gateway-infra/maas-api-5747fd5c8c-zx8np"
Sep 20 02:03:42.469502 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:42.469478    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:03:42.472346 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:42.472324    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v67kq\" (UniqueName: \"kubernetes.io/projected/b05ebed5-1002-4b61-8fac-d1227f949a39-kube-api-access-v67kq\") pod \"maas-api-5747fd5c8c-zx8np\" (UID: \"b05ebed5-1002-4b61-8fac-d1227f949a39\") " pod="odh-ai-gateway-infra/maas-api-5747fd5c8c-zx8np"
Sep 20 02:03:42.611368 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:42.611321    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5747fd5c8c-zx8np"
Sep 20 02:03:42.655371 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:42.655335    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:03:42.770373 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:42.770332    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:03:42.802992 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:42.802954    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-z42zz"]
Sep 20 02:03:42.807534 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:42.807497    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-z42zz"
Sep 20 02:03:42.810068 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:42.810041    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Sep 20 02:03:42.826084 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:42.826052    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-z42zz"]
Sep 20 02:03:42.868285 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:42.868111    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zn2ss\" (UniqueName: \"kubernetes.io/projected/6bb258c3-e7d4-4656-9653-db0c9f68186e-kube-api-access-zn2ss\") pod \"payload-pre-processing-59fbc47bf8-z42zz\" (UID: \"6bb258c3-e7d4-4656-9653-db0c9f68186e\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-z42zz"
Sep 20 02:03:42.868285 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:42.868170    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6bb258c3-e7d4-4656-9653-db0c9f68186e-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-z42zz\" (UID: \"6bb258c3-e7d4-4656-9653-db0c9f68186e\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-z42zz"
Sep 20 02:03:42.900281 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:42.900236    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5747fd5c8c-zx8np"]
Sep 20 02:03:42.903791 ip-10-0-130-159 kubenswrapper[2584]: W0920 02:03:42.903762    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb05ebed5_1002_4b61_8fac_d1227f949a39.slice/crio-172733bb90a6af8bf625ca467780d1c2869964ec24ab2c2426af2c3321845319 WatchSource:0}: Error finding container 172733bb90a6af8bf625ca467780d1c2869964ec24ab2c2426af2c3321845319: Status 404 returned error can't find the container with id 172733bb90a6af8bf625ca467780d1c2869964ec24ab2c2426af2c3321845319
Sep 20 02:03:42.968746 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:42.968706    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zn2ss\" (UniqueName: \"kubernetes.io/projected/6bb258c3-e7d4-4656-9653-db0c9f68186e-kube-api-access-zn2ss\") pod \"payload-pre-processing-59fbc47bf8-z42zz\" (UID: \"6bb258c3-e7d4-4656-9653-db0c9f68186e\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-z42zz"
Sep 20 02:03:42.968942 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:42.968757    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6bb258c3-e7d4-4656-9653-db0c9f68186e-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-z42zz\" (UID: \"6bb258c3-e7d4-4656-9653-db0c9f68186e\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-z42zz"
Sep 20 02:03:42.970003 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:42.969955    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6bb258c3-e7d4-4656-9653-db0c9f68186e-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-z42zz\" (UID: \"6bb258c3-e7d4-4656-9653-db0c9f68186e\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-z42zz"
Sep 20 02:03:42.977429 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:42.977396    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zn2ss\" (UniqueName: \"kubernetes.io/projected/6bb258c3-e7d4-4656-9653-db0c9f68186e-kube-api-access-zn2ss\") pod \"payload-pre-processing-59fbc47bf8-z42zz\" (UID: \"6bb258c3-e7d4-4656-9653-db0c9f68186e\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-z42zz"
Sep 20 02:03:43.119646 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:43.119555    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-z42zz"
Sep 20 02:03:43.383764 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:43.383686    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5747fd5c8c-zx8np" event={"ID":"b05ebed5-1002-4b61-8fac-d1227f949a39","Type":"ContainerStarted","Data":"172733bb90a6af8bf625ca467780d1c2869964ec24ab2c2426af2c3321845319"}
Sep 20 02:03:43.392584 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:43.392555    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-z42zz"]
Sep 20 02:03:43.395030 ip-10-0-130-159 kubenswrapper[2584]: W0920 02:03:43.395007    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6bb258c3_e7d4_4656_9653_db0c9f68186e.slice/crio-b5942e0feae877df842185a1f6a55398570d8517c61bb6f39edd5cc7bd1fc2b5 WatchSource:0}: Error finding container b5942e0feae877df842185a1f6a55398570d8517c61bb6f39edd5cc7bd1fc2b5: Status 404 returned error can't find the container with id b5942e0feae877df842185a1f6a55398570d8517c61bb6f39edd5cc7bd1fc2b5
Sep 20 02:03:44.392017 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:44.391972    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-z42zz" event={"ID":"6bb258c3-e7d4-4656-9653-db0c9f68186e","Type":"ContainerStarted","Data":"b5942e0feae877df842185a1f6a55398570d8517c61bb6f39edd5cc7bd1fc2b5"}
Sep 20 02:03:45.401325 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:45.401283    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5747fd5c8c-zx8np" event={"ID":"b05ebed5-1002-4b61-8fac-d1227f949a39","Type":"ContainerStarted","Data":"cac7921b024a1727da1a9194f2158a14cc4a1fb89a61592f9486cb7879ad5493"}
Sep 20 02:03:45.401827 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:45.401394    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-5747fd5c8c-zx8np"
Sep 20 02:03:45.420421 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:45.420356    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-5747fd5c8c-zx8np" podStartSLOduration=1.5586538509999999 podStartE2EDuration="3.420333107s" totalImagesPullingTime="1.861679256s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-20 02:03:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-20 02:03:45.417753315 +0000 UTC m=+682.160432521" watchObservedRunningTime="2026-09-20 02:03:45.420333107 +0000 UTC m=+682.163012312"
Sep 20 02:03:46.407971 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:46.407877    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-z42zz" event={"ID":"6bb258c3-e7d4-4656-9653-db0c9f68186e","Type":"ContainerStarted","Data":"48bc80ed754a8abc363817a3c97500ab5e601f0acec385252c56d1e27e56cb4c"}
Sep 20 02:03:46.408462 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:46.408090    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-z42zz"
Sep 20 02:03:46.425441 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:46.425383    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-z42zz" podStartSLOduration=1.860220905 podStartE2EDuration="4.425364811s" totalImagesPullingTime="2.565143906s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-20 02:03:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-20 02:03:46.424178532 +0000 UTC m=+683.166857738" watchObservedRunningTime="2026-09-20 02:03:46.425364811 +0000 UTC m=+683.168044016"
Sep 20 02:03:51.414306 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:51.414271    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-5747fd5c8c-zx8np"
Sep 20 02:03:52.485131 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:52.485090    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:03:52.582173 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:52.582132    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:03:57.414885 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:03:57.414849    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-z42zz"
Sep 20 02:04:15.038485 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:04:15.038405    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:04:26.508199 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:04:26.508163    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:04:33.510884 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:04:33.510841    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-sdq5l"]
Sep 20 02:04:33.518457 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:04:33.518428    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-sdq5l"
Sep 20 02:04:33.521549 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:04:33.521513    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 20 02:04:33.521703 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:04:33.521586    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-embedding-simulated-kserve-self-signed-certs\""
Sep 20 02:04:33.522297 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:04:33.522279    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 20 02:04:33.522391 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:04:33.522304    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-ln8ck\""
Sep 20 02:04:33.527302 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:04:33.527276    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-sdq5l"]
Sep 20 02:04:33.577913 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:04:33.577882    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:04:33.605022 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:04:33.604990    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7b10b1cd-1ae2-4ff2-be1d-b9c158059274-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-sdq5l\" (UID: \"7b10b1cd-1ae2-4ff2-be1d-b9c158059274\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-sdq5l"
Sep 20 02:04:33.605181 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:04:33.605038    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7b10b1cd-1ae2-4ff2-be1d-b9c158059274-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-sdq5l\" (UID: \"7b10b1cd-1ae2-4ff2-be1d-b9c158059274\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-sdq5l"
Sep 20 02:04:33.605181 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:04:33.605062    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7b10b1cd-1ae2-4ff2-be1d-b9c158059274-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-sdq5l\" (UID: \"7b10b1cd-1ae2-4ff2-be1d-b9c158059274\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-sdq5l"
Sep 20 02:04:33.605181 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:04:33.605108    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7b10b1cd-1ae2-4ff2-be1d-b9c158059274-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-sdq5l\" (UID: \"7b10b1cd-1ae2-4ff2-be1d-b9c158059274\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-sdq5l"
Sep 20 02:04:33.605181 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:04:33.605141    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8879g\" (UniqueName: \"kubernetes.io/projected/7b10b1cd-1ae2-4ff2-be1d-b9c158059274-kube-api-access-8879g\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-sdq5l\" (UID: \"7b10b1cd-1ae2-4ff2-be1d-b9c158059274\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-sdq5l"
Sep 20 02:04:33.605181 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:04:33.605165    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7b10b1cd-1ae2-4ff2-be1d-b9c158059274-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-sdq5l\" (UID: \"7b10b1cd-1ae2-4ff2-be1d-b9c158059274\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-sdq5l"
Sep 20 02:04:33.706408 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:04:33.706364    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7b10b1cd-1ae2-4ff2-be1d-b9c158059274-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-sdq5l\" (UID: \"7b10b1cd-1ae2-4ff2-be1d-b9c158059274\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-sdq5l"
Sep 20 02:04:33.706408 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:04:33.706409    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7b10b1cd-1ae2-4ff2-be1d-b9c158059274-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-sdq5l\" (UID: \"7b10b1cd-1ae2-4ff2-be1d-b9c158059274\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-sdq5l"
Sep 20 02:04:33.706683 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:04:33.706427    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7b10b1cd-1ae2-4ff2-be1d-b9c158059274-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-sdq5l\" (UID: \"7b10b1cd-1ae2-4ff2-be1d-b9c158059274\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-sdq5l"
Sep 20 02:04:33.706683 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:04:33.706450    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7b10b1cd-1ae2-4ff2-be1d-b9c158059274-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-sdq5l\" (UID: \"7b10b1cd-1ae2-4ff2-be1d-b9c158059274\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-sdq5l"
Sep 20 02:04:33.706683 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:04:33.706479    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8879g\" (UniqueName: \"kubernetes.io/projected/7b10b1cd-1ae2-4ff2-be1d-b9c158059274-kube-api-access-8879g\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-sdq5l\" (UID: \"7b10b1cd-1ae2-4ff2-be1d-b9c158059274\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-sdq5l"
Sep 20 02:04:33.706683 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:04:33.706496    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7b10b1cd-1ae2-4ff2-be1d-b9c158059274-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-sdq5l\" (UID: \"7b10b1cd-1ae2-4ff2-be1d-b9c158059274\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-sdq5l"
Sep 20 02:04:33.706892 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:04:33.706875    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7b10b1cd-1ae2-4ff2-be1d-b9c158059274-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-sdq5l\" (UID: \"7b10b1cd-1ae2-4ff2-be1d-b9c158059274\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-sdq5l"
Sep 20 02:04:33.706941 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:04:33.706903    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7b10b1cd-1ae2-4ff2-be1d-b9c158059274-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-sdq5l\" (UID: \"7b10b1cd-1ae2-4ff2-be1d-b9c158059274\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-sdq5l"
Sep 20 02:04:33.706941 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:04:33.706930    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7b10b1cd-1ae2-4ff2-be1d-b9c158059274-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-sdq5l\" (UID: \"7b10b1cd-1ae2-4ff2-be1d-b9c158059274\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-sdq5l"
Sep 20 02:04:33.708879 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:04:33.708858    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7b10b1cd-1ae2-4ff2-be1d-b9c158059274-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-sdq5l\" (UID: \"7b10b1cd-1ae2-4ff2-be1d-b9c158059274\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-sdq5l"
Sep 20 02:04:33.709154 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:04:33.709135    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7b10b1cd-1ae2-4ff2-be1d-b9c158059274-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-sdq5l\" (UID: \"7b10b1cd-1ae2-4ff2-be1d-b9c158059274\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-sdq5l"
Sep 20 02:04:33.714242 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:04:33.714217    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8879g\" (UniqueName: \"kubernetes.io/projected/7b10b1cd-1ae2-4ff2-be1d-b9c158059274-kube-api-access-8879g\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-sdq5l\" (UID: \"7b10b1cd-1ae2-4ff2-be1d-b9c158059274\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-sdq5l"
Sep 20 02:04:33.830768 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:04:33.830679    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-sdq5l"
Sep 20 02:04:34.104273 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:04:34.104215    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-sdq5l"]
Sep 20 02:04:34.106805 ip-10-0-130-159 kubenswrapper[2584]: W0920 02:04:34.106780    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b10b1cd_1ae2_4ff2_be1d_b9c158059274.slice/crio-23120e634605ebf4f69be4df5dcfa746146379d8b29127104907a5cbb243ef4f WatchSource:0}: Error finding container 23120e634605ebf4f69be4df5dcfa746146379d8b29127104907a5cbb243ef4f: Status 404 returned error can't find the container with id 23120e634605ebf4f69be4df5dcfa746146379d8b29127104907a5cbb243ef4f
Sep 20 02:04:34.108130 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:04:34.108110    2584 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 20 02:04:34.644772 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:04:34.644734    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-sdq5l" event={"ID":"7b10b1cd-1ae2-4ff2-be1d-b9c158059274","Type":"ContainerStarted","Data":"23120e634605ebf4f69be4df5dcfa746146379d8b29127104907a5cbb243ef4f"}
Sep 20 02:04:35.651390 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:04:35.651344    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-sdq5l" event={"ID":"7b10b1cd-1ae2-4ff2-be1d-b9c158059274","Type":"ContainerStarted","Data":"65403b1e3f823a4ab345da7fbdb0cba267721c6b01b3ac3e01c81540e968ac3c"}
Sep 20 02:04:35.669078 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:04:35.669022    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-sdq5l" podStartSLOduration=1.717184777 podStartE2EDuration="2.669004563s" totalImagesPullingTime="951.819786ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-20 02:04:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-20 02:04:35.666994428 +0000 UTC m=+732.409673634" watchObservedRunningTime="2026-09-20 02:04:35.669004563 +0000 UTC m=+732.411683768"
Sep 20 02:04:42.144060 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:04:42.144021    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:04:43.830909 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:04:43.830868    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-sdq5l"
Sep 20 02:04:43.831350 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:04:43.830936    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-sdq5l"
Sep 20 02:04:43.843600 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:04:43.843570    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-sdq5l"
Sep 20 02:04:44.707731 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:04:44.707701    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-sdq5l"
Sep 20 02:04:54.646163 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:04:54.646124    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:04:55.812741 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:04:55.812701    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:04:59.307280 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:04:59.307248    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:05:15.934409 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:05:15.934373    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:06:29.095086 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:06:29.095055    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-84755fcbb8-rxcv5"]
Sep 20 02:06:29.098749 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:06:29.098727    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-84755fcbb8-rxcv5"
Sep 20 02:06:29.101955 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:06:29.101927    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-gnzt4\""
Sep 20 02:06:29.101955 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:06:29.101950    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Sep 20 02:06:29.104710 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:06:29.104685    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-84755fcbb8-rxcv5"]
Sep 20 02:06:29.213900 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:06:29.213862    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q2tqh\" (UniqueName: \"kubernetes.io/projected/807e839b-d2f0-4e52-b9b4-e98e4d2f05b5-kube-api-access-q2tqh\") pod \"authorino-84755fcbb8-rxcv5\" (UID: \"807e839b-d2f0-4e52-b9b4-e98e4d2f05b5\") " pod="kuadrant-system/authorino-84755fcbb8-rxcv5"
Sep 20 02:06:29.213900 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:06:29.213896    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/807e839b-d2f0-4e52-b9b4-e98e4d2f05b5-tls-cert\") pod \"authorino-84755fcbb8-rxcv5\" (UID: \"807e839b-d2f0-4e52-b9b4-e98e4d2f05b5\") " pod="kuadrant-system/authorino-84755fcbb8-rxcv5"
Sep 20 02:06:29.314403 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:06:29.314365    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q2tqh\" (UniqueName: \"kubernetes.io/projected/807e839b-d2f0-4e52-b9b4-e98e4d2f05b5-kube-api-access-q2tqh\") pod \"authorino-84755fcbb8-rxcv5\" (UID: \"807e839b-d2f0-4e52-b9b4-e98e4d2f05b5\") " pod="kuadrant-system/authorino-84755fcbb8-rxcv5"
Sep 20 02:06:29.314403 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:06:29.314407    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/807e839b-d2f0-4e52-b9b4-e98e4d2f05b5-tls-cert\") pod \"authorino-84755fcbb8-rxcv5\" (UID: \"807e839b-d2f0-4e52-b9b4-e98e4d2f05b5\") " pod="kuadrant-system/authorino-84755fcbb8-rxcv5"
Sep 20 02:06:29.317128 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:06:29.317101    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/807e839b-d2f0-4e52-b9b4-e98e4d2f05b5-tls-cert\") pod \"authorino-84755fcbb8-rxcv5\" (UID: \"807e839b-d2f0-4e52-b9b4-e98e4d2f05b5\") " pod="kuadrant-system/authorino-84755fcbb8-rxcv5"
Sep 20 02:06:29.322091 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:06:29.322066    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q2tqh\" (UniqueName: \"kubernetes.io/projected/807e839b-d2f0-4e52-b9b4-e98e4d2f05b5-kube-api-access-q2tqh\") pod \"authorino-84755fcbb8-rxcv5\" (UID: \"807e839b-d2f0-4e52-b9b4-e98e4d2f05b5\") " pod="kuadrant-system/authorino-84755fcbb8-rxcv5"
Sep 20 02:06:29.416342 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:06:29.416239    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-84755fcbb8-rxcv5"
Sep 20 02:06:29.689817 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:06:29.689789    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-84755fcbb8-rxcv5"]
Sep 20 02:06:29.692255 ip-10-0-130-159 kubenswrapper[2584]: W0920 02:06:29.692226    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod807e839b_d2f0_4e52_b9b4_e98e4d2f05b5.slice/crio-5176c5390b039e24a340e7bc82ad1b9679be0fc030b5bfe8498b02e7d5148b2a WatchSource:0}: Error finding container 5176c5390b039e24a340e7bc82ad1b9679be0fc030b5bfe8498b02e7d5148b2a: Status 404 returned error can't find the container with id 5176c5390b039e24a340e7bc82ad1b9679be0fc030b5bfe8498b02e7d5148b2a
Sep 20 02:06:30.219472 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:06:30.219437    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-84755fcbb8-rxcv5" event={"ID":"807e839b-d2f0-4e52-b9b4-e98e4d2f05b5","Type":"ContainerStarted","Data":"5176c5390b039e24a340e7bc82ad1b9679be0fc030b5bfe8498b02e7d5148b2a"}
Sep 20 02:06:31.227468 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:06:31.227429    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-84755fcbb8-rxcv5" event={"ID":"807e839b-d2f0-4e52-b9b4-e98e4d2f05b5","Type":"ContainerStarted","Data":"000122fd5968306ce6271cf5ea96db111c409666e91ab7664d95a78b1100941c"}
Sep 20 02:06:31.244631 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:06:31.244575    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-84755fcbb8-rxcv5" podStartSLOduration=1.015166691 podStartE2EDuration="2.244557513s" totalImagesPullingTime="1.229390822s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-20 02:06:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-20 02:06:31.242348641 +0000 UTC m=+847.985027846" watchObservedRunningTime="2026-09-20 02:06:31.244557513 +0000 UTC m=+847.987236719"
Sep 20 02:06:31.944023 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:06:31.943992    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:06:32.042911 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:06:32.042871    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:06:33.068311 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:06:33.068270    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:06:33.165691 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:06:33.165659    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:06:37.360252 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:06:37.360220    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:06:55.023401 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:06:55.023363    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:06:55.123611 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:06:55.123582    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:06:55.226790 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:06:55.226752    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:06:55.341018 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:06:55.340931    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-04a2fd-openshift-default-74d645d896-c89xp"]
Sep 20 02:06:55.345243 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:06:55.345218    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-04a2fd-openshift-default-74d645d896-c89xp"
Sep 20 02:06:55.347593 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:06:55.347566    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-04a2fd-openshift-default-dockercfg-s7477\""
Sep 20 02:06:55.347746 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:06:55.347635    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 20 02:06:55.358991 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:06:55.358916    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-04a2fd-openshift-default-74d645d896-c89xp"]
Sep 20 02:06:55.384621 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:06:55.384583    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:06:55.446653 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:06:55.446620    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f8979a7e-a149-4c3f-a82c-32c19f551021-workload-socket\") pod \"e2e-inf-a-04a2fd-openshift-default-74d645d896-c89xp\" (UID: \"f8979a7e-a149-4c3f-a82c-32c19f551021\") " pod="openshift-ingress/e2e-inf-a-04a2fd-openshift-default-74d645d896-c89xp"
Sep 20 02:06:55.446834 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:06:55.446670    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f8979a7e-a149-4c3f-a82c-32c19f551021-istio-envoy\") pod \"e2e-inf-a-04a2fd-openshift-default-74d645d896-c89xp\" (UID: \"f8979a7e-a149-4c3f-a82c-32c19f551021\") " pod="openshift-ingress/e2e-inf-a-04a2fd-openshift-default-74d645d896-c89xp"
Sep 20 02:06:55.446834 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:06:55.446697    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f8979a7e-a149-4c3f-a82c-32c19f551021-istiod-ca-cert\") pod \"e2e-inf-a-04a2fd-openshift-default-74d645d896-c89xp\" (UID: \"f8979a7e-a149-4c3f-a82c-32c19f551021\") " pod="openshift-ingress/e2e-inf-a-04a2fd-openshift-default-74d645d896-c89xp"
Sep 20 02:06:55.446834 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:06:55.446726    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f8979a7e-a149-4c3f-a82c-32c19f551021-credential-socket\") pod \"e2e-inf-a-04a2fd-openshift-default-74d645d896-c89xp\" (UID: \"f8979a7e-a149-4c3f-a82c-32c19f551021\") " pod="openshift-ingress/e2e-inf-a-04a2fd-openshift-default-74d645d896-c89xp"
Sep 20 02:06:55.446834 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:06:55.446811    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f8979a7e-a149-4c3f-a82c-32c19f551021-workload-certs\") pod \"e2e-inf-a-04a2fd-openshift-default-74d645d896-c89xp\" (UID: \"f8979a7e-a149-4c3f-a82c-32c19f551021\") " pod="openshift-ingress/e2e-inf-a-04a2fd-openshift-default-74d645d896-c89xp"
Sep 20 02:06:55.447004 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:06:55.446867    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f8979a7e-a149-4c3f-a82c-32c19f551021-istio-token\") pod \"e2e-inf-a-04a2fd-openshift-default-74d645d896-c89xp\" (UID: \"f8979a7e-a149-4c3f-a82c-32c19f551021\") " pod="openshift-ingress/e2e-inf-a-04a2fd-openshift-default-74d645d896-c89xp"
Sep 20 02:06:55.447004 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:06:55.446889    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f8979a7e-a149-4c3f-a82c-32c19f551021-istio-data\") pod \"e2e-inf-a-04a2fd-openshift-default-74d645d896-c89xp\" (UID: \"f8979a7e-a149-4c3f-a82c-32c19f551021\") " pod="openshift-ingress/e2e-inf-a-04a2fd-openshift-default-74d645d896-c89xp"
Sep 20 02:06:55.447004 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:06:55.446919    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5ljq4\" (UniqueName: \"kubernetes.io/projected/f8979a7e-a149-4c3f-a82c-32c19f551021-kube-api-access-5ljq4\") pod \"e2e-inf-a-04a2fd-openshift-default-74d645d896-c89xp\" (UID: \"f8979a7e-a149-4c3f-a82c-32c19f551021\") " pod="openshift-ingress/e2e-inf-a-04a2fd-openshift-default-74d645d896-c89xp"
Sep 20 02:06:55.447111 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:06:55.446996    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f8979a7e-a149-4c3f-a82c-32c19f551021-istio-podinfo\") pod \"e2e-inf-a-04a2fd-openshift-default-74d645d896-c89xp\" (UID: \"f8979a7e-a149-4c3f-a82c-32c19f551021\") " pod="openshift-ingress/e2e-inf-a-04a2fd-openshift-default-74d645d896-c89xp"
Sep 20 02:06:55.482204 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:06:55.482173    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:06:55.547666 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:06:55.547624    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f8979a7e-a149-4c3f-a82c-32c19f551021-istio-token\") pod \"e2e-inf-a-04a2fd-openshift-default-74d645d896-c89xp\" (UID: \"f8979a7e-a149-4c3f-a82c-32c19f551021\") " pod="openshift-ingress/e2e-inf-a-04a2fd-openshift-default-74d645d896-c89xp"
Sep 20 02:06:55.547847 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:06:55.547673    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f8979a7e-a149-4c3f-a82c-32c19f551021-istio-data\") pod \"e2e-inf-a-04a2fd-openshift-default-74d645d896-c89xp\" (UID: \"f8979a7e-a149-4c3f-a82c-32c19f551021\") " pod="openshift-ingress/e2e-inf-a-04a2fd-openshift-default-74d645d896-c89xp"
Sep 20 02:06:55.547847 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:06:55.547707    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5ljq4\" (UniqueName: \"kubernetes.io/projected/f8979a7e-a149-4c3f-a82c-32c19f551021-kube-api-access-5ljq4\") pod \"e2e-inf-a-04a2fd-openshift-default-74d645d896-c89xp\" (UID: \"f8979a7e-a149-4c3f-a82c-32c19f551021\") " pod="openshift-ingress/e2e-inf-a-04a2fd-openshift-default-74d645d896-c89xp"
Sep 20 02:06:55.547847 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:06:55.547743    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f8979a7e-a149-4c3f-a82c-32c19f551021-istio-podinfo\") pod \"e2e-inf-a-04a2fd-openshift-default-74d645d896-c89xp\" (UID: \"f8979a7e-a149-4c3f-a82c-32c19f551021\") " pod="openshift-ingress/e2e-inf-a-04a2fd-openshift-default-74d645d896-c89xp"
Sep 20 02:06:55.547847 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:06:55.547761    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f8979a7e-a149-4c3f-a82c-32c19f551021-workload-socket\") pod \"e2e-inf-a-04a2fd-openshift-default-74d645d896-c89xp\" (UID: \"f8979a7e-a149-4c3f-a82c-32c19f551021\") " pod="openshift-ingress/e2e-inf-a-04a2fd-openshift-default-74d645d896-c89xp"
Sep 20 02:06:55.548050 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:06:55.547874    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f8979a7e-a149-4c3f-a82c-32c19f551021-istio-envoy\") pod \"e2e-inf-a-04a2fd-openshift-default-74d645d896-c89xp\" (UID: \"f8979a7e-a149-4c3f-a82c-32c19f551021\") " pod="openshift-ingress/e2e-inf-a-04a2fd-openshift-default-74d645d896-c89xp"
Sep 20 02:06:55.548050 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:06:55.547897    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f8979a7e-a149-4c3f-a82c-32c19f551021-istiod-ca-cert\") pod \"e2e-inf-a-04a2fd-openshift-default-74d645d896-c89xp\" (UID: \"f8979a7e-a149-4c3f-a82c-32c19f551021\") " pod="openshift-ingress/e2e-inf-a-04a2fd-openshift-default-74d645d896-c89xp"
Sep 20 02:06:55.548050 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:06:55.547935    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f8979a7e-a149-4c3f-a82c-32c19f551021-credential-socket\") pod \"e2e-inf-a-04a2fd-openshift-default-74d645d896-c89xp\" (UID: \"f8979a7e-a149-4c3f-a82c-32c19f551021\") " pod="openshift-ingress/e2e-inf-a-04a2fd-openshift-default-74d645d896-c89xp"
Sep 20 02:06:55.548050 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:06:55.547989    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f8979a7e-a149-4c3f-a82c-32c19f551021-workload-certs\") pod \"e2e-inf-a-04a2fd-openshift-default-74d645d896-c89xp\" (UID: \"f8979a7e-a149-4c3f-a82c-32c19f551021\") " pod="openshift-ingress/e2e-inf-a-04a2fd-openshift-default-74d645d896-c89xp"
Sep 20 02:06:55.548260 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:06:55.548151    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f8979a7e-a149-4c3f-a82c-32c19f551021-istio-data\") pod \"e2e-inf-a-04a2fd-openshift-default-74d645d896-c89xp\" (UID: \"f8979a7e-a149-4c3f-a82c-32c19f551021\") " pod="openshift-ingress/e2e-inf-a-04a2fd-openshift-default-74d645d896-c89xp"
Sep 20 02:06:55.548331 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:06:55.548252    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f8979a7e-a149-4c3f-a82c-32c19f551021-workload-socket\") pod \"e2e-inf-a-04a2fd-openshift-default-74d645d896-c89xp\" (UID: \"f8979a7e-a149-4c3f-a82c-32c19f551021\") " pod="openshift-ingress/e2e-inf-a-04a2fd-openshift-default-74d645d896-c89xp"
Sep 20 02:06:55.548376 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:06:55.548358    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f8979a7e-a149-4c3f-a82c-32c19f551021-credential-socket\") pod \"e2e-inf-a-04a2fd-openshift-default-74d645d896-c89xp\" (UID: \"f8979a7e-a149-4c3f-a82c-32c19f551021\") " pod="openshift-ingress/e2e-inf-a-04a2fd-openshift-default-74d645d896-c89xp"
Sep 20 02:06:55.548422 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:06:55.548373    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f8979a7e-a149-4c3f-a82c-32c19f551021-workload-certs\") pod \"e2e-inf-a-04a2fd-openshift-default-74d645d896-c89xp\" (UID: \"f8979a7e-a149-4c3f-a82c-32c19f551021\") " pod="openshift-ingress/e2e-inf-a-04a2fd-openshift-default-74d645d896-c89xp"
Sep 20 02:06:55.548731 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:06:55.548712    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f8979a7e-a149-4c3f-a82c-32c19f551021-istiod-ca-cert\") pod \"e2e-inf-a-04a2fd-openshift-default-74d645d896-c89xp\" (UID: \"f8979a7e-a149-4c3f-a82c-32c19f551021\") " pod="openshift-ingress/e2e-inf-a-04a2fd-openshift-default-74d645d896-c89xp"
Sep 20 02:06:55.550302 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:06:55.550277    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f8979a7e-a149-4c3f-a82c-32c19f551021-istio-envoy\") pod \"e2e-inf-a-04a2fd-openshift-default-74d645d896-c89xp\" (UID: \"f8979a7e-a149-4c3f-a82c-32c19f551021\") " pod="openshift-ingress/e2e-inf-a-04a2fd-openshift-default-74d645d896-c89xp"
Sep 20 02:06:55.550504 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:06:55.550487    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f8979a7e-a149-4c3f-a82c-32c19f551021-istio-podinfo\") pod \"e2e-inf-a-04a2fd-openshift-default-74d645d896-c89xp\" (UID: \"f8979a7e-a149-4c3f-a82c-32c19f551021\") " pod="openshift-ingress/e2e-inf-a-04a2fd-openshift-default-74d645d896-c89xp"
Sep 20 02:06:55.557569 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:06:55.557546    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f8979a7e-a149-4c3f-a82c-32c19f551021-istio-token\") pod \"e2e-inf-a-04a2fd-openshift-default-74d645d896-c89xp\" (UID: \"f8979a7e-a149-4c3f-a82c-32c19f551021\") " pod="openshift-ingress/e2e-inf-a-04a2fd-openshift-default-74d645d896-c89xp"
Sep 20 02:06:55.557889 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:06:55.557863    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5ljq4\" (UniqueName: \"kubernetes.io/projected/f8979a7e-a149-4c3f-a82c-32c19f551021-kube-api-access-5ljq4\") pod \"e2e-inf-a-04a2fd-openshift-default-74d645d896-c89xp\" (UID: \"f8979a7e-a149-4c3f-a82c-32c19f551021\") " pod="openshift-ingress/e2e-inf-a-04a2fd-openshift-default-74d645d896-c89xp"
Sep 20 02:06:55.583276 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:06:55.583236    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:06:55.660583 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:06:55.660482    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-04a2fd-openshift-default-74d645d896-c89xp"
Sep 20 02:06:55.691054 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:06:55.691024    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:06:55.791792 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:06:55.791757    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:06:55.939011 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:06:55.938987    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-04a2fd-openshift-default-74d645d896-c89xp"]
Sep 20 02:06:55.941274 ip-10-0-130-159 kubenswrapper[2584]: W0920 02:06:55.941248    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8979a7e_a149_4c3f_a82c_32c19f551021.slice/crio-e216e8109032fdf5ca8f8fc06b5b941b6132ad0154f2661c163998bdf433c3a5 WatchSource:0}: Error finding container e216e8109032fdf5ca8f8fc06b5b941b6132ad0154f2661c163998bdf433c3a5: Status 404 returned error can't find the container with id e216e8109032fdf5ca8f8fc06b5b941b6132ad0154f2661c163998bdf433c3a5
Sep 20 02:06:56.081427 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:06:56.081389    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:06:56.358438 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:06:56.358401    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-04a2fd-openshift-default-74d645d896-c89xp" event={"ID":"f8979a7e-a149-4c3f-a82c-32c19f551021","Type":"ContainerStarted","Data":"e216e8109032fdf5ca8f8fc06b5b941b6132ad0154f2661c163998bdf433c3a5"}
Sep 20 02:06:56.519248 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:06:56.519212    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:06:56.620499 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:06:56.620409    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:06:57.062242 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:06:57.062206    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:06:58.937319 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:06:58.937285    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:06:59.043289 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:06:59.043243    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:06:59.138124 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:06:59.138082    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:06:59.467283 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:06:59.467238    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 20 02:06:59.467661 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:06:59.467315    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 20 02:06:59.467661 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:06:59.467351    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 20 02:07:00.336567 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:00.336513    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-b15051-f8f665d5b-694xh"]
Sep 20 02:07:00.340456 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:00.340431    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-b15051-f8f665d5b-694xh"
Sep 20 02:07:00.342607 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:00.342583    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w4-b15051-serving-cert\""
Sep 20 02:07:00.351426 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:00.351393    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-b15051-f8f665d5b-694xh"]
Sep 20 02:07:00.373047 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:00.373011    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:07:00.388097 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:00.388060    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9f64dd1e-7983-4545-b7ab-f8d684791433-maas-api-tls\") pod \"maas-api-e2e-worker-w4-b15051-f8f665d5b-694xh\" (UID: \"9f64dd1e-7983-4545-b7ab-f8d684791433\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-b15051-f8f665d5b-694xh"
Sep 20 02:07:00.388277 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:00.388159    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9f64dd1e-7983-4545-b7ab-f8d684791433-metrics-certs\") pod \"maas-api-e2e-worker-w4-b15051-f8f665d5b-694xh\" (UID: \"9f64dd1e-7983-4545-b7ab-f8d684791433\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-b15051-f8f665d5b-694xh"
Sep 20 02:07:00.388277 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:00.388263    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rfl45\" (UniqueName: \"kubernetes.io/projected/9f64dd1e-7983-4545-b7ab-f8d684791433-kube-api-access-rfl45\") pod \"maas-api-e2e-worker-w4-b15051-f8f665d5b-694xh\" (UID: \"9f64dd1e-7983-4545-b7ab-f8d684791433\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-b15051-f8f665d5b-694xh"
Sep 20 02:07:00.388376 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:00.388308    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9f64dd1e-7983-4545-b7ab-f8d684791433-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-b15051-f8f665d5b-694xh\" (UID: \"9f64dd1e-7983-4545-b7ab-f8d684791433\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-b15051-f8f665d5b-694xh"
Sep 20 02:07:00.391164 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:00.391130    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-04a2fd-openshift-default-74d645d896-c89xp" event={"ID":"f8979a7e-a149-4c3f-a82c-32c19f551021","Type":"ContainerStarted","Data":"fdd39165c51488978744128bbe1b15fa0a23677794ef5e669306b84089024620"}
Sep 20 02:07:00.410661 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:00.410605    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-04a2fd-openshift-default-74d645d896-c89xp" podStartSLOduration=1.8874849249999999 podStartE2EDuration="5.410584187s" totalImagesPullingTime="3.523099262s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-20 02:06:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-20 02:07:00.410185606 +0000 UTC m=+877.152864811" watchObservedRunningTime="2026-09-20 02:07:00.410584187 +0000 UTC m=+877.153263394"
Sep 20 02:07:00.473034 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:00.472998    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:07:00.489433 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:00.489393    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9f64dd1e-7983-4545-b7ab-f8d684791433-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-b15051-f8f665d5b-694xh\" (UID: \"9f64dd1e-7983-4545-b7ab-f8d684791433\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-b15051-f8f665d5b-694xh"
Sep 20 02:07:00.489717 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:00.489448    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9f64dd1e-7983-4545-b7ab-f8d684791433-maas-api-tls\") pod \"maas-api-e2e-worker-w4-b15051-f8f665d5b-694xh\" (UID: \"9f64dd1e-7983-4545-b7ab-f8d684791433\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-b15051-f8f665d5b-694xh"
Sep 20 02:07:00.489717 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:00.489485    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9f64dd1e-7983-4545-b7ab-f8d684791433-metrics-certs\") pod \"maas-api-e2e-worker-w4-b15051-f8f665d5b-694xh\" (UID: \"9f64dd1e-7983-4545-b7ab-f8d684791433\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-b15051-f8f665d5b-694xh"
Sep 20 02:07:00.489717 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:00.489578    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rfl45\" (UniqueName: \"kubernetes.io/projected/9f64dd1e-7983-4545-b7ab-f8d684791433-kube-api-access-rfl45\") pod \"maas-api-e2e-worker-w4-b15051-f8f665d5b-694xh\" (UID: \"9f64dd1e-7983-4545-b7ab-f8d684791433\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-b15051-f8f665d5b-694xh"
Sep 20 02:07:00.489717 ip-10-0-130-159 kubenswrapper[2584]: E0920 02:07:00.489661    2584 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w4-b15051-serving-cert: secret "maas-api-e2e-worker-w4-b15051-serving-cert" not found
Sep 20 02:07:00.489959 ip-10-0-130-159 kubenswrapper[2584]: E0920 02:07:00.489744    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9f64dd1e-7983-4545-b7ab-f8d684791433-maas-api-tls podName:9f64dd1e-7983-4545-b7ab-f8d684791433 nodeName:}" failed. No retries permitted until 2026-09-20 02:07:00.989719958 +0000 UTC m=+877.732399158 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/9f64dd1e-7983-4545-b7ab-f8d684791433-maas-api-tls") pod "maas-api-e2e-worker-w4-b15051-f8f665d5b-694xh" (UID: "9f64dd1e-7983-4545-b7ab-f8d684791433") : secret "maas-api-e2e-worker-w4-b15051-serving-cert" not found
Sep 20 02:07:00.490291 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:00.490264    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9f64dd1e-7983-4545-b7ab-f8d684791433-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-b15051-f8f665d5b-694xh\" (UID: \"9f64dd1e-7983-4545-b7ab-f8d684791433\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-b15051-f8f665d5b-694xh"
Sep 20 02:07:00.492123 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:00.492098    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9f64dd1e-7983-4545-b7ab-f8d684791433-metrics-certs\") pod \"maas-api-e2e-worker-w4-b15051-f8f665d5b-694xh\" (UID: \"9f64dd1e-7983-4545-b7ab-f8d684791433\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-b15051-f8f665d5b-694xh"
Sep 20 02:07:00.497701 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:00.497680    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rfl45\" (UniqueName: \"kubernetes.io/projected/9f64dd1e-7983-4545-b7ab-f8d684791433-kube-api-access-rfl45\") pod \"maas-api-e2e-worker-w4-b15051-f8f665d5b-694xh\" (UID: \"9f64dd1e-7983-4545-b7ab-f8d684791433\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-b15051-f8f665d5b-694xh"
Sep 20 02:07:00.661012 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:00.660919    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-04a2fd-openshift-default-74d645d896-c89xp"
Sep 20 02:07:00.668617 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:00.668585    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-04a2fd-openshift-default-74d645d896-c89xp"
Sep 20 02:07:00.671947 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:00.671921    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:07:00.994037 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:00.993987    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9f64dd1e-7983-4545-b7ab-f8d684791433-maas-api-tls\") pod \"maas-api-e2e-worker-w4-b15051-f8f665d5b-694xh\" (UID: \"9f64dd1e-7983-4545-b7ab-f8d684791433\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-b15051-f8f665d5b-694xh"
Sep 20 02:07:00.996702 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:00.996672    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9f64dd1e-7983-4545-b7ab-f8d684791433-maas-api-tls\") pod \"maas-api-e2e-worker-w4-b15051-f8f665d5b-694xh\" (UID: \"9f64dd1e-7983-4545-b7ab-f8d684791433\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-b15051-f8f665d5b-694xh"
Sep 20 02:07:01.029428 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:01.029381    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:07:01.123809 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:01.123771    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:07:01.131531 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:01.131477    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-b15051-75f77884c4-klpfp"]
Sep 20 02:07:01.135451 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:01.135426    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-b15051-75f77884c4-klpfp"
Sep 20 02:07:01.137554 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:01.137498    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w4-b15051-dockercfg-qdn27\""
Sep 20 02:07:01.137703 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:01.137501    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w4-b15051\""
Sep 20 02:07:01.143187 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:01.143160    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-b15051-75f77884c4-klpfp"]
Sep 20 02:07:01.169377 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:01.169347    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-b15051-554564495-2c89t"]
Sep 20 02:07:01.172862 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:01.172845    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-b15051-554564495-2c89t"
Sep 20 02:07:01.185044 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:01.185015    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-b15051-554564495-2c89t"]
Sep 20 02:07:01.195145 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:01.195105    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/883d6aff-0186-4fdc-84da-6c9a122abe0c-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-b15051-75f77884c4-klpfp\" (UID: \"883d6aff-0186-4fdc-84da-6c9a122abe0c\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-b15051-75f77884c4-klpfp"
Sep 20 02:07:01.195145 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:01.195146    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2dd7d85a-5316-4cc9-ba92-1bea129040c1-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-b15051-554564495-2c89t\" (UID: \"2dd7d85a-5316-4cc9-ba92-1bea129040c1\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-b15051-554564495-2c89t"
Sep 20 02:07:01.195408 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:01.195237    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-84dvl\" (UniqueName: \"kubernetes.io/projected/883d6aff-0186-4fdc-84da-6c9a122abe0c-kube-api-access-84dvl\") pod \"payload-pre-processing-e2e-worker-w4-b15051-75f77884c4-klpfp\" (UID: \"883d6aff-0186-4fdc-84da-6c9a122abe0c\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-b15051-75f77884c4-klpfp"
Sep 20 02:07:01.195408 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:01.195280    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zrbtm\" (UniqueName: \"kubernetes.io/projected/2dd7d85a-5316-4cc9-ba92-1bea129040c1-kube-api-access-zrbtm\") pod \"payload-processing-e2e-worker-w4-b15051-554564495-2c89t\" (UID: \"2dd7d85a-5316-4cc9-ba92-1bea129040c1\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-b15051-554564495-2c89t"
Sep 20 02:07:01.255081 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:01.254991    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-b15051-f8f665d5b-694xh"
Sep 20 02:07:01.296271 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:01.296235    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-84dvl\" (UniqueName: \"kubernetes.io/projected/883d6aff-0186-4fdc-84da-6c9a122abe0c-kube-api-access-84dvl\") pod \"payload-pre-processing-e2e-worker-w4-b15051-75f77884c4-klpfp\" (UID: \"883d6aff-0186-4fdc-84da-6c9a122abe0c\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-b15051-75f77884c4-klpfp"
Sep 20 02:07:01.296487 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:01.296311    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zrbtm\" (UniqueName: \"kubernetes.io/projected/2dd7d85a-5316-4cc9-ba92-1bea129040c1-kube-api-access-zrbtm\") pod \"payload-processing-e2e-worker-w4-b15051-554564495-2c89t\" (UID: \"2dd7d85a-5316-4cc9-ba92-1bea129040c1\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-b15051-554564495-2c89t"
Sep 20 02:07:01.296487 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:01.296373    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/883d6aff-0186-4fdc-84da-6c9a122abe0c-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-b15051-75f77884c4-klpfp\" (UID: \"883d6aff-0186-4fdc-84da-6c9a122abe0c\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-b15051-75f77884c4-klpfp"
Sep 20 02:07:01.296487 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:01.296420    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2dd7d85a-5316-4cc9-ba92-1bea129040c1-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-b15051-554564495-2c89t\" (UID: \"2dd7d85a-5316-4cc9-ba92-1bea129040c1\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-b15051-554564495-2c89t"
Sep 20 02:07:01.297503 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:01.297315    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/883d6aff-0186-4fdc-84da-6c9a122abe0c-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-b15051-75f77884c4-klpfp\" (UID: \"883d6aff-0186-4fdc-84da-6c9a122abe0c\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-b15051-75f77884c4-klpfp"
Sep 20 02:07:01.297503 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:01.297398    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2dd7d85a-5316-4cc9-ba92-1bea129040c1-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-b15051-554564495-2c89t\" (UID: \"2dd7d85a-5316-4cc9-ba92-1bea129040c1\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-b15051-554564495-2c89t"
Sep 20 02:07:01.304789 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:01.304762    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-84dvl\" (UniqueName: \"kubernetes.io/projected/883d6aff-0186-4fdc-84da-6c9a122abe0c-kube-api-access-84dvl\") pod \"payload-pre-processing-e2e-worker-w4-b15051-75f77884c4-klpfp\" (UID: \"883d6aff-0186-4fdc-84da-6c9a122abe0c\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-b15051-75f77884c4-klpfp"
Sep 20 02:07:01.304941 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:01.304909    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zrbtm\" (UniqueName: \"kubernetes.io/projected/2dd7d85a-5316-4cc9-ba92-1bea129040c1-kube-api-access-zrbtm\") pod \"payload-processing-e2e-worker-w4-b15051-554564495-2c89t\" (UID: \"2dd7d85a-5316-4cc9-ba92-1bea129040c1\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-b15051-554564495-2c89t"
Sep 20 02:07:01.395550 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:01.395514    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-04a2fd-openshift-default-74d645d896-c89xp"
Sep 20 02:07:01.396640 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:01.396617    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-04a2fd-openshift-default-74d645d896-c89xp"
Sep 20 02:07:01.448410 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:01.448360    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-b15051-75f77884c4-klpfp"
Sep 20 02:07:01.485114 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:01.485074    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-b15051-554564495-2c89t"
Sep 20 02:07:01.541487 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:01.541457    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-b15051-f8f665d5b-694xh"]
Sep 20 02:07:01.544271 ip-10-0-130-159 kubenswrapper[2584]: W0920 02:07:01.544228    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f64dd1e_7983_4545_b7ab_f8d684791433.slice/crio-32aa72bd7bcfdb3bf9855c5fef8080aa6d96fe488fe81edf07f0c044a8c6d38e WatchSource:0}: Error finding container 32aa72bd7bcfdb3bf9855c5fef8080aa6d96fe488fe81edf07f0c044a8c6d38e: Status 404 returned error can't find the container with id 32aa72bd7bcfdb3bf9855c5fef8080aa6d96fe488fe81edf07f0c044a8c6d38e
Sep 20 02:07:01.718607 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:01.718552    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:07:01.726640 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:01.726616    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-b15051-75f77884c4-klpfp"]
Sep 20 02:07:01.729341 ip-10-0-130-159 kubenswrapper[2584]: W0920 02:07:01.729310    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod883d6aff_0186_4fdc_84da_6c9a122abe0c.slice/crio-020c7abbff65ffae946d28bf7caf25a9c76ca5b0de98092f93a995d9134d27bd WatchSource:0}: Error finding container 020c7abbff65ffae946d28bf7caf25a9c76ca5b0de98092f93a995d9134d27bd: Status 404 returned error can't find the container with id 020c7abbff65ffae946d28bf7caf25a9c76ca5b0de98092f93a995d9134d27bd
Sep 20 02:07:01.785439 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:01.785399    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-b15051-554564495-2c89t"]
Sep 20 02:07:01.787956 ip-10-0-130-159 kubenswrapper[2584]: W0920 02:07:01.787926    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2dd7d85a_5316_4cc9_ba92_1bea129040c1.slice/crio-58beb0cb5630d4c2ed71af7b85fb96987e3958730203c2513065f4eaf034fadc WatchSource:0}: Error finding container 58beb0cb5630d4c2ed71af7b85fb96987e3958730203c2513065f4eaf034fadc: Status 404 returned error can't find the container with id 58beb0cb5630d4c2ed71af7b85fb96987e3958730203c2513065f4eaf034fadc
Sep 20 02:07:02.355560 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:02.355444    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:07:02.403160 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:02.403119    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-b15051-554564495-2c89t" event={"ID":"2dd7d85a-5316-4cc9-ba92-1bea129040c1","Type":"ContainerStarted","Data":"d785b644368da37dbf0e699928d2217898918d2a6b8fdbb1430ffbd870099a61"}
Sep 20 02:07:02.403645 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:02.403183    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-b15051-554564495-2c89t" event={"ID":"2dd7d85a-5316-4cc9-ba92-1bea129040c1","Type":"ContainerStarted","Data":"58beb0cb5630d4c2ed71af7b85fb96987e3958730203c2513065f4eaf034fadc"}
Sep 20 02:07:02.403645 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:02.403332    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w4-b15051-554564495-2c89t"
Sep 20 02:07:02.404898 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:02.404843    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-b15051-f8f665d5b-694xh" event={"ID":"9f64dd1e-7983-4545-b7ab-f8d684791433","Type":"ContainerStarted","Data":"17f45bdd4bc3d3ec0cea272b545396bdb7cacc8213cca9bf7e979aa1fac4eb84"}
Sep 20 02:07:02.405047 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:02.404923    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-b15051-f8f665d5b-694xh" event={"ID":"9f64dd1e-7983-4545-b7ab-f8d684791433","Type":"ContainerStarted","Data":"32aa72bd7bcfdb3bf9855c5fef8080aa6d96fe488fe81edf07f0c044a8c6d38e"}
Sep 20 02:07:02.405047 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:02.404969    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-b15051-f8f665d5b-694xh"
Sep 20 02:07:02.406564 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:02.406471    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-b15051-75f77884c4-klpfp" event={"ID":"883d6aff-0186-4fdc-84da-6c9a122abe0c","Type":"ContainerStarted","Data":"5f14d2c14cfbed6ffcc443bf7d96ea66e1da3dc7c495c710ac47f45cae1f1d05"}
Sep 20 02:07:02.406564 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:02.406509    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-b15051-75f77884c4-klpfp" event={"ID":"883d6aff-0186-4fdc-84da-6c9a122abe0c","Type":"ContainerStarted","Data":"020c7abbff65ffae946d28bf7caf25a9c76ca5b0de98092f93a995d9134d27bd"}
Sep 20 02:07:02.425703 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:02.425629    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w4-b15051-554564495-2c89t" podStartSLOduration=1.425605673 podStartE2EDuration="1.425605673s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-20 02:07:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-20 02:07:02.425569822 +0000 UTC m=+879.168249026" watchObservedRunningTime="2026-09-20 02:07:02.425605673 +0000 UTC m=+879.168284879"
Sep 20 02:07:02.438171 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:02.438140    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:07:02.454119 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:02.454063    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-b15051-f8f665d5b-694xh" podStartSLOduration=1.898908455 podStartE2EDuration="2.454041992s" totalImagesPullingTime="555.133537ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-20 02:07:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-20 02:07:02.451675524 +0000 UTC m=+879.194354729" watchObservedRunningTime="2026-09-20 02:07:02.454041992 +0000 UTC m=+879.196721218"
Sep 20 02:07:02.471103 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:02.471036    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-b15051-75f77884c4-klpfp" podStartSLOduration=1.471010994 podStartE2EDuration="1.471010994s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-20 02:07:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-20 02:07:02.470716763 +0000 UTC m=+879.213395969" watchObservedRunningTime="2026-09-20 02:07:02.471010994 +0000 UTC m=+879.213690200"
Sep 20 02:07:02.515594 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:02.515551    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:07:02.517154 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:02.517130    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-8195ba-9b58d5bd6-5m5b8"]
Sep 20 02:07:02.521439 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:02.521413    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-8195ba-9b58d5bd6-5m5b8"
Sep 20 02:07:02.523645 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:02.523622    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w0-8195ba-serving-cert\""
Sep 20 02:07:02.532898 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:02.532742    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-8195ba-9b58d5bd6-5m5b8"]
Sep 20 02:07:02.607626 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:02.607516    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/634c1084-aa3b-4904-b893-1281899f03c3-maas-api-tls\") pod \"maas-api-e2e-worker-w0-8195ba-9b58d5bd6-5m5b8\" (UID: \"634c1084-aa3b-4904-b893-1281899f03c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-8195ba-9b58d5bd6-5m5b8"
Sep 20 02:07:02.607626 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:02.607589    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/634c1084-aa3b-4904-b893-1281899f03c3-metrics-certs\") pod \"maas-api-e2e-worker-w0-8195ba-9b58d5bd6-5m5b8\" (UID: \"634c1084-aa3b-4904-b893-1281899f03c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-8195ba-9b58d5bd6-5m5b8"
Sep 20 02:07:02.607839 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:02.607659    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/634c1084-aa3b-4904-b893-1281899f03c3-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-8195ba-9b58d5bd6-5m5b8\" (UID: \"634c1084-aa3b-4904-b893-1281899f03c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-8195ba-9b58d5bd6-5m5b8"
Sep 20 02:07:02.607839 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:02.607686    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-km77p\" (UniqueName: \"kubernetes.io/projected/634c1084-aa3b-4904-b893-1281899f03c3-kube-api-access-km77p\") pod \"maas-api-e2e-worker-w0-8195ba-9b58d5bd6-5m5b8\" (UID: \"634c1084-aa3b-4904-b893-1281899f03c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-8195ba-9b58d5bd6-5m5b8"
Sep 20 02:07:02.608107 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:02.608064    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:07:02.708329 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:02.708297    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/634c1084-aa3b-4904-b893-1281899f03c3-maas-api-tls\") pod \"maas-api-e2e-worker-w0-8195ba-9b58d5bd6-5m5b8\" (UID: \"634c1084-aa3b-4904-b893-1281899f03c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-8195ba-9b58d5bd6-5m5b8"
Sep 20 02:07:02.708632 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:02.708358    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/634c1084-aa3b-4904-b893-1281899f03c3-metrics-certs\") pod \"maas-api-e2e-worker-w0-8195ba-9b58d5bd6-5m5b8\" (UID: \"634c1084-aa3b-4904-b893-1281899f03c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-8195ba-9b58d5bd6-5m5b8"
Sep 20 02:07:02.708632 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:02.708409    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/634c1084-aa3b-4904-b893-1281899f03c3-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-8195ba-9b58d5bd6-5m5b8\" (UID: \"634c1084-aa3b-4904-b893-1281899f03c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-8195ba-9b58d5bd6-5m5b8"
Sep 20 02:07:02.708632 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:02.708435    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-km77p\" (UniqueName: \"kubernetes.io/projected/634c1084-aa3b-4904-b893-1281899f03c3-kube-api-access-km77p\") pod \"maas-api-e2e-worker-w0-8195ba-9b58d5bd6-5m5b8\" (UID: \"634c1084-aa3b-4904-b893-1281899f03c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-8195ba-9b58d5bd6-5m5b8"
Sep 20 02:07:02.708632 ip-10-0-130-159 kubenswrapper[2584]: E0920 02:07:02.708449    2584 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w0-8195ba-serving-cert: secret "maas-api-e2e-worker-w0-8195ba-serving-cert" not found
Sep 20 02:07:02.708632 ip-10-0-130-159 kubenswrapper[2584]: E0920 02:07:02.708550    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/634c1084-aa3b-4904-b893-1281899f03c3-maas-api-tls podName:634c1084-aa3b-4904-b893-1281899f03c3 nodeName:}" failed. No retries permitted until 2026-09-20 02:07:03.208494721 +0000 UTC m=+879.951173926 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/634c1084-aa3b-4904-b893-1281899f03c3-maas-api-tls") pod "maas-api-e2e-worker-w0-8195ba-9b58d5bd6-5m5b8" (UID: "634c1084-aa3b-4904-b893-1281899f03c3") : secret "maas-api-e2e-worker-w0-8195ba-serving-cert" not found
Sep 20 02:07:02.708632 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:02.708593    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:07:02.709278 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:02.709255    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/634c1084-aa3b-4904-b893-1281899f03c3-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-8195ba-9b58d5bd6-5m5b8\" (UID: \"634c1084-aa3b-4904-b893-1281899f03c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-8195ba-9b58d5bd6-5m5b8"
Sep 20 02:07:02.711025 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:02.711004    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/634c1084-aa3b-4904-b893-1281899f03c3-metrics-certs\") pod \"maas-api-e2e-worker-w0-8195ba-9b58d5bd6-5m5b8\" (UID: \"634c1084-aa3b-4904-b893-1281899f03c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-8195ba-9b58d5bd6-5m5b8"
Sep 20 02:07:02.718809 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:02.718785    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-km77p\" (UniqueName: \"kubernetes.io/projected/634c1084-aa3b-4904-b893-1281899f03c3-kube-api-access-km77p\") pod \"maas-api-e2e-worker-w0-8195ba-9b58d5bd6-5m5b8\" (UID: \"634c1084-aa3b-4904-b893-1281899f03c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-8195ba-9b58d5bd6-5m5b8"
Sep 20 02:07:02.890029 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:02.889891    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:07:03.214507 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:03.214466    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/634c1084-aa3b-4904-b893-1281899f03c3-maas-api-tls\") pod \"maas-api-e2e-worker-w0-8195ba-9b58d5bd6-5m5b8\" (UID: \"634c1084-aa3b-4904-b893-1281899f03c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-8195ba-9b58d5bd6-5m5b8"
Sep 20 02:07:03.216089 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:03.216040    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:07:03.217327 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:03.217302    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/634c1084-aa3b-4904-b893-1281899f03c3-maas-api-tls\") pod \"maas-api-e2e-worker-w0-8195ba-9b58d5bd6-5m5b8\" (UID: \"634c1084-aa3b-4904-b893-1281899f03c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-8195ba-9b58d5bd6-5m5b8"
Sep 20 02:07:03.317952 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:03.317918    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:07:03.418032 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:03.417996    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-b15051-75f77884c4-klpfp"
Sep 20 02:07:03.437434 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:03.437393    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-8195ba-9b58d5bd6-5m5b8"
Sep 20 02:07:03.723193 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:03.720667    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-8195ba-9b58d5bd6-5m5b8"]
Sep 20 02:07:03.723901 ip-10-0-130-159 kubenswrapper[2584]: W0920 02:07:03.723868    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod634c1084_aa3b_4904_b893_1281899f03c3.slice/crio-678aeb132f526073fda84385cf31f9e1a7c16754fc3a54ce37a904d3a90b9489 WatchSource:0}: Error finding container 678aeb132f526073fda84385cf31f9e1a7c16754fc3a54ce37a904d3a90b9489: Status 404 returned error can't find the container with id 678aeb132f526073fda84385cf31f9e1a7c16754fc3a54ce37a904d3a90b9489
Sep 20 02:07:03.835399 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:03.835313    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:07:04.394653 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:04.394611    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:07:04.425405 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:04.425369    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-8195ba-9b58d5bd6-5m5b8" event={"ID":"634c1084-aa3b-4904-b893-1281899f03c3","Type":"ContainerStarted","Data":"678aeb132f526073fda84385cf31f9e1a7c16754fc3a54ce37a904d3a90b9489"}
Sep 20 02:07:04.476161 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:04.476125    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:07:04.638165 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:04.638130    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:07:04.744178 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:04.744146    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:07:04.956699 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:04.956657    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:07:05.145641 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:05.145560    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:07:05.432815 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:05.432721    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-8195ba-9b58d5bd6-5m5b8" event={"ID":"634c1084-aa3b-4904-b893-1281899f03c3","Type":"ContainerStarted","Data":"3e9dbe0e237e1fc9ba94d91d040ad22b99a55cfd90fa6f3aa9d0dc304dff47a5"}
Sep 20 02:07:05.433233 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:05.432853    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-8195ba-9b58d5bd6-5m5b8"
Sep 20 02:07:05.448810 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:05.448759    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-8195ba-9b58d5bd6-5m5b8" podStartSLOduration=2.650702624 podStartE2EDuration="3.448740778s" totalImagesPullingTime="798.038154ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-20 02:07:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-20 02:07:05.4471598 +0000 UTC m=+882.189839004" watchObservedRunningTime="2026-09-20 02:07:05.448740778 +0000 UTC m=+882.191420016"
Sep 20 02:07:05.519458 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:05.519416    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:07:06.237606 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:06.237570    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-04a2fd-7c57fb68bb-lrlnw"]
Sep 20 02:07:06.241432 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:06.241401    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-04a2fd-7c57fb68bb-lrlnw"
Sep 20 02:07:06.243810 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:06.243781    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-04a2fd-serving-cert\""
Sep 20 02:07:06.251615 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:06.251585    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-04a2fd-7c57fb68bb-lrlnw"]
Sep 20 02:07:06.273281 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:06.273249    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:07:06.342966 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:06.342934    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q8nxp\" (UniqueName: \"kubernetes.io/projected/9108b468-ac9c-42e9-83e1-178dcc575f01-kube-api-access-q8nxp\") pod \"maas-api-e2e-inf-a-04a2fd-7c57fb68bb-lrlnw\" (UID: \"9108b468-ac9c-42e9-83e1-178dcc575f01\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-04a2fd-7c57fb68bb-lrlnw"
Sep 20 02:07:06.343165 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:06.342998    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9108b468-ac9c-42e9-83e1-178dcc575f01-maas-api-tls\") pod \"maas-api-e2e-inf-a-04a2fd-7c57fb68bb-lrlnw\" (UID: \"9108b468-ac9c-42e9-83e1-178dcc575f01\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-04a2fd-7c57fb68bb-lrlnw"
Sep 20 02:07:06.343165 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:06.343027    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9108b468-ac9c-42e9-83e1-178dcc575f01-metrics-certs\") pod \"maas-api-e2e-inf-a-04a2fd-7c57fb68bb-lrlnw\" (UID: \"9108b468-ac9c-42e9-83e1-178dcc575f01\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-04a2fd-7c57fb68bb-lrlnw"
Sep 20 02:07:06.343165 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:06.343073    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9108b468-ac9c-42e9-83e1-178dcc575f01-openshift-service-ca\") pod \"maas-api-e2e-inf-a-04a2fd-7c57fb68bb-lrlnw\" (UID: \"9108b468-ac9c-42e9-83e1-178dcc575f01\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-04a2fd-7c57fb68bb-lrlnw"
Sep 20 02:07:06.374143 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:06.374109    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:07:06.444387 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:06.444358    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9108b468-ac9c-42e9-83e1-178dcc575f01-openshift-service-ca\") pod \"maas-api-e2e-inf-a-04a2fd-7c57fb68bb-lrlnw\" (UID: \"9108b468-ac9c-42e9-83e1-178dcc575f01\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-04a2fd-7c57fb68bb-lrlnw"
Sep 20 02:07:06.444791 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:06.444415    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q8nxp\" (UniqueName: \"kubernetes.io/projected/9108b468-ac9c-42e9-83e1-178dcc575f01-kube-api-access-q8nxp\") pod \"maas-api-e2e-inf-a-04a2fd-7c57fb68bb-lrlnw\" (UID: \"9108b468-ac9c-42e9-83e1-178dcc575f01\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-04a2fd-7c57fb68bb-lrlnw"
Sep 20 02:07:06.444791 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:06.444476    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9108b468-ac9c-42e9-83e1-178dcc575f01-maas-api-tls\") pod \"maas-api-e2e-inf-a-04a2fd-7c57fb68bb-lrlnw\" (UID: \"9108b468-ac9c-42e9-83e1-178dcc575f01\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-04a2fd-7c57fb68bb-lrlnw"
Sep 20 02:07:06.444791 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:06.444494    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9108b468-ac9c-42e9-83e1-178dcc575f01-metrics-certs\") pod \"maas-api-e2e-inf-a-04a2fd-7c57fb68bb-lrlnw\" (UID: \"9108b468-ac9c-42e9-83e1-178dcc575f01\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-04a2fd-7c57fb68bb-lrlnw"
Sep 20 02:07:06.445038 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:06.445011    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9108b468-ac9c-42e9-83e1-178dcc575f01-openshift-service-ca\") pod \"maas-api-e2e-inf-a-04a2fd-7c57fb68bb-lrlnw\" (UID: \"9108b468-ac9c-42e9-83e1-178dcc575f01\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-04a2fd-7c57fb68bb-lrlnw"
Sep 20 02:07:06.447052 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:06.447031    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9108b468-ac9c-42e9-83e1-178dcc575f01-metrics-certs\") pod \"maas-api-e2e-inf-a-04a2fd-7c57fb68bb-lrlnw\" (UID: \"9108b468-ac9c-42e9-83e1-178dcc575f01\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-04a2fd-7c57fb68bb-lrlnw"
Sep 20 02:07:06.447168 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:06.447051    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9108b468-ac9c-42e9-83e1-178dcc575f01-maas-api-tls\") pod \"maas-api-e2e-inf-a-04a2fd-7c57fb68bb-lrlnw\" (UID: \"9108b468-ac9c-42e9-83e1-178dcc575f01\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-04a2fd-7c57fb68bb-lrlnw"
Sep 20 02:07:06.452838 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:06.452814    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q8nxp\" (UniqueName: \"kubernetes.io/projected/9108b468-ac9c-42e9-83e1-178dcc575f01-kube-api-access-q8nxp\") pod \"maas-api-e2e-inf-a-04a2fd-7c57fb68bb-lrlnw\" (UID: \"9108b468-ac9c-42e9-83e1-178dcc575f01\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-04a2fd-7c57fb68bb-lrlnw"
Sep 20 02:07:06.482111 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:06.482080    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:07:06.556986 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:06.556878    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-04a2fd-7c57fb68bb-lrlnw"
Sep 20 02:07:06.753378 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:06.753344    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:07:06.833892 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:06.833864    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-04a2fd-7c57fb68bb-lrlnw"]
Sep 20 02:07:06.835477 ip-10-0-130-159 kubenswrapper[2584]: W0920 02:07:06.835449    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9108b468_ac9c_42e9_83e1_178dcc575f01.slice/crio-7466f46e58e78bd23c02176fc4f8fa914d3f60fdd8610bcf02259fd73e303092 WatchSource:0}: Error finding container 7466f46e58e78bd23c02176fc4f8fa914d3f60fdd8610bcf02259fd73e303092: Status 404 returned error can't find the container with id 7466f46e58e78bd23c02176fc4f8fa914d3f60fdd8610bcf02259fd73e303092
Sep 20 02:07:06.838358 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:06.838335    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:07:07.444884 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:07.444846    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-04a2fd-7c57fb68bb-lrlnw" event={"ID":"9108b468-ac9c-42e9-83e1-178dcc575f01","Type":"ContainerStarted","Data":"7466f46e58e78bd23c02176fc4f8fa914d3f60fdd8610bcf02259fd73e303092"}
Sep 20 02:07:07.726723 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:07.726633    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:07:08.062776 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:08.062641    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:07:08.303240 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:08.303149    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:07:08.423953 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:08.423878    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-b15051-f8f665d5b-694xh"
Sep 20 02:07:08.452757 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:08.452714    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-04a2fd-7c57fb68bb-lrlnw" event={"ID":"9108b468-ac9c-42e9-83e1-178dcc575f01","Type":"ContainerStarted","Data":"13f883c699ec73eb6a1be56fcb4986fd71f89a075dd1aad8ce3c2785f46ad9ea"}
Sep 20 02:07:08.453197 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:08.453028    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-04a2fd-7c57fb68bb-lrlnw"
Sep 20 02:07:08.472813 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:08.472762    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-04a2fd-7c57fb68bb-lrlnw" podStartSLOduration=1.808801941 podStartE2EDuration="2.472745466s" totalImagesPullingTime="663.943525ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-20 02:07:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-20 02:07:08.470301168 +0000 UTC m=+885.212980374" watchObservedRunningTime="2026-09-20 02:07:08.472745466 +0000 UTC m=+885.215424671"
Sep 20 02:07:11.246846 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:11.246801    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:07:11.443643 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:11.443618    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-8195ba-9b58d5bd6-5m5b8"
Sep 20 02:07:11.937413 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:11.937376    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:07:12.044497 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:12.044461    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:07:12.550431 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:12.550399    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:07:12.650749 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:12.650717    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:07:12.750943 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:12.750908    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:07:13.419683 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:13.419652    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w4-b15051-554564495-2c89t"
Sep 20 02:07:13.526456 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:13.526422    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:07:14.428473 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:14.428437    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-b15051-75f77884c4-klpfp"
Sep 20 02:07:14.469275 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:14.469245    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-04a2fd-7c57fb68bb-lrlnw"
Sep 20 02:07:14.651059 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:14.651024    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:07:15.438542 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:15.438498    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:07:15.489740 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:15.489690    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:07:15.767329 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:15.767275    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:07:15.950276 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:15.950246    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:07:16.049841 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:16.049750    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:07:16.087779 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:16.087742    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-4cfa5a-7c8c4fc9d6-dzsc7"]
Sep 20 02:07:16.092077 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:16.092052    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-4cfa5a-7c8c4fc9d6-dzsc7"
Sep 20 02:07:16.094387 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:16.094360    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-autores-4cfa5a\""
Sep 20 02:07:16.099013 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:16.098981    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-4cfa5a-7c8c4fc9d6-dzsc7"]
Sep 20 02:07:16.139104 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:16.139072    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nwmrq\" (UniqueName: \"kubernetes.io/projected/ea286772-f46d-49c3-869c-1161c5cab4f4-kube-api-access-nwmrq\") pod \"payload-pre-processing-e2e-autores-4cfa5a-7c8c4fc9d6-dzsc7\" (UID: \"ea286772-f46d-49c3-869c-1161c5cab4f4\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-4cfa5a-7c8c4fc9d6-dzsc7"
Sep 20 02:07:16.139314 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:16.139112    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ea286772-f46d-49c3-869c-1161c5cab4f4-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-4cfa5a-7c8c4fc9d6-dzsc7\" (UID: \"ea286772-f46d-49c3-869c-1161c5cab4f4\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-4cfa5a-7c8c4fc9d6-dzsc7"
Sep 20 02:07:16.213322 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:16.213288    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-4cfa5a-75ddbf85cc-s7tkt"]
Sep 20 02:07:16.217573 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:16.217481    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-4cfa5a-75ddbf85cc-s7tkt"
Sep 20 02:07:16.219734 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:16.219711    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-autores-4cfa5a-dockercfg-msd9k\""
Sep 20 02:07:16.227282 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:16.227257    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-4cfa5a-75ddbf85cc-s7tkt"]
Sep 20 02:07:16.240109 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:16.240070    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nwmrq\" (UniqueName: \"kubernetes.io/projected/ea286772-f46d-49c3-869c-1161c5cab4f4-kube-api-access-nwmrq\") pod \"payload-pre-processing-e2e-autores-4cfa5a-7c8c4fc9d6-dzsc7\" (UID: \"ea286772-f46d-49c3-869c-1161c5cab4f4\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-4cfa5a-7c8c4fc9d6-dzsc7"
Sep 20 02:07:16.240109 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:16.240124    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ea286772-f46d-49c3-869c-1161c5cab4f4-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-4cfa5a-7c8c4fc9d6-dzsc7\" (UID: \"ea286772-f46d-49c3-869c-1161c5cab4f4\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-4cfa5a-7c8c4fc9d6-dzsc7"
Sep 20 02:07:16.241078 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:16.241042    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ea286772-f46d-49c3-869c-1161c5cab4f4-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-4cfa5a-7c8c4fc9d6-dzsc7\" (UID: \"ea286772-f46d-49c3-869c-1161c5cab4f4\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-4cfa5a-7c8c4fc9d6-dzsc7"
Sep 20 02:07:16.252913 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:16.252878    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nwmrq\" (UniqueName: \"kubernetes.io/projected/ea286772-f46d-49c3-869c-1161c5cab4f4-kube-api-access-nwmrq\") pod \"payload-pre-processing-e2e-autores-4cfa5a-7c8c4fc9d6-dzsc7\" (UID: \"ea286772-f46d-49c3-869c-1161c5cab4f4\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-4cfa5a-7c8c4fc9d6-dzsc7"
Sep 20 02:07:16.266865 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:16.266724    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:07:16.341320 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:16.341230    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/00534def-114d-4913-9a34-e65cda8d616c-plugins-config-volume\") pod \"payload-processing-e2e-autores-4cfa5a-75ddbf85cc-s7tkt\" (UID: \"00534def-114d-4913-9a34-e65cda8d616c\") " pod="openshift-ingress/payload-processing-e2e-autores-4cfa5a-75ddbf85cc-s7tkt"
Sep 20 02:07:16.341466 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:16.341375    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-564w2\" (UniqueName: \"kubernetes.io/projected/00534def-114d-4913-9a34-e65cda8d616c-kube-api-access-564w2\") pod \"payload-processing-e2e-autores-4cfa5a-75ddbf85cc-s7tkt\" (UID: \"00534def-114d-4913-9a34-e65cda8d616c\") " pod="openshift-ingress/payload-processing-e2e-autores-4cfa5a-75ddbf85cc-s7tkt"
Sep 20 02:07:16.405153 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:16.405114    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-4cfa5a-7c8c4fc9d6-dzsc7"
Sep 20 02:07:16.425429 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:16.425047    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:07:16.443048 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:16.442865    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-564w2\" (UniqueName: \"kubernetes.io/projected/00534def-114d-4913-9a34-e65cda8d616c-kube-api-access-564w2\") pod \"payload-processing-e2e-autores-4cfa5a-75ddbf85cc-s7tkt\" (UID: \"00534def-114d-4913-9a34-e65cda8d616c\") " pod="openshift-ingress/payload-processing-e2e-autores-4cfa5a-75ddbf85cc-s7tkt"
Sep 20 02:07:16.443048 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:16.442938    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/00534def-114d-4913-9a34-e65cda8d616c-plugins-config-volume\") pod \"payload-processing-e2e-autores-4cfa5a-75ddbf85cc-s7tkt\" (UID: \"00534def-114d-4913-9a34-e65cda8d616c\") " pod="openshift-ingress/payload-processing-e2e-autores-4cfa5a-75ddbf85cc-s7tkt"
Sep 20 02:07:16.443808 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:16.443787    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/00534def-114d-4913-9a34-e65cda8d616c-plugins-config-volume\") pod \"payload-processing-e2e-autores-4cfa5a-75ddbf85cc-s7tkt\" (UID: \"00534def-114d-4913-9a34-e65cda8d616c\") " pod="openshift-ingress/payload-processing-e2e-autores-4cfa5a-75ddbf85cc-s7tkt"
Sep 20 02:07:16.455085 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:16.453382    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-564w2\" (UniqueName: \"kubernetes.io/projected/00534def-114d-4913-9a34-e65cda8d616c-kube-api-access-564w2\") pod \"payload-processing-e2e-autores-4cfa5a-75ddbf85cc-s7tkt\" (UID: \"00534def-114d-4913-9a34-e65cda8d616c\") " pod="openshift-ingress/payload-processing-e2e-autores-4cfa5a-75ddbf85cc-s7tkt"
Sep 20 02:07:16.512488 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:16.512448    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:07:16.531587 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:16.531454    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-4cfa5a-75ddbf85cc-s7tkt"
Sep 20 02:07:16.610006 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:16.609973    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:07:16.698390 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:16.698362    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-4cfa5a-7c8c4fc9d6-dzsc7"]
Sep 20 02:07:16.699780 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:16.699755    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:07:16.700925 ip-10-0-130-159 kubenswrapper[2584]: W0920 02:07:16.700898    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea286772_f46d_49c3_869c_1161c5cab4f4.slice/crio-940331e74b742247462124f010d0d153862396525a30f72b88c4677a652d0d57 WatchSource:0}: Error finding container 940331e74b742247462124f010d0d153862396525a30f72b88c4677a652d0d57: Status 404 returned error can't find the container with id 940331e74b742247462124f010d0d153862396525a30f72b88c4677a652d0d57
Sep 20 02:07:16.815587 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:16.815560    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-4cfa5a-75ddbf85cc-s7tkt"]
Sep 20 02:07:16.817342 ip-10-0-130-159 kubenswrapper[2584]: W0920 02:07:16.817307    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00534def_114d_4913_9a34_e65cda8d616c.slice/crio-ee2b5b7e0738fc626d03b31b36f194eeb43e3b90dd49e06feaec637ac14426ff WatchSource:0}: Error finding container ee2b5b7e0738fc626d03b31b36f194eeb43e3b90dd49e06feaec637ac14426ff: Status 404 returned error can't find the container with id ee2b5b7e0738fc626d03b31b36f194eeb43e3b90dd49e06feaec637ac14426ff
Sep 20 02:07:16.942893 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:16.942833    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:07:17.090969 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:17.090932    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:07:17.191925 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:17.191897    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:07:17.284306 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:17.284260    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:07:17.512048 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:17.511939    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-4cfa5a-75ddbf85cc-s7tkt" event={"ID":"00534def-114d-4913-9a34-e65cda8d616c","Type":"ContainerStarted","Data":"f5490723a3edf3a3e0e4247c5063c4c78fcc88b2550683d738c9998e9baf154b"}
Sep 20 02:07:17.512048 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:17.512006    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-4cfa5a-75ddbf85cc-s7tkt" event={"ID":"00534def-114d-4913-9a34-e65cda8d616c","Type":"ContainerStarted","Data":"ee2b5b7e0738fc626d03b31b36f194eeb43e3b90dd49e06feaec637ac14426ff"}
Sep 20 02:07:17.512594 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:17.512077    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-autores-4cfa5a-75ddbf85cc-s7tkt"
Sep 20 02:07:17.513765 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:17.513733    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-4cfa5a-7c8c4fc9d6-dzsc7" event={"ID":"ea286772-f46d-49c3-869c-1161c5cab4f4","Type":"ContainerStarted","Data":"14c34011aaceb8826bac07125b383d93710b9e57e72a8d294066e3a2e084033c"}
Sep 20 02:07:17.513900 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:17.513777    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-4cfa5a-7c8c4fc9d6-dzsc7" event={"ID":"ea286772-f46d-49c3-869c-1161c5cab4f4","Type":"ContainerStarted","Data":"940331e74b742247462124f010d0d153862396525a30f72b88c4677a652d0d57"}
Sep 20 02:07:17.513900 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:17.513816    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-4cfa5a-7c8c4fc9d6-dzsc7"
Sep 20 02:07:17.523271 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:17.523243    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:07:17.529488 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:17.529438    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-autores-4cfa5a-75ddbf85cc-s7tkt" podStartSLOduration=1.529420353 podStartE2EDuration="1.529420353s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-20 02:07:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-20 02:07:17.528941437 +0000 UTC m=+894.271620643" watchObservedRunningTime="2026-09-20 02:07:17.529420353 +0000 UTC m=+894.272099558"
Sep 20 02:07:17.546838 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:17.546782    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-autores-4cfa5a-7c8c4fc9d6-dzsc7" podStartSLOduration=1.546761761 podStartE2EDuration="1.546761761s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-20 02:07:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-20 02:07:17.545385121 +0000 UTC m=+894.288064328" watchObservedRunningTime="2026-09-20 02:07:17.546761761 +0000 UTC m=+894.289440967"
Sep 20 02:07:17.624016 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:17.623982    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:07:17.861333 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:17.861239    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:07:18.189911 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:18.189825    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:07:18.426552 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:18.426495    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:07:18.869021 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:18.868980    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:07:18.969332 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:18.969287    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:07:19.368553 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:19.368503    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:07:19.458501 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:19.458462    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:07:19.557324 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:19.557265    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:07:19.657964 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:19.657837    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:07:20.384331 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:20.384278    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:07:21.238748 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:21.238681    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:07:21.672745 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:21.672583    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:07:22.249490 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:22.249453    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:07:22.673869 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:22.673780    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:07:22.765864 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:22.765814    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:07:23.139553 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:23.139511    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:07:23.238846 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:23.238815    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:07:23.391042 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:23.390964    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:07:24.141638 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:24.141606    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-703998-6755f65cb6-sbmsr"]
Sep 20 02:07:24.146318 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:24.146298    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-703998-6755f65cb6-sbmsr"
Sep 20 02:07:24.148667 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:24.148643    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-703998-serving-cert\""
Sep 20 02:07:24.153870 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:24.153845    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-703998-6755f65cb6-sbmsr"]
Sep 20 02:07:24.227611 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:24.227570    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:07:24.231620 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:24.231594    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b8656bfb-4ba7-47df-ad3f-fde87101cf1b-metrics-certs\") pod \"maas-api-e2e-inf-b-703998-6755f65cb6-sbmsr\" (UID: \"b8656bfb-4ba7-47df-ad3f-fde87101cf1b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-703998-6755f65cb6-sbmsr"
Sep 20 02:07:24.231751 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:24.231642    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b8656bfb-4ba7-47df-ad3f-fde87101cf1b-openshift-service-ca\") pod \"maas-api-e2e-inf-b-703998-6755f65cb6-sbmsr\" (UID: \"b8656bfb-4ba7-47df-ad3f-fde87101cf1b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-703998-6755f65cb6-sbmsr"
Sep 20 02:07:24.231789 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:24.231749    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wck2j\" (UniqueName: \"kubernetes.io/projected/b8656bfb-4ba7-47df-ad3f-fde87101cf1b-kube-api-access-wck2j\") pod \"maas-api-e2e-inf-b-703998-6755f65cb6-sbmsr\" (UID: \"b8656bfb-4ba7-47df-ad3f-fde87101cf1b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-703998-6755f65cb6-sbmsr"
Sep 20 02:07:24.231822 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:24.231790    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b8656bfb-4ba7-47df-ad3f-fde87101cf1b-maas-api-tls\") pod \"maas-api-e2e-inf-b-703998-6755f65cb6-sbmsr\" (UID: \"b8656bfb-4ba7-47df-ad3f-fde87101cf1b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-703998-6755f65cb6-sbmsr"
Sep 20 02:07:24.329077 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:24.329041    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:07:24.333216 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:24.333191    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b8656bfb-4ba7-47df-ad3f-fde87101cf1b-metrics-certs\") pod \"maas-api-e2e-inf-b-703998-6755f65cb6-sbmsr\" (UID: \"b8656bfb-4ba7-47df-ad3f-fde87101cf1b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-703998-6755f65cb6-sbmsr"
Sep 20 02:07:24.333306 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:24.333237    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b8656bfb-4ba7-47df-ad3f-fde87101cf1b-openshift-service-ca\") pod \"maas-api-e2e-inf-b-703998-6755f65cb6-sbmsr\" (UID: \"b8656bfb-4ba7-47df-ad3f-fde87101cf1b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-703998-6755f65cb6-sbmsr"
Sep 20 02:07:24.333306 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:24.333281    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wck2j\" (UniqueName: \"kubernetes.io/projected/b8656bfb-4ba7-47df-ad3f-fde87101cf1b-kube-api-access-wck2j\") pod \"maas-api-e2e-inf-b-703998-6755f65cb6-sbmsr\" (UID: \"b8656bfb-4ba7-47df-ad3f-fde87101cf1b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-703998-6755f65cb6-sbmsr"
Sep 20 02:07:24.333306 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:24.333301    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b8656bfb-4ba7-47df-ad3f-fde87101cf1b-maas-api-tls\") pod \"maas-api-e2e-inf-b-703998-6755f65cb6-sbmsr\" (UID: \"b8656bfb-4ba7-47df-ad3f-fde87101cf1b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-703998-6755f65cb6-sbmsr"
Sep 20 02:07:24.333427 ip-10-0-130-159 kubenswrapper[2584]: E0920 02:07:24.333414    2584 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-703998-serving-cert: secret "maas-api-e2e-inf-b-703998-serving-cert" not found
Sep 20 02:07:24.333484 ip-10-0-130-159 kubenswrapper[2584]: E0920 02:07:24.333473    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b8656bfb-4ba7-47df-ad3f-fde87101cf1b-maas-api-tls podName:b8656bfb-4ba7-47df-ad3f-fde87101cf1b nodeName:}" failed. No retries permitted until 2026-09-20 02:07:24.833453957 +0000 UTC m=+901.576133157 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b8656bfb-4ba7-47df-ad3f-fde87101cf1b-maas-api-tls") pod "maas-api-e2e-inf-b-703998-6755f65cb6-sbmsr" (UID: "b8656bfb-4ba7-47df-ad3f-fde87101cf1b") : secret "maas-api-e2e-inf-b-703998-serving-cert" not found
Sep 20 02:07:24.334060 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:24.334038    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b8656bfb-4ba7-47df-ad3f-fde87101cf1b-openshift-service-ca\") pod \"maas-api-e2e-inf-b-703998-6755f65cb6-sbmsr\" (UID: \"b8656bfb-4ba7-47df-ad3f-fde87101cf1b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-703998-6755f65cb6-sbmsr"
Sep 20 02:07:24.335867 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:24.335838    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b8656bfb-4ba7-47df-ad3f-fde87101cf1b-metrics-certs\") pod \"maas-api-e2e-inf-b-703998-6755f65cb6-sbmsr\" (UID: \"b8656bfb-4ba7-47df-ad3f-fde87101cf1b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-703998-6755f65cb6-sbmsr"
Sep 20 02:07:24.343511 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:24.343483    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wck2j\" (UniqueName: \"kubernetes.io/projected/b8656bfb-4ba7-47df-ad3f-fde87101cf1b-kube-api-access-wck2j\") pod \"maas-api-e2e-inf-b-703998-6755f65cb6-sbmsr\" (UID: \"b8656bfb-4ba7-47df-ad3f-fde87101cf1b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-703998-6755f65cb6-sbmsr"
Sep 20 02:07:24.564870 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:24.564812    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:07:24.693413 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:24.693361    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:07:24.796468 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:24.796431    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:07:24.830282 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:24.830180    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-703998-56676dd7-vhtqd"]
Sep 20 02:07:24.835179 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:24.835149    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-703998-56676dd7-vhtqd"
Sep 20 02:07:24.837676 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:24.837647    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-703998\""
Sep 20 02:07:24.837825 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:24.837700    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-703998-dockercfg-jv2br\""
Sep 20 02:07:24.839746 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:24.839198    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b8656bfb-4ba7-47df-ad3f-fde87101cf1b-maas-api-tls\") pod \"maas-api-e2e-inf-b-703998-6755f65cb6-sbmsr\" (UID: \"b8656bfb-4ba7-47df-ad3f-fde87101cf1b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-703998-6755f65cb6-sbmsr"
Sep 20 02:07:24.842776 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:24.842742    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b8656bfb-4ba7-47df-ad3f-fde87101cf1b-maas-api-tls\") pod \"maas-api-e2e-inf-b-703998-6755f65cb6-sbmsr\" (UID: \"b8656bfb-4ba7-47df-ad3f-fde87101cf1b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-703998-6755f65cb6-sbmsr"
Sep 20 02:07:24.844015 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:24.843992    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-703998-56676dd7-vhtqd"]
Sep 20 02:07:24.940054 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:24.940004    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aae1344b-f684-4f6d-8c9f-eb4353885f54-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-703998-56676dd7-vhtqd\" (UID: \"aae1344b-f684-4f6d-8c9f-eb4353885f54\") " pod="openshift-ingress/payload-processing-e2e-inf-b-703998-56676dd7-vhtqd"
Sep 20 02:07:24.940332 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:24.940219    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fjn62\" (UniqueName: \"kubernetes.io/projected/aae1344b-f684-4f6d-8c9f-eb4353885f54-kube-api-access-fjn62\") pod \"payload-processing-e2e-inf-b-703998-56676dd7-vhtqd\" (UID: \"aae1344b-f684-4f6d-8c9f-eb4353885f54\") " pod="openshift-ingress/payload-processing-e2e-inf-b-703998-56676dd7-vhtqd"
Sep 20 02:07:25.040917 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:25.040874    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aae1344b-f684-4f6d-8c9f-eb4353885f54-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-703998-56676dd7-vhtqd\" (UID: \"aae1344b-f684-4f6d-8c9f-eb4353885f54\") " pod="openshift-ingress/payload-processing-e2e-inf-b-703998-56676dd7-vhtqd"
Sep 20 02:07:25.041115 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:25.041043    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fjn62\" (UniqueName: \"kubernetes.io/projected/aae1344b-f684-4f6d-8c9f-eb4353885f54-kube-api-access-fjn62\") pod \"payload-processing-e2e-inf-b-703998-56676dd7-vhtqd\" (UID: \"aae1344b-f684-4f6d-8c9f-eb4353885f54\") " pod="openshift-ingress/payload-processing-e2e-inf-b-703998-56676dd7-vhtqd"
Sep 20 02:07:25.043416 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:25.043397    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-703998\""
Sep 20 02:07:25.049484 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:25.049440    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fjn62\" (UniqueName: \"kubernetes.io/projected/aae1344b-f684-4f6d-8c9f-eb4353885f54-kube-api-access-fjn62\") pod \"payload-processing-e2e-inf-b-703998-56676dd7-vhtqd\" (UID: \"aae1344b-f684-4f6d-8c9f-eb4353885f54\") " pod="openshift-ingress/payload-processing-e2e-inf-b-703998-56676dd7-vhtqd"
Sep 20 02:07:25.051997 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:25.051963    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aae1344b-f684-4f6d-8c9f-eb4353885f54-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-703998-56676dd7-vhtqd\" (UID: \"aae1344b-f684-4f6d-8c9f-eb4353885f54\") " pod="openshift-ingress/payload-processing-e2e-inf-b-703998-56676dd7-vhtqd"
Sep 20 02:07:25.060485 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:25.060433    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-703998-6755f65cb6-sbmsr"
Sep 20 02:07:25.117616 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:25.117533    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-ztdt2_ece82f19-68b5-4810-be50-d8678cfdee02/console-operator/1.log"
Sep 20 02:07:25.117752 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:25.117670    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-ztdt2_ece82f19-68b5-4810-be50-d8678cfdee02/console-operator/1.log"
Sep 20 02:07:25.147023 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:25.144144    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:07:25.168140 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:25.167727    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-703998-dockercfg-jv2br\""
Sep 20 02:07:25.176357 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:25.175922    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-703998-56676dd7-vhtqd"
Sep 20 02:07:25.348106 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:25.348077    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-703998-6755f65cb6-sbmsr"]
Sep 20 02:07:25.351103 ip-10-0-130-159 kubenswrapper[2584]: W0920 02:07:25.351073    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8656bfb_4ba7_47df_ad3f_fde87101cf1b.slice/crio-a1a441728bd3593094c3bdd4c57347e871750a7be745c89ceac9c29de69116ec WatchSource:0}: Error finding container a1a441728bd3593094c3bdd4c57347e871750a7be745c89ceac9c29de69116ec: Status 404 returned error can't find the container with id a1a441728bd3593094c3bdd4c57347e871750a7be745c89ceac9c29de69116ec
Sep 20 02:07:25.460145 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:25.460094    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-703998-56676dd7-vhtqd"]
Sep 20 02:07:25.463784 ip-10-0-130-159 kubenswrapper[2584]: W0920 02:07:25.463755    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaae1344b_f684_4f6d_8c9f_eb4353885f54.slice/crio-2ddc9c71dffe597257deec3a06aeeeff6c27cc6d1cdb3953756f9ac0afbb297b WatchSource:0}: Error finding container 2ddc9c71dffe597257deec3a06aeeeff6c27cc6d1cdb3953756f9ac0afbb297b: Status 404 returned error can't find the container with id 2ddc9c71dffe597257deec3a06aeeeff6c27cc6d1cdb3953756f9ac0afbb297b
Sep 20 02:07:25.561963 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:25.561921    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-703998-56676dd7-vhtqd" event={"ID":"aae1344b-f684-4f6d-8c9f-eb4353885f54","Type":"ContainerStarted","Data":"204634417f4bdd38deea31f57ba3b40d357dc60af558e7434c70d5ad9e9a928e"}
Sep 20 02:07:25.561963 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:25.561986    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-703998-56676dd7-vhtqd" event={"ID":"aae1344b-f684-4f6d-8c9f-eb4353885f54","Type":"ContainerStarted","Data":"2ddc9c71dffe597257deec3a06aeeeff6c27cc6d1cdb3953756f9ac0afbb297b"}
Sep 20 02:07:25.561963 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:25.562017    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-703998-56676dd7-vhtqd"
Sep 20 02:07:25.563512 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:25.563487    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-703998-6755f65cb6-sbmsr" event={"ID":"b8656bfb-4ba7-47df-ad3f-fde87101cf1b","Type":"ContainerStarted","Data":"a1a441728bd3593094c3bdd4c57347e871750a7be745c89ceac9c29de69116ec"}
Sep 20 02:07:25.595500 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:25.595440    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-703998-56676dd7-vhtqd" podStartSLOduration=1.595421173 podStartE2EDuration="1.595421173s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-20 02:07:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-20 02:07:25.594429802 +0000 UTC m=+902.337109006" watchObservedRunningTime="2026-09-20 02:07:25.595421173 +0000 UTC m=+902.338100391"
Sep 20 02:07:25.946136 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:25.946102    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:07:26.572704 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:26.572669    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-703998-6755f65cb6-sbmsr" event={"ID":"b8656bfb-4ba7-47df-ad3f-fde87101cf1b","Type":"ContainerStarted","Data":"1f435424e044a7c4376fd966889e3f469a4502c992727176c02549ba3e7810a8"}
Sep 20 02:07:26.573231 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:26.573010    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-703998-6755f65cb6-sbmsr"
Sep 20 02:07:26.591717 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:26.591654    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-703998-6755f65cb6-sbmsr" podStartSLOduration=2.25287004 podStartE2EDuration="2.591631454s" totalImagesPullingTime="338.761414ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-20 02:07:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-20 02:07:26.589696099 +0000 UTC m=+903.332375304" watchObservedRunningTime="2026-09-20 02:07:26.591631454 +0000 UTC m=+903.334310660"
Sep 20 02:07:26.753865 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:26.753833    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:07:27.584548 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:27.584482    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:07:27.633779 ip-10-0-130-159 kubenswrapper[2584]: E0920 02:07:27.633741    2584 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8979a7e_a149_4c3f_a82c_32c19f551021.slice/crio-conmon-fdd39165c51488978744128bbe1b15fa0a23677794ef5e669306b84089024620.scope\": RecentStats: unable to find data in memory cache]"
Sep 20 02:07:28.248659 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:28.248624    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:07:28.522683 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:28.522604    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-autores-4cfa5a-75ddbf85cc-s7tkt"
Sep 20 02:07:28.522683 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:28.522669    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-4cfa5a-7c8c4fc9d6-dzsc7"
Sep 20 02:07:28.596910 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:28.596879    2584 generic.go:356] "Generic (PLEG): container finished" podID="f8979a7e-a149-4c3f-a82c-32c19f551021" containerID="fdd39165c51488978744128bbe1b15fa0a23677794ef5e669306b84089024620" exitCode=0
Sep 20 02:07:28.597306 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:28.596956    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-04a2fd-openshift-default-74d645d896-c89xp" event={"ID":"f8979a7e-a149-4c3f-a82c-32c19f551021","Type":"ContainerDied","Data":"fdd39165c51488978744128bbe1b15fa0a23677794ef5e669306b84089024620"}
Sep 20 02:07:28.597443 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:28.597430    2584 scope.go:117] "RemoveContainer" containerID="fdd39165c51488978744128bbe1b15fa0a23677794ef5e669306b84089024620"
Sep 20 02:07:28.598617 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:28.598589    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 20 02:07:28.598685 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:28.598654    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 20 02:07:28.598722 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:28.598686    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 20 02:07:28.991203 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:28.991171    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:07:29.471545 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:29.471488    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:07:29.570170 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:29.570138    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:07:29.604223 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:29.604189    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-04a2fd-openshift-default-74d645d896-c89xp" event={"ID":"f8979a7e-a149-4c3f-a82c-32c19f551021","Type":"ContainerStarted","Data":"6710642a1127d57ffa1eda51b44e8f4b124fb8ccb0f401fff2cb321534aaa5d1"}
Sep 20 02:07:29.661140 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:29.661094    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-04a2fd-openshift-default-74d645d896-c89xp"
Sep 20 02:07:29.662377 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:29.662345    2584 patch_prober.go:28] interesting pod/e2e-inf-a-04a2fd-openshift-default-74d645d896-c89xp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body=
Sep 20 02:07:29.662565 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:29.662407    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-04a2fd-openshift-default-74d645d896-c89xp" podUID="f8979a7e-a149-4c3f-a82c-32c19f551021" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused"
Sep 20 02:07:29.670337 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:29.670307    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:07:29.772033 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:29.771917    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:07:29.881190 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:29.881156    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:07:30.661606 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:30.661565    2584 patch_prober.go:28] interesting pod/e2e-inf-a-04a2fd-openshift-default-74d645d896-c89xp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body=
Sep 20 02:07:30.662241 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:30.662197    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-04a2fd-openshift-default-74d645d896-c89xp" podUID="f8979a7e-a149-4c3f-a82c-32c19f551021" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused"
Sep 20 02:07:30.765844 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:30.765803    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:07:31.395654 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:31.395624    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-04a2fd-openshift-default-74d645d896-c89xp"
Sep 20 02:07:31.644466 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:31.644434    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:07:31.661459 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:31.661364    2584 patch_prober.go:28] interesting pod/e2e-inf-a-04a2fd-openshift-default-74d645d896-c89xp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body=
Sep 20 02:07:31.661768 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:31.661736    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-04a2fd-openshift-default-74d645d896-c89xp" podUID="f8979a7e-a149-4c3f-a82c-32c19f551021" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused"
Sep 20 02:07:32.532611 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:32.532559    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:07:32.595488 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:32.595456    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-703998-6755f65cb6-sbmsr"
Sep 20 02:07:32.618535 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:32.618493    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:07:32.661995 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:32.661949    2584 patch_prober.go:28] interesting pod/e2e-inf-a-04a2fd-openshift-default-74d645d896-c89xp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body=
Sep 20 02:07:32.662483 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:32.662033    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-04a2fd-openshift-default-74d645d896-c89xp" podUID="f8979a7e-a149-4c3f-a82c-32c19f551021" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused"
Sep 20 02:07:33.384832 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:33.384794    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:07:33.661958 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:33.661410    2584 patch_prober.go:28] interesting pod/e2e-inf-a-04a2fd-openshift-default-74d645d896-c89xp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body=
Sep 20 02:07:33.661958 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:33.661478    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-04a2fd-openshift-default-74d645d896-c89xp" podUID="f8979a7e-a149-4c3f-a82c-32c19f551021" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused"
Sep 20 02:07:34.158540 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:34.158491    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:07:34.661949 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:34.661898    2584 patch_prober.go:28] interesting pod/e2e-inf-a-04a2fd-openshift-default-74d645d896-c89xp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body=
Sep 20 02:07:34.662176 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:34.661976    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-04a2fd-openshift-default-74d645d896-c89xp" podUID="f8979a7e-a149-4c3f-a82c-32c19f551021" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused"
Sep 20 02:07:35.066734 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:35.066701    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:07:35.666286 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:35.666250    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-04a2fd-openshift-default-74d645d896-c89xp"
Sep 20 02:07:35.667509 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:35.667488    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-04a2fd-openshift-default-74d645d896-c89xp"
Sep 20 02:07:35.849281 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:35.849250    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:07:36.026711 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:36.026673    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:07:36.127727 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:36.127693    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:07:36.575379 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:36.575347    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-703998-56676dd7-vhtqd"
Sep 20 02:07:36.668683 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:36.668650    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:07:37.385786 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:37.385753    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:07:37.485162 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:37.485122    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:07:38.254661 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:38.254630    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:07:39.050693 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:39.050659    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:07:39.700325 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:39.700276    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:07:40.424981 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:40.424951    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:07:41.082264 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:41.082228    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:07:41.833032 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:41.832987    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:07:42.661583 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:42.661548    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:07:45.101150 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:45.101116    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:07:47.016702 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:47.016665    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:07:53.438585 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:53.438544    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:07:54.035889 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:54.035858    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:07:54.104626 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:54.104584    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:07:54.162197 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:54.162166    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:07:54.227590 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:54.227546    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:07:54.283255 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:54.283219    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:07:54.384203 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:54.384110    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:07:54.649747 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:54.649626    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:07:54.898827 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:54.898789    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:07:55.004493 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:55.004421    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:07:55.128690 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:55.128653    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:07:55.475598 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:55.475559    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:07:55.588834 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:55.588800    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:07:55.759141 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:55.759039    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:07:56.456247 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:56.456210    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:07:57.151405 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:57.151376    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:07:57.880133 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:57.880095    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:07:58.519482 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:58.519443    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:07:58.619791 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:58.619760    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:07:58.719846 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:58.719806    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:07:59.456235 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:59.456179    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:07:59.509951 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:59.509915    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:07:59.675711 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:07:59.675678    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:08:00.087794 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:08:00.087762    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:08:00.191204 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:08:00.191167    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:08:00.656367 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:08:00.656327    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:08:01.395093 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:08:01.395059    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:08:02.041466 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:08:02.041427    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:08:02.773469 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:08:02.773436    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:08:03.437225 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:08:03.437196    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:08:04.762863 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:08:04.762829    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:08:08.115409 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:08:08.115373    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:08:19.816593 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:08:19.816561    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:08:21.648667 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:08:21.648619    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:08:22.342894 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:08:22.342861    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:08:23.160882 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:08:23.160848    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:08:23.953411 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:08:23.953372    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:08:24.738762 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:08:24.738720    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:08:25.439466 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:08:25.439433    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:08:26.125551 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:08:26.125504    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:08:26.903355 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:08:26.903321    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:08:27.779483 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:08:27.779453    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:08:28.560355 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:08:28.560318    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:08:29.136184 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:08:29.136149    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:08:29.199734 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:08:29.199697    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:08:29.261769 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:08:29.261729    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:08:29.322253 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:08:29.322220    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:08:29.383043 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:08:29.383011    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:08:29.462606 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:08:29.461582    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:08:29.821399 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:08:29.821313    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:08:29.941636 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:08:29.941598    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:08:30.288532 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:08:30.288480    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:08:30.505129 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:08:30.505097    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:08:30.639126 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:08:30.639046    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:08:31.737636 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:08:31.737604    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:08:32.550706 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:08:32.550674    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:08:32.650613 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:08:32.650578    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:08:33.546977 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:08:33.546942    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:08:34.305113 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:08:34.305077    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:08:35.053096 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:08:35.053061    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:08:35.154035 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:08:35.153997    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:08:35.329458 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:08:35.329363    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:08:35.514053 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:08:35.514021    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:08:35.615325 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:08:35.615243    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:08:35.946575 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:08:35.946542    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:08:36.683085 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:08:36.683036    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:08:37.350845 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:08:37.350785    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:08:38.167600 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:08:38.167563    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:08:39.095859 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:08:39.095827    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-04a2fd/test-model-04a2fd-kserve-7bd95f6bdc-nhf75"]
Sep 20 02:08:39.099903 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:08:39.099877    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-04a2fd/test-model-04a2fd-kserve-7bd95f6bdc-nhf75"
Sep 20 02:08:39.105513 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:08:39.105481    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-04a2fd\"/\"test-model-04a2fd-kserve-self-signed-certs\""
Sep 20 02:08:39.105683 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:08:39.105488    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-04a2fd\"/\"kube-root-ca.crt\""
Sep 20 02:08:39.105683 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:08:39.105486    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-04a2fd\"/\"openshift-service-ca.crt\""
Sep 20 02:08:39.105683 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:08:39.105514    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-04a2fd\"/\"default-dockercfg-7lpsr\""
Sep 20 02:08:39.109818 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:08:39.109796    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-04a2fd/test-model-04a2fd-kserve-7bd95f6bdc-nhf75"]
Sep 20 02:08:39.154782 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:08:39.154746    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ab8ecb10-3859-406f-b614-ad045a1f69a1-tmp-dir\") pod \"test-model-04a2fd-kserve-7bd95f6bdc-nhf75\" (UID: \"ab8ecb10-3859-406f-b614-ad045a1f69a1\") " pod="ai-tenant-e2e-inf-a-04a2fd/test-model-04a2fd-kserve-7bd95f6bdc-nhf75"
Sep 20 02:08:39.154981 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:08:39.154800    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ab8ecb10-3859-406f-b614-ad045a1f69a1-home\") pod \"test-model-04a2fd-kserve-7bd95f6bdc-nhf75\" (UID: \"ab8ecb10-3859-406f-b614-ad045a1f69a1\") " pod="ai-tenant-e2e-inf-a-04a2fd/test-model-04a2fd-kserve-7bd95f6bdc-nhf75"
Sep 20 02:08:39.154981 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:08:39.154817    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ab8ecb10-3859-406f-b614-ad045a1f69a1-dshm\") pod \"test-model-04a2fd-kserve-7bd95f6bdc-nhf75\" (UID: \"ab8ecb10-3859-406f-b614-ad045a1f69a1\") " pod="ai-tenant-e2e-inf-a-04a2fd/test-model-04a2fd-kserve-7bd95f6bdc-nhf75"
Sep 20 02:08:39.154981 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:08:39.154840    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4s82c\" (UniqueName: \"kubernetes.io/projected/ab8ecb10-3859-406f-b614-ad045a1f69a1-kube-api-access-4s82c\") pod \"test-model-04a2fd-kserve-7bd95f6bdc-nhf75\" (UID: \"ab8ecb10-3859-406f-b614-ad045a1f69a1\") " pod="ai-tenant-e2e-inf-a-04a2fd/test-model-04a2fd-kserve-7bd95f6bdc-nhf75"
Sep 20 02:08:39.154981 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:08:39.154946    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ab8ecb10-3859-406f-b614-ad045a1f69a1-model-cache\") pod \"test-model-04a2fd-kserve-7bd95f6bdc-nhf75\" (UID: \"ab8ecb10-3859-406f-b614-ad045a1f69a1\") " pod="ai-tenant-e2e-inf-a-04a2fd/test-model-04a2fd-kserve-7bd95f6bdc-nhf75"
Sep 20 02:08:39.155230 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:08:39.154993    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ab8ecb10-3859-406f-b614-ad045a1f69a1-tls-certs\") pod \"test-model-04a2fd-kserve-7bd95f6bdc-nhf75\" (UID: \"ab8ecb10-3859-406f-b614-ad045a1f69a1\") " pod="ai-tenant-e2e-inf-a-04a2fd/test-model-04a2fd-kserve-7bd95f6bdc-nhf75"
Sep 20 02:08:39.256663 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:08:39.256618    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ab8ecb10-3859-406f-b614-ad045a1f69a1-model-cache\") pod \"test-model-04a2fd-kserve-7bd95f6bdc-nhf75\" (UID: \"ab8ecb10-3859-406f-b614-ad045a1f69a1\") " pod="ai-tenant-e2e-inf-a-04a2fd/test-model-04a2fd-kserve-7bd95f6bdc-nhf75"
Sep 20 02:08:39.257236 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:08:39.256682    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ab8ecb10-3859-406f-b614-ad045a1f69a1-tls-certs\") pod \"test-model-04a2fd-kserve-7bd95f6bdc-nhf75\" (UID: \"ab8ecb10-3859-406f-b614-ad045a1f69a1\") " pod="ai-tenant-e2e-inf-a-04a2fd/test-model-04a2fd-kserve-7bd95f6bdc-nhf75"
Sep 20 02:08:39.257236 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:08:39.256766    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ab8ecb10-3859-406f-b614-ad045a1f69a1-tmp-dir\") pod \"test-model-04a2fd-kserve-7bd95f6bdc-nhf75\" (UID: \"ab8ecb10-3859-406f-b614-ad045a1f69a1\") " pod="ai-tenant-e2e-inf-a-04a2fd/test-model-04a2fd-kserve-7bd95f6bdc-nhf75"
Sep 20 02:08:39.257236 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:08:39.256798    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ab8ecb10-3859-406f-b614-ad045a1f69a1-home\") pod \"test-model-04a2fd-kserve-7bd95f6bdc-nhf75\" (UID: \"ab8ecb10-3859-406f-b614-ad045a1f69a1\") " pod="ai-tenant-e2e-inf-a-04a2fd/test-model-04a2fd-kserve-7bd95f6bdc-nhf75"
Sep 20 02:08:39.257236 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:08:39.256821    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ab8ecb10-3859-406f-b614-ad045a1f69a1-dshm\") pod \"test-model-04a2fd-kserve-7bd95f6bdc-nhf75\" (UID: \"ab8ecb10-3859-406f-b614-ad045a1f69a1\") " pod="ai-tenant-e2e-inf-a-04a2fd/test-model-04a2fd-kserve-7bd95f6bdc-nhf75"
Sep 20 02:08:39.257236 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:08:39.256857    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4s82c\" (UniqueName: \"kubernetes.io/projected/ab8ecb10-3859-406f-b614-ad045a1f69a1-kube-api-access-4s82c\") pod \"test-model-04a2fd-kserve-7bd95f6bdc-nhf75\" (UID: \"ab8ecb10-3859-406f-b614-ad045a1f69a1\") " pod="ai-tenant-e2e-inf-a-04a2fd/test-model-04a2fd-kserve-7bd95f6bdc-nhf75"
Sep 20 02:08:39.257574 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:08:39.257238    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ab8ecb10-3859-406f-b614-ad045a1f69a1-home\") pod \"test-model-04a2fd-kserve-7bd95f6bdc-nhf75\" (UID: \"ab8ecb10-3859-406f-b614-ad045a1f69a1\") " pod="ai-tenant-e2e-inf-a-04a2fd/test-model-04a2fd-kserve-7bd95f6bdc-nhf75"
Sep 20 02:08:39.257574 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:08:39.257481    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ab8ecb10-3859-406f-b614-ad045a1f69a1-tmp-dir\") pod \"test-model-04a2fd-kserve-7bd95f6bdc-nhf75\" (UID: \"ab8ecb10-3859-406f-b614-ad045a1f69a1\") " pod="ai-tenant-e2e-inf-a-04a2fd/test-model-04a2fd-kserve-7bd95f6bdc-nhf75"
Sep 20 02:08:39.257688 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:08:39.257669    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ab8ecb10-3859-406f-b614-ad045a1f69a1-model-cache\") pod \"test-model-04a2fd-kserve-7bd95f6bdc-nhf75\" (UID: \"ab8ecb10-3859-406f-b614-ad045a1f69a1\") " pod="ai-tenant-e2e-inf-a-04a2fd/test-model-04a2fd-kserve-7bd95f6bdc-nhf75"
Sep 20 02:08:39.259908 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:08:39.259871    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ab8ecb10-3859-406f-b614-ad045a1f69a1-dshm\") pod \"test-model-04a2fd-kserve-7bd95f6bdc-nhf75\" (UID: \"ab8ecb10-3859-406f-b614-ad045a1f69a1\") " pod="ai-tenant-e2e-inf-a-04a2fd/test-model-04a2fd-kserve-7bd95f6bdc-nhf75"
Sep 20 02:08:39.260277 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:08:39.260254    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ab8ecb10-3859-406f-b614-ad045a1f69a1-tls-certs\") pod \"test-model-04a2fd-kserve-7bd95f6bdc-nhf75\" (UID: \"ab8ecb10-3859-406f-b614-ad045a1f69a1\") " pod="ai-tenant-e2e-inf-a-04a2fd/test-model-04a2fd-kserve-7bd95f6bdc-nhf75"
Sep 20 02:08:39.266859 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:08:39.266830    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4s82c\" (UniqueName: \"kubernetes.io/projected/ab8ecb10-3859-406f-b614-ad045a1f69a1-kube-api-access-4s82c\") pod \"test-model-04a2fd-kserve-7bd95f6bdc-nhf75\" (UID: \"ab8ecb10-3859-406f-b614-ad045a1f69a1\") " pod="ai-tenant-e2e-inf-a-04a2fd/test-model-04a2fd-kserve-7bd95f6bdc-nhf75"
Sep 20 02:08:39.415903 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:08:39.415795    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-04a2fd/test-model-04a2fd-kserve-7bd95f6bdc-nhf75"
Sep 20 02:08:39.500694 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:08:39.500666    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:08:39.690915 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:08:39.690889    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-04a2fd/test-model-04a2fd-kserve-7bd95f6bdc-nhf75"]
Sep 20 02:08:39.692810 ip-10-0-130-159 kubenswrapper[2584]: W0920 02:08:39.692783    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podab8ecb10_3859_406f_b614_ad045a1f69a1.slice/crio-1a2c86605c79f37b4eb3ea855eef35873fc254fd94a150f264ab954949f68b9c WatchSource:0}: Error finding container 1a2c86605c79f37b4eb3ea855eef35873fc254fd94a150f264ab954949f68b9c: Status 404 returned error can't find the container with id 1a2c86605c79f37b4eb3ea855eef35873fc254fd94a150f264ab954949f68b9c
Sep 20 02:08:40.009294 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:08:40.009256    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-04a2fd/test-model-04a2fd-kserve-7bd95f6bdc-nhf75" event={"ID":"ab8ecb10-3859-406f-b614-ad045a1f69a1","Type":"ContainerStarted","Data":"d28457a04b3db308fa83726a923cb90458e44eabbe5b32b1f39699255c4134cf"}
Sep 20 02:08:40.009505 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:08:40.009306    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-04a2fd/test-model-04a2fd-kserve-7bd95f6bdc-nhf75" event={"ID":"ab8ecb10-3859-406f-b614-ad045a1f69a1","Type":"ContainerStarted","Data":"1a2c86605c79f37b4eb3ea855eef35873fc254fd94a150f264ab954949f68b9c"}
Sep 20 02:08:40.027244 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:08:40.027179    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-04a2fd/test-model-04a2fd-kserve-7bd95f6bdc-nhf75" podStartSLOduration=1.027161511 podStartE2EDuration="1.027161511s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-20 02:08:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-20 02:08:40.024800906 +0000 UTC m=+976.767480111" watchObservedRunningTime="2026-09-20 02:08:40.027161511 +0000 UTC m=+976.769840749"
Sep 20 02:08:49.416371 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:08:49.416325    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-04a2fd/test-model-04a2fd-kserve-7bd95f6bdc-nhf75"
Sep 20 02:08:49.416902 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:08:49.416394    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-04a2fd/test-model-04a2fd-kserve-7bd95f6bdc-nhf75"
Sep 20 02:08:49.429253 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:08:49.429227    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-04a2fd/test-model-04a2fd-kserve-7bd95f6bdc-nhf75"
Sep 20 02:08:50.080598 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:08:50.080570    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-04a2fd/test-model-04a2fd-kserve-7bd95f6bdc-nhf75"
Sep 20 02:08:51.312030 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:08:51.311996    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:08:52.296362 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:08:52.296331    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:08:55.642752 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:08:55.642711    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:08:56.441350 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:08:56.441304    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:08:57.260150 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:08:57.260087    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:08:57.944137 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:08:57.944070    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:08:58.823502 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:08:58.823459    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:08:59.461454 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:08:59.461417    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:09:00.231823 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:09:00.231788    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:09:00.920423 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:09:00.920379    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:09:01.623025 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:09:01.622992    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:09:02.335973 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:09:02.335928    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:09:03.051304 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:09:03.051270    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:09:03.942294 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:09:03.942255    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:09:04.762817 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:09:04.762780    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:09:05.439223 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:09:05.439194    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:09:06.008887 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:09:06.008855    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:09:06.029833 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:09:06.029786    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-4cfa5a-75ddbf85cc-s7tkt"]
Sep 20 02:09:06.030451 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:09:06.030411    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-autores-4cfa5a-75ddbf85cc-s7tkt" podUID="00534def-114d-4913-9a34-e65cda8d616c" containerName="payload-processing" containerID="cri-o://f5490723a3edf3a3e0e4247c5063c4c78fcc88b2550683d738c9998e9baf154b" gracePeriod=30
Sep 20 02:09:06.033610 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:09:06.033584    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-4cfa5a-7c8c4fc9d6-dzsc7"]
Sep 20 02:09:06.033865 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:09:06.033824    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-autores-4cfa5a-7c8c4fc9d6-dzsc7" podUID="ea286772-f46d-49c3-869c-1161c5cab4f4" containerName="payload-pre-processing" containerID="cri-o://14c34011aaceb8826bac07125b383d93710b9e57e72a8d294066e3a2e084033c" gracePeriod=30
Sep 20 02:09:06.072385 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:09:06.072349    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:09:06.137978 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:09:06.137941    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:09:06.164915 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:09:06.164804    2584 generic.go:356] "Generic (PLEG): container finished" podID="00534def-114d-4913-9a34-e65cda8d616c" containerID="f5490723a3edf3a3e0e4247c5063c4c78fcc88b2550683d738c9998e9baf154b" exitCode=0
Sep 20 02:09:06.164915 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:09:06.164838    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-4cfa5a-75ddbf85cc-s7tkt" event={"ID":"00534def-114d-4913-9a34-e65cda8d616c","Type":"ContainerDied","Data":"f5490723a3edf3a3e0e4247c5063c4c78fcc88b2550683d738c9998e9baf154b"}
Sep 20 02:09:06.167065 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:09:06.166972    2584 generic.go:356] "Generic (PLEG): container finished" podID="ea286772-f46d-49c3-869c-1161c5cab4f4" containerID="14c34011aaceb8826bac07125b383d93710b9e57e72a8d294066e3a2e084033c" exitCode=0
Sep 20 02:09:06.167217 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:09:06.167065    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-4cfa5a-7c8c4fc9d6-dzsc7" event={"ID":"ea286772-f46d-49c3-869c-1161c5cab4f4","Type":"ContainerDied","Data":"14c34011aaceb8826bac07125b383d93710b9e57e72a8d294066e3a2e084033c"}
Sep 20 02:09:06.237597 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:09:06.237557    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:09:06.298483 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:09:06.298441    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:09:06.325006 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:09:06.324979    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-4cfa5a-7c8c4fc9d6-dzsc7"
Sep 20 02:09:06.351113 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:09:06.351089    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-4cfa5a-75ddbf85cc-s7tkt"
Sep 20 02:09:06.420593 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:09:06.420563    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/00534def-114d-4913-9a34-e65cda8d616c-plugins-config-volume\") pod \"00534def-114d-4913-9a34-e65cda8d616c\" (UID: \"00534def-114d-4913-9a34-e65cda8d616c\") "
Sep 20 02:09:06.420794 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:09:06.420606    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-564w2\" (UniqueName: \"kubernetes.io/projected/00534def-114d-4913-9a34-e65cda8d616c-kube-api-access-564w2\") pod \"00534def-114d-4913-9a34-e65cda8d616c\" (UID: \"00534def-114d-4913-9a34-e65cda8d616c\") "
Sep 20 02:09:06.420794 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:09:06.420674    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ea286772-f46d-49c3-869c-1161c5cab4f4-plugins-config-volume\") pod \"ea286772-f46d-49c3-869c-1161c5cab4f4\" (UID: \"ea286772-f46d-49c3-869c-1161c5cab4f4\") "
Sep 20 02:09:06.420794 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:09:06.420709    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nwmrq\" (UniqueName: \"kubernetes.io/projected/ea286772-f46d-49c3-869c-1161c5cab4f4-kube-api-access-nwmrq\") pod \"ea286772-f46d-49c3-869c-1161c5cab4f4\" (UID: \"ea286772-f46d-49c3-869c-1161c5cab4f4\") "
Sep 20 02:09:06.421032 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:09:06.421000    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/00534def-114d-4913-9a34-e65cda8d616c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "00534def-114d-4913-9a34-e65cda8d616c" (UID: "00534def-114d-4913-9a34-e65cda8d616c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 20 02:09:06.421092 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:09:06.421066    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ea286772-f46d-49c3-869c-1161c5cab4f4-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ea286772-f46d-49c3-869c-1161c5cab4f4" (UID: "ea286772-f46d-49c3-869c-1161c5cab4f4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 20 02:09:06.422940 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:09:06.422914    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea286772-f46d-49c3-869c-1161c5cab4f4-kube-api-access-nwmrq" (OuterVolumeSpecName: "kube-api-access-nwmrq") pod "ea286772-f46d-49c3-869c-1161c5cab4f4" (UID: "ea286772-f46d-49c3-869c-1161c5cab4f4"). InnerVolumeSpecName "kube-api-access-nwmrq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 20 02:09:06.423067 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:09:06.423037    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/00534def-114d-4913-9a34-e65cda8d616c-kube-api-access-564w2" (OuterVolumeSpecName: "kube-api-access-564w2") pod "00534def-114d-4913-9a34-e65cda8d616c" (UID: "00534def-114d-4913-9a34-e65cda8d616c"). InnerVolumeSpecName "kube-api-access-564w2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 20 02:09:06.437781 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:09:06.437749    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:09:06.521819 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:09:06.521783    2584 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ea286772-f46d-49c3-869c-1161c5cab4f4-plugins-config-volume\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:09:06.521819 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:09:06.521810    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nwmrq\" (UniqueName: \"kubernetes.io/projected/ea286772-f46d-49c3-869c-1161c5cab4f4-kube-api-access-nwmrq\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:09:06.521819 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:09:06.521821    2584 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/00534def-114d-4913-9a34-e65cda8d616c-plugins-config-volume\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:09:06.521819 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:09:06.521830    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-564w2\" (UniqueName: \"kubernetes.io/projected/00534def-114d-4913-9a34-e65cda8d616c-kube-api-access-564w2\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:09:06.664048 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:09:06.664010    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:09:07.175426 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:09:07.175391    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-4cfa5a-75ddbf85cc-s7tkt"
Sep 20 02:09:07.176009 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:09:07.175379    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-4cfa5a-75ddbf85cc-s7tkt" event={"ID":"00534def-114d-4913-9a34-e65cda8d616c","Type":"ContainerDied","Data":"ee2b5b7e0738fc626d03b31b36f194eeb43e3b90dd49e06feaec637ac14426ff"}
Sep 20 02:09:07.176009 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:09:07.175615    2584 scope.go:117] "RemoveContainer" containerID="f5490723a3edf3a3e0e4247c5063c4c78fcc88b2550683d738c9998e9baf154b"
Sep 20 02:09:07.177637 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:09:07.177360    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-4cfa5a-7c8c4fc9d6-dzsc7"
Sep 20 02:09:07.177637 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:09:07.177420    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-4cfa5a-7c8c4fc9d6-dzsc7" event={"ID":"ea286772-f46d-49c3-869c-1161c5cab4f4","Type":"ContainerDied","Data":"940331e74b742247462124f010d0d153862396525a30f72b88c4677a652d0d57"}
Sep 20 02:09:07.187885 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:09:07.187861    2584 scope.go:117] "RemoveContainer" containerID="14c34011aaceb8826bac07125b383d93710b9e57e72a8d294066e3a2e084033c"
Sep 20 02:09:07.200625 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:09:07.200580    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-4cfa5a-7c8c4fc9d6-dzsc7"]
Sep 20 02:09:07.205573 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:09:07.205504    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-4cfa5a-7c8c4fc9d6-dzsc7"]
Sep 20 02:09:07.214935 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:09:07.214890    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-4cfa5a-75ddbf85cc-s7tkt"]
Sep 20 02:09:07.218945 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:09:07.218903    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-4cfa5a-75ddbf85cc-s7tkt"]
Sep 20 02:09:07.389716 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:09:07.389617    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:09:08.061301 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:09:08.061268    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:09:08.733764 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:09:08.733721    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:09:09.094645 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:09:09.094560    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="00534def-114d-4913-9a34-e65cda8d616c" path="/var/lib/kubelet/pods/00534def-114d-4913-9a34-e65cda8d616c/volumes"
Sep 20 02:09:09.095122 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:09:09.095101    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ea286772-f46d-49c3-869c-1161c5cab4f4" path="/var/lib/kubelet/pods/ea286772-f46d-49c3-869c-1161c5cab4f4/volumes"
Sep 20 02:09:09.452791 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:09:09.452763    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:09:10.128691 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:09:10.128664    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:09:10.949003 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:09:10.948969    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:09:11.577933 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:09:11.577903    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:09:11.678999 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:09:11.678966    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:09:14.264833 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:09:14.264789    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:09:14.340764 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:09:14.340732    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:09:14.404706 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:09:14.404672    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:09:14.466722 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:09:14.466682    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:09:14.529865 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:09:14.529754    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:09:14.932679 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:09:14.932563    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:09:15.649975 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:09:15.649944    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:09:15.848668 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:09:15.848633    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:09:16.071295 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:09:16.071264    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:09:16.365084 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:09:16.364987    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:09:16.459894 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:09:16.459862    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:09:16.571760 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:09:16.571721    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:09:17.325696 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:09:17.325657    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:09:18.254661 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:09:18.254627    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:09:18.926471 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:09:18.926438    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:09:19.022710 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:09:19.022676    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:09:19.842402 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:09:19.842363    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:09:20.637413 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:09:20.637376    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:09:21.452748 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:09:21.452716    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:09:22.152502 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:09:22.152452    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:09:22.211691 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:09:22.211650    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:09:22.312127 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:09:22.312089    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:09:22.625128 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:09:22.625081    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:09:22.711567 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:09:22.711503    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:09:29.915729 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:09:29.915692    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:09:29.967091 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:09:29.967053    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:09:31.878628 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:09:31.878594    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:09:31.932289 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:09:31.932257    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:09:45.044494 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:09:45.044456    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:09:45.740650 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:09:45.740615    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:09:46.561960 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:09:46.561913    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:09:47.316694 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:09:47.316643    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:09:48.149511 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:09:48.149476    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:09:48.459863 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:09:48.459830    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:09:48.719434 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:09:48.719334    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:09:48.886565 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:09:48.886497    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:09:49.189373 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:09:49.189280    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:09:49.510313 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:09:49.510277    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:09:49.633722 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:09:49.633673    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:09:49.826868 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:09:49.826772    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:09:50.530928 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:09:50.530892    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:09:51.231419 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:09:51.231381    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:09:52.066958 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:09:52.066923    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:09:52.723824 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:09:52.723790    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:09:52.830359 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:09:52.830328    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:09:52.945183 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:09:52.945149    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:09:53.772957 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:09:53.772921    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:09:54.558117 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:09:54.558078    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:09:54.661122 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:09:54.661062    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:09:54.760406 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:09:54.760366    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:09:55.140769 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:09:55.140733    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:09:55.232378 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:09:55.232342    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:09:55.662898 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:09:55.662857    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:09:56.375143 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:09:56.375109    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:09:57.034215 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:09:57.034180    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:09:57.730394 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:09:57.730361    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:09:58.386606 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:09:58.386575    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:09:59.151632 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:09:59.151604    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:10:00.251987 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:10:00.251945    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:10:00.809131 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:10:00.809097    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:10:00.874575 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:10:00.874543    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:10:00.934909 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:10:00.934874    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:10:00.994962 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:10:00.994921    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:10:01.054378 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:10:01.054342    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:10:01.167121 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:10:01.167021    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:10:01.449389 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:10:01.449362    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:10:01.913707 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:10:01.913612    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:10:02.013903 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:10:02.013870    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:10:02.251511 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:10:02.251475    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:10:02.936278 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:10:02.936242    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:10:03.550563 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:10:03.550513    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:10:03.696398 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:10:03.696365    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:10:04.387314 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:10:04.387282    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:10:04.974862 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:10:04.974828    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:10:06.161660 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:10:06.161625    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:10:06.854735 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:10:06.854707    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:10:07.181431 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:10:07.181396    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:10:07.281382 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:10:07.281350    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:10:07.428116 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:10:07.428077    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:10:07.733511 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:10:07.733483    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:10:07.910913 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:10:07.910883    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:10:08.458152 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:10:08.458052    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:10:09.355870 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:10:09.355832    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:10:10.139774 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:10:10.139740    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:10:10.585003 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:10:10.584971    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:10:10.686970 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:10:10.686891    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:10:10.782855 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:10:10.782820    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:10:10.883325 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:10:10.883236    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:10:11.353878 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:10:11.353843    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:10:26.944291 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:10:26.944258    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:10:36.105429 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:10:36.105394    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:10:37.329060 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:10:37.329029    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:10:43.159141 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:10:43.159103    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:10:43.223272 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:10:43.223240    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:10:43.292981 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:10:43.292933    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:10:43.393652 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:10:43.393619    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:10:50.812457 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:10:50.812412    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-explicit-725649/explicit-model-725649-kserve-57b4877bcf-qzcb2"]
Sep 20 02:10:50.813101 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:10:50.813080    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="00534def-114d-4913-9a34-e65cda8d616c" containerName="payload-processing"
Sep 20 02:10:50.813146 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:10:50.813103    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="00534def-114d-4913-9a34-e65cda8d616c" containerName="payload-processing"
Sep 20 02:10:50.813146 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:10:50.813117    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea286772-f46d-49c3-869c-1161c5cab4f4" containerName="payload-pre-processing"
Sep 20 02:10:50.813146 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:10:50.813125    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea286772-f46d-49c3-869c-1161c5cab4f4" containerName="payload-pre-processing"
Sep 20 02:10:50.813247 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:10:50.813202    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="00534def-114d-4913-9a34-e65cda8d616c" containerName="payload-processing"
Sep 20 02:10:50.813247 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:10:50.813213    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="ea286772-f46d-49c3-869c-1161c5cab4f4" containerName="payload-pre-processing"
Sep 20 02:10:50.816579 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:10:50.816556    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-725649/explicit-model-725649-kserve-57b4877bcf-qzcb2"
Sep 20 02:10:50.819856 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:10:50.819828    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-explicit-725649\"/\"explicit-model-725649-kserve-self-signed-certs\""
Sep 20 02:10:50.820017 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:10:50.819989    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-explicit-725649\"/\"kube-root-ca.crt\""
Sep 20 02:10:50.820098 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:10:50.820045    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-explicit-725649\"/\"default-dockercfg-n5qwx\""
Sep 20 02:10:50.820098 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:10:50.820045    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-explicit-725649\"/\"openshift-service-ca.crt\""
Sep 20 02:10:50.826176 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:10:50.826148    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-explicit-725649/explicit-model-725649-kserve-57b4877bcf-qzcb2"]
Sep 20 02:10:50.842938 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:10:50.842895    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:10:50.932551 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:10:50.932477    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rmr22\" (UniqueName: \"kubernetes.io/projected/1faea3ab-644e-496a-849a-a6d769c4467a-kube-api-access-rmr22\") pod \"explicit-model-725649-kserve-57b4877bcf-qzcb2\" (UID: \"1faea3ab-644e-496a-849a-a6d769c4467a\") " pod="ai-tenant-e2e-explicit-725649/explicit-model-725649-kserve-57b4877bcf-qzcb2"
Sep 20 02:10:50.932551 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:10:50.932550    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1faea3ab-644e-496a-849a-a6d769c4467a-dshm\") pod \"explicit-model-725649-kserve-57b4877bcf-qzcb2\" (UID: \"1faea3ab-644e-496a-849a-a6d769c4467a\") " pod="ai-tenant-e2e-explicit-725649/explicit-model-725649-kserve-57b4877bcf-qzcb2"
Sep 20 02:10:50.932833 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:10:50.932621    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1faea3ab-644e-496a-849a-a6d769c4467a-home\") pod \"explicit-model-725649-kserve-57b4877bcf-qzcb2\" (UID: \"1faea3ab-644e-496a-849a-a6d769c4467a\") " pod="ai-tenant-e2e-explicit-725649/explicit-model-725649-kserve-57b4877bcf-qzcb2"
Sep 20 02:10:50.932833 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:10:50.932655    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1faea3ab-644e-496a-849a-a6d769c4467a-tmp-dir\") pod \"explicit-model-725649-kserve-57b4877bcf-qzcb2\" (UID: \"1faea3ab-644e-496a-849a-a6d769c4467a\") " pod="ai-tenant-e2e-explicit-725649/explicit-model-725649-kserve-57b4877bcf-qzcb2"
Sep 20 02:10:50.932833 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:10:50.932690    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1faea3ab-644e-496a-849a-a6d769c4467a-model-cache\") pod \"explicit-model-725649-kserve-57b4877bcf-qzcb2\" (UID: \"1faea3ab-644e-496a-849a-a6d769c4467a\") " pod="ai-tenant-e2e-explicit-725649/explicit-model-725649-kserve-57b4877bcf-qzcb2"
Sep 20 02:10:50.932833 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:10:50.932720    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1faea3ab-644e-496a-849a-a6d769c4467a-tls-certs\") pod \"explicit-model-725649-kserve-57b4877bcf-qzcb2\" (UID: \"1faea3ab-644e-496a-849a-a6d769c4467a\") " pod="ai-tenant-e2e-explicit-725649/explicit-model-725649-kserve-57b4877bcf-qzcb2"
Sep 20 02:10:51.033657 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:10:51.033613    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rmr22\" (UniqueName: \"kubernetes.io/projected/1faea3ab-644e-496a-849a-a6d769c4467a-kube-api-access-rmr22\") pod \"explicit-model-725649-kserve-57b4877bcf-qzcb2\" (UID: \"1faea3ab-644e-496a-849a-a6d769c4467a\") " pod="ai-tenant-e2e-explicit-725649/explicit-model-725649-kserve-57b4877bcf-qzcb2"
Sep 20 02:10:51.033657 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:10:51.033665    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1faea3ab-644e-496a-849a-a6d769c4467a-dshm\") pod \"explicit-model-725649-kserve-57b4877bcf-qzcb2\" (UID: \"1faea3ab-644e-496a-849a-a6d769c4467a\") " pod="ai-tenant-e2e-explicit-725649/explicit-model-725649-kserve-57b4877bcf-qzcb2"
Sep 20 02:10:51.033943 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:10:51.033689    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1faea3ab-644e-496a-849a-a6d769c4467a-home\") pod \"explicit-model-725649-kserve-57b4877bcf-qzcb2\" (UID: \"1faea3ab-644e-496a-849a-a6d769c4467a\") " pod="ai-tenant-e2e-explicit-725649/explicit-model-725649-kserve-57b4877bcf-qzcb2"
Sep 20 02:10:51.033943 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:10:51.033710    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1faea3ab-644e-496a-849a-a6d769c4467a-tmp-dir\") pod \"explicit-model-725649-kserve-57b4877bcf-qzcb2\" (UID: \"1faea3ab-644e-496a-849a-a6d769c4467a\") " pod="ai-tenant-e2e-explicit-725649/explicit-model-725649-kserve-57b4877bcf-qzcb2"
Sep 20 02:10:51.033943 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:10:51.033748    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1faea3ab-644e-496a-849a-a6d769c4467a-model-cache\") pod \"explicit-model-725649-kserve-57b4877bcf-qzcb2\" (UID: \"1faea3ab-644e-496a-849a-a6d769c4467a\") " pod="ai-tenant-e2e-explicit-725649/explicit-model-725649-kserve-57b4877bcf-qzcb2"
Sep 20 02:10:51.033943 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:10:51.033779    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1faea3ab-644e-496a-849a-a6d769c4467a-tls-certs\") pod \"explicit-model-725649-kserve-57b4877bcf-qzcb2\" (UID: \"1faea3ab-644e-496a-849a-a6d769c4467a\") " pod="ai-tenant-e2e-explicit-725649/explicit-model-725649-kserve-57b4877bcf-qzcb2"
Sep 20 02:10:51.034181 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:10:51.034158    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1faea3ab-644e-496a-849a-a6d769c4467a-tmp-dir\") pod \"explicit-model-725649-kserve-57b4877bcf-qzcb2\" (UID: \"1faea3ab-644e-496a-849a-a6d769c4467a\") " pod="ai-tenant-e2e-explicit-725649/explicit-model-725649-kserve-57b4877bcf-qzcb2"
Sep 20 02:10:51.034221 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:10:51.034192    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1faea3ab-644e-496a-849a-a6d769c4467a-home\") pod \"explicit-model-725649-kserve-57b4877bcf-qzcb2\" (UID: \"1faea3ab-644e-496a-849a-a6d769c4467a\") " pod="ai-tenant-e2e-explicit-725649/explicit-model-725649-kserve-57b4877bcf-qzcb2"
Sep 20 02:10:51.034256 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:10:51.034215    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1faea3ab-644e-496a-849a-a6d769c4467a-model-cache\") pod \"explicit-model-725649-kserve-57b4877bcf-qzcb2\" (UID: \"1faea3ab-644e-496a-849a-a6d769c4467a\") " pod="ai-tenant-e2e-explicit-725649/explicit-model-725649-kserve-57b4877bcf-qzcb2"
Sep 20 02:10:51.036203 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:10:51.036179    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1faea3ab-644e-496a-849a-a6d769c4467a-dshm\") pod \"explicit-model-725649-kserve-57b4877bcf-qzcb2\" (UID: \"1faea3ab-644e-496a-849a-a6d769c4467a\") " pod="ai-tenant-e2e-explicit-725649/explicit-model-725649-kserve-57b4877bcf-qzcb2"
Sep 20 02:10:51.036395 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:10:51.036377    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1faea3ab-644e-496a-849a-a6d769c4467a-tls-certs\") pod \"explicit-model-725649-kserve-57b4877bcf-qzcb2\" (UID: \"1faea3ab-644e-496a-849a-a6d769c4467a\") " pod="ai-tenant-e2e-explicit-725649/explicit-model-725649-kserve-57b4877bcf-qzcb2"
Sep 20 02:10:51.045558 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:10:51.042304    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rmr22\" (UniqueName: \"kubernetes.io/projected/1faea3ab-644e-496a-849a-a6d769c4467a-kube-api-access-rmr22\") pod \"explicit-model-725649-kserve-57b4877bcf-qzcb2\" (UID: \"1faea3ab-644e-496a-849a-a6d769c4467a\") " pod="ai-tenant-e2e-explicit-725649/explicit-model-725649-kserve-57b4877bcf-qzcb2"
Sep 20 02:10:51.127064 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:10:51.126978    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-725649/explicit-model-725649-kserve-57b4877bcf-qzcb2"
Sep 20 02:10:51.402913 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:10:51.402883    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-explicit-725649/explicit-model-725649-kserve-57b4877bcf-qzcb2"]
Sep 20 02:10:51.404749 ip-10-0-130-159 kubenswrapper[2584]: W0920 02:10:51.404714    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1faea3ab_644e_496a_849a_a6d769c4467a.slice/crio-7407d720a5422dc8ffe0a8cb6d158da1f816101950e1f26601800262ab74a69b WatchSource:0}: Error finding container 7407d720a5422dc8ffe0a8cb6d158da1f816101950e1f26601800262ab74a69b: Status 404 returned error can't find the container with id 7407d720a5422dc8ffe0a8cb6d158da1f816101950e1f26601800262ab74a69b
Sep 20 02:10:51.744464 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:10:51.744430    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-725649/explicit-model-725649-kserve-57b4877bcf-qzcb2" event={"ID":"1faea3ab-644e-496a-849a-a6d769c4467a","Type":"ContainerStarted","Data":"aff3e33e33914cac291c11861f1659122757ee71084842e68845b0a6bc965dd4"}
Sep 20 02:10:51.744675 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:10:51.744476    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-725649/explicit-model-725649-kserve-57b4877bcf-qzcb2" event={"ID":"1faea3ab-644e-496a-849a-a6d769c4467a","Type":"ContainerStarted","Data":"7407d720a5422dc8ffe0a8cb6d158da1f816101950e1f26601800262ab74a69b"}
Sep 20 02:10:51.762983 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:10:51.762933    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-explicit-725649/explicit-model-725649-kserve-57b4877bcf-qzcb2" podStartSLOduration=1.762917619 podStartE2EDuration="1.762917619s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-20 02:10:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-20 02:10:51.760463412 +0000 UTC m=+1108.503142617" watchObservedRunningTime="2026-09-20 02:10:51.762917619 +0000 UTC m=+1108.505596842"
Sep 20 02:10:59.291467 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:10:59.291428    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:11:01.128115 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:11:01.128071    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-explicit-725649/explicit-model-725649-kserve-57b4877bcf-qzcb2"
Sep 20 02:11:01.128542 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:11:01.128139    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-explicit-725649/explicit-model-725649-kserve-57b4877bcf-qzcb2"
Sep 20 02:11:01.142239 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:11:01.142208    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-explicit-725649/explicit-model-725649-kserve-57b4877bcf-qzcb2"
Sep 20 02:11:01.811462 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:11:01.811432    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-explicit-725649/explicit-model-725649-kserve-57b4877bcf-qzcb2"
Sep 20 02:11:05.009621 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:11:05.009588    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:11:05.117023 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:11:05.116983    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:11:05.209853 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:11:05.209817    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:11:11.639960 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:11:11.639921    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-explicit-725649/explicit-model-725649-kserve-57b4877bcf-qzcb2"]
Sep 20 02:11:11.662655 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:11:11.662624    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:11:12.741558 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:11:12.741498    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:11:13.541740 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:11:13.541700    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:11:14.277805 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:11:14.277773    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:11:15.061503 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:11:15.061472    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:11:15.723789 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:11:15.723750    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:11:15.824057 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:11:15.824023    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:11:16.443260 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:11:16.443222    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:11:16.544012 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:11:16.543981    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:11:22.305781 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:11:22.305745    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:11:22.397923 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:11:22.397882    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:11:22.490758 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:11:22.490715    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:11:22.573690 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:11:22.573603    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:11:22.719096 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:11:22.719063    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:11:22.967743 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:11:22.967711    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:11:23.226998 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:11:23.226914    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:11:23.452238 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:11:23.452168    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:11:23.551251 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:11:23.551138    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:11:23.649200 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:11:23.649167    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:11:24.252109 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:11:24.252073    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:11:25.054391 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:11:25.054363    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:11:25.160373 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:11:25.160317    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:11:25.745163 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:11:25.745129    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:11:26.587918 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:11:26.587883    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:11:26.662316 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:11:26.662267    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-explicit-725649/explicit-model-725649-kserve-57b4877bcf-qzcb2" podUID="1faea3ab-644e-496a-849a-a6d769c4467a" containerName="main" containerID="cri-o://aff3e33e33914cac291c11861f1659122757ee71084842e68845b0a6bc965dd4" gracePeriod=45
Sep 20 02:11:26.923053 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:11:26.923028    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-725649/explicit-model-725649-kserve-57b4877bcf-qzcb2"
Sep 20 02:11:26.952464 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:11:26.952431    2584 generic.go:356] "Generic (PLEG): container finished" podID="1faea3ab-644e-496a-849a-a6d769c4467a" containerID="aff3e33e33914cac291c11861f1659122757ee71084842e68845b0a6bc965dd4" exitCode=0
Sep 20 02:11:26.952654 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:11:26.952483    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-725649/explicit-model-725649-kserve-57b4877bcf-qzcb2" event={"ID":"1faea3ab-644e-496a-849a-a6d769c4467a","Type":"ContainerDied","Data":"aff3e33e33914cac291c11861f1659122757ee71084842e68845b0a6bc965dd4"}
Sep 20 02:11:26.952654 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:11:26.952484    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-725649/explicit-model-725649-kserve-57b4877bcf-qzcb2"
Sep 20 02:11:26.952654 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:11:26.952551    2584 scope.go:117] "RemoveContainer" containerID="aff3e33e33914cac291c11861f1659122757ee71084842e68845b0a6bc965dd4"
Sep 20 02:11:26.952822 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:11:26.952515    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-725649/explicit-model-725649-kserve-57b4877bcf-qzcb2" event={"ID":"1faea3ab-644e-496a-849a-a6d769c4467a","Type":"ContainerDied","Data":"7407d720a5422dc8ffe0a8cb6d158da1f816101950e1f26601800262ab74a69b"}
Sep 20 02:11:26.963425 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:11:26.963401    2584 scope.go:117] "RemoveContainer" containerID="aff3e33e33914cac291c11861f1659122757ee71084842e68845b0a6bc965dd4"
Sep 20 02:11:26.963809 ip-10-0-130-159 kubenswrapper[2584]: E0920 02:11:26.963783    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aff3e33e33914cac291c11861f1659122757ee71084842e68845b0a6bc965dd4\": container with ID starting with aff3e33e33914cac291c11861f1659122757ee71084842e68845b0a6bc965dd4 not found: ID does not exist" containerID="aff3e33e33914cac291c11861f1659122757ee71084842e68845b0a6bc965dd4"
Sep 20 02:11:26.963910 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:11:26.963821    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aff3e33e33914cac291c11861f1659122757ee71084842e68845b0a6bc965dd4"} err="failed to get container status \"aff3e33e33914cac291c11861f1659122757ee71084842e68845b0a6bc965dd4\": rpc error: code = NotFound desc = could not find container \"aff3e33e33914cac291c11861f1659122757ee71084842e68845b0a6bc965dd4\": container with ID starting with aff3e33e33914cac291c11861f1659122757ee71084842e68845b0a6bc965dd4 not found: ID does not exist"
Sep 20 02:11:26.997914 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:11:26.997882    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rmr22\" (UniqueName: \"kubernetes.io/projected/1faea3ab-644e-496a-849a-a6d769c4467a-kube-api-access-rmr22\") pod \"1faea3ab-644e-496a-849a-a6d769c4467a\" (UID: \"1faea3ab-644e-496a-849a-a6d769c4467a\") "
Sep 20 02:11:26.998087 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:11:26.997981    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1faea3ab-644e-496a-849a-a6d769c4467a-dshm\") pod \"1faea3ab-644e-496a-849a-a6d769c4467a\" (UID: \"1faea3ab-644e-496a-849a-a6d769c4467a\") "
Sep 20 02:11:26.998087 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:11:26.998027    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1faea3ab-644e-496a-849a-a6d769c4467a-home\") pod \"1faea3ab-644e-496a-849a-a6d769c4467a\" (UID: \"1faea3ab-644e-496a-849a-a6d769c4467a\") "
Sep 20 02:11:26.998087 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:11:26.998050    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1faea3ab-644e-496a-849a-a6d769c4467a-tls-certs\") pod \"1faea3ab-644e-496a-849a-a6d769c4467a\" (UID: \"1faea3ab-644e-496a-849a-a6d769c4467a\") "
Sep 20 02:11:26.998087 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:11:26.998079    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1faea3ab-644e-496a-849a-a6d769c4467a-tmp-dir\") pod \"1faea3ab-644e-496a-849a-a6d769c4467a\" (UID: \"1faea3ab-644e-496a-849a-a6d769c4467a\") "
Sep 20 02:11:26.998350 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:11:26.998168    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1faea3ab-644e-496a-849a-a6d769c4467a-model-cache\") pod \"1faea3ab-644e-496a-849a-a6d769c4467a\" (UID: \"1faea3ab-644e-496a-849a-a6d769c4467a\") "
Sep 20 02:11:26.998350 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:11:26.998275    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1faea3ab-644e-496a-849a-a6d769c4467a-home" (OuterVolumeSpecName: "home") pod "1faea3ab-644e-496a-849a-a6d769c4467a" (UID: "1faea3ab-644e-496a-849a-a6d769c4467a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 20 02:11:26.998464 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:11:26.998406    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1faea3ab-644e-496a-849a-a6d769c4467a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "1faea3ab-644e-496a-849a-a6d769c4467a" (UID: "1faea3ab-644e-496a-849a-a6d769c4467a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 20 02:11:26.998548 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:11:26.998500    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1faea3ab-644e-496a-849a-a6d769c4467a-model-cache" (OuterVolumeSpecName: "model-cache") pod "1faea3ab-644e-496a-849a-a6d769c4467a" (UID: "1faea3ab-644e-496a-849a-a6d769c4467a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 20 02:11:26.998623 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:11:26.998575    2584 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1faea3ab-644e-496a-849a-a6d769c4467a-home\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:11:26.998623 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:11:26.998593    2584 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1faea3ab-644e-496a-849a-a6d769c4467a-tmp-dir\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:11:27.000371 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:11:27.000331    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1faea3ab-644e-496a-849a-a6d769c4467a-dshm" (OuterVolumeSpecName: "dshm") pod "1faea3ab-644e-496a-849a-a6d769c4467a" (UID: "1faea3ab-644e-496a-849a-a6d769c4467a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 20 02:11:27.000494 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:11:27.000387    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1faea3ab-644e-496a-849a-a6d769c4467a-kube-api-access-rmr22" (OuterVolumeSpecName: "kube-api-access-rmr22") pod "1faea3ab-644e-496a-849a-a6d769c4467a" (UID: "1faea3ab-644e-496a-849a-a6d769c4467a"). InnerVolumeSpecName "kube-api-access-rmr22". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 20 02:11:27.000494 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:11:27.000407    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1faea3ab-644e-496a-849a-a6d769c4467a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "1faea3ab-644e-496a-849a-a6d769c4467a" (UID: "1faea3ab-644e-496a-849a-a6d769c4467a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 20 02:11:27.099067 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:11:27.099028    2584 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1faea3ab-644e-496a-849a-a6d769c4467a-dshm\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:11:27.099067 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:11:27.099057    2584 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1faea3ab-644e-496a-849a-a6d769c4467a-tls-certs\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:11:27.099067 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:11:27.099072    2584 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1faea3ab-644e-496a-849a-a6d769c4467a-model-cache\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:11:27.099304 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:11:27.099083    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rmr22\" (UniqueName: \"kubernetes.io/projected/1faea3ab-644e-496a-849a-a6d769c4467a-kube-api-access-rmr22\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:11:27.269537 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:11:27.269498    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-explicit-725649/explicit-model-725649-kserve-57b4877bcf-qzcb2"]
Sep 20 02:11:27.273211 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:11:27.273183    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-explicit-725649/explicit-model-725649-kserve-57b4877bcf-qzcb2"]
Sep 20 02:11:28.920902 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:11:28.920870    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:11:29.021902 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:11:29.021864    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:11:29.093401 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:11:29.093356    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1faea3ab-644e-496a-849a-a6d769c4467a" path="/var/lib/kubelet/pods/1faea3ab-644e-496a-849a-a6d769c4467a/volumes"
Sep 20 02:11:29.180988 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:11:29.180904    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:11:29.616648 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:11:29.616606    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:11:29.982866 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:11:29.982825    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:11:30.277871 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:11:30.277787    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:11:31.252669 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:11:31.252639    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:11:31.317256 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:11:31.317230    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:11:31.682603 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:11:31.682486    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:11:31.783469 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:11:31.783435    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:11:31.921814 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:11:31.921778    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:11:31.981834 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:11:31.981803    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:11:37.815974 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:11:37.815936    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:11:37.915760 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:11:37.915727    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:11:38.051748 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:11:38.051714    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:11:38.351626 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:11:38.351549    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:11:38.724444 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:11:38.724412    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:11:43.628980 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:11:43.628944    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:11:44.241955 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:11:44.241922    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:11:44.340576 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:11:44.340512    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:11:44.439745 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:11:44.439715    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:11:45.355350 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:11:45.355320    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:11:46.179588 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:11:46.179548    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:11:47.022292 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:11:47.022258    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:11:47.740723 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:11:47.740693    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:11:47.800954 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:11:47.800915    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:11:47.947499 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:11:47.947459    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:11:48.191414 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:11:48.191380    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:11:48.296270 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:11:48.296240    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:12:09.651473 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:12:09.651436    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:12:10.456670 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:12:10.456599    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:12:11.143568 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:12:11.143531    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:12:11.853734 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:12:11.853697    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:12:12.662400 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:12:12.662367    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:12:13.345696 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:12:13.345654    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:12:14.350482 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:12:14.350445    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:12:15.028260 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:12:15.028225    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:12:15.032911 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:12:15.032881    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-b15051/e2e-conflict-a-62c06db8-kserve-bf8cfc568-wzjn7"]
Sep 20 02:12:15.033315 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:12:15.033302    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1faea3ab-644e-496a-849a-a6d769c4467a" containerName="main"
Sep 20 02:12:15.033362 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:12:15.033316    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="1faea3ab-644e-496a-849a-a6d769c4467a" containerName="main"
Sep 20 02:12:15.033409 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:12:15.033387    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="1faea3ab-644e-496a-849a-a6d769c4467a" containerName="main"
Sep 20 02:12:15.036730 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:12:15.036711    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-b15051/e2e-conflict-a-62c06db8-kserve-bf8cfc568-wzjn7"
Sep 20 02:12:15.039859 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:12:15.039841    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-b15051\"/\"openshift-service-ca.crt\""
Sep 20 02:12:15.040837 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:12:15.040814    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-b15051\"/\"e2e-conflict-a-62c06db8-kserve-self-signed-certs\""
Sep 20 02:12:15.040954 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:12:15.040833    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-b15051\"/\"default-dockercfg-xgpxq\""
Sep 20 02:12:15.040954 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:12:15.040814    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-b15051\"/\"kube-root-ca.crt\""
Sep 20 02:12:15.049705 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:12:15.049678    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-b15051/e2e-conflict-a-62c06db8-kserve-bf8cfc568-wzjn7"]
Sep 20 02:12:15.107732 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:12:15.107695    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:12:15.179945 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:12:15.179901    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5xq67\" (UniqueName: \"kubernetes.io/projected/79805ee4-493b-4936-a8c9-ae0044f2f215-kube-api-access-5xq67\") pod \"e2e-conflict-a-62c06db8-kserve-bf8cfc568-wzjn7\" (UID: \"79805ee4-493b-4936-a8c9-ae0044f2f215\") " pod="e2e-models-e2e-worker-w4-b15051/e2e-conflict-a-62c06db8-kserve-bf8cfc568-wzjn7"
Sep 20 02:12:15.179945 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:12:15.179949    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/79805ee4-493b-4936-a8c9-ae0044f2f215-model-cache\") pod \"e2e-conflict-a-62c06db8-kserve-bf8cfc568-wzjn7\" (UID: \"79805ee4-493b-4936-a8c9-ae0044f2f215\") " pod="e2e-models-e2e-worker-w4-b15051/e2e-conflict-a-62c06db8-kserve-bf8cfc568-wzjn7"
Sep 20 02:12:15.180183 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:12:15.180001    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/79805ee4-493b-4936-a8c9-ae0044f2f215-dshm\") pod \"e2e-conflict-a-62c06db8-kserve-bf8cfc568-wzjn7\" (UID: \"79805ee4-493b-4936-a8c9-ae0044f2f215\") " pod="e2e-models-e2e-worker-w4-b15051/e2e-conflict-a-62c06db8-kserve-bf8cfc568-wzjn7"
Sep 20 02:12:15.180183 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:12:15.180046    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/79805ee4-493b-4936-a8c9-ae0044f2f215-tls-certs\") pod \"e2e-conflict-a-62c06db8-kserve-bf8cfc568-wzjn7\" (UID: \"79805ee4-493b-4936-a8c9-ae0044f2f215\") " pod="e2e-models-e2e-worker-w4-b15051/e2e-conflict-a-62c06db8-kserve-bf8cfc568-wzjn7"
Sep 20 02:12:15.180183 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:12:15.180104    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/79805ee4-493b-4936-a8c9-ae0044f2f215-home\") pod \"e2e-conflict-a-62c06db8-kserve-bf8cfc568-wzjn7\" (UID: \"79805ee4-493b-4936-a8c9-ae0044f2f215\") " pod="e2e-models-e2e-worker-w4-b15051/e2e-conflict-a-62c06db8-kserve-bf8cfc568-wzjn7"
Sep 20 02:12:15.180285 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:12:15.180197    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/79805ee4-493b-4936-a8c9-ae0044f2f215-tmp-dir\") pod \"e2e-conflict-a-62c06db8-kserve-bf8cfc568-wzjn7\" (UID: \"79805ee4-493b-4936-a8c9-ae0044f2f215\") " pod="e2e-models-e2e-worker-w4-b15051/e2e-conflict-a-62c06db8-kserve-bf8cfc568-wzjn7"
Sep 20 02:12:15.209444 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:12:15.209404    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:12:15.281152 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:12:15.281038    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/79805ee4-493b-4936-a8c9-ae0044f2f215-model-cache\") pod \"e2e-conflict-a-62c06db8-kserve-bf8cfc568-wzjn7\" (UID: \"79805ee4-493b-4936-a8c9-ae0044f2f215\") " pod="e2e-models-e2e-worker-w4-b15051/e2e-conflict-a-62c06db8-kserve-bf8cfc568-wzjn7"
Sep 20 02:12:15.281152 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:12:15.281085    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/79805ee4-493b-4936-a8c9-ae0044f2f215-dshm\") pod \"e2e-conflict-a-62c06db8-kserve-bf8cfc568-wzjn7\" (UID: \"79805ee4-493b-4936-a8c9-ae0044f2f215\") " pod="e2e-models-e2e-worker-w4-b15051/e2e-conflict-a-62c06db8-kserve-bf8cfc568-wzjn7"
Sep 20 02:12:15.281152 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:12:15.281108    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/79805ee4-493b-4936-a8c9-ae0044f2f215-tls-certs\") pod \"e2e-conflict-a-62c06db8-kserve-bf8cfc568-wzjn7\" (UID: \"79805ee4-493b-4936-a8c9-ae0044f2f215\") " pod="e2e-models-e2e-worker-w4-b15051/e2e-conflict-a-62c06db8-kserve-bf8cfc568-wzjn7"
Sep 20 02:12:15.281152 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:12:15.281136    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/79805ee4-493b-4936-a8c9-ae0044f2f215-home\") pod \"e2e-conflict-a-62c06db8-kserve-bf8cfc568-wzjn7\" (UID: \"79805ee4-493b-4936-a8c9-ae0044f2f215\") " pod="e2e-models-e2e-worker-w4-b15051/e2e-conflict-a-62c06db8-kserve-bf8cfc568-wzjn7"
Sep 20 02:12:15.281569 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:12:15.281215    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/79805ee4-493b-4936-a8c9-ae0044f2f215-tmp-dir\") pod \"e2e-conflict-a-62c06db8-kserve-bf8cfc568-wzjn7\" (UID: \"79805ee4-493b-4936-a8c9-ae0044f2f215\") " pod="e2e-models-e2e-worker-w4-b15051/e2e-conflict-a-62c06db8-kserve-bf8cfc568-wzjn7"
Sep 20 02:12:15.281569 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:12:15.281278    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5xq67\" (UniqueName: \"kubernetes.io/projected/79805ee4-493b-4936-a8c9-ae0044f2f215-kube-api-access-5xq67\") pod \"e2e-conflict-a-62c06db8-kserve-bf8cfc568-wzjn7\" (UID: \"79805ee4-493b-4936-a8c9-ae0044f2f215\") " pod="e2e-models-e2e-worker-w4-b15051/e2e-conflict-a-62c06db8-kserve-bf8cfc568-wzjn7"
Sep 20 02:12:15.281569 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:12:15.281483    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/79805ee4-493b-4936-a8c9-ae0044f2f215-model-cache\") pod \"e2e-conflict-a-62c06db8-kserve-bf8cfc568-wzjn7\" (UID: \"79805ee4-493b-4936-a8c9-ae0044f2f215\") " pod="e2e-models-e2e-worker-w4-b15051/e2e-conflict-a-62c06db8-kserve-bf8cfc568-wzjn7"
Sep 20 02:12:15.281711 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:12:15.281636    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/79805ee4-493b-4936-a8c9-ae0044f2f215-home\") pod \"e2e-conflict-a-62c06db8-kserve-bf8cfc568-wzjn7\" (UID: \"79805ee4-493b-4936-a8c9-ae0044f2f215\") " pod="e2e-models-e2e-worker-w4-b15051/e2e-conflict-a-62c06db8-kserve-bf8cfc568-wzjn7"
Sep 20 02:12:15.281750 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:12:15.281706    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/79805ee4-493b-4936-a8c9-ae0044f2f215-tmp-dir\") pod \"e2e-conflict-a-62c06db8-kserve-bf8cfc568-wzjn7\" (UID: \"79805ee4-493b-4936-a8c9-ae0044f2f215\") " pod="e2e-models-e2e-worker-w4-b15051/e2e-conflict-a-62c06db8-kserve-bf8cfc568-wzjn7"
Sep 20 02:12:15.283630 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:12:15.283598    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/79805ee4-493b-4936-a8c9-ae0044f2f215-dshm\") pod \"e2e-conflict-a-62c06db8-kserve-bf8cfc568-wzjn7\" (UID: \"79805ee4-493b-4936-a8c9-ae0044f2f215\") " pod="e2e-models-e2e-worker-w4-b15051/e2e-conflict-a-62c06db8-kserve-bf8cfc568-wzjn7"
Sep 20 02:12:15.283879 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:12:15.283855    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/79805ee4-493b-4936-a8c9-ae0044f2f215-tls-certs\") pod \"e2e-conflict-a-62c06db8-kserve-bf8cfc568-wzjn7\" (UID: \"79805ee4-493b-4936-a8c9-ae0044f2f215\") " pod="e2e-models-e2e-worker-w4-b15051/e2e-conflict-a-62c06db8-kserve-bf8cfc568-wzjn7"
Sep 20 02:12:15.290067 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:12:15.290037    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5xq67\" (UniqueName: \"kubernetes.io/projected/79805ee4-493b-4936-a8c9-ae0044f2f215-kube-api-access-5xq67\") pod \"e2e-conflict-a-62c06db8-kserve-bf8cfc568-wzjn7\" (UID: \"79805ee4-493b-4936-a8c9-ae0044f2f215\") " pod="e2e-models-e2e-worker-w4-b15051/e2e-conflict-a-62c06db8-kserve-bf8cfc568-wzjn7"
Sep 20 02:12:15.346631 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:12:15.346585    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-b15051/e2e-conflict-a-62c06db8-kserve-bf8cfc568-wzjn7"
Sep 20 02:12:15.542606 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:12:15.542481    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:12:15.633652 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:12:15.633618    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-b15051/e2e-conflict-a-62c06db8-kserve-bf8cfc568-wzjn7"]
Sep 20 02:12:15.635739 ip-10-0-130-159 kubenswrapper[2584]: W0920 02:12:15.635704    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod79805ee4_493b_4936_a8c9_ae0044f2f215.slice/crio-18c54230ad3ea54572cbdda62ee07d4a210fa1ad4a2eb84abc0a8a090b2ea131 WatchSource:0}: Error finding container 18c54230ad3ea54572cbdda62ee07d4a210fa1ad4a2eb84abc0a8a090b2ea131: Status 404 returned error can't find the container with id 18c54230ad3ea54572cbdda62ee07d4a210fa1ad4a2eb84abc0a8a090b2ea131
Sep 20 02:12:16.225049 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:12:16.225011    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-b15051/e2e-conflict-a-62c06db8-kserve-bf8cfc568-wzjn7" event={"ID":"79805ee4-493b-4936-a8c9-ae0044f2f215","Type":"ContainerStarted","Data":"528f92f9dfad55c6ebf66b683c9205c85c5da428d36c3301ace699afb08297b3"}
Sep 20 02:12:16.225267 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:12:16.225062    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-b15051/e2e-conflict-a-62c06db8-kserve-bf8cfc568-wzjn7" event={"ID":"79805ee4-493b-4936-a8c9-ae0044f2f215","Type":"ContainerStarted","Data":"18c54230ad3ea54572cbdda62ee07d4a210fa1ad4a2eb84abc0a8a090b2ea131"}
Sep 20 02:12:16.244481 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:12:16.244420    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-b15051/e2e-conflict-a-62c06db8-kserve-bf8cfc568-wzjn7" podStartSLOduration=1.244398867 podStartE2EDuration="1.244398867s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-20 02:12:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-20 02:12:16.242510288 +0000 UTC m=+1192.985189495" watchObservedRunningTime="2026-09-20 02:12:16.244398867 +0000 UTC m=+1192.987078074"
Sep 20 02:12:21.154973 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:12:21.154939    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:12:21.258036 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:12:21.257997    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:12:21.330677 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:12:21.330627    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:12:22.447092 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:12:22.447043    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:12:23.235925 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:12:23.235887    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:12:23.974718 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:12:23.974677    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:12:24.756031 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:12:24.755988    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:12:25.159606 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:12:25.159503    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-ztdt2_ece82f19-68b5-4810-be50-d8678cfdee02/console-operator/1.log"
Sep 20 02:12:25.165512 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:12:25.165491    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-ztdt2_ece82f19-68b5-4810-be50-d8678cfdee02/console-operator/1.log"
Sep 20 02:12:25.347016 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:12:25.346979    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-b15051/e2e-conflict-a-62c06db8-kserve-bf8cfc568-wzjn7"
Sep 20 02:12:25.347016 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:12:25.347022    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-b15051/e2e-conflict-a-62c06db8-kserve-bf8cfc568-wzjn7"
Sep 20 02:12:25.360364 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:12:25.360335    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-b15051/e2e-conflict-a-62c06db8-kserve-bf8cfc568-wzjn7"
Sep 20 02:12:25.542135 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:12:25.542097    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:12:26.217622 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:12:26.217574    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:12:26.293796 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:12:26.293764    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-b15051/e2e-conflict-a-62c06db8-kserve-bf8cfc568-wzjn7"
Sep 20 02:12:26.938627 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:12:26.938583    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:12:29.111913 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:12:29.111879    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-noten-1750e4/noten-model-1750e4-kserve-69967c4f79-kh4gq"]
Sep 20 02:12:29.115379 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:12:29.115358    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-1750e4/noten-model-1750e4-kserve-69967c4f79-kh4gq"
Sep 20 02:12:29.117771 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:12:29.117746    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-noten-1750e4\"/\"kube-root-ca.crt\""
Sep 20 02:12:29.117894 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:12:29.117844    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-noten-1750e4\"/\"openshift-service-ca.crt\""
Sep 20 02:12:29.118757 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:12:29.118735    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-noten-1750e4\"/\"default-dockercfg-7qpb6\""
Sep 20 02:12:29.118857 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:12:29.118737    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-noten-1750e4\"/\"noten-model-1750e4-kserve-self-signed-certs\""
Sep 20 02:12:29.124115 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:12:29.124087    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-noten-1750e4/noten-model-1750e4-kserve-69967c4f79-kh4gq"]
Sep 20 02:12:29.138896 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:12:29.138867    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:12:29.208396 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:12:29.208344    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/403c3382-04c0-482e-99e0-86c3d0eb8cf1-model-cache\") pod \"noten-model-1750e4-kserve-69967c4f79-kh4gq\" (UID: \"403c3382-04c0-482e-99e0-86c3d0eb8cf1\") " pod="ai-tenant-e2e-noten-1750e4/noten-model-1750e4-kserve-69967c4f79-kh4gq"
Sep 20 02:12:29.208627 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:12:29.208410    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/403c3382-04c0-482e-99e0-86c3d0eb8cf1-tls-certs\") pod \"noten-model-1750e4-kserve-69967c4f79-kh4gq\" (UID: \"403c3382-04c0-482e-99e0-86c3d0eb8cf1\") " pod="ai-tenant-e2e-noten-1750e4/noten-model-1750e4-kserve-69967c4f79-kh4gq"
Sep 20 02:12:29.208627 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:12:29.208503    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/403c3382-04c0-482e-99e0-86c3d0eb8cf1-dshm\") pod \"noten-model-1750e4-kserve-69967c4f79-kh4gq\" (UID: \"403c3382-04c0-482e-99e0-86c3d0eb8cf1\") " pod="ai-tenant-e2e-noten-1750e4/noten-model-1750e4-kserve-69967c4f79-kh4gq"
Sep 20 02:12:29.208711 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:12:29.208640    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/403c3382-04c0-482e-99e0-86c3d0eb8cf1-tmp-dir\") pod \"noten-model-1750e4-kserve-69967c4f79-kh4gq\" (UID: \"403c3382-04c0-482e-99e0-86c3d0eb8cf1\") " pod="ai-tenant-e2e-noten-1750e4/noten-model-1750e4-kserve-69967c4f79-kh4gq"
Sep 20 02:12:29.208711 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:12:29.208688    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/403c3382-04c0-482e-99e0-86c3d0eb8cf1-home\") pod \"noten-model-1750e4-kserve-69967c4f79-kh4gq\" (UID: \"403c3382-04c0-482e-99e0-86c3d0eb8cf1\") " pod="ai-tenant-e2e-noten-1750e4/noten-model-1750e4-kserve-69967c4f79-kh4gq"
Sep 20 02:12:29.208781 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:12:29.208713    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cpwk5\" (UniqueName: \"kubernetes.io/projected/403c3382-04c0-482e-99e0-86c3d0eb8cf1-kube-api-access-cpwk5\") pod \"noten-model-1750e4-kserve-69967c4f79-kh4gq\" (UID: \"403c3382-04c0-482e-99e0-86c3d0eb8cf1\") " pod="ai-tenant-e2e-noten-1750e4/noten-model-1750e4-kserve-69967c4f79-kh4gq"
Sep 20 02:12:29.309183 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:12:29.309146    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/403c3382-04c0-482e-99e0-86c3d0eb8cf1-dshm\") pod \"noten-model-1750e4-kserve-69967c4f79-kh4gq\" (UID: \"403c3382-04c0-482e-99e0-86c3d0eb8cf1\") " pod="ai-tenant-e2e-noten-1750e4/noten-model-1750e4-kserve-69967c4f79-kh4gq"
Sep 20 02:12:29.309402 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:12:29.309226    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/403c3382-04c0-482e-99e0-86c3d0eb8cf1-tmp-dir\") pod \"noten-model-1750e4-kserve-69967c4f79-kh4gq\" (UID: \"403c3382-04c0-482e-99e0-86c3d0eb8cf1\") " pod="ai-tenant-e2e-noten-1750e4/noten-model-1750e4-kserve-69967c4f79-kh4gq"
Sep 20 02:12:29.309402 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:12:29.309258    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/403c3382-04c0-482e-99e0-86c3d0eb8cf1-home\") pod \"noten-model-1750e4-kserve-69967c4f79-kh4gq\" (UID: \"403c3382-04c0-482e-99e0-86c3d0eb8cf1\") " pod="ai-tenant-e2e-noten-1750e4/noten-model-1750e4-kserve-69967c4f79-kh4gq"
Sep 20 02:12:29.309402 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:12:29.309274    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cpwk5\" (UniqueName: \"kubernetes.io/projected/403c3382-04c0-482e-99e0-86c3d0eb8cf1-kube-api-access-cpwk5\") pod \"noten-model-1750e4-kserve-69967c4f79-kh4gq\" (UID: \"403c3382-04c0-482e-99e0-86c3d0eb8cf1\") " pod="ai-tenant-e2e-noten-1750e4/noten-model-1750e4-kserve-69967c4f79-kh4gq"
Sep 20 02:12:29.309402 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:12:29.309313    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/403c3382-04c0-482e-99e0-86c3d0eb8cf1-model-cache\") pod \"noten-model-1750e4-kserve-69967c4f79-kh4gq\" (UID: \"403c3382-04c0-482e-99e0-86c3d0eb8cf1\") " pod="ai-tenant-e2e-noten-1750e4/noten-model-1750e4-kserve-69967c4f79-kh4gq"
Sep 20 02:12:29.309402 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:12:29.309365    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/403c3382-04c0-482e-99e0-86c3d0eb8cf1-tls-certs\") pod \"noten-model-1750e4-kserve-69967c4f79-kh4gq\" (UID: \"403c3382-04c0-482e-99e0-86c3d0eb8cf1\") " pod="ai-tenant-e2e-noten-1750e4/noten-model-1750e4-kserve-69967c4f79-kh4gq"
Sep 20 02:12:29.309731 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:12:29.309703    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/403c3382-04c0-482e-99e0-86c3d0eb8cf1-tmp-dir\") pod \"noten-model-1750e4-kserve-69967c4f79-kh4gq\" (UID: \"403c3382-04c0-482e-99e0-86c3d0eb8cf1\") " pod="ai-tenant-e2e-noten-1750e4/noten-model-1750e4-kserve-69967c4f79-kh4gq"
Sep 20 02:12:29.309784 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:12:29.309753    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/403c3382-04c0-482e-99e0-86c3d0eb8cf1-home\") pod \"noten-model-1750e4-kserve-69967c4f79-kh4gq\" (UID: \"403c3382-04c0-482e-99e0-86c3d0eb8cf1\") " pod="ai-tenant-e2e-noten-1750e4/noten-model-1750e4-kserve-69967c4f79-kh4gq"
Sep 20 02:12:29.309784 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:12:29.309774    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/403c3382-04c0-482e-99e0-86c3d0eb8cf1-model-cache\") pod \"noten-model-1750e4-kserve-69967c4f79-kh4gq\" (UID: \"403c3382-04c0-482e-99e0-86c3d0eb8cf1\") " pod="ai-tenant-e2e-noten-1750e4/noten-model-1750e4-kserve-69967c4f79-kh4gq"
Sep 20 02:12:29.311605 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:12:29.311584    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/403c3382-04c0-482e-99e0-86c3d0eb8cf1-dshm\") pod \"noten-model-1750e4-kserve-69967c4f79-kh4gq\" (UID: \"403c3382-04c0-482e-99e0-86c3d0eb8cf1\") " pod="ai-tenant-e2e-noten-1750e4/noten-model-1750e4-kserve-69967c4f79-kh4gq"
Sep 20 02:12:29.311786 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:12:29.311770    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/403c3382-04c0-482e-99e0-86c3d0eb8cf1-tls-certs\") pod \"noten-model-1750e4-kserve-69967c4f79-kh4gq\" (UID: \"403c3382-04c0-482e-99e0-86c3d0eb8cf1\") " pod="ai-tenant-e2e-noten-1750e4/noten-model-1750e4-kserve-69967c4f79-kh4gq"
Sep 20 02:12:29.316774 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:12:29.316743    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cpwk5\" (UniqueName: \"kubernetes.io/projected/403c3382-04c0-482e-99e0-86c3d0eb8cf1-kube-api-access-cpwk5\") pod \"noten-model-1750e4-kserve-69967c4f79-kh4gq\" (UID: \"403c3382-04c0-482e-99e0-86c3d0eb8cf1\") " pod="ai-tenant-e2e-noten-1750e4/noten-model-1750e4-kserve-69967c4f79-kh4gq"
Sep 20 02:12:29.426510 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:12:29.426402    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-1750e4/noten-model-1750e4-kserve-69967c4f79-kh4gq"
Sep 20 02:12:29.656979 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:12:29.656942    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:12:29.705725 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:12:29.705696    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-noten-1750e4/noten-model-1750e4-kserve-69967c4f79-kh4gq"]
Sep 20 02:12:29.707913 ip-10-0-130-159 kubenswrapper[2584]: W0920 02:12:29.707887    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod403c3382_04c0_482e_99e0_86c3d0eb8cf1.slice/crio-4e6fa9d478c7d08efe26b0e7dcbf255e37feec70c57c95172e1574d3f9030cad WatchSource:0}: Error finding container 4e6fa9d478c7d08efe26b0e7dcbf255e37feec70c57c95172e1574d3f9030cad: Status 404 returned error can't find the container with id 4e6fa9d478c7d08efe26b0e7dcbf255e37feec70c57c95172e1574d3f9030cad
Sep 20 02:12:30.307326 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:12:30.307291    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-1750e4/noten-model-1750e4-kserve-69967c4f79-kh4gq" event={"ID":"403c3382-04c0-482e-99e0-86c3d0eb8cf1","Type":"ContainerStarted","Data":"44b08cf51fd735a2d846d88cd9cb1318201e96adc36859fd9f1cf9d476c81476"}
Sep 20 02:12:30.307326 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:12:30.307335    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-1750e4/noten-model-1750e4-kserve-69967c4f79-kh4gq" event={"ID":"403c3382-04c0-482e-99e0-86c3d0eb8cf1","Type":"ContainerStarted","Data":"4e6fa9d478c7d08efe26b0e7dcbf255e37feec70c57c95172e1574d3f9030cad"}
Sep 20 02:12:30.323139 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:12:30.323090    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-noten-1750e4/noten-model-1750e4-kserve-69967c4f79-kh4gq" podStartSLOduration=1.323073628 podStartE2EDuration="1.323073628s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-20 02:12:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-20 02:12:30.322249437 +0000 UTC m=+1207.064928641" watchObservedRunningTime="2026-09-20 02:12:30.323073628 +0000 UTC m=+1207.065752832"
Sep 20 02:12:31.646556 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:12:31.646505    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:12:32.213254 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:12:32.213228    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:12:32.284598 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:12:32.284562    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:12:32.347336 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:12:32.347298    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:12:32.407359 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:12:32.407325    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:12:32.754276 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:12:32.754242    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:12:32.857852 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:12:32.857811    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:12:33.566761 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:12:33.566692    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:12:34.348799 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:12:34.348764    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:12:35.038026 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:12:35.037969    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:12:35.742476 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:12:35.742429    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:12:38.396783 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:12:38.396742    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:12:38.535370 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:12:38.535335    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:12:38.660041 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:12:38.659935    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:12:38.979542 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:12:38.979483    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:12:39.079082 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:12:39.079039    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:12:39.222190 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:12:39.222160    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:12:39.427424 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:12:39.427328    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-noten-1750e4/noten-model-1750e4-kserve-69967c4f79-kh4gq"
Sep 20 02:12:39.427424 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:12:39.427373    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-noten-1750e4/noten-model-1750e4-kserve-69967c4f79-kh4gq"
Sep 20 02:12:39.440659 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:12:39.440631    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-noten-1750e4/noten-model-1750e4-kserve-69967c4f79-kh4gq"
Sep 20 02:12:40.376969 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:12:40.376941    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-noten-1750e4/noten-model-1750e4-kserve-69967c4f79-kh4gq"
Sep 20 02:12:45.083456 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:12:45.083417    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:12:45.145674 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:12:45.145643    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:12:45.236855 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:12:45.236807    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:12:45.321915 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:12:45.321881    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:12:45.425335 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:12:45.425237    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:12:45.763286 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:12:45.763241    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:12:51.060219 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:12:51.060183    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:12:51.115189 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:12:51.115153    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:12:51.966907 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:12:51.966870    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:12:52.275198 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:12:52.275101    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:12:52.387870 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:12:52.387837    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:12:54.159478 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:12:54.159441    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:12:54.820887 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:12:54.820853    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:12:54.919228 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:12:54.919182    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:12:55.878981 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:12:55.878937    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:12:56.735147 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:12:56.735105    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:12:57.449532 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:12:57.449498    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:12:58.145925 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:12:58.145890    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:12:58.245716 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:12:58.245683    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:12:58.345469 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:12:58.345436    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:12:58.625634 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:12:58.625603    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:12:58.709950 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:12:58.709919    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:13:00.177734 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:00.177698    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:13:00.411978 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:00.411930    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:13:00.519345 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:00.519286    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:13:00.905613 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:00.905507    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:13:05.502082 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:05.502051    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-noten-1750e4/noten-model-1750e4-kserve-69967c4f79-kh4gq"]
Sep 20 02:13:05.597588 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:05.597539    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:13:05.695063 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:05.695024    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-04a2fd/test-model-04a2fd-kserve-7bd95f6bdc-nhf75"]
Sep 20 02:13:05.711626 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:05.711592    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:13:07.121664 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:07.121628    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:13:07.878948 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:07.878914    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:13:08.559662 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:08.559575    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:13:09.276047 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:09.276010    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:13:10.062080 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:10.061998    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:13:11.038702 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:11.038626    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:13:11.861273 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:11.861233    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:13:12.504802 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:12.504759    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:13:13.351668 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:13.351630    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:13:14.038652 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:14.038618    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:13:14.671106 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:14.671070    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:13:14.851381 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:14.851336    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:13:14.987626 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:14.987590    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:13:15.280733 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:15.280646    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:13:15.668589 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:15.668452    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:13:16.239511 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:16.239465    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:13:16.971107 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:16.971065    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:13:17.927709 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:17.927675    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:13:18.075999 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:18.075960    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:13:18.174727 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:18.174691    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:13:18.825355 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:18.825318    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:13:19.538641 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:19.538608    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:13:20.218595 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:20.218514    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:13:20.523741 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:20.523646    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-noten-1750e4/noten-model-1750e4-kserve-69967c4f79-kh4gq" podUID="403c3382-04c0-482e-99e0-86c3d0eb8cf1" containerName="main" containerID="cri-o://44b08cf51fd735a2d846d88cd9cb1318201e96adc36859fd9f1cf9d476c81476" gracePeriod=45
Sep 20 02:13:20.717840 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:20.717795    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-04a2fd/test-model-04a2fd-kserve-7bd95f6bdc-nhf75" podUID="ab8ecb10-3859-406f-b614-ad045a1f69a1" containerName="main" containerID="cri-o://d28457a04b3db308fa83726a923cb90458e44eabbe5b32b1f39699255c4134cf" gracePeriod=45
Sep 20 02:13:20.848536 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:20.848488    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-1750e4/noten-model-1750e4-kserve-69967c4f79-kh4gq"
Sep 20 02:13:20.944334 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:20.944307    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cpwk5\" (UniqueName: \"kubernetes.io/projected/403c3382-04c0-482e-99e0-86c3d0eb8cf1-kube-api-access-cpwk5\") pod \"403c3382-04c0-482e-99e0-86c3d0eb8cf1\" (UID: \"403c3382-04c0-482e-99e0-86c3d0eb8cf1\") "
Sep 20 02:13:20.944553 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:20.944365    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/403c3382-04c0-482e-99e0-86c3d0eb8cf1-tmp-dir\") pod \"403c3382-04c0-482e-99e0-86c3d0eb8cf1\" (UID: \"403c3382-04c0-482e-99e0-86c3d0eb8cf1\") "
Sep 20 02:13:20.944553 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:20.944417    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/403c3382-04c0-482e-99e0-86c3d0eb8cf1-model-cache\") pod \"403c3382-04c0-482e-99e0-86c3d0eb8cf1\" (UID: \"403c3382-04c0-482e-99e0-86c3d0eb8cf1\") "
Sep 20 02:13:20.944553 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:20.944446    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/403c3382-04c0-482e-99e0-86c3d0eb8cf1-tls-certs\") pod \"403c3382-04c0-482e-99e0-86c3d0eb8cf1\" (UID: \"403c3382-04c0-482e-99e0-86c3d0eb8cf1\") "
Sep 20 02:13:20.944553 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:20.944461    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/403c3382-04c0-482e-99e0-86c3d0eb8cf1-dshm\") pod \"403c3382-04c0-482e-99e0-86c3d0eb8cf1\" (UID: \"403c3382-04c0-482e-99e0-86c3d0eb8cf1\") "
Sep 20 02:13:20.944553 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:20.944539    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/403c3382-04c0-482e-99e0-86c3d0eb8cf1-home\") pod \"403c3382-04c0-482e-99e0-86c3d0eb8cf1\" (UID: \"403c3382-04c0-482e-99e0-86c3d0eb8cf1\") "
Sep 20 02:13:20.944990 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:20.944966    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/403c3382-04c0-482e-99e0-86c3d0eb8cf1-model-cache" (OuterVolumeSpecName: "model-cache") pod "403c3382-04c0-482e-99e0-86c3d0eb8cf1" (UID: "403c3382-04c0-482e-99e0-86c3d0eb8cf1"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 20 02:13:20.945059 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:20.944990    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/403c3382-04c0-482e-99e0-86c3d0eb8cf1-home" (OuterVolumeSpecName: "home") pod "403c3382-04c0-482e-99e0-86c3d0eb8cf1" (UID: "403c3382-04c0-482e-99e0-86c3d0eb8cf1"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 20 02:13:20.945118 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:20.945070    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/403c3382-04c0-482e-99e0-86c3d0eb8cf1-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "403c3382-04c0-482e-99e0-86c3d0eb8cf1" (UID: "403c3382-04c0-482e-99e0-86c3d0eb8cf1"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 20 02:13:20.947529 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:20.947493    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/403c3382-04c0-482e-99e0-86c3d0eb8cf1-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "403c3382-04c0-482e-99e0-86c3d0eb8cf1" (UID: "403c3382-04c0-482e-99e0-86c3d0eb8cf1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 20 02:13:20.948713 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:20.948675    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/403c3382-04c0-482e-99e0-86c3d0eb8cf1-dshm" (OuterVolumeSpecName: "dshm") pod "403c3382-04c0-482e-99e0-86c3d0eb8cf1" (UID: "403c3382-04c0-482e-99e0-86c3d0eb8cf1"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 20 02:13:20.948713 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:20.948696    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/403c3382-04c0-482e-99e0-86c3d0eb8cf1-kube-api-access-cpwk5" (OuterVolumeSpecName: "kube-api-access-cpwk5") pod "403c3382-04c0-482e-99e0-86c3d0eb8cf1" (UID: "403c3382-04c0-482e-99e0-86c3d0eb8cf1"). InnerVolumeSpecName "kube-api-access-cpwk5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 20 02:13:20.949608 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:20.949577    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:13:20.979397 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:20.979376    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-04a2fd/test-model-04a2fd-kserve-7bd95f6bdc-nhf75"
Sep 20 02:13:21.045397 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:21.045312    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ab8ecb10-3859-406f-b614-ad045a1f69a1-model-cache\") pod \"ab8ecb10-3859-406f-b614-ad045a1f69a1\" (UID: \"ab8ecb10-3859-406f-b614-ad045a1f69a1\") "
Sep 20 02:13:21.045397 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:21.045381    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ab8ecb10-3859-406f-b614-ad045a1f69a1-tmp-dir\") pod \"ab8ecb10-3859-406f-b614-ad045a1f69a1\" (UID: \"ab8ecb10-3859-406f-b614-ad045a1f69a1\") "
Sep 20 02:13:21.045657 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:21.045414    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ab8ecb10-3859-406f-b614-ad045a1f69a1-dshm\") pod \"ab8ecb10-3859-406f-b614-ad045a1f69a1\" (UID: \"ab8ecb10-3859-406f-b614-ad045a1f69a1\") "
Sep 20 02:13:21.045657 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:21.045451    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ab8ecb10-3859-406f-b614-ad045a1f69a1-home\") pod \"ab8ecb10-3859-406f-b614-ad045a1f69a1\" (UID: \"ab8ecb10-3859-406f-b614-ad045a1f69a1\") "
Sep 20 02:13:21.045657 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:21.045479    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ab8ecb10-3859-406f-b614-ad045a1f69a1-tls-certs\") pod \"ab8ecb10-3859-406f-b614-ad045a1f69a1\" (UID: \"ab8ecb10-3859-406f-b614-ad045a1f69a1\") "
Sep 20 02:13:21.045657 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:21.045553    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4s82c\" (UniqueName: \"kubernetes.io/projected/ab8ecb10-3859-406f-b614-ad045a1f69a1-kube-api-access-4s82c\") pod \"ab8ecb10-3859-406f-b614-ad045a1f69a1\" (UID: \"ab8ecb10-3859-406f-b614-ad045a1f69a1\") "
Sep 20 02:13:21.045657 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:21.045624    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab8ecb10-3859-406f-b614-ad045a1f69a1-model-cache" (OuterVolumeSpecName: "model-cache") pod "ab8ecb10-3859-406f-b614-ad045a1f69a1" (UID: "ab8ecb10-3859-406f-b614-ad045a1f69a1"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 20 02:13:21.045914 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:21.045722    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab8ecb10-3859-406f-b614-ad045a1f69a1-home" (OuterVolumeSpecName: "home") pod "ab8ecb10-3859-406f-b614-ad045a1f69a1" (UID: "ab8ecb10-3859-406f-b614-ad045a1f69a1"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 20 02:13:21.045914 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:21.045739    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab8ecb10-3859-406f-b614-ad045a1f69a1-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ab8ecb10-3859-406f-b614-ad045a1f69a1" (UID: "ab8ecb10-3859-406f-b614-ad045a1f69a1"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 20 02:13:21.045914 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:21.045904    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cpwk5\" (UniqueName: \"kubernetes.io/projected/403c3382-04c0-482e-99e0-86c3d0eb8cf1-kube-api-access-cpwk5\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:13:21.046027 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:21.045919    2584 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ab8ecb10-3859-406f-b614-ad045a1f69a1-tmp-dir\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:13:21.046027 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:21.045938    2584 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/403c3382-04c0-482e-99e0-86c3d0eb8cf1-tmp-dir\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:13:21.046027 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:21.045953    2584 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ab8ecb10-3859-406f-b614-ad045a1f69a1-home\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:13:21.046027 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:21.045968    2584 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/403c3382-04c0-482e-99e0-86c3d0eb8cf1-model-cache\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:13:21.046027 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:21.045981    2584 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/403c3382-04c0-482e-99e0-86c3d0eb8cf1-tls-certs\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:13:21.046027 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:21.045995    2584 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/403c3382-04c0-482e-99e0-86c3d0eb8cf1-dshm\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:13:21.046027 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:21.046008    2584 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/403c3382-04c0-482e-99e0-86c3d0eb8cf1-home\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:13:21.046027 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:21.046019    2584 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ab8ecb10-3859-406f-b614-ad045a1f69a1-model-cache\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:13:21.047775 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:21.047751    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ab8ecb10-3859-406f-b614-ad045a1f69a1-kube-api-access-4s82c" (OuterVolumeSpecName: "kube-api-access-4s82c") pod "ab8ecb10-3859-406f-b614-ad045a1f69a1" (UID: "ab8ecb10-3859-406f-b614-ad045a1f69a1"). InnerVolumeSpecName "kube-api-access-4s82c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 20 02:13:21.047870 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:21.047749    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab8ecb10-3859-406f-b614-ad045a1f69a1-dshm" (OuterVolumeSpecName: "dshm") pod "ab8ecb10-3859-406f-b614-ad045a1f69a1" (UID: "ab8ecb10-3859-406f-b614-ad045a1f69a1"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 20 02:13:21.047870 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:21.047787    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ab8ecb10-3859-406f-b614-ad045a1f69a1-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ab8ecb10-3859-406f-b614-ad045a1f69a1" (UID: "ab8ecb10-3859-406f-b614-ad045a1f69a1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 20 02:13:21.147901 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:21.147868    2584 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ab8ecb10-3859-406f-b614-ad045a1f69a1-dshm\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:13:21.147901 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:21.147901    2584 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ab8ecb10-3859-406f-b614-ad045a1f69a1-tls-certs\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:13:21.148115 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:21.147917    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4s82c\" (UniqueName: \"kubernetes.io/projected/ab8ecb10-3859-406f-b614-ad045a1f69a1-kube-api-access-4s82c\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:13:21.596204 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:21.596168    2584 generic.go:356] "Generic (PLEG): container finished" podID="403c3382-04c0-482e-99e0-86c3d0eb8cf1" containerID="44b08cf51fd735a2d846d88cd9cb1318201e96adc36859fd9f1cf9d476c81476" exitCode=0
Sep 20 02:13:21.596398 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:21.596224    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-1750e4/noten-model-1750e4-kserve-69967c4f79-kh4gq"
Sep 20 02:13:21.596398 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:21.596245    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-1750e4/noten-model-1750e4-kserve-69967c4f79-kh4gq" event={"ID":"403c3382-04c0-482e-99e0-86c3d0eb8cf1","Type":"ContainerDied","Data":"44b08cf51fd735a2d846d88cd9cb1318201e96adc36859fd9f1cf9d476c81476"}
Sep 20 02:13:21.596398 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:21.596283    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-1750e4/noten-model-1750e4-kserve-69967c4f79-kh4gq" event={"ID":"403c3382-04c0-482e-99e0-86c3d0eb8cf1","Type":"ContainerDied","Data":"4e6fa9d478c7d08efe26b0e7dcbf255e37feec70c57c95172e1574d3f9030cad"}
Sep 20 02:13:21.596398 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:21.596301    2584 scope.go:117] "RemoveContainer" containerID="44b08cf51fd735a2d846d88cd9cb1318201e96adc36859fd9f1cf9d476c81476"
Sep 20 02:13:21.597863 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:21.597837    2584 generic.go:356] "Generic (PLEG): container finished" podID="ab8ecb10-3859-406f-b614-ad045a1f69a1" containerID="d28457a04b3db308fa83726a923cb90458e44eabbe5b32b1f39699255c4134cf" exitCode=0
Sep 20 02:13:21.597993 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:21.597920    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-04a2fd/test-model-04a2fd-kserve-7bd95f6bdc-nhf75"
Sep 20 02:13:21.597993 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:21.597924    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-04a2fd/test-model-04a2fd-kserve-7bd95f6bdc-nhf75" event={"ID":"ab8ecb10-3859-406f-b614-ad045a1f69a1","Type":"ContainerDied","Data":"d28457a04b3db308fa83726a923cb90458e44eabbe5b32b1f39699255c4134cf"}
Sep 20 02:13:21.597993 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:21.597965    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-04a2fd/test-model-04a2fd-kserve-7bd95f6bdc-nhf75" event={"ID":"ab8ecb10-3859-406f-b614-ad045a1f69a1","Type":"ContainerDied","Data":"1a2c86605c79f37b4eb3ea855eef35873fc254fd94a150f264ab954949f68b9c"}
Sep 20 02:13:21.605760 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:21.605741    2584 scope.go:117] "RemoveContainer" containerID="44b08cf51fd735a2d846d88cd9cb1318201e96adc36859fd9f1cf9d476c81476"
Sep 20 02:13:21.606018 ip-10-0-130-159 kubenswrapper[2584]: E0920 02:13:21.606000    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"44b08cf51fd735a2d846d88cd9cb1318201e96adc36859fd9f1cf9d476c81476\": container with ID starting with 44b08cf51fd735a2d846d88cd9cb1318201e96adc36859fd9f1cf9d476c81476 not found: ID does not exist" containerID="44b08cf51fd735a2d846d88cd9cb1318201e96adc36859fd9f1cf9d476c81476"
Sep 20 02:13:21.606099 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:21.606026    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"44b08cf51fd735a2d846d88cd9cb1318201e96adc36859fd9f1cf9d476c81476"} err="failed to get container status \"44b08cf51fd735a2d846d88cd9cb1318201e96adc36859fd9f1cf9d476c81476\": rpc error: code = NotFound desc = could not find container \"44b08cf51fd735a2d846d88cd9cb1318201e96adc36859fd9f1cf9d476c81476\": container with ID starting with 44b08cf51fd735a2d846d88cd9cb1318201e96adc36859fd9f1cf9d476c81476 not found: ID does not exist"
Sep 20 02:13:21.606099 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:21.606042    2584 scope.go:117] "RemoveContainer" containerID="d28457a04b3db308fa83726a923cb90458e44eabbe5b32b1f39699255c4134cf"
Sep 20 02:13:21.614026 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:21.614009    2584 scope.go:117] "RemoveContainer" containerID="d28457a04b3db308fa83726a923cb90458e44eabbe5b32b1f39699255c4134cf"
Sep 20 02:13:21.614272 ip-10-0-130-159 kubenswrapper[2584]: E0920 02:13:21.614253    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d28457a04b3db308fa83726a923cb90458e44eabbe5b32b1f39699255c4134cf\": container with ID starting with d28457a04b3db308fa83726a923cb90458e44eabbe5b32b1f39699255c4134cf not found: ID does not exist" containerID="d28457a04b3db308fa83726a923cb90458e44eabbe5b32b1f39699255c4134cf"
Sep 20 02:13:21.614325 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:21.614278    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d28457a04b3db308fa83726a923cb90458e44eabbe5b32b1f39699255c4134cf"} err="failed to get container status \"d28457a04b3db308fa83726a923cb90458e44eabbe5b32b1f39699255c4134cf\": rpc error: code = NotFound desc = could not find container \"d28457a04b3db308fa83726a923cb90458e44eabbe5b32b1f39699255c4134cf\": container with ID starting with d28457a04b3db308fa83726a923cb90458e44eabbe5b32b1f39699255c4134cf not found: ID does not exist"
Sep 20 02:13:21.620504 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:21.620461    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-noten-1750e4/noten-model-1750e4-kserve-69967c4f79-kh4gq"]
Sep 20 02:13:21.622649 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:21.622628    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-noten-1750e4/noten-model-1750e4-kserve-69967c4f79-kh4gq"]
Sep 20 02:13:21.633240 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:21.633218    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-04a2fd/test-model-04a2fd-kserve-7bd95f6bdc-nhf75"]
Sep 20 02:13:21.638657 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:21.638633    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-04a2fd/test-model-04a2fd-kserve-7bd95f6bdc-nhf75"]
Sep 20 02:13:21.680296 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:21.680265    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-04a2fd-7c57fb68bb-lrlnw"]
Sep 20 02:13:21.680597 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:21.680570    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-04a2fd-7c57fb68bb-lrlnw" podUID="9108b468-ac9c-42e9-83e1-178dcc575f01" containerName="maas-api" containerID="cri-o://13f883c699ec73eb6a1be56fcb4986fd71f89a075dd1aad8ce3c2785f46ad9ea" gracePeriod=30
Sep 20 02:13:21.713285 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:21.713255    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:13:21.794047 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:21.793947    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:13:21.897822 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:21.897782    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:13:21.947182 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:21.947160    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-04a2fd-7c57fb68bb-lrlnw"
Sep 20 02:13:21.995347 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:21.995316    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:13:22.031611 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:22.031570    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-90811b-66749f8cf9-svlvk"]
Sep 20 02:13:22.032031 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:22.032014    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab8ecb10-3859-406f-b614-ad045a1f69a1" containerName="main"
Sep 20 02:13:22.032031 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:22.032030    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab8ecb10-3859-406f-b614-ad045a1f69a1" containerName="main"
Sep 20 02:13:22.032195 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:22.032039    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9108b468-ac9c-42e9-83e1-178dcc575f01" containerName="maas-api"
Sep 20 02:13:22.032195 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:22.032045    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="9108b468-ac9c-42e9-83e1-178dcc575f01" containerName="maas-api"
Sep 20 02:13:22.032195 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:22.032054    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="403c3382-04c0-482e-99e0-86c3d0eb8cf1" containerName="main"
Sep 20 02:13:22.032195 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:22.032059    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="403c3382-04c0-482e-99e0-86c3d0eb8cf1" containerName="main"
Sep 20 02:13:22.032195 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:22.032131    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="403c3382-04c0-482e-99e0-86c3d0eb8cf1" containerName="main"
Sep 20 02:13:22.032195 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:22.032140    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="ab8ecb10-3859-406f-b614-ad045a1f69a1" containerName="main"
Sep 20 02:13:22.032195 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:22.032149    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="9108b468-ac9c-42e9-83e1-178dcc575f01" containerName="maas-api"
Sep 20 02:13:22.035137 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:22.035115    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-90811b-66749f8cf9-svlvk"
Sep 20 02:13:22.037537 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:22.037500    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w2-90811b-serving-cert\""
Sep 20 02:13:22.048734 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:22.048666    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-90811b-66749f8cf9-svlvk"]
Sep 20 02:13:22.061102 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:22.061078    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9108b468-ac9c-42e9-83e1-178dcc575f01-maas-api-tls\") pod \"9108b468-ac9c-42e9-83e1-178dcc575f01\" (UID: \"9108b468-ac9c-42e9-83e1-178dcc575f01\") "
Sep 20 02:13:22.061377 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:22.061361    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q8nxp\" (UniqueName: \"kubernetes.io/projected/9108b468-ac9c-42e9-83e1-178dcc575f01-kube-api-access-q8nxp\") pod \"9108b468-ac9c-42e9-83e1-178dcc575f01\" (UID: \"9108b468-ac9c-42e9-83e1-178dcc575f01\") "
Sep 20 02:13:22.061895 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:22.061878    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9108b468-ac9c-42e9-83e1-178dcc575f01-metrics-certs\") pod \"9108b468-ac9c-42e9-83e1-178dcc575f01\" (UID: \"9108b468-ac9c-42e9-83e1-178dcc575f01\") "
Sep 20 02:13:22.062348 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:22.062333    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9108b468-ac9c-42e9-83e1-178dcc575f01-openshift-service-ca\") pod \"9108b468-ac9c-42e9-83e1-178dcc575f01\" (UID: \"9108b468-ac9c-42e9-83e1-178dcc575f01\") "
Sep 20 02:13:22.063038 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:22.062828    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9108b468-ac9c-42e9-83e1-178dcc575f01-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "9108b468-ac9c-42e9-83e1-178dcc575f01" (UID: "9108b468-ac9c-42e9-83e1-178dcc575f01"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 20 02:13:22.063413 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:22.063395    2584 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9108b468-ac9c-42e9-83e1-178dcc575f01-openshift-service-ca\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:13:22.068007 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:22.067979    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9108b468-ac9c-42e9-83e1-178dcc575f01-kube-api-access-q8nxp" (OuterVolumeSpecName: "kube-api-access-q8nxp") pod "9108b468-ac9c-42e9-83e1-178dcc575f01" (UID: "9108b468-ac9c-42e9-83e1-178dcc575f01"). InnerVolumeSpecName "kube-api-access-q8nxp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 20 02:13:22.068245 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:22.068219    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9108b468-ac9c-42e9-83e1-178dcc575f01-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "9108b468-ac9c-42e9-83e1-178dcc575f01" (UID: "9108b468-ac9c-42e9-83e1-178dcc575f01"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 20 02:13:22.068946 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:22.068912    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9108b468-ac9c-42e9-83e1-178dcc575f01-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "9108b468-ac9c-42e9-83e1-178dcc575f01" (UID: "9108b468-ac9c-42e9-83e1-178dcc575f01"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 20 02:13:22.080355 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:22.080327    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:13:22.139264 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:22.139230    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:13:22.164848 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:22.164813    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e906d174-1565-4ce9-819b-df16a4977b15-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-90811b-66749f8cf9-svlvk\" (UID: \"e906d174-1565-4ce9-819b-df16a4977b15\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-90811b-66749f8cf9-svlvk"
Sep 20 02:13:22.164848 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:22.164846    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k7jqw\" (UniqueName: \"kubernetes.io/projected/e906d174-1565-4ce9-819b-df16a4977b15-kube-api-access-k7jqw\") pod \"maas-api-e2e-shared-b-w2-90811b-66749f8cf9-svlvk\" (UID: \"e906d174-1565-4ce9-819b-df16a4977b15\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-90811b-66749f8cf9-svlvk"
Sep 20 02:13:22.165033 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:22.164943    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e906d174-1565-4ce9-819b-df16a4977b15-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-90811b-66749f8cf9-svlvk\" (UID: \"e906d174-1565-4ce9-819b-df16a4977b15\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-90811b-66749f8cf9-svlvk"
Sep 20 02:13:22.165033 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:22.164982    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e906d174-1565-4ce9-819b-df16a4977b15-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-90811b-66749f8cf9-svlvk\" (UID: \"e906d174-1565-4ce9-819b-df16a4977b15\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-90811b-66749f8cf9-svlvk"
Sep 20 02:13:22.165104 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:22.165089    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q8nxp\" (UniqueName: \"kubernetes.io/projected/9108b468-ac9c-42e9-83e1-178dcc575f01-kube-api-access-q8nxp\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:13:22.165139 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:22.165104    2584 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9108b468-ac9c-42e9-83e1-178dcc575f01-metrics-certs\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:13:22.165139 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:22.165120    2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9108b468-ac9c-42e9-83e1-178dcc575f01-maas-api-tls\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:13:22.265766 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:22.265728    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e906d174-1565-4ce9-819b-df16a4977b15-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-90811b-66749f8cf9-svlvk\" (UID: \"e906d174-1565-4ce9-819b-df16a4977b15\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-90811b-66749f8cf9-svlvk"
Sep 20 02:13:22.265766 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:22.265764    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k7jqw\" (UniqueName: \"kubernetes.io/projected/e906d174-1565-4ce9-819b-df16a4977b15-kube-api-access-k7jqw\") pod \"maas-api-e2e-shared-b-w2-90811b-66749f8cf9-svlvk\" (UID: \"e906d174-1565-4ce9-819b-df16a4977b15\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-90811b-66749f8cf9-svlvk"
Sep 20 02:13:22.266015 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:22.265838    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e906d174-1565-4ce9-819b-df16a4977b15-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-90811b-66749f8cf9-svlvk\" (UID: \"e906d174-1565-4ce9-819b-df16a4977b15\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-90811b-66749f8cf9-svlvk"
Sep 20 02:13:22.266015 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:22.265865    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e906d174-1565-4ce9-819b-df16a4977b15-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-90811b-66749f8cf9-svlvk\" (UID: \"e906d174-1565-4ce9-819b-df16a4977b15\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-90811b-66749f8cf9-svlvk"
Sep 20 02:13:22.266130 ip-10-0-130-159 kubenswrapper[2584]: E0920 02:13:22.266052    2584 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-90811b-serving-cert: secret "maas-api-e2e-shared-b-w2-90811b-serving-cert" not found
Sep 20 02:13:22.266185 ip-10-0-130-159 kubenswrapper[2584]: E0920 02:13:22.266134    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e906d174-1565-4ce9-819b-df16a4977b15-maas-api-tls podName:e906d174-1565-4ce9-819b-df16a4977b15 nodeName:}" failed. No retries permitted until 2026-09-20 02:13:22.766097555 +0000 UTC m=+1259.508776739 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e906d174-1565-4ce9-819b-df16a4977b15-maas-api-tls") pod "maas-api-e2e-shared-b-w2-90811b-66749f8cf9-svlvk" (UID: "e906d174-1565-4ce9-819b-df16a4977b15") : secret "maas-api-e2e-shared-b-w2-90811b-serving-cert" not found
Sep 20 02:13:22.266506 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:22.266477    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e906d174-1565-4ce9-819b-df16a4977b15-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-90811b-66749f8cf9-svlvk\" (UID: \"e906d174-1565-4ce9-819b-df16a4977b15\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-90811b-66749f8cf9-svlvk"
Sep 20 02:13:22.268439 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:22.268413    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e906d174-1565-4ce9-819b-df16a4977b15-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-90811b-66749f8cf9-svlvk\" (UID: \"e906d174-1565-4ce9-819b-df16a4977b15\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-90811b-66749f8cf9-svlvk"
Sep 20 02:13:22.276328 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:22.276304    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k7jqw\" (UniqueName: \"kubernetes.io/projected/e906d174-1565-4ce9-819b-df16a4977b15-kube-api-access-k7jqw\") pod \"maas-api-e2e-shared-b-w2-90811b-66749f8cf9-svlvk\" (UID: \"e906d174-1565-4ce9-819b-df16a4977b15\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-90811b-66749f8cf9-svlvk"
Sep 20 02:13:22.379928 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:22.379855    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:13:22.605991 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:22.605956    2584 generic.go:356] "Generic (PLEG): container finished" podID="9108b468-ac9c-42e9-83e1-178dcc575f01" containerID="13f883c699ec73eb6a1be56fcb4986fd71f89a075dd1aad8ce3c2785f46ad9ea" exitCode=0
Sep 20 02:13:22.606175 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:22.605998    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-04a2fd-7c57fb68bb-lrlnw"
Sep 20 02:13:22.606175 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:22.606024    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-04a2fd-7c57fb68bb-lrlnw" event={"ID":"9108b468-ac9c-42e9-83e1-178dcc575f01","Type":"ContainerDied","Data":"13f883c699ec73eb6a1be56fcb4986fd71f89a075dd1aad8ce3c2785f46ad9ea"}
Sep 20 02:13:22.606175 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:22.606070    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-04a2fd-7c57fb68bb-lrlnw" event={"ID":"9108b468-ac9c-42e9-83e1-178dcc575f01","Type":"ContainerDied","Data":"7466f46e58e78bd23c02176fc4f8fa914d3f60fdd8610bcf02259fd73e303092"}
Sep 20 02:13:22.606175 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:22.606093    2584 scope.go:117] "RemoveContainer" containerID="13f883c699ec73eb6a1be56fcb4986fd71f89a075dd1aad8ce3c2785f46ad9ea"
Sep 20 02:13:22.615892 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:22.615870    2584 scope.go:117] "RemoveContainer" containerID="13f883c699ec73eb6a1be56fcb4986fd71f89a075dd1aad8ce3c2785f46ad9ea"
Sep 20 02:13:22.616168 ip-10-0-130-159 kubenswrapper[2584]: E0920 02:13:22.616146    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"13f883c699ec73eb6a1be56fcb4986fd71f89a075dd1aad8ce3c2785f46ad9ea\": container with ID starting with 13f883c699ec73eb6a1be56fcb4986fd71f89a075dd1aad8ce3c2785f46ad9ea not found: ID does not exist" containerID="13f883c699ec73eb6a1be56fcb4986fd71f89a075dd1aad8ce3c2785f46ad9ea"
Sep 20 02:13:22.616272 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:22.616174    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"13f883c699ec73eb6a1be56fcb4986fd71f89a075dd1aad8ce3c2785f46ad9ea"} err="failed to get container status \"13f883c699ec73eb6a1be56fcb4986fd71f89a075dd1aad8ce3c2785f46ad9ea\": rpc error: code = NotFound desc = could not find container \"13f883c699ec73eb6a1be56fcb4986fd71f89a075dd1aad8ce3c2785f46ad9ea\": container with ID starting with 13f883c699ec73eb6a1be56fcb4986fd71f89a075dd1aad8ce3c2785f46ad9ea not found: ID does not exist"
Sep 20 02:13:22.627347 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:22.627322    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-04a2fd-7c57fb68bb-lrlnw"]
Sep 20 02:13:22.638193 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:22.638141    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-04a2fd-7c57fb68bb-lrlnw"]
Sep 20 02:13:22.687696 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:22.687668    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:13:22.771931 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:22.771902    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e906d174-1565-4ce9-819b-df16a4977b15-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-90811b-66749f8cf9-svlvk\" (UID: \"e906d174-1565-4ce9-819b-df16a4977b15\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-90811b-66749f8cf9-svlvk"
Sep 20 02:13:22.774508 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:22.774475    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e906d174-1565-4ce9-819b-df16a4977b15-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-90811b-66749f8cf9-svlvk\" (UID: \"e906d174-1565-4ce9-819b-df16a4977b15\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-90811b-66749f8cf9-svlvk"
Sep 20 02:13:22.789267 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:22.789241    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:13:22.865461 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:22.865413    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:13:22.949543 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:22.949488    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-90811b-66749f8cf9-svlvk"
Sep 20 02:13:23.094344 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:23.094311    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="403c3382-04c0-482e-99e0-86c3d0eb8cf1" path="/var/lib/kubelet/pods/403c3382-04c0-482e-99e0-86c3d0eb8cf1/volumes"
Sep 20 02:13:23.094827 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:23.094804    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9108b468-ac9c-42e9-83e1-178dcc575f01" path="/var/lib/kubelet/pods/9108b468-ac9c-42e9-83e1-178dcc575f01/volumes"
Sep 20 02:13:23.095248 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:23.095228    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ab8ecb10-3859-406f-b614-ad045a1f69a1" path="/var/lib/kubelet/pods/ab8ecb10-3859-406f-b614-ad045a1f69a1/volumes"
Sep 20 02:13:23.161815 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:23.161787    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:13:23.230184 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:23.230147    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-90811b-66749f8cf9-svlvk"]
Sep 20 02:13:23.231939 ip-10-0-130-159 kubenswrapper[2584]: W0920 02:13:23.231917    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode906d174_1565_4ce9_819b_df16a4977b15.slice/crio-7ee3aa233ef044aafc59762642f80c06af8f9ca6ab391addab2ced7aff8d4efd WatchSource:0}: Error finding container 7ee3aa233ef044aafc59762642f80c06af8f9ca6ab391addab2ced7aff8d4efd: Status 404 returned error can't find the container with id 7ee3aa233ef044aafc59762642f80c06af8f9ca6ab391addab2ced7aff8d4efd
Sep 20 02:13:23.233267 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:23.233251    2584 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 20 02:13:23.613811 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:23.613722    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-90811b-66749f8cf9-svlvk" event={"ID":"e906d174-1565-4ce9-819b-df16a4977b15","Type":"ContainerStarted","Data":"7ee3aa233ef044aafc59762642f80c06af8f9ca6ab391addab2ced7aff8d4efd"}
Sep 20 02:13:24.049943 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:24.047951    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:13:24.620002 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:24.619915    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-90811b-66749f8cf9-svlvk" event={"ID":"e906d174-1565-4ce9-819b-df16a4977b15","Type":"ContainerStarted","Data":"4b63083e7969e1f4c689c8b6c2de092272e1724bb7127fe77f8ab438cac311ea"}
Sep 20 02:13:24.620002 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:24.619991    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-90811b-66749f8cf9-svlvk"
Sep 20 02:13:24.644243 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:24.644194    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-90811b-66749f8cf9-svlvk" podStartSLOduration=1.578576772 podStartE2EDuration="2.644178306s" totalImagesPullingTime="1.065601534s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-20 02:13:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-20 02:13:24.642022623 +0000 UTC m=+1261.384701842" watchObservedRunningTime="2026-09-20 02:13:24.644178306 +0000 UTC m=+1261.386857511"
Sep 20 02:13:24.847354 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:24.847320    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:13:25.575417 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:25.575318    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:13:26.259276 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:26.259245    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:13:26.938781 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:26.938746    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:13:28.060675 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:28.060639    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:13:28.162495 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:28.162465    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:13:28.301134 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:28.301100    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-04a2fd-openshift-default-74d645d896-c89xp"]
Sep 20 02:13:28.301328 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:28.301307    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-04a2fd-openshift-default-74d645d896-c89xp" podUID="f8979a7e-a149-4c3f-a82c-32c19f551021" containerName="istio-proxy" containerID="cri-o://6710642a1127d57ffa1eda51b44e8f4b124fb8ccb0f401fff2cb321534aaa5d1" gracePeriod=30
Sep 20 02:13:28.320381 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:28.320290    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:13:28.947656 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:28.947625    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:13:29.731690 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:29.731657    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:13:30.631440 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:30.631415    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-90811b-66749f8cf9-svlvk"
Sep 20 02:13:30.639927 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:30.639902    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:13:31.332435 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:31.332398    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:13:32.124834 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:32.124803    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:13:32.774182 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:32.774149    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:13:32.874684 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:32.874653    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:13:33.544787 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:33.544756    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7531-edd643"]
Sep 20 02:13:33.548476 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:33.548455    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7531-edd643"
Sep 20 02:13:33.550724 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:33.550699    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-7l75z\""
Sep 20 02:13:33.551921 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:33.551897    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7531-edd643"]
Sep 20 02:13:33.568793 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:33.568687    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-04a2fd-openshift-default-74d645d896-c89xp"
Sep 20 02:13:33.584627 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:33.584598    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5ljq4\" (UniqueName: \"kubernetes.io/projected/f8979a7e-a149-4c3f-a82c-32c19f551021-kube-api-access-5ljq4\") pod \"f8979a7e-a149-4c3f-a82c-32c19f551021\" (UID: \"f8979a7e-a149-4c3f-a82c-32c19f551021\") "
Sep 20 02:13:33.584779 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:33.584644    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f8979a7e-a149-4c3f-a82c-32c19f551021-credential-socket\") pod \"f8979a7e-a149-4c3f-a82c-32c19f551021\" (UID: \"f8979a7e-a149-4c3f-a82c-32c19f551021\") "
Sep 20 02:13:33.584779 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:33.584675    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f8979a7e-a149-4c3f-a82c-32c19f551021-istio-token\") pod \"f8979a7e-a149-4c3f-a82c-32c19f551021\" (UID: \"f8979a7e-a149-4c3f-a82c-32c19f551021\") "
Sep 20 02:13:33.584779 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:33.584734    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f8979a7e-a149-4c3f-a82c-32c19f551021-istio-data\") pod \"f8979a7e-a149-4c3f-a82c-32c19f551021\" (UID: \"f8979a7e-a149-4c3f-a82c-32c19f551021\") "
Sep 20 02:13:33.584779 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:33.584769    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f8979a7e-a149-4c3f-a82c-32c19f551021-workload-socket\") pod \"f8979a7e-a149-4c3f-a82c-32c19f551021\" (UID: \"f8979a7e-a149-4c3f-a82c-32c19f551021\") "
Sep 20 02:13:33.584991 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:33.584793    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f8979a7e-a149-4c3f-a82c-32c19f551021-istio-envoy\") pod \"f8979a7e-a149-4c3f-a82c-32c19f551021\" (UID: \"f8979a7e-a149-4c3f-a82c-32c19f551021\") "
Sep 20 02:13:33.584991 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:33.584820    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f8979a7e-a149-4c3f-a82c-32c19f551021-istiod-ca-cert\") pod \"f8979a7e-a149-4c3f-a82c-32c19f551021\" (UID: \"f8979a7e-a149-4c3f-a82c-32c19f551021\") "
Sep 20 02:13:33.584991 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:33.584852    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f8979a7e-a149-4c3f-a82c-32c19f551021-istio-podinfo\") pod \"f8979a7e-a149-4c3f-a82c-32c19f551021\" (UID: \"f8979a7e-a149-4c3f-a82c-32c19f551021\") "
Sep 20 02:13:33.584991 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:33.584885    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f8979a7e-a149-4c3f-a82c-32c19f551021-workload-certs\") pod \"f8979a7e-a149-4c3f-a82c-32c19f551021\" (UID: \"f8979a7e-a149-4c3f-a82c-32c19f551021\") "
Sep 20 02:13:33.585200 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:33.584987    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f8979a7e-a149-4c3f-a82c-32c19f551021-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f8979a7e-a149-4c3f-a82c-32c19f551021" (UID: "f8979a7e-a149-4c3f-a82c-32c19f551021"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 20 02:13:33.585200 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:33.585020    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f8979a7e-a149-4c3f-a82c-32c19f551021-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f8979a7e-a149-4c3f-a82c-32c19f551021" (UID: "f8979a7e-a149-4c3f-a82c-32c19f551021"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 20 02:13:33.585200 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:33.585129    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xf4wl\" (UniqueName: \"kubernetes.io/projected/e5827699-49d7-433a-8102-e7e32eda4cf1-kube-api-access-xf4wl\") pod \"test-curl-7531-edd643\" (UID: \"e5827699-49d7-433a-8102-e7e32eda4cf1\") " pod="opendatahub/test-curl-7531-edd643"
Sep 20 02:13:33.585362 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:33.585213    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f8979a7e-a149-4c3f-a82c-32c19f551021-istio-data" (OuterVolumeSpecName: "istio-data") pod "f8979a7e-a149-4c3f-a82c-32c19f551021" (UID: "f8979a7e-a149-4c3f-a82c-32c19f551021"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 20 02:13:33.585362 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:33.585289    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f8979a7e-a149-4c3f-a82c-32c19f551021-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f8979a7e-a149-4c3f-a82c-32c19f551021" (UID: "f8979a7e-a149-4c3f-a82c-32c19f551021"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 20 02:13:33.585362 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:33.585302    2584 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f8979a7e-a149-4c3f-a82c-32c19f551021-istio-data\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:13:33.585362 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:33.585348    2584 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f8979a7e-a149-4c3f-a82c-32c19f551021-workload-socket\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:13:33.585599 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:33.585365    2584 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f8979a7e-a149-4c3f-a82c-32c19f551021-credential-socket\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:13:33.585599 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:33.585468    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f8979a7e-a149-4c3f-a82c-32c19f551021-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f8979a7e-a149-4c3f-a82c-32c19f551021" (UID: "f8979a7e-a149-4c3f-a82c-32c19f551021"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 20 02:13:33.587261 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:33.587237    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f8979a7e-a149-4c3f-a82c-32c19f551021-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f8979a7e-a149-4c3f-a82c-32c19f551021" (UID: "f8979a7e-a149-4c3f-a82c-32c19f551021"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 20 02:13:33.587462 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:33.587266    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f8979a7e-a149-4c3f-a82c-32c19f551021-istio-token" (OuterVolumeSpecName: "istio-token") pod "f8979a7e-a149-4c3f-a82c-32c19f551021" (UID: "f8979a7e-a149-4c3f-a82c-32c19f551021"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 20 02:13:33.587462 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:33.587349    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f8979a7e-a149-4c3f-a82c-32c19f551021-kube-api-access-5ljq4" (OuterVolumeSpecName: "kube-api-access-5ljq4") pod "f8979a7e-a149-4c3f-a82c-32c19f551021" (UID: "f8979a7e-a149-4c3f-a82c-32c19f551021"). InnerVolumeSpecName "kube-api-access-5ljq4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 20 02:13:33.588216 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:33.588191    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f8979a7e-a149-4c3f-a82c-32c19f551021-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f8979a7e-a149-4c3f-a82c-32c19f551021" (UID: "f8979a7e-a149-4c3f-a82c-32c19f551021"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 20 02:13:33.671583 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:33.671547    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:13:33.677408 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:33.677378    2584 generic.go:356] "Generic (PLEG): container finished" podID="f8979a7e-a149-4c3f-a82c-32c19f551021" containerID="6710642a1127d57ffa1eda51b44e8f4b124fb8ccb0f401fff2cb321534aaa5d1" exitCode=0
Sep 20 02:13:33.677547 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:33.677433    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-04a2fd-openshift-default-74d645d896-c89xp"
Sep 20 02:13:33.677547 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:33.677460    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-04a2fd-openshift-default-74d645d896-c89xp" event={"ID":"f8979a7e-a149-4c3f-a82c-32c19f551021","Type":"ContainerDied","Data":"6710642a1127d57ffa1eda51b44e8f4b124fb8ccb0f401fff2cb321534aaa5d1"}
Sep 20 02:13:33.677547 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:33.677512    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-04a2fd-openshift-default-74d645d896-c89xp" event={"ID":"f8979a7e-a149-4c3f-a82c-32c19f551021","Type":"ContainerDied","Data":"e216e8109032fdf5ca8f8fc06b5b941b6132ad0154f2661c163998bdf433c3a5"}
Sep 20 02:13:33.677701 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:33.677548    2584 scope.go:117] "RemoveContainer" containerID="6710642a1127d57ffa1eda51b44e8f4b124fb8ccb0f401fff2cb321534aaa5d1"
Sep 20 02:13:33.686168 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:33.685955    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xf4wl\" (UniqueName: \"kubernetes.io/projected/e5827699-49d7-433a-8102-e7e32eda4cf1-kube-api-access-xf4wl\") pod \"test-curl-7531-edd643\" (UID: \"e5827699-49d7-433a-8102-e7e32eda4cf1\") " pod="opendatahub/test-curl-7531-edd643"
Sep 20 02:13:33.686168 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:33.686124    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5ljq4\" (UniqueName: \"kubernetes.io/projected/f8979a7e-a149-4c3f-a82c-32c19f551021-kube-api-access-5ljq4\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:13:33.686168 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:33.686143    2584 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f8979a7e-a149-4c3f-a82c-32c19f551021-istio-token\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:13:33.686168 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:33.686158    2584 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f8979a7e-a149-4c3f-a82c-32c19f551021-istio-envoy\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:13:33.686411 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:33.686180    2584 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f8979a7e-a149-4c3f-a82c-32c19f551021-istiod-ca-cert\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:13:33.686411 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:33.686193    2584 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f8979a7e-a149-4c3f-a82c-32c19f551021-istio-podinfo\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:13:33.686411 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:33.686205    2584 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f8979a7e-a149-4c3f-a82c-32c19f551021-workload-certs\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:13:33.687956 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:33.687940    2584 scope.go:117] "RemoveContainer" containerID="fdd39165c51488978744128bbe1b15fa0a23677794ef5e669306b84089024620"
Sep 20 02:13:33.696715 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:33.696693    2584 scope.go:117] "RemoveContainer" containerID="6710642a1127d57ffa1eda51b44e8f4b124fb8ccb0f401fff2cb321534aaa5d1"
Sep 20 02:13:33.696874 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:33.696858    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xf4wl\" (UniqueName: \"kubernetes.io/projected/e5827699-49d7-433a-8102-e7e32eda4cf1-kube-api-access-xf4wl\") pod \"test-curl-7531-edd643\" (UID: \"e5827699-49d7-433a-8102-e7e32eda4cf1\") " pod="opendatahub/test-curl-7531-edd643"
Sep 20 02:13:33.696983 ip-10-0-130-159 kubenswrapper[2584]: E0920 02:13:33.696968    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6710642a1127d57ffa1eda51b44e8f4b124fb8ccb0f401fff2cb321534aaa5d1\": container with ID starting with 6710642a1127d57ffa1eda51b44e8f4b124fb8ccb0f401fff2cb321534aaa5d1 not found: ID does not exist" containerID="6710642a1127d57ffa1eda51b44e8f4b124fb8ccb0f401fff2cb321534aaa5d1"
Sep 20 02:13:33.697026 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:33.696990    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6710642a1127d57ffa1eda51b44e8f4b124fb8ccb0f401fff2cb321534aaa5d1"} err="failed to get container status \"6710642a1127d57ffa1eda51b44e8f4b124fb8ccb0f401fff2cb321534aaa5d1\": rpc error: code = NotFound desc = could not find container \"6710642a1127d57ffa1eda51b44e8f4b124fb8ccb0f401fff2cb321534aaa5d1\": container with ID starting with 6710642a1127d57ffa1eda51b44e8f4b124fb8ccb0f401fff2cb321534aaa5d1 not found: ID does not exist"
Sep 20 02:13:33.697026 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:33.697006    2584 scope.go:117] "RemoveContainer" containerID="fdd39165c51488978744128bbe1b15fa0a23677794ef5e669306b84089024620"
Sep 20 02:13:33.697255 ip-10-0-130-159 kubenswrapper[2584]: E0920 02:13:33.697239    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fdd39165c51488978744128bbe1b15fa0a23677794ef5e669306b84089024620\": container with ID starting with fdd39165c51488978744128bbe1b15fa0a23677794ef5e669306b84089024620 not found: ID does not exist" containerID="fdd39165c51488978744128bbe1b15fa0a23677794ef5e669306b84089024620"
Sep 20 02:13:33.697293 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:33.697262    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fdd39165c51488978744128bbe1b15fa0a23677794ef5e669306b84089024620"} err="failed to get container status \"fdd39165c51488978744128bbe1b15fa0a23677794ef5e669306b84089024620\": rpc error: code = NotFound desc = could not find container \"fdd39165c51488978744128bbe1b15fa0a23677794ef5e669306b84089024620\": container with ID starting with fdd39165c51488978744128bbe1b15fa0a23677794ef5e669306b84089024620 not found: ID does not exist"
Sep 20 02:13:33.703600 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:33.703577    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-04a2fd-openshift-default-74d645d896-c89xp"]
Sep 20 02:13:33.707401 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:33.707382    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-04a2fd-openshift-default-74d645d896-c89xp"]
Sep 20 02:13:33.875202 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:33.875123    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7531-edd643"
Sep 20 02:13:34.146158 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:34.146132    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7531-edd643"]
Sep 20 02:13:34.148149 ip-10-0-130-159 kubenswrapper[2584]: W0920 02:13:34.148121    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode5827699_49d7_433a_8102_e7e32eda4cf1.slice/crio-7496d977bf4e5508ee0f527ebb00cc13c63bfdae7620377bc28503bf63cb1380 WatchSource:0}: Error finding container 7496d977bf4e5508ee0f527ebb00cc13c63bfdae7620377bc28503bf63cb1380: Status 404 returned error can't find the container with id 7496d977bf4e5508ee0f527ebb00cc13c63bfdae7620377bc28503bf63cb1380
Sep 20 02:13:34.461794 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:34.461758    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:13:34.683924 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:34.683881    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7531-edd643" event={"ID":"e5827699-49d7-433a-8102-e7e32eda4cf1","Type":"ContainerStarted","Data":"7496d977bf4e5508ee0f527ebb00cc13c63bfdae7620377bc28503bf63cb1380"}
Sep 20 02:13:35.095451 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:35.095416    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f8979a7e-a149-4c3f-a82c-32c19f551021" path="/var/lib/kubelet/pods/f8979a7e-a149-4c3f-a82c-32c19f551021/volumes"
Sep 20 02:13:35.104432 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:35.104383    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:13:35.839196 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:35.839160    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:13:36.550370 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:36.550341    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:13:37.704878 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:37.704840    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7531-edd643" event={"ID":"e5827699-49d7-433a-8102-e7e32eda4cf1","Type":"ContainerStarted","Data":"7e60589723c3144a628245ade19715ea40906020518f13c16913ea170224022d"}
Sep 20 02:13:37.719054 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:37.719009    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7531-edd643" podStartSLOduration=1.5645786739999998 podStartE2EDuration="4.718994026s" totalImagesPullingTime="3.154415352s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-20 02:13:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-20 02:13:37.716883987 +0000 UTC m=+1274.459563192" watchObservedRunningTime="2026-09-20 02:13:37.718994026 +0000 UTC m=+1274.461673231"
Sep 20 02:13:38.303591 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:38.303555    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7531-edd643"]
Sep 20 02:13:38.307495 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:38.307464    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7531-edd643"]
Sep 20 02:13:38.432725 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:38.432648    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7531-51e133"]
Sep 20 02:13:38.433194 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:38.433179    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f8979a7e-a149-4c3f-a82c-32c19f551021" containerName="istio-proxy"
Sep 20 02:13:38.433247 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:38.433196    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="f8979a7e-a149-4c3f-a82c-32c19f551021" containerName="istio-proxy"
Sep 20 02:13:38.433247 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:38.433205    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e5827699-49d7-433a-8102-e7e32eda4cf1" containerName="test-curl-7531-edd643"
Sep 20 02:13:38.433247 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:38.433211    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="e5827699-49d7-433a-8102-e7e32eda4cf1" containerName="test-curl-7531-edd643"
Sep 20 02:13:38.433247 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:38.433226    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f8979a7e-a149-4c3f-a82c-32c19f551021" containerName="istio-proxy"
Sep 20 02:13:38.433247 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:38.433231    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="f8979a7e-a149-4c3f-a82c-32c19f551021" containerName="istio-proxy"
Sep 20 02:13:38.433392 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:38.433293    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="f8979a7e-a149-4c3f-a82c-32c19f551021" containerName="istio-proxy"
Sep 20 02:13:38.433392 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:38.433301    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="e5827699-49d7-433a-8102-e7e32eda4cf1" containerName="test-curl-7531-edd643"
Sep 20 02:13:38.433392 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:38.433307    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="f8979a7e-a149-4c3f-a82c-32c19f551021" containerName="istio-proxy"
Sep 20 02:13:38.436534 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:38.436508    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7531-51e133"
Sep 20 02:13:38.440305 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:38.440285    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7531-51e133"]
Sep 20 02:13:38.534391 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:38.534354    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-plhwh\" (UniqueName: \"kubernetes.io/projected/4fb35884-ebe0-44fa-8878-f4756de75298-kube-api-access-plhwh\") pod \"test-curl-7531-51e133\" (UID: \"4fb35884-ebe0-44fa-8878-f4756de75298\") " pod="opendatahub/test-curl-7531-51e133"
Sep 20 02:13:38.635837 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:38.635805    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-plhwh\" (UniqueName: \"kubernetes.io/projected/4fb35884-ebe0-44fa-8878-f4756de75298-kube-api-access-plhwh\") pod \"test-curl-7531-51e133\" (UID: \"4fb35884-ebe0-44fa-8878-f4756de75298\") " pod="opendatahub/test-curl-7531-51e133"
Sep 20 02:13:38.644039 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:38.644014    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-plhwh\" (UniqueName: \"kubernetes.io/projected/4fb35884-ebe0-44fa-8878-f4756de75298-kube-api-access-plhwh\") pod \"test-curl-7531-51e133\" (UID: \"4fb35884-ebe0-44fa-8878-f4756de75298\") " pod="opendatahub/test-curl-7531-51e133"
Sep 20 02:13:38.711397 ip-10-0-130-159 kubenswrapper[2584]: E0920 02:13:38.711367    2584 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7531-edd643" container="test-curl-7531-edd643"
Sep 20 02:13:38.748303 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:38.748277    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7531-51e133"
Sep 20 02:13:38.830723 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:38.830693    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:13:39.021101 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:39.021076    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7531-51e133"]
Sep 20 02:13:39.022288 ip-10-0-130-159 kubenswrapper[2584]: W0920 02:13:39.022262    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4fb35884_ebe0_44fa_8878_f4756de75298.slice/crio-3350b338e55dbb42fa89f6ec3a06ea423bb17bc5725b0f43610aebf69a3545f1 WatchSource:0}: Error finding container 3350b338e55dbb42fa89f6ec3a06ea423bb17bc5725b0f43610aebf69a3545f1: Status 404 returned error can't find the container with id 3350b338e55dbb42fa89f6ec3a06ea423bb17bc5725b0f43610aebf69a3545f1
Sep 20 02:13:39.369756 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:39.369673    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-703998-6755f65cb6-sbmsr"]
Sep 20 02:13:39.369937 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:39.369848    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-703998-6755f65cb6-sbmsr" podUID="b8656bfb-4ba7-47df-ad3f-fde87101cf1b" containerName="maas-api" containerID="cri-o://1f435424e044a7c4376fd966889e3f469a4502c992727176c02549ba3e7810a8" gracePeriod=30
Sep 20 02:13:39.407108 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:39.407078    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:13:39.429259 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:39.429228    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-703998-56676dd7-vhtqd"]
Sep 20 02:13:39.429493 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:39.429454    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-703998-56676dd7-vhtqd" podUID="aae1344b-f684-4f6d-8c9f-eb4353885f54" containerName="payload-processing" containerID="cri-o://204634417f4bdd38deea31f57ba3b40d357dc60af558e7434c70d5ad9e9a928e" gracePeriod=30
Sep 20 02:13:39.508630 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:39.508597    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:13:39.596505 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:39.596408    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:13:39.653454 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:39.652931    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:13:39.692933 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:39.692912    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-703998-6755f65cb6-sbmsr"
Sep 20 02:13:39.721944 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:39.721911    2584 generic.go:356] "Generic (PLEG): container finished" podID="b8656bfb-4ba7-47df-ad3f-fde87101cf1b" containerID="1f435424e044a7c4376fd966889e3f469a4502c992727176c02549ba3e7810a8" exitCode=0
Sep 20 02:13:39.722329 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:39.722054    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-703998-6755f65cb6-sbmsr" event={"ID":"b8656bfb-4ba7-47df-ad3f-fde87101cf1b","Type":"ContainerDied","Data":"1f435424e044a7c4376fd966889e3f469a4502c992727176c02549ba3e7810a8"}
Sep 20 02:13:39.722329 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:39.722139    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-703998-6755f65cb6-sbmsr" event={"ID":"b8656bfb-4ba7-47df-ad3f-fde87101cf1b","Type":"ContainerDied","Data":"a1a441728bd3593094c3bdd4c57347e871750a7be745c89ceac9c29de69116ec"}
Sep 20 02:13:39.722329 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:39.722172    2584 scope.go:117] "RemoveContainer" containerID="1f435424e044a7c4376fd966889e3f469a4502c992727176c02549ba3e7810a8"
Sep 20 02:13:39.722329 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:39.722323    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-703998-6755f65cb6-sbmsr"
Sep 20 02:13:39.724885 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:39.724857    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7531-51e133" event={"ID":"4fb35884-ebe0-44fa-8878-f4756de75298","Type":"ContainerStarted","Data":"066724bd0b104c5bf5c743192a726265b7bacd3053b5cc435d8304f8b731809a"}
Sep 20 02:13:39.725008 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:39.724895    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7531-51e133" event={"ID":"4fb35884-ebe0-44fa-8878-f4756de75298","Type":"ContainerStarted","Data":"3350b338e55dbb42fa89f6ec3a06ea423bb17bc5725b0f43610aebf69a3545f1"}
Sep 20 02:13:39.726621 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:39.726502    2584 generic.go:356] "Generic (PLEG): container finished" podID="aae1344b-f684-4f6d-8c9f-eb4353885f54" containerID="204634417f4bdd38deea31f57ba3b40d357dc60af558e7434c70d5ad9e9a928e" exitCode=0
Sep 20 02:13:39.726621 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:39.726561    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-703998-56676dd7-vhtqd" event={"ID":"aae1344b-f684-4f6d-8c9f-eb4353885f54","Type":"ContainerDied","Data":"204634417f4bdd38deea31f57ba3b40d357dc60af558e7434c70d5ad9e9a928e"}
Sep 20 02:13:39.726923 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:39.726896    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7531-edd643" podUID="e5827699-49d7-433a-8102-e7e32eda4cf1" containerName="test-curl-7531-edd643" containerID="cri-o://7e60589723c3144a628245ade19715ea40906020518f13c16913ea170224022d" gracePeriod=2
Sep 20 02:13:39.736448 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:39.736426    2584 scope.go:117] "RemoveContainer" containerID="1f435424e044a7c4376fd966889e3f469a4502c992727176c02549ba3e7810a8"
Sep 20 02:13:39.736758 ip-10-0-130-159 kubenswrapper[2584]: E0920 02:13:39.736735    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1f435424e044a7c4376fd966889e3f469a4502c992727176c02549ba3e7810a8\": container with ID starting with 1f435424e044a7c4376fd966889e3f469a4502c992727176c02549ba3e7810a8 not found: ID does not exist" containerID="1f435424e044a7c4376fd966889e3f469a4502c992727176c02549ba3e7810a8"
Sep 20 02:13:39.736829 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:39.736769    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f435424e044a7c4376fd966889e3f469a4502c992727176c02549ba3e7810a8"} err="failed to get container status \"1f435424e044a7c4376fd966889e3f469a4502c992727176c02549ba3e7810a8\": rpc error: code = NotFound desc = could not find container \"1f435424e044a7c4376fd966889e3f469a4502c992727176c02549ba3e7810a8\": container with ID starting with 1f435424e044a7c4376fd966889e3f469a4502c992727176c02549ba3e7810a8 not found: ID does not exist"
Sep 20 02:13:39.739405 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:39.739387    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-703998-56676dd7-vhtqd"
Sep 20 02:13:39.739940 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:39.739894    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7531-51e133" podStartSLOduration=1.739875707 podStartE2EDuration="1.739875707s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-20 02:13:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-20 02:13:39.73871231 +0000 UTC m=+1276.481391515" watchObservedRunningTime="2026-09-20 02:13:39.739875707 +0000 UTC m=+1276.482554913"
Sep 20 02:13:39.747799 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:39.747778    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b8656bfb-4ba7-47df-ad3f-fde87101cf1b-maas-api-tls\") pod \"b8656bfb-4ba7-47df-ad3f-fde87101cf1b\" (UID: \"b8656bfb-4ba7-47df-ad3f-fde87101cf1b\") "
Sep 20 02:13:39.747877 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:39.747860    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b8656bfb-4ba7-47df-ad3f-fde87101cf1b-openshift-service-ca\") pod \"b8656bfb-4ba7-47df-ad3f-fde87101cf1b\" (UID: \"b8656bfb-4ba7-47df-ad3f-fde87101cf1b\") "
Sep 20 02:13:39.747937 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:39.747895    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wck2j\" (UniqueName: \"kubernetes.io/projected/b8656bfb-4ba7-47df-ad3f-fde87101cf1b-kube-api-access-wck2j\") pod \"b8656bfb-4ba7-47df-ad3f-fde87101cf1b\" (UID: \"b8656bfb-4ba7-47df-ad3f-fde87101cf1b\") "
Sep 20 02:13:39.747997 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:39.747976    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b8656bfb-4ba7-47df-ad3f-fde87101cf1b-metrics-certs\") pod \"b8656bfb-4ba7-47df-ad3f-fde87101cf1b\" (UID: \"b8656bfb-4ba7-47df-ad3f-fde87101cf1b\") "
Sep 20 02:13:39.748299 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:39.748196    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b8656bfb-4ba7-47df-ad3f-fde87101cf1b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b8656bfb-4ba7-47df-ad3f-fde87101cf1b" (UID: "b8656bfb-4ba7-47df-ad3f-fde87101cf1b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 20 02:13:39.748374 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:39.748314    2584 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b8656bfb-4ba7-47df-ad3f-fde87101cf1b-openshift-service-ca\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:13:39.749783 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:39.749764    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b8656bfb-4ba7-47df-ad3f-fde87101cf1b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b8656bfb-4ba7-47df-ad3f-fde87101cf1b" (UID: "b8656bfb-4ba7-47df-ad3f-fde87101cf1b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 20 02:13:39.749941 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:39.749922    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b8656bfb-4ba7-47df-ad3f-fde87101cf1b-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "b8656bfb-4ba7-47df-ad3f-fde87101cf1b" (UID: "b8656bfb-4ba7-47df-ad3f-fde87101cf1b"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 20 02:13:39.750314 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:39.750295    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b8656bfb-4ba7-47df-ad3f-fde87101cf1b-kube-api-access-wck2j" (OuterVolumeSpecName: "kube-api-access-wck2j") pod "b8656bfb-4ba7-47df-ad3f-fde87101cf1b" (UID: "b8656bfb-4ba7-47df-ad3f-fde87101cf1b"). InnerVolumeSpecName "kube-api-access-wck2j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 20 02:13:39.849285 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:39.849253    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aae1344b-f684-4f6d-8c9f-eb4353885f54-plugins-config-volume\") pod \"aae1344b-f684-4f6d-8c9f-eb4353885f54\" (UID: \"aae1344b-f684-4f6d-8c9f-eb4353885f54\") "
Sep 20 02:13:39.849480 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:39.849293    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fjn62\" (UniqueName: \"kubernetes.io/projected/aae1344b-f684-4f6d-8c9f-eb4353885f54-kube-api-access-fjn62\") pod \"aae1344b-f684-4f6d-8c9f-eb4353885f54\" (UID: \"aae1344b-f684-4f6d-8c9f-eb4353885f54\") "
Sep 20 02:13:39.849480 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:39.849446    2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b8656bfb-4ba7-47df-ad3f-fde87101cf1b-maas-api-tls\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:13:39.849480 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:39.849457    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wck2j\" (UniqueName: \"kubernetes.io/projected/b8656bfb-4ba7-47df-ad3f-fde87101cf1b-kube-api-access-wck2j\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:13:39.849480 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:39.849469    2584 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b8656bfb-4ba7-47df-ad3f-fde87101cf1b-metrics-certs\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:13:39.849734 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:39.849710    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aae1344b-f684-4f6d-8c9f-eb4353885f54-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "aae1344b-f684-4f6d-8c9f-eb4353885f54" (UID: "aae1344b-f684-4f6d-8c9f-eb4353885f54"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 20 02:13:39.851340 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:39.851319    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aae1344b-f684-4f6d-8c9f-eb4353885f54-kube-api-access-fjn62" (OuterVolumeSpecName: "kube-api-access-fjn62") pod "aae1344b-f684-4f6d-8c9f-eb4353885f54" (UID: "aae1344b-f684-4f6d-8c9f-eb4353885f54"). InnerVolumeSpecName "kube-api-access-fjn62". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 20 02:13:39.950540 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:39.950500    2584 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aae1344b-f684-4f6d-8c9f-eb4353885f54-plugins-config-volume\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:13:39.950540 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:39.950542    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fjn62\" (UniqueName: \"kubernetes.io/projected/aae1344b-f684-4f6d-8c9f-eb4353885f54-kube-api-access-fjn62\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:13:39.958768 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:39.958695    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:13:40.044378 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:40.044350    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-703998-6755f65cb6-sbmsr"]
Sep 20 02:13:40.049539 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:40.049492    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-703998-6755f65cb6-sbmsr"]
Sep 20 02:13:40.063801 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:40.063777    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:13:40.355236 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:40.355156    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7531-51e133"]
Sep 20 02:13:40.360053 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:40.360012    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7531-51e133"]
Sep 20 02:13:40.469028 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:40.468998    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7531-707851"]
Sep 20 02:13:40.469426 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:40.469414    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aae1344b-f684-4f6d-8c9f-eb4353885f54" containerName="payload-processing"
Sep 20 02:13:40.469483 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:40.469428    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="aae1344b-f684-4f6d-8c9f-eb4353885f54" containerName="payload-processing"
Sep 20 02:13:40.469483 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:40.469435    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b8656bfb-4ba7-47df-ad3f-fde87101cf1b" containerName="maas-api"
Sep 20 02:13:40.469483 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:40.469440    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="b8656bfb-4ba7-47df-ad3f-fde87101cf1b" containerName="maas-api"
Sep 20 02:13:40.469483 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:40.469449    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4fb35884-ebe0-44fa-8878-f4756de75298" containerName="test-curl-7531-51e133"
Sep 20 02:13:40.469483 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:40.469455    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="4fb35884-ebe0-44fa-8878-f4756de75298" containerName="test-curl-7531-51e133"
Sep 20 02:13:40.469675 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:40.469557    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="b8656bfb-4ba7-47df-ad3f-fde87101cf1b" containerName="maas-api"
Sep 20 02:13:40.469675 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:40.469567    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="aae1344b-f684-4f6d-8c9f-eb4353885f54" containerName="payload-processing"
Sep 20 02:13:40.469675 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:40.469574    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="4fb35884-ebe0-44fa-8878-f4756de75298" containerName="test-curl-7531-51e133"
Sep 20 02:13:40.473726 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:40.473706    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7531-707851"
Sep 20 02:13:40.477863 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:40.477841    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7531-707851"]
Sep 20 02:13:40.556948 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:40.556913    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6xtcp\" (UniqueName: \"kubernetes.io/projected/a27fb5fc-f712-448e-b8fd-f28beb7c5590-kube-api-access-6xtcp\") pod \"test-curl-7531-707851\" (UID: \"a27fb5fc-f712-448e-b8fd-f28beb7c5590\") " pod="opendatahub/test-curl-7531-707851"
Sep 20 02:13:40.658124 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:40.658094    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6xtcp\" (UniqueName: \"kubernetes.io/projected/a27fb5fc-f712-448e-b8fd-f28beb7c5590-kube-api-access-6xtcp\") pod \"test-curl-7531-707851\" (UID: \"a27fb5fc-f712-448e-b8fd-f28beb7c5590\") " pod="opendatahub/test-curl-7531-707851"
Sep 20 02:13:40.668839 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:40.668812    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6xtcp\" (UniqueName: \"kubernetes.io/projected/a27fb5fc-f712-448e-b8fd-f28beb7c5590-kube-api-access-6xtcp\") pod \"test-curl-7531-707851\" (UID: \"a27fb5fc-f712-448e-b8fd-f28beb7c5590\") " pod="opendatahub/test-curl-7531-707851"
Sep 20 02:13:40.726147 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:40.726118    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:13:40.732876 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:40.732849    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-703998-56676dd7-vhtqd"
Sep 20 02:13:40.733023 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:40.732874    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-703998-56676dd7-vhtqd" event={"ID":"aae1344b-f684-4f6d-8c9f-eb4353885f54","Type":"ContainerDied","Data":"2ddc9c71dffe597257deec3a06aeeeff6c27cc6d1cdb3953756f9ac0afbb297b"}
Sep 20 02:13:40.733023 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:40.732940    2584 scope.go:117] "RemoveContainer" containerID="204634417f4bdd38deea31f57ba3b40d357dc60af558e7434c70d5ad9e9a928e"
Sep 20 02:13:40.734149 ip-10-0-130-159 kubenswrapper[2584]: E0920 02:13:40.734131    2584 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7531-51e133" container="test-curl-7531-51e133"
Sep 20 02:13:40.754218 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:40.754198    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-703998-56676dd7-vhtqd"]
Sep 20 02:13:40.761756 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:40.761736    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-703998-56676dd7-vhtqd"]
Sep 20 02:13:40.784590 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:40.784567    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7531-707851"
Sep 20 02:13:41.057279 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:41.057247    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7531-707851"]
Sep 20 02:13:41.059321 ip-10-0-130-159 kubenswrapper[2584]: W0920 02:13:41.059286    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda27fb5fc_f712_448e_b8fd_f28beb7c5590.slice/crio-57037b973b872b9ab5fa24d8dede7da62e9ec278eba54acbc99c06df6edb1d7e WatchSource:0}: Error finding container 57037b973b872b9ab5fa24d8dede7da62e9ec278eba54acbc99c06df6edb1d7e: Status 404 returned error can't find the container with id 57037b973b872b9ab5fa24d8dede7da62e9ec278eba54acbc99c06df6edb1d7e
Sep 20 02:13:41.095270 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:41.095246    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aae1344b-f684-4f6d-8c9f-eb4353885f54" path="/var/lib/kubelet/pods/aae1344b-f684-4f6d-8c9f-eb4353885f54/volumes"
Sep 20 02:13:41.095671 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:41.095652    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b8656bfb-4ba7-47df-ad3f-fde87101cf1b" path="/var/lib/kubelet/pods/b8656bfb-4ba7-47df-ad3f-fde87101cf1b/volumes"
Sep 20 02:13:41.565708 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:41.565661    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:13:41.742717 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:41.742680    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7531-707851" event={"ID":"a27fb5fc-f712-448e-b8fd-f28beb7c5590","Type":"ContainerStarted","Data":"21cea5f3301e6cf09b18c3fbf47d6978b755a77f7b526b50d652066fad604811"}
Sep 20 02:13:41.743113 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:41.742732    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7531-707851" event={"ID":"a27fb5fc-f712-448e-b8fd-f28beb7c5590","Type":"ContainerStarted","Data":"57037b973b872b9ab5fa24d8dede7da62e9ec278eba54acbc99c06df6edb1d7e"}
Sep 20 02:13:41.743113 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:41.742767    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7531-51e133" podUID="4fb35884-ebe0-44fa-8878-f4756de75298" containerName="test-curl-7531-51e133" containerID="cri-o://066724bd0b104c5bf5c743192a726265b7bacd3053b5cc435d8304f8b731809a" gracePeriod=2
Sep 20 02:13:41.759305 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:41.759246    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7531-707851" podStartSLOduration=1.75922434 podStartE2EDuration="1.75922434s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-20 02:13:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-20 02:13:41.756856742 +0000 UTC m=+1278.499535940" watchObservedRunningTime="2026-09-20 02:13:41.75922434 +0000 UTC m=+1278.501903546"
Sep 20 02:13:41.920276 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:41.920238    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7531-edd643"
Sep 20 02:13:41.959494 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:41.959435    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:13:41.969260 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:41.969220    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xf4wl\" (UniqueName: \"kubernetes.io/projected/e5827699-49d7-433a-8102-e7e32eda4cf1-kube-api-access-xf4wl\") pod \"e5827699-49d7-433a-8102-e7e32eda4cf1\" (UID: \"e5827699-49d7-433a-8102-e7e32eda4cf1\") "
Sep 20 02:13:41.971371 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:41.971339    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e5827699-49d7-433a-8102-e7e32eda4cf1-kube-api-access-xf4wl" (OuterVolumeSpecName: "kube-api-access-xf4wl") pod "e5827699-49d7-433a-8102-e7e32eda4cf1" (UID: "e5827699-49d7-433a-8102-e7e32eda4cf1"). InnerVolumeSpecName "kube-api-access-xf4wl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 20 02:13:42.070833 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:42.070791    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xf4wl\" (UniqueName: \"kubernetes.io/projected/e5827699-49d7-433a-8102-e7e32eda4cf1-kube-api-access-xf4wl\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:13:42.343902 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:42.343870    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7531-707851"]
Sep 20 02:13:42.349716 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:42.349685    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7531-707851"]
Sep 20 02:13:42.367227 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:42.367195    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:13:42.457434 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:42.457401    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7531-24adfd"]
Sep 20 02:13:42.457856 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:42.457841    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a27fb5fc-f712-448e-b8fd-f28beb7c5590" containerName="test-curl-7531-707851"
Sep 20 02:13:42.457918 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:42.457858    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="a27fb5fc-f712-448e-b8fd-f28beb7c5590" containerName="test-curl-7531-707851"
Sep 20 02:13:42.457962 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:42.457934    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="a27fb5fc-f712-448e-b8fd-f28beb7c5590" containerName="test-curl-7531-707851"
Sep 20 02:13:42.461135 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:42.461118    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7531-24adfd"
Sep 20 02:13:42.467095 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:42.467075    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7531-24adfd"]
Sep 20 02:13:42.577223 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:42.577180    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dj2ll\" (UniqueName: \"kubernetes.io/projected/5a2b14a3-f842-48df-9b3e-046b9771f1e5-kube-api-access-dj2ll\") pod \"test-curl-7531-24adfd\" (UID: \"5a2b14a3-f842-48df-9b3e-046b9771f1e5\") " pod="opendatahub/test-curl-7531-24adfd"
Sep 20 02:13:42.678356 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:42.678313    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dj2ll\" (UniqueName: \"kubernetes.io/projected/5a2b14a3-f842-48df-9b3e-046b9771f1e5-kube-api-access-dj2ll\") pod \"test-curl-7531-24adfd\" (UID: \"5a2b14a3-f842-48df-9b3e-046b9771f1e5\") " pod="opendatahub/test-curl-7531-24adfd"
Sep 20 02:13:42.686768 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:42.686740    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dj2ll\" (UniqueName: \"kubernetes.io/projected/5a2b14a3-f842-48df-9b3e-046b9771f1e5-kube-api-access-dj2ll\") pod \"test-curl-7531-24adfd\" (UID: \"5a2b14a3-f842-48df-9b3e-046b9771f1e5\") " pod="opendatahub/test-curl-7531-24adfd"
Sep 20 02:13:42.749621 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:42.749501    2584 generic.go:356] "Generic (PLEG): container finished" podID="e5827699-49d7-433a-8102-e7e32eda4cf1" containerID="7e60589723c3144a628245ade19715ea40906020518f13c16913ea170224022d" exitCode=137
Sep 20 02:13:42.749621 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:42.749556    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7531-edd643"
Sep 20 02:13:42.749621 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:42.749606    2584 scope.go:117] "RemoveContainer" containerID="7e60589723c3144a628245ade19715ea40906020518f13c16913ea170224022d"
Sep 20 02:13:42.750203 ip-10-0-130-159 kubenswrapper[2584]: E0920 02:13:42.749943    2584 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7531-707851" container="test-curl-7531-707851"
Sep 20 02:13:42.759800 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:42.759780    2584 scope.go:117] "RemoveContainer" containerID="7e60589723c3144a628245ade19715ea40906020518f13c16913ea170224022d"
Sep 20 02:13:42.760145 ip-10-0-130-159 kubenswrapper[2584]: E0920 02:13:42.760121    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7e60589723c3144a628245ade19715ea40906020518f13c16913ea170224022d\": container with ID starting with 7e60589723c3144a628245ade19715ea40906020518f13c16913ea170224022d not found: ID does not exist" containerID="7e60589723c3144a628245ade19715ea40906020518f13c16913ea170224022d"
Sep 20 02:13:42.760243 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:42.760160    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7e60589723c3144a628245ade19715ea40906020518f13c16913ea170224022d"} err="failed to get container status \"7e60589723c3144a628245ade19715ea40906020518f13c16913ea170224022d\": rpc error: code = NotFound desc = could not find container \"7e60589723c3144a628245ade19715ea40906020518f13c16913ea170224022d\": container with ID starting with 7e60589723c3144a628245ade19715ea40906020518f13c16913ea170224022d not found: ID does not exist"
Sep 20 02:13:42.760910 ip-10-0-130-159 kubenswrapper[2584]: E0920 02:13:42.760877    2584 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7531-edd643\" is forbidden: User \"system:node:ip-10-0-130-159.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-159.ec2.internal' and this object" podUID="e5827699-49d7-433a-8102-e7e32eda4cf1" pod="opendatahub/test-curl-7531-edd643"
Sep 20 02:13:42.772108 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:42.772086    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7531-24adfd"
Sep 20 02:13:43.054166 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:43.054133    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7531-24adfd"]
Sep 20 02:13:43.056300 ip-10-0-130-159 kubenswrapper[2584]: W0920 02:13:43.056270    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5a2b14a3_f842_48df_9b3e_046b9771f1e5.slice/crio-c74f77f2ef22558d7afb09c1c94dbd71d9c51efaf77aab84fbfd63e58c94241a WatchSource:0}: Error finding container c74f77f2ef22558d7afb09c1c94dbd71d9c51efaf77aab84fbfd63e58c94241a: Status 404 returned error can't find the container with id c74f77f2ef22558d7afb09c1c94dbd71d9c51efaf77aab84fbfd63e58c94241a
Sep 20 02:13:43.061205 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:43.061181    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:13:43.094048 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:43.094018    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e5827699-49d7-433a-8102-e7e32eda4cf1" path="/var/lib/kubelet/pods/e5827699-49d7-433a-8102-e7e32eda4cf1/volumes"
Sep 20 02:13:43.151876 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:43.151846    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:13:43.758645 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:43.758304    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7531-24adfd" event={"ID":"5a2b14a3-f842-48df-9b3e-046b9771f1e5","Type":"ContainerStarted","Data":"5a624c59314742ea4f55e6cc820aa8b802ece8dc963313e40e9d2b7d6c215b30"}
Sep 20 02:13:43.758645 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:43.758359    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7531-24adfd" event={"ID":"5a2b14a3-f842-48df-9b3e-046b9771f1e5","Type":"ContainerStarted","Data":"c74f77f2ef22558d7afb09c1c94dbd71d9c51efaf77aab84fbfd63e58c94241a"}
Sep 20 02:13:43.759804 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:43.759777    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7531-707851" podUID="a27fb5fc-f712-448e-b8fd-f28beb7c5590" containerName="test-curl-7531-707851" containerID="cri-o://21cea5f3301e6cf09b18c3fbf47d6978b755a77f7b526b50d652066fad604811" gracePeriod=2
Sep 20 02:13:43.771866 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:43.771813    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7531-24adfd" podStartSLOduration=1.7717941659999998 podStartE2EDuration="1.771794166s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-20 02:13:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-20 02:13:43.771022198 +0000 UTC m=+1280.513701406" watchObservedRunningTime="2026-09-20 02:13:43.771794166 +0000 UTC m=+1280.514473370"
Sep 20 02:13:43.836607 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:43.836575    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:13:43.939859 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:43.939833    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7531-51e133"
Sep 20 02:13:43.991093 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:43.991063    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-plhwh\" (UniqueName: \"kubernetes.io/projected/4fb35884-ebe0-44fa-8878-f4756de75298-kube-api-access-plhwh\") pod \"4fb35884-ebe0-44fa-8878-f4756de75298\" (UID: \"4fb35884-ebe0-44fa-8878-f4756de75298\") "
Sep 20 02:13:43.993217 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:43.993185    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4fb35884-ebe0-44fa-8878-f4756de75298-kube-api-access-plhwh" (OuterVolumeSpecName: "kube-api-access-plhwh") pod "4fb35884-ebe0-44fa-8878-f4756de75298" (UID: "4fb35884-ebe0-44fa-8878-f4756de75298"). InnerVolumeSpecName "kube-api-access-plhwh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 20 02:13:44.091989 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:44.091951    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-plhwh\" (UniqueName: \"kubernetes.io/projected/4fb35884-ebe0-44fa-8878-f4756de75298-kube-api-access-plhwh\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:13:44.328797 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:44.328706    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7531-24adfd"]
Sep 20 02:13:44.332898 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:44.332861    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7531-24adfd"]
Sep 20 02:13:44.767733 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:44.767697    2584 generic.go:356] "Generic (PLEG): container finished" podID="4fb35884-ebe0-44fa-8878-f4756de75298" containerID="066724bd0b104c5bf5c743192a726265b7bacd3053b5cc435d8304f8b731809a" exitCode=137
Sep 20 02:13:44.768190 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:44.767745    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7531-51e133"
Sep 20 02:13:44.768190 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:44.767797    2584 scope.go:117] "RemoveContainer" containerID="066724bd0b104c5bf5c743192a726265b7bacd3053b5cc435d8304f8b731809a"
Sep 20 02:13:44.768190 ip-10-0-130-159 kubenswrapper[2584]: E0920 02:13:44.768006    2584 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7531-24adfd" container="test-curl-7531-24adfd"
Sep 20 02:13:44.778696 ip-10-0-130-159 kubenswrapper[2584]: E0920 02:13:44.778572    2584 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7531-51e133\" is forbidden: User \"system:node:ip-10-0-130-159.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-159.ec2.internal' and this object" podUID="4fb35884-ebe0-44fa-8878-f4756de75298" pod="opendatahub/test-curl-7531-51e133"
Sep 20 02:13:44.778779 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:44.778718    2584 scope.go:117] "RemoveContainer" containerID="066724bd0b104c5bf5c743192a726265b7bacd3053b5cc435d8304f8b731809a"
Sep 20 02:13:44.779035 ip-10-0-130-159 kubenswrapper[2584]: E0920 02:13:44.779018    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"066724bd0b104c5bf5c743192a726265b7bacd3053b5cc435d8304f8b731809a\": container with ID starting with 066724bd0b104c5bf5c743192a726265b7bacd3053b5cc435d8304f8b731809a not found: ID does not exist" containerID="066724bd0b104c5bf5c743192a726265b7bacd3053b5cc435d8304f8b731809a"
Sep 20 02:13:44.779098 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:44.779047    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"066724bd0b104c5bf5c743192a726265b7bacd3053b5cc435d8304f8b731809a"} err="failed to get container status \"066724bd0b104c5bf5c743192a726265b7bacd3053b5cc435d8304f8b731809a\": rpc error: code = NotFound desc = could not find container \"066724bd0b104c5bf5c743192a726265b7bacd3053b5cc435d8304f8b731809a\": container with ID starting with 066724bd0b104c5bf5c743192a726265b7bacd3053b5cc435d8304f8b731809a not found: ID does not exist"
Sep 20 02:13:44.903793 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:44.903762    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:13:45.056904 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:45.056811    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:13:45.090880 ip-10-0-130-159 kubenswrapper[2584]: E0920 02:13:45.090842    2584 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7531-51e133\" is forbidden: User \"system:node:ip-10-0-130-159.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-159.ec2.internal' and this object" podUID="4fb35884-ebe0-44fa-8878-f4756de75298" pod="opendatahub/test-curl-7531-51e133"
Sep 20 02:13:45.093512 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:45.093482    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4fb35884-ebe0-44fa-8878-f4756de75298" path="/var/lib/kubelet/pods/4fb35884-ebe0-44fa-8878-f4756de75298/volumes"
Sep 20 02:13:45.169613 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:45.169580    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-f5813346-openshift-default-dcd586ff6-cvqn5"]
Sep 20 02:13:45.170072 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:45.170053    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5a2b14a3-f842-48df-9b3e-046b9771f1e5" containerName="test-curl-7531-24adfd"
Sep 20 02:13:45.170159 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:45.170073    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a2b14a3-f842-48df-9b3e-046b9771f1e5" containerName="test-curl-7531-24adfd"
Sep 20 02:13:45.170223 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:45.170164    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="5a2b14a3-f842-48df-9b3e-046b9771f1e5" containerName="test-curl-7531-24adfd"
Sep 20 02:13:45.173687 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:45.173662    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-f5813346-openshift-default-dcd586ff6-cvqn5"
Sep 20 02:13:45.175897 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:45.175875    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 20 02:13:45.176011 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:45.175882    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-f5813346-openshift-default-dockercfg-6vf7m\""
Sep 20 02:13:45.185919 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:45.185896    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-f5813346-openshift-default-dcd586ff6-cvqn5"]
Sep 20 02:13:45.215200 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:45.215171    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:13:45.303435 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:45.303399    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2ce47e33-9674-4f3e-956b-d0083215f70f-workload-certs\") pod \"e2e-mt-f5813346-openshift-default-dcd586ff6-cvqn5\" (UID: \"2ce47e33-9674-4f3e-956b-d0083215f70f\") " pod="openshift-ingress/e2e-mt-f5813346-openshift-default-dcd586ff6-cvqn5"
Sep 20 02:13:45.303646 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:45.303441    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ce47e33-9674-4f3e-956b-d0083215f70f-credential-socket\") pod \"e2e-mt-f5813346-openshift-default-dcd586ff6-cvqn5\" (UID: \"2ce47e33-9674-4f3e-956b-d0083215f70f\") " pod="openshift-ingress/e2e-mt-f5813346-openshift-default-dcd586ff6-cvqn5"
Sep 20 02:13:45.303646 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:45.303480    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ce47e33-9674-4f3e-956b-d0083215f70f-workload-socket\") pod \"e2e-mt-f5813346-openshift-default-dcd586ff6-cvqn5\" (UID: \"2ce47e33-9674-4f3e-956b-d0083215f70f\") " pod="openshift-ingress/e2e-mt-f5813346-openshift-default-dcd586ff6-cvqn5"
Sep 20 02:13:45.303646 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:45.303528    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2ce47e33-9674-4f3e-956b-d0083215f70f-istio-podinfo\") pod \"e2e-mt-f5813346-openshift-default-dcd586ff6-cvqn5\" (UID: \"2ce47e33-9674-4f3e-956b-d0083215f70f\") " pod="openshift-ingress/e2e-mt-f5813346-openshift-default-dcd586ff6-cvqn5"
Sep 20 02:13:45.303646 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:45.303575    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2ce47e33-9674-4f3e-956b-d0083215f70f-istio-envoy\") pod \"e2e-mt-f5813346-openshift-default-dcd586ff6-cvqn5\" (UID: \"2ce47e33-9674-4f3e-956b-d0083215f70f\") " pod="openshift-ingress/e2e-mt-f5813346-openshift-default-dcd586ff6-cvqn5"
Sep 20 02:13:45.303829 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:45.303668    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mzl5k\" (UniqueName: \"kubernetes.io/projected/2ce47e33-9674-4f3e-956b-d0083215f70f-kube-api-access-mzl5k\") pod \"e2e-mt-f5813346-openshift-default-dcd586ff6-cvqn5\" (UID: \"2ce47e33-9674-4f3e-956b-d0083215f70f\") " pod="openshift-ingress/e2e-mt-f5813346-openshift-default-dcd586ff6-cvqn5"
Sep 20 02:13:45.303829 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:45.303688    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2ce47e33-9674-4f3e-956b-d0083215f70f-istiod-ca-cert\") pod \"e2e-mt-f5813346-openshift-default-dcd586ff6-cvqn5\" (UID: \"2ce47e33-9674-4f3e-956b-d0083215f70f\") " pod="openshift-ingress/e2e-mt-f5813346-openshift-default-dcd586ff6-cvqn5"
Sep 20 02:13:45.303829 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:45.303714    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2ce47e33-9674-4f3e-956b-d0083215f70f-istio-data\") pod \"e2e-mt-f5813346-openshift-default-dcd586ff6-cvqn5\" (UID: \"2ce47e33-9674-4f3e-956b-d0083215f70f\") " pod="openshift-ingress/e2e-mt-f5813346-openshift-default-dcd586ff6-cvqn5"
Sep 20 02:13:45.303829 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:45.303729    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2ce47e33-9674-4f3e-956b-d0083215f70f-istio-token\") pod \"e2e-mt-f5813346-openshift-default-dcd586ff6-cvqn5\" (UID: \"2ce47e33-9674-4f3e-956b-d0083215f70f\") " pod="openshift-ingress/e2e-mt-f5813346-openshift-default-dcd586ff6-cvqn5"
Sep 20 02:13:45.404308 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:45.404225    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2ce47e33-9674-4f3e-956b-d0083215f70f-istio-podinfo\") pod \"e2e-mt-f5813346-openshift-default-dcd586ff6-cvqn5\" (UID: \"2ce47e33-9674-4f3e-956b-d0083215f70f\") " pod="openshift-ingress/e2e-mt-f5813346-openshift-default-dcd586ff6-cvqn5"
Sep 20 02:13:45.404308 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:45.404272    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2ce47e33-9674-4f3e-956b-d0083215f70f-istio-envoy\") pod \"e2e-mt-f5813346-openshift-default-dcd586ff6-cvqn5\" (UID: \"2ce47e33-9674-4f3e-956b-d0083215f70f\") " pod="openshift-ingress/e2e-mt-f5813346-openshift-default-dcd586ff6-cvqn5"
Sep 20 02:13:45.404556 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:45.404337    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mzl5k\" (UniqueName: \"kubernetes.io/projected/2ce47e33-9674-4f3e-956b-d0083215f70f-kube-api-access-mzl5k\") pod \"e2e-mt-f5813346-openshift-default-dcd586ff6-cvqn5\" (UID: \"2ce47e33-9674-4f3e-956b-d0083215f70f\") " pod="openshift-ingress/e2e-mt-f5813346-openshift-default-dcd586ff6-cvqn5"
Sep 20 02:13:45.404556 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:45.404357    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2ce47e33-9674-4f3e-956b-d0083215f70f-istiod-ca-cert\") pod \"e2e-mt-f5813346-openshift-default-dcd586ff6-cvqn5\" (UID: \"2ce47e33-9674-4f3e-956b-d0083215f70f\") " pod="openshift-ingress/e2e-mt-f5813346-openshift-default-dcd586ff6-cvqn5"
Sep 20 02:13:45.404556 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:45.404382    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2ce47e33-9674-4f3e-956b-d0083215f70f-istio-data\") pod \"e2e-mt-f5813346-openshift-default-dcd586ff6-cvqn5\" (UID: \"2ce47e33-9674-4f3e-956b-d0083215f70f\") " pod="openshift-ingress/e2e-mt-f5813346-openshift-default-dcd586ff6-cvqn5"
Sep 20 02:13:45.404556 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:45.404414    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2ce47e33-9674-4f3e-956b-d0083215f70f-istio-token\") pod \"e2e-mt-f5813346-openshift-default-dcd586ff6-cvqn5\" (UID: \"2ce47e33-9674-4f3e-956b-d0083215f70f\") " pod="openshift-ingress/e2e-mt-f5813346-openshift-default-dcd586ff6-cvqn5"
Sep 20 02:13:45.404556 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:45.404443    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2ce47e33-9674-4f3e-956b-d0083215f70f-workload-certs\") pod \"e2e-mt-f5813346-openshift-default-dcd586ff6-cvqn5\" (UID: \"2ce47e33-9674-4f3e-956b-d0083215f70f\") " pod="openshift-ingress/e2e-mt-f5813346-openshift-default-dcd586ff6-cvqn5"
Sep 20 02:13:45.404556 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:45.404467    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ce47e33-9674-4f3e-956b-d0083215f70f-credential-socket\") pod \"e2e-mt-f5813346-openshift-default-dcd586ff6-cvqn5\" (UID: \"2ce47e33-9674-4f3e-956b-d0083215f70f\") " pod="openshift-ingress/e2e-mt-f5813346-openshift-default-dcd586ff6-cvqn5"
Sep 20 02:13:45.404556 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:45.404498    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ce47e33-9674-4f3e-956b-d0083215f70f-workload-socket\") pod \"e2e-mt-f5813346-openshift-default-dcd586ff6-cvqn5\" (UID: \"2ce47e33-9674-4f3e-956b-d0083215f70f\") " pod="openshift-ingress/e2e-mt-f5813346-openshift-default-dcd586ff6-cvqn5"
Sep 20 02:13:45.404978 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:45.404949    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2ce47e33-9674-4f3e-956b-d0083215f70f-istio-data\") pod \"e2e-mt-f5813346-openshift-default-dcd586ff6-cvqn5\" (UID: \"2ce47e33-9674-4f3e-956b-d0083215f70f\") " pod="openshift-ingress/e2e-mt-f5813346-openshift-default-dcd586ff6-cvqn5"
Sep 20 02:13:45.405032 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:45.404985    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ce47e33-9674-4f3e-956b-d0083215f70f-credential-socket\") pod \"e2e-mt-f5813346-openshift-default-dcd586ff6-cvqn5\" (UID: \"2ce47e33-9674-4f3e-956b-d0083215f70f\") " pod="openshift-ingress/e2e-mt-f5813346-openshift-default-dcd586ff6-cvqn5"
Sep 20 02:13:45.405098 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:45.405036    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2ce47e33-9674-4f3e-956b-d0083215f70f-workload-certs\") pod \"e2e-mt-f5813346-openshift-default-dcd586ff6-cvqn5\" (UID: \"2ce47e33-9674-4f3e-956b-d0083215f70f\") " pod="openshift-ingress/e2e-mt-f5813346-openshift-default-dcd586ff6-cvqn5"
Sep 20 02:13:45.405098 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:45.405087    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ce47e33-9674-4f3e-956b-d0083215f70f-workload-socket\") pod \"e2e-mt-f5813346-openshift-default-dcd586ff6-cvqn5\" (UID: \"2ce47e33-9674-4f3e-956b-d0083215f70f\") " pod="openshift-ingress/e2e-mt-f5813346-openshift-default-dcd586ff6-cvqn5"
Sep 20 02:13:45.405349 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:45.405333    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2ce47e33-9674-4f3e-956b-d0083215f70f-istiod-ca-cert\") pod \"e2e-mt-f5813346-openshift-default-dcd586ff6-cvqn5\" (UID: \"2ce47e33-9674-4f3e-956b-d0083215f70f\") " pod="openshift-ingress/e2e-mt-f5813346-openshift-default-dcd586ff6-cvqn5"
Sep 20 02:13:45.406817 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:45.406799    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2ce47e33-9674-4f3e-956b-d0083215f70f-istio-envoy\") pod \"e2e-mt-f5813346-openshift-default-dcd586ff6-cvqn5\" (UID: \"2ce47e33-9674-4f3e-956b-d0083215f70f\") " pod="openshift-ingress/e2e-mt-f5813346-openshift-default-dcd586ff6-cvqn5"
Sep 20 02:13:45.406952 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:45.406935    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2ce47e33-9674-4f3e-956b-d0083215f70f-istio-podinfo\") pod \"e2e-mt-f5813346-openshift-default-dcd586ff6-cvqn5\" (UID: \"2ce47e33-9674-4f3e-956b-d0083215f70f\") " pod="openshift-ingress/e2e-mt-f5813346-openshift-default-dcd586ff6-cvqn5"
Sep 20 02:13:45.412319 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:45.412296    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2ce47e33-9674-4f3e-956b-d0083215f70f-istio-token\") pod \"e2e-mt-f5813346-openshift-default-dcd586ff6-cvqn5\" (UID: \"2ce47e33-9674-4f3e-956b-d0083215f70f\") " pod="openshift-ingress/e2e-mt-f5813346-openshift-default-dcd586ff6-cvqn5"
Sep 20 02:13:45.412466 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:45.412448    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mzl5k\" (UniqueName: \"kubernetes.io/projected/2ce47e33-9674-4f3e-956b-d0083215f70f-kube-api-access-mzl5k\") pod \"e2e-mt-f5813346-openshift-default-dcd586ff6-cvqn5\" (UID: \"2ce47e33-9674-4f3e-956b-d0083215f70f\") " pod="openshift-ingress/e2e-mt-f5813346-openshift-default-dcd586ff6-cvqn5"
Sep 20 02:13:45.486163 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:45.486124    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-f5813346-openshift-default-dcd586ff6-cvqn5"
Sep 20 02:13:45.515639 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:45.515601    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:13:45.766428 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:45.766088    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-f5813346-openshift-default-dcd586ff6-cvqn5"]
Sep 20 02:13:45.769612 ip-10-0-130-159 kubenswrapper[2584]: W0920 02:13:45.769584    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ce47e33_9674_4f3e_956b_d0083215f70f.slice/crio-6e56d904eac1b315d2fe9693fce4ef0094788640086d032c993d18907b74eee0 WatchSource:0}: Error finding container 6e56d904eac1b315d2fe9693fce4ef0094788640086d032c993d18907b74eee0: Status 404 returned error can't find the container with id 6e56d904eac1b315d2fe9693fce4ef0094788640086d032c993d18907b74eee0
Sep 20 02:13:45.783966 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:45.783929    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7531-24adfd" podUID="5a2b14a3-f842-48df-9b3e-046b9771f1e5" containerName="test-curl-7531-24adfd" containerID="cri-o://5a624c59314742ea4f55e6cc820aa8b802ece8dc963313e40e9d2b7d6c215b30" gracePeriod=2
Sep 20 02:13:45.810999 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:45.810960    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 20 02:13:45.811101 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:45.811033    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 20 02:13:45.811101 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:45.811063    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 20 02:13:45.940503 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:45.940475    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7531-707851"
Sep 20 02:13:46.010250 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:46.010211    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6xtcp\" (UniqueName: \"kubernetes.io/projected/a27fb5fc-f712-448e-b8fd-f28beb7c5590-kube-api-access-6xtcp\") pod \"a27fb5fc-f712-448e-b8fd-f28beb7c5590\" (UID: \"a27fb5fc-f712-448e-b8fd-f28beb7c5590\") "
Sep 20 02:13:46.012204 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:46.012167    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a27fb5fc-f712-448e-b8fd-f28beb7c5590-kube-api-access-6xtcp" (OuterVolumeSpecName: "kube-api-access-6xtcp") pod "a27fb5fc-f712-448e-b8fd-f28beb7c5590" (UID: "a27fb5fc-f712-448e-b8fd-f28beb7c5590"). InnerVolumeSpecName "kube-api-access-6xtcp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 20 02:13:46.112000 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:46.111921    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6xtcp\" (UniqueName: \"kubernetes.io/projected/a27fb5fc-f712-448e-b8fd-f28beb7c5590-kube-api-access-6xtcp\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:13:46.791514 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:46.791483    2584 generic.go:356] "Generic (PLEG): container finished" podID="a27fb5fc-f712-448e-b8fd-f28beb7c5590" containerID="21cea5f3301e6cf09b18c3fbf47d6978b755a77f7b526b50d652066fad604811" exitCode=137
Sep 20 02:13:46.791975 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:46.791535    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7531-707851"
Sep 20 02:13:46.791975 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:46.791564    2584 scope.go:117] "RemoveContainer" containerID="21cea5f3301e6cf09b18c3fbf47d6978b755a77f7b526b50d652066fad604811"
Sep 20 02:13:46.793349 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:46.793323    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f5813346-openshift-default-dcd586ff6-cvqn5" event={"ID":"2ce47e33-9674-4f3e-956b-d0083215f70f","Type":"ContainerStarted","Data":"bca3d5c8741a70b6c889361d94901593472e7a9704e4445def400e8b08425ec6"}
Sep 20 02:13:46.793471 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:46.793370    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f5813346-openshift-default-dcd586ff6-cvqn5" event={"ID":"2ce47e33-9674-4f3e-956b-d0083215f70f","Type":"ContainerStarted","Data":"6e56d904eac1b315d2fe9693fce4ef0094788640086d032c993d18907b74eee0"}
Sep 20 02:13:46.800717 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:46.800701    2584 scope.go:117] "RemoveContainer" containerID="21cea5f3301e6cf09b18c3fbf47d6978b755a77f7b526b50d652066fad604811"
Sep 20 02:13:46.800986 ip-10-0-130-159 kubenswrapper[2584]: E0920 02:13:46.800964    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"21cea5f3301e6cf09b18c3fbf47d6978b755a77f7b526b50d652066fad604811\": container with ID starting with 21cea5f3301e6cf09b18c3fbf47d6978b755a77f7b526b50d652066fad604811 not found: ID does not exist" containerID="21cea5f3301e6cf09b18c3fbf47d6978b755a77f7b526b50d652066fad604811"
Sep 20 02:13:46.801047 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:46.800992    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"21cea5f3301e6cf09b18c3fbf47d6978b755a77f7b526b50d652066fad604811"} err="failed to get container status \"21cea5f3301e6cf09b18c3fbf47d6978b755a77f7b526b50d652066fad604811\": rpc error: code = NotFound desc = could not find container \"21cea5f3301e6cf09b18c3fbf47d6978b755a77f7b526b50d652066fad604811\": container with ID starting with 21cea5f3301e6cf09b18c3fbf47d6978b755a77f7b526b50d652066fad604811 not found: ID does not exist"
Sep 20 02:13:46.813700 ip-10-0-130-159 kubenswrapper[2584]: E0920 02:13:46.813674    2584 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7531-707851\" is forbidden: User \"system:node:ip-10-0-130-159.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-159.ec2.internal' and this object" podUID="a27fb5fc-f712-448e-b8fd-f28beb7c5590" pod="opendatahub/test-curl-7531-707851"
Sep 20 02:13:46.814413 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:46.814375    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-f5813346-openshift-default-dcd586ff6-cvqn5" podStartSLOduration=1.814362917 podStartE2EDuration="1.814362917s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-20 02:13:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-20 02:13:46.811487892 +0000 UTC m=+1283.554167098" watchObservedRunningTime="2026-09-20 02:13:46.814362917 +0000 UTC m=+1283.557042121"
Sep 20 02:13:47.093803 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:47.093724    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a27fb5fc-f712-448e-b8fd-f28beb7c5590" path="/var/lib/kubelet/pods/a27fb5fc-f712-448e-b8fd-f28beb7c5590/volumes"
Sep 20 02:13:47.486458 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:47.486418    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-f5813346-openshift-default-dcd586ff6-cvqn5"
Sep 20 02:13:47.491499 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:47.491475    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-f5813346-openshift-default-dcd586ff6-cvqn5"
Sep 20 02:13:47.801760 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:47.801739    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-f5813346-openshift-default-dcd586ff6-cvqn5"
Sep 20 02:13:47.802700 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:47.802684    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-f5813346-openshift-default-dcd586ff6-cvqn5"
Sep 20 02:13:47.976005 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:47.975979    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7531-24adfd"
Sep 20 02:13:48.031102 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:48.031070    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dj2ll\" (UniqueName: \"kubernetes.io/projected/5a2b14a3-f842-48df-9b3e-046b9771f1e5-kube-api-access-dj2ll\") pod \"5a2b14a3-f842-48df-9b3e-046b9771f1e5\" (UID: \"5a2b14a3-f842-48df-9b3e-046b9771f1e5\") "
Sep 20 02:13:48.033215 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:48.033186    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a2b14a3-f842-48df-9b3e-046b9771f1e5-kube-api-access-dj2ll" (OuterVolumeSpecName: "kube-api-access-dj2ll") pod "5a2b14a3-f842-48df-9b3e-046b9771f1e5" (UID: "5a2b14a3-f842-48df-9b3e-046b9771f1e5"). InnerVolumeSpecName "kube-api-access-dj2ll". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 20 02:13:48.132710 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:48.132680    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dj2ll\" (UniqueName: \"kubernetes.io/projected/5a2b14a3-f842-48df-9b3e-046b9771f1e5-kube-api-access-dj2ll\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:13:48.503607 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:48.503571    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:13:48.713795 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:48.713761    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:13:48.809252 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:48.809151    2584 generic.go:356] "Generic (PLEG): container finished" podID="5a2b14a3-f842-48df-9b3e-046b9771f1e5" containerID="5a624c59314742ea4f55e6cc820aa8b802ece8dc963313e40e9d2b7d6c215b30" exitCode=137
Sep 20 02:13:48.809252 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:48.809208    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7531-24adfd"
Sep 20 02:13:48.809252 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:48.809248    2584 scope.go:117] "RemoveContainer" containerID="5a624c59314742ea4f55e6cc820aa8b802ece8dc963313e40e9d2b7d6c215b30"
Sep 20 02:13:48.814679 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:48.814650    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:13:48.818069 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:48.818051    2584 scope.go:117] "RemoveContainer" containerID="5a624c59314742ea4f55e6cc820aa8b802ece8dc963313e40e9d2b7d6c215b30"
Sep 20 02:13:48.818342 ip-10-0-130-159 kubenswrapper[2584]: E0920 02:13:48.818322    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5a624c59314742ea4f55e6cc820aa8b802ece8dc963313e40e9d2b7d6c215b30\": container with ID starting with 5a624c59314742ea4f55e6cc820aa8b802ece8dc963313e40e9d2b7d6c215b30 not found: ID does not exist" containerID="5a624c59314742ea4f55e6cc820aa8b802ece8dc963313e40e9d2b7d6c215b30"
Sep 20 02:13:48.818412 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:48.818355    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5a624c59314742ea4f55e6cc820aa8b802ece8dc963313e40e9d2b7d6c215b30"} err="failed to get container status \"5a624c59314742ea4f55e6cc820aa8b802ece8dc963313e40e9d2b7d6c215b30\": rpc error: code = NotFound desc = could not find container \"5a624c59314742ea4f55e6cc820aa8b802ece8dc963313e40e9d2b7d6c215b30\": container with ID starting with 5a624c59314742ea4f55e6cc820aa8b802ece8dc963313e40e9d2b7d6c215b30 not found: ID does not exist"
Sep 20 02:13:48.819476 ip-10-0-130-159 kubenswrapper[2584]: E0920 02:13:48.819452    2584 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7531-24adfd\" is forbidden: User \"system:node:ip-10-0-130-159.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-159.ec2.internal' and this object" podUID="5a2b14a3-f842-48df-9b3e-046b9771f1e5" pod="opendatahub/test-curl-7531-24adfd"
Sep 20 02:13:49.093348 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:49.093270    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5a2b14a3-f842-48df-9b3e-046b9771f1e5" path="/var/lib/kubelet/pods/5a2b14a3-f842-48df-9b3e-046b9771f1e5/volumes"
Sep 20 02:13:49.406545 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:49.406392    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:13:50.241403 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:50.241370    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:13:50.934789 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:50.934750    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:13:51.562755 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:51.562718    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f5813346-7ff9bcd677-zf6tr"]
Sep 20 02:13:51.567926 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:51.567898    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f5813346-7ff9bcd677-zf6tr"
Sep 20 02:13:51.570090 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:51.570067    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-f5813346-serving-cert\""
Sep 20 02:13:51.577146 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:51.577123    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f5813346-7ff9bcd677-zf6tr"]
Sep 20 02:13:51.666865 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:51.666833    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hx42x\" (UniqueName: \"kubernetes.io/projected/99a6dc33-0436-461b-b728-44523b5811b2-kube-api-access-hx42x\") pod \"maas-api-e2e-mt-f5813346-7ff9bcd677-zf6tr\" (UID: \"99a6dc33-0436-461b-b728-44523b5811b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f5813346-7ff9bcd677-zf6tr"
Sep 20 02:13:51.667052 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:51.666905    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/99a6dc33-0436-461b-b728-44523b5811b2-openshift-service-ca\") pod \"maas-api-e2e-mt-f5813346-7ff9bcd677-zf6tr\" (UID: \"99a6dc33-0436-461b-b728-44523b5811b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f5813346-7ff9bcd677-zf6tr"
Sep 20 02:13:51.667052 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:51.666983    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/99a6dc33-0436-461b-b728-44523b5811b2-metrics-certs\") pod \"maas-api-e2e-mt-f5813346-7ff9bcd677-zf6tr\" (UID: \"99a6dc33-0436-461b-b728-44523b5811b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f5813346-7ff9bcd677-zf6tr"
Sep 20 02:13:51.667137 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:51.667087    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/99a6dc33-0436-461b-b728-44523b5811b2-maas-api-tls\") pod \"maas-api-e2e-mt-f5813346-7ff9bcd677-zf6tr\" (UID: \"99a6dc33-0436-461b-b728-44523b5811b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f5813346-7ff9bcd677-zf6tr"
Sep 20 02:13:51.702400 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:51.702355    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:13:51.762372 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:51.762337    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:13:51.767546 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:51.767502    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/99a6dc33-0436-461b-b728-44523b5811b2-metrics-certs\") pod \"maas-api-e2e-mt-f5813346-7ff9bcd677-zf6tr\" (UID: \"99a6dc33-0436-461b-b728-44523b5811b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f5813346-7ff9bcd677-zf6tr"
Sep 20 02:13:51.767679 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:51.767586    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/99a6dc33-0436-461b-b728-44523b5811b2-maas-api-tls\") pod \"maas-api-e2e-mt-f5813346-7ff9bcd677-zf6tr\" (UID: \"99a6dc33-0436-461b-b728-44523b5811b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f5813346-7ff9bcd677-zf6tr"
Sep 20 02:13:51.767679 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:51.767636    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hx42x\" (UniqueName: \"kubernetes.io/projected/99a6dc33-0436-461b-b728-44523b5811b2-kube-api-access-hx42x\") pod \"maas-api-e2e-mt-f5813346-7ff9bcd677-zf6tr\" (UID: \"99a6dc33-0436-461b-b728-44523b5811b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f5813346-7ff9bcd677-zf6tr"
Sep 20 02:13:51.767798 ip-10-0-130-159 kubenswrapper[2584]: E0920 02:13:51.767726    2584 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-f5813346-serving-cert: secret "maas-api-e2e-mt-f5813346-serving-cert" not found
Sep 20 02:13:51.767798 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:51.767736    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/99a6dc33-0436-461b-b728-44523b5811b2-openshift-service-ca\") pod \"maas-api-e2e-mt-f5813346-7ff9bcd677-zf6tr\" (UID: \"99a6dc33-0436-461b-b728-44523b5811b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f5813346-7ff9bcd677-zf6tr"
Sep 20 02:13:51.767798 ip-10-0-130-159 kubenswrapper[2584]: E0920 02:13:51.767795    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/99a6dc33-0436-461b-b728-44523b5811b2-maas-api-tls podName:99a6dc33-0436-461b-b728-44523b5811b2 nodeName:}" failed. No retries permitted until 2026-09-20 02:13:52.267776283 +0000 UTC m=+1289.010455483 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/99a6dc33-0436-461b-b728-44523b5811b2-maas-api-tls") pod "maas-api-e2e-mt-f5813346-7ff9bcd677-zf6tr" (UID: "99a6dc33-0436-461b-b728-44523b5811b2") : secret "maas-api-e2e-mt-f5813346-serving-cert" not found
Sep 20 02:13:51.768322 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:51.768301    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/99a6dc33-0436-461b-b728-44523b5811b2-openshift-service-ca\") pod \"maas-api-e2e-mt-f5813346-7ff9bcd677-zf6tr\" (UID: \"99a6dc33-0436-461b-b728-44523b5811b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f5813346-7ff9bcd677-zf6tr"
Sep 20 02:13:51.770168 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:51.770150    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/99a6dc33-0436-461b-b728-44523b5811b2-metrics-certs\") pod \"maas-api-e2e-mt-f5813346-7ff9bcd677-zf6tr\" (UID: \"99a6dc33-0436-461b-b728-44523b5811b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f5813346-7ff9bcd677-zf6tr"
Sep 20 02:13:51.776646 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:51.776624    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hx42x\" (UniqueName: \"kubernetes.io/projected/99a6dc33-0436-461b-b728-44523b5811b2-kube-api-access-hx42x\") pod \"maas-api-e2e-mt-f5813346-7ff9bcd677-zf6tr\" (UID: \"99a6dc33-0436-461b-b728-44523b5811b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f5813346-7ff9bcd677-zf6tr"
Sep 20 02:13:52.079370 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:52.079340    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:13:52.186939 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:52.186907    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:13:52.204822 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:52.204791    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-f5813346-5897d44497-fc64d"]
Sep 20 02:13:52.208814 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:52.208785    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-f5813346-5897d44497-fc64d"
Sep 20 02:13:52.211027 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:52.211001    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-f5813346\""
Sep 20 02:13:52.211027 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:52.211028    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-f5813346-dockercfg-wkgzb\""
Sep 20 02:13:52.219144 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:52.219120    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-f5813346-5897d44497-fc64d"]
Sep 20 02:13:52.274191 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:52.274153    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/65d3c0b7-83df-4dfb-abc3-8e4bb010709b-plugins-config-volume\") pod \"payload-processing-e2e-mt-f5813346-5897d44497-fc64d\" (UID: \"65d3c0b7-83df-4dfb-abc3-8e4bb010709b\") " pod="openshift-ingress/payload-processing-e2e-mt-f5813346-5897d44497-fc64d"
Sep 20 02:13:52.274191 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:52.274190    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s9dz5\" (UniqueName: \"kubernetes.io/projected/65d3c0b7-83df-4dfb-abc3-8e4bb010709b-kube-api-access-s9dz5\") pod \"payload-processing-e2e-mt-f5813346-5897d44497-fc64d\" (UID: \"65d3c0b7-83df-4dfb-abc3-8e4bb010709b\") " pod="openshift-ingress/payload-processing-e2e-mt-f5813346-5897d44497-fc64d"
Sep 20 02:13:52.274431 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:52.274260    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/99a6dc33-0436-461b-b728-44523b5811b2-maas-api-tls\") pod \"maas-api-e2e-mt-f5813346-7ff9bcd677-zf6tr\" (UID: \"99a6dc33-0436-461b-b728-44523b5811b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f5813346-7ff9bcd677-zf6tr"
Sep 20 02:13:52.276777 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:52.276753    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/99a6dc33-0436-461b-b728-44523b5811b2-maas-api-tls\") pod \"maas-api-e2e-mt-f5813346-7ff9bcd677-zf6tr\" (UID: \"99a6dc33-0436-461b-b728-44523b5811b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f5813346-7ff9bcd677-zf6tr"
Sep 20 02:13:52.375579 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:52.375474    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/65d3c0b7-83df-4dfb-abc3-8e4bb010709b-plugins-config-volume\") pod \"payload-processing-e2e-mt-f5813346-5897d44497-fc64d\" (UID: \"65d3c0b7-83df-4dfb-abc3-8e4bb010709b\") " pod="openshift-ingress/payload-processing-e2e-mt-f5813346-5897d44497-fc64d"
Sep 20 02:13:52.375579 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:52.375513    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s9dz5\" (UniqueName: \"kubernetes.io/projected/65d3c0b7-83df-4dfb-abc3-8e4bb010709b-kube-api-access-s9dz5\") pod \"payload-processing-e2e-mt-f5813346-5897d44497-fc64d\" (UID: \"65d3c0b7-83df-4dfb-abc3-8e4bb010709b\") " pod="openshift-ingress/payload-processing-e2e-mt-f5813346-5897d44497-fc64d"
Sep 20 02:13:52.376298 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:52.376272    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/65d3c0b7-83df-4dfb-abc3-8e4bb010709b-plugins-config-volume\") pod \"payload-processing-e2e-mt-f5813346-5897d44497-fc64d\" (UID: \"65d3c0b7-83df-4dfb-abc3-8e4bb010709b\") " pod="openshift-ingress/payload-processing-e2e-mt-f5813346-5897d44497-fc64d"
Sep 20 02:13:52.384745 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:52.384721    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s9dz5\" (UniqueName: \"kubernetes.io/projected/65d3c0b7-83df-4dfb-abc3-8e4bb010709b-kube-api-access-s9dz5\") pod \"payload-processing-e2e-mt-f5813346-5897d44497-fc64d\" (UID: \"65d3c0b7-83df-4dfb-abc3-8e4bb010709b\") " pod="openshift-ingress/payload-processing-e2e-mt-f5813346-5897d44497-fc64d"
Sep 20 02:13:52.480121 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:52.480081    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f5813346-7ff9bcd677-zf6tr"
Sep 20 02:13:52.521890 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:52.521857    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-f5813346-5897d44497-fc64d"
Sep 20 02:13:52.753268 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:52.753242    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f5813346-7ff9bcd677-zf6tr"]
Sep 20 02:13:52.754867 ip-10-0-130-159 kubenswrapper[2584]: W0920 02:13:52.754843    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99a6dc33_0436_461b_b728_44523b5811b2.slice/crio-82c2128e04a73039c9799251ff6f2ed21e9104e41625d01ba9bafe254582a1a4 WatchSource:0}: Error finding container 82c2128e04a73039c9799251ff6f2ed21e9104e41625d01ba9bafe254582a1a4: Status 404 returned error can't find the container with id 82c2128e04a73039c9799251ff6f2ed21e9104e41625d01ba9bafe254582a1a4
Sep 20 02:13:52.806691 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:52.806667    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-f5813346-5897d44497-fc64d"]
Sep 20 02:13:52.808780 ip-10-0-130-159 kubenswrapper[2584]: W0920 02:13:52.808751    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65d3c0b7_83df_4dfb_abc3_8e4bb010709b.slice/crio-5259f6b59fe1a49bc011339b85f61fccf978f235a74dbf9eda7c8adfe4d8e829 WatchSource:0}: Error finding container 5259f6b59fe1a49bc011339b85f61fccf978f235a74dbf9eda7c8adfe4d8e829: Status 404 returned error can't find the container with id 5259f6b59fe1a49bc011339b85f61fccf978f235a74dbf9eda7c8adfe4d8e829
Sep 20 02:13:52.833812 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:52.833772    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-f5813346-5897d44497-fc64d" event={"ID":"65d3c0b7-83df-4dfb-abc3-8e4bb010709b","Type":"ContainerStarted","Data":"5259f6b59fe1a49bc011339b85f61fccf978f235a74dbf9eda7c8adfe4d8e829"}
Sep 20 02:13:52.835349 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:52.835319    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f5813346-7ff9bcd677-zf6tr" event={"ID":"99a6dc33-0436-461b-b728-44523b5811b2","Type":"ContainerStarted","Data":"82c2128e04a73039c9799251ff6f2ed21e9104e41625d01ba9bafe254582a1a4"}
Sep 20 02:13:53.842805 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:53.842755    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-f5813346-5897d44497-fc64d" event={"ID":"65d3c0b7-83df-4dfb-abc3-8e4bb010709b","Type":"ContainerStarted","Data":"010a404b3048546ccf8855b21065874e02538b5326c998ca4face7f16bf41822"}
Sep 20 02:13:53.843292 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:53.842825    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-f5813346-5897d44497-fc64d"
Sep 20 02:13:53.844396 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:53.844364    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f5813346-7ff9bcd677-zf6tr" event={"ID":"99a6dc33-0436-461b-b728-44523b5811b2","Type":"ContainerStarted","Data":"359e66a2f2fe5659acc006650895b2a16aa7ca43594053f16aa9359af3212cf2"}
Sep 20 02:13:53.844538 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:53.844486    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f5813346-7ff9bcd677-zf6tr"
Sep 20 02:13:53.859631 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:53.859558    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-f5813346-5897d44497-fc64d" podStartSLOduration=1.8595375619999999 podStartE2EDuration="1.859537562s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-20 02:13:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-20 02:13:53.857856679 +0000 UTC m=+1290.600535884" watchObservedRunningTime="2026-09-20 02:13:53.859537562 +0000 UTC m=+1290.602216762"
Sep 20 02:13:53.874364 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:53.874320    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f5813346-7ff9bcd677-zf6tr" podStartSLOduration=2.557838808 podStartE2EDuration="2.874305256s" totalImagesPullingTime="316.466448ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-20 02:13:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-20 02:13:53.873063308 +0000 UTC m=+1290.615742514" watchObservedRunningTime="2026-09-20 02:13:53.874305256 +0000 UTC m=+1290.616984460"
Sep 20 02:13:59.856490 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:13:59.856458    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f5813346-7ff9bcd677-zf6tr"
Sep 20 02:14:04.852677 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:14:04.852644    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-f5813346-5897d44497-fc64d"
Sep 20 02:14:08.236880 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:14:08.236843    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:14:10.823139 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:14:10.823098    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:14:12.495809 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:14:12.495771    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:14:12.965172 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:14:12.965146    2584 generic.go:356] "Generic (PLEG): container finished" podID="2ce47e33-9674-4f3e-956b-d0083215f70f" containerID="bca3d5c8741a70b6c889361d94901593472e7a9704e4445def400e8b08425ec6" exitCode=0
Sep 20 02:14:12.965336 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:14:12.965215    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f5813346-openshift-default-dcd586ff6-cvqn5" event={"ID":"2ce47e33-9674-4f3e-956b-d0083215f70f","Type":"ContainerDied","Data":"bca3d5c8741a70b6c889361d94901593472e7a9704e4445def400e8b08425ec6"}
Sep 20 02:14:12.965652 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:14:12.965629    2584 scope.go:117] "RemoveContainer" containerID="bca3d5c8741a70b6c889361d94901593472e7a9704e4445def400e8b08425ec6"
Sep 20 02:14:12.966936 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:14:12.966907    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 20 02:14:12.967010 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:14:12.966978    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 20 02:14:12.967050 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:14:12.967008    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 20 02:14:13.631356 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:14:13.631311    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:14:13.815851 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:14:13.815812    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:14:13.978959 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:14:13.978911    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f5813346-openshift-default-dcd586ff6-cvqn5" event={"ID":"2ce47e33-9674-4f3e-956b-d0083215f70f","Type":"ContainerStarted","Data":"a53a4e1965e80180d9e01965a3fc86c2699c827300230335979c48becc473101"}
Sep 20 02:14:14.487665 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:14:14.487614    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-f5813346-openshift-default-dcd586ff6-cvqn5"
Sep 20 02:14:14.488215 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:14:14.488184    2584 patch_prober.go:28] interesting pod/e2e-mt-f5813346-openshift-default-dcd586ff6-cvqn5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Sep 20 02:14:14.488317 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:14:14.488253    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-f5813346-openshift-default-dcd586ff6-cvqn5" podUID="2ce47e33-9674-4f3e-956b-d0083215f70f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Sep 20 02:14:14.546980 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:14:14.546947    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:14:14.647804 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:14:14.647768    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:14:14.744801 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:14:14.744706    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:14:15.458169 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:14:15.458132    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:14:15.487297 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:14:15.487266    2584 patch_prober.go:28] interesting pod/e2e-mt-f5813346-openshift-default-dcd586ff6-cvqn5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Sep 20 02:14:15.487500 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:14:15.487450    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-f5813346-openshift-default-dcd586ff6-cvqn5" podUID="2ce47e33-9674-4f3e-956b-d0083215f70f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Sep 20 02:14:16.367011 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:14:16.366936    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:14:16.487078 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:14:16.487044    2584 patch_prober.go:28] interesting pod/e2e-mt-f5813346-openshift-default-dcd586ff6-cvqn5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Sep 20 02:14:16.487271 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:14:16.487099    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-f5813346-openshift-default-dcd586ff6-cvqn5" podUID="2ce47e33-9674-4f3e-956b-d0083215f70f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Sep 20 02:14:17.260753 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:14:17.260713    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:14:17.486668 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:14:17.486618    2584 patch_prober.go:28] interesting pod/e2e-mt-f5813346-openshift-default-dcd586ff6-cvqn5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Sep 20 02:14:17.487066 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:14:17.486695    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-f5813346-openshift-default-dcd586ff6-cvqn5" podUID="2ce47e33-9674-4f3e-956b-d0083215f70f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Sep 20 02:14:17.801991 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:14:17.801953    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-f5813346-openshift-default-dcd586ff6-cvqn5"
Sep 20 02:14:18.044366 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:14:18.044332    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:14:18.487156 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:14:18.487116    2584 patch_prober.go:28] interesting pod/e2e-mt-f5813346-openshift-default-dcd586ff6-cvqn5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Sep 20 02:14:18.487726 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:14:18.487697    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-f5813346-openshift-default-dcd586ff6-cvqn5" podUID="2ce47e33-9674-4f3e-956b-d0083215f70f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Sep 20 02:14:19.524306 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:14:19.524253    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-f5813346-openshift-default-dcd586ff6-cvqn5"
Sep 20 02:14:19.526446 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:14:19.526421    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-f5813346-openshift-default-dcd586ff6-cvqn5"
Sep 20 02:14:21.779888 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:14:21.779844    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:14:22.007663 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:14:22.007622    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:14:22.579686 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:14:22.579655    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:14:22.909203 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:14:22.909109    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:14:23.095208 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:14:23.095178    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:14:25.733608 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:14:25.733574    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:14:26.416007 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:14:26.415976    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:14:26.652350 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:14:26.652311    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:14:26.768586 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:14:26.768499    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:14:27.576803 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:14:27.576765    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:14:28.511251 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:14:28.511214    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:14:29.247435 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:14:29.247399    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:14:29.940025 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:14:29.939986    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:14:30.751913 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:14:30.751879    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:14:30.854014 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:14:30.853986    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:14:30.905195 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:14:30.905162    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:14:31.220757 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:14:31.220724    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:14:31.315328 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:14:31.315294    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:14:43.112197 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:14:43.112109    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-b15051/e2e-conflict-b-62c06db8-kserve-c7f7dbf8b-bskwp"]
Sep 20 02:14:43.116178 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:14:43.116156    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-b15051/e2e-conflict-b-62c06db8-kserve-c7f7dbf8b-bskwp"
Sep 20 02:14:43.118480 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:14:43.118463    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-b15051\"/\"e2e-conflict-b-62c06db8-kserve-self-signed-certs\""
Sep 20 02:14:43.124355 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:14:43.124333    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-b15051/e2e-conflict-b-62c06db8-kserve-c7f7dbf8b-bskwp"]
Sep 20 02:14:43.208634 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:14:43.208581    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:14:43.277406 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:14:43.277358    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a8313df0-7262-44a2-b0c1-7d2302d419f8-tmp-dir\") pod \"e2e-conflict-b-62c06db8-kserve-c7f7dbf8b-bskwp\" (UID: \"a8313df0-7262-44a2-b0c1-7d2302d419f8\") " pod="e2e-models-e2e-worker-w4-b15051/e2e-conflict-b-62c06db8-kserve-c7f7dbf8b-bskwp"
Sep 20 02:14:43.277641 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:14:43.277507    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a8313df0-7262-44a2-b0c1-7d2302d419f8-tls-certs\") pod \"e2e-conflict-b-62c06db8-kserve-c7f7dbf8b-bskwp\" (UID: \"a8313df0-7262-44a2-b0c1-7d2302d419f8\") " pod="e2e-models-e2e-worker-w4-b15051/e2e-conflict-b-62c06db8-kserve-c7f7dbf8b-bskwp"
Sep 20 02:14:43.277641 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:14:43.277582    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a8313df0-7262-44a2-b0c1-7d2302d419f8-model-cache\") pod \"e2e-conflict-b-62c06db8-kserve-c7f7dbf8b-bskwp\" (UID: \"a8313df0-7262-44a2-b0c1-7d2302d419f8\") " pod="e2e-models-e2e-worker-w4-b15051/e2e-conflict-b-62c06db8-kserve-c7f7dbf8b-bskwp"
Sep 20 02:14:43.277777 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:14:43.277756    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a8313df0-7262-44a2-b0c1-7d2302d419f8-dshm\") pod \"e2e-conflict-b-62c06db8-kserve-c7f7dbf8b-bskwp\" (UID: \"a8313df0-7262-44a2-b0c1-7d2302d419f8\") " pod="e2e-models-e2e-worker-w4-b15051/e2e-conflict-b-62c06db8-kserve-c7f7dbf8b-bskwp"
Sep 20 02:14:43.277838 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:14:43.277783    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a8313df0-7262-44a2-b0c1-7d2302d419f8-home\") pod \"e2e-conflict-b-62c06db8-kserve-c7f7dbf8b-bskwp\" (UID: \"a8313df0-7262-44a2-b0c1-7d2302d419f8\") " pod="e2e-models-e2e-worker-w4-b15051/e2e-conflict-b-62c06db8-kserve-c7f7dbf8b-bskwp"
Sep 20 02:14:43.277905 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:14:43.277872    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lb8c4\" (UniqueName: \"kubernetes.io/projected/a8313df0-7262-44a2-b0c1-7d2302d419f8-kube-api-access-lb8c4\") pod \"e2e-conflict-b-62c06db8-kserve-c7f7dbf8b-bskwp\" (UID: \"a8313df0-7262-44a2-b0c1-7d2302d419f8\") " pod="e2e-models-e2e-worker-w4-b15051/e2e-conflict-b-62c06db8-kserve-c7f7dbf8b-bskwp"
Sep 20 02:14:43.378456 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:14:43.378365    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a8313df0-7262-44a2-b0c1-7d2302d419f8-tmp-dir\") pod \"e2e-conflict-b-62c06db8-kserve-c7f7dbf8b-bskwp\" (UID: \"a8313df0-7262-44a2-b0c1-7d2302d419f8\") " pod="e2e-models-e2e-worker-w4-b15051/e2e-conflict-b-62c06db8-kserve-c7f7dbf8b-bskwp"
Sep 20 02:14:43.378456 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:14:43.378438    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a8313df0-7262-44a2-b0c1-7d2302d419f8-tls-certs\") pod \"e2e-conflict-b-62c06db8-kserve-c7f7dbf8b-bskwp\" (UID: \"a8313df0-7262-44a2-b0c1-7d2302d419f8\") " pod="e2e-models-e2e-worker-w4-b15051/e2e-conflict-b-62c06db8-kserve-c7f7dbf8b-bskwp"
Sep 20 02:14:43.378721 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:14:43.378570    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a8313df0-7262-44a2-b0c1-7d2302d419f8-model-cache\") pod \"e2e-conflict-b-62c06db8-kserve-c7f7dbf8b-bskwp\" (UID: \"a8313df0-7262-44a2-b0c1-7d2302d419f8\") " pod="e2e-models-e2e-worker-w4-b15051/e2e-conflict-b-62c06db8-kserve-c7f7dbf8b-bskwp"
Sep 20 02:14:43.378721 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:14:43.378663    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a8313df0-7262-44a2-b0c1-7d2302d419f8-dshm\") pod \"e2e-conflict-b-62c06db8-kserve-c7f7dbf8b-bskwp\" (UID: \"a8313df0-7262-44a2-b0c1-7d2302d419f8\") " pod="e2e-models-e2e-worker-w4-b15051/e2e-conflict-b-62c06db8-kserve-c7f7dbf8b-bskwp"
Sep 20 02:14:43.378721 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:14:43.378698    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a8313df0-7262-44a2-b0c1-7d2302d419f8-home\") pod \"e2e-conflict-b-62c06db8-kserve-c7f7dbf8b-bskwp\" (UID: \"a8313df0-7262-44a2-b0c1-7d2302d419f8\") " pod="e2e-models-e2e-worker-w4-b15051/e2e-conflict-b-62c06db8-kserve-c7f7dbf8b-bskwp"
Sep 20 02:14:43.378891 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:14:43.378801    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a8313df0-7262-44a2-b0c1-7d2302d419f8-tmp-dir\") pod \"e2e-conflict-b-62c06db8-kserve-c7f7dbf8b-bskwp\" (UID: \"a8313df0-7262-44a2-b0c1-7d2302d419f8\") " pod="e2e-models-e2e-worker-w4-b15051/e2e-conflict-b-62c06db8-kserve-c7f7dbf8b-bskwp"
Sep 20 02:14:43.378891 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:14:43.378831    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lb8c4\" (UniqueName: \"kubernetes.io/projected/a8313df0-7262-44a2-b0c1-7d2302d419f8-kube-api-access-lb8c4\") pod \"e2e-conflict-b-62c06db8-kserve-c7f7dbf8b-bskwp\" (UID: \"a8313df0-7262-44a2-b0c1-7d2302d419f8\") " pod="e2e-models-e2e-worker-w4-b15051/e2e-conflict-b-62c06db8-kserve-c7f7dbf8b-bskwp"
Sep 20 02:14:43.378891 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:14:43.378873    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a8313df0-7262-44a2-b0c1-7d2302d419f8-model-cache\") pod \"e2e-conflict-b-62c06db8-kserve-c7f7dbf8b-bskwp\" (UID: \"a8313df0-7262-44a2-b0c1-7d2302d419f8\") " pod="e2e-models-e2e-worker-w4-b15051/e2e-conflict-b-62c06db8-kserve-c7f7dbf8b-bskwp"
Sep 20 02:14:43.379058 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:14:43.378997    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a8313df0-7262-44a2-b0c1-7d2302d419f8-home\") pod \"e2e-conflict-b-62c06db8-kserve-c7f7dbf8b-bskwp\" (UID: \"a8313df0-7262-44a2-b0c1-7d2302d419f8\") " pod="e2e-models-e2e-worker-w4-b15051/e2e-conflict-b-62c06db8-kserve-c7f7dbf8b-bskwp"
Sep 20 02:14:43.380809 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:14:43.380787    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a8313df0-7262-44a2-b0c1-7d2302d419f8-dshm\") pod \"e2e-conflict-b-62c06db8-kserve-c7f7dbf8b-bskwp\" (UID: \"a8313df0-7262-44a2-b0c1-7d2302d419f8\") " pod="e2e-models-e2e-worker-w4-b15051/e2e-conflict-b-62c06db8-kserve-c7f7dbf8b-bskwp"
Sep 20 02:14:43.381081 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:14:43.381064    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a8313df0-7262-44a2-b0c1-7d2302d419f8-tls-certs\") pod \"e2e-conflict-b-62c06db8-kserve-c7f7dbf8b-bskwp\" (UID: \"a8313df0-7262-44a2-b0c1-7d2302d419f8\") " pod="e2e-models-e2e-worker-w4-b15051/e2e-conflict-b-62c06db8-kserve-c7f7dbf8b-bskwp"
Sep 20 02:14:43.389638 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:14:43.389616    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lb8c4\" (UniqueName: \"kubernetes.io/projected/a8313df0-7262-44a2-b0c1-7d2302d419f8-kube-api-access-lb8c4\") pod \"e2e-conflict-b-62c06db8-kserve-c7f7dbf8b-bskwp\" (UID: \"a8313df0-7262-44a2-b0c1-7d2302d419f8\") " pod="e2e-models-e2e-worker-w4-b15051/e2e-conflict-b-62c06db8-kserve-c7f7dbf8b-bskwp"
Sep 20 02:14:43.427058 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:14:43.427027    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-b15051/e2e-conflict-b-62c06db8-kserve-c7f7dbf8b-bskwp"
Sep 20 02:14:43.702029 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:14:43.701999    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-b15051/e2e-conflict-b-62c06db8-kserve-c7f7dbf8b-bskwp"]
Sep 20 02:14:43.704255 ip-10-0-130-159 kubenswrapper[2584]: W0920 02:14:43.704229    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda8313df0_7262_44a2_b0c1_7d2302d419f8.slice/crio-8175be3c47ffd6e7b316c272ac963f456e319601aacfbab6df289406ff4fc63b WatchSource:0}: Error finding container 8175be3c47ffd6e7b316c272ac963f456e319601aacfbab6df289406ff4fc63b: Status 404 returned error can't find the container with id 8175be3c47ffd6e7b316c272ac963f456e319601aacfbab6df289406ff4fc63b
Sep 20 02:14:44.149585 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:14:44.149484    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-b15051/e2e-conflict-b-62c06db8-kserve-c7f7dbf8b-bskwp" event={"ID":"a8313df0-7262-44a2-b0c1-7d2302d419f8","Type":"ContainerStarted","Data":"011ea31ecc88652a0db13fcd66dca0e6078aca0ddc4a0fa0fffabe7805a33799"}
Sep 20 02:14:44.149585 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:14:44.149540    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-b15051/e2e-conflict-b-62c06db8-kserve-c7f7dbf8b-bskwp" event={"ID":"a8313df0-7262-44a2-b0c1-7d2302d419f8","Type":"ContainerStarted","Data":"8175be3c47ffd6e7b316c272ac963f456e319601aacfbab6df289406ff4fc63b"}
Sep 20 02:14:44.169057 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:14:44.169002    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-b15051/e2e-conflict-b-62c06db8-kserve-c7f7dbf8b-bskwp" podStartSLOduration=1.168983958 podStartE2EDuration="1.168983958s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-20 02:14:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-20 02:14:44.166320538 +0000 UTC m=+1340.908999743" watchObservedRunningTime="2026-09-20 02:14:44.168983958 +0000 UTC m=+1340.911663162"
Sep 20 02:14:52.892920 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:14:52.892883    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:14:53.414655 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:14:53.414619    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:14:53.428144 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:14:53.428114    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-b15051/e2e-conflict-b-62c06db8-kserve-c7f7dbf8b-bskwp"
Sep 20 02:14:53.428286 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:14:53.428167    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-b15051/e2e-conflict-b-62c06db8-kserve-c7f7dbf8b-bskwp"
Sep 20 02:14:53.441352 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:14:53.441328    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-b15051/e2e-conflict-b-62c06db8-kserve-c7f7dbf8b-bskwp"
Sep 20 02:14:54.051345 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:14:54.051309    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:14:54.215904 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:14:54.215877    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-b15051/e2e-conflict-b-62c06db8-kserve-c7f7dbf8b-bskwp"
Sep 20 02:14:54.727701 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:14:54.727668    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:14:55.552987 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:14:55.552953    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:14:56.241832 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:14:56.241796    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:14:56.517846 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:14:56.517760    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:14:56.861066 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:14:56.860961    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:14:56.961802 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:14:56.961743    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:14:57.175635 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:14:57.175542    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:14:57.316873 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:14:57.316843    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:15:00.557870 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:15:00.557835    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:15:00.701352 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:15:00.701323    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:15:00.793501 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:15:00.793474    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:15:01.621799 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:15:01.621759    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:15:02.231989 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:15:02.231957    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:15:03.040163 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:15:03.040133    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:15:03.739610 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:15:03.739578    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:15:04.385346 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:15:04.385313    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:15:05.234241 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:15:05.234206    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-317780-8d7f49969-l2qbf"]
Sep 20 02:15:05.238617 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:15:05.238589    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-317780-8d7f49969-l2qbf"
Sep 20 02:15:05.240728 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:15:05.240703    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w3-317780-serving-cert\""
Sep 20 02:15:05.249382 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:15:05.249353    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-317780-8d7f49969-l2qbf"]
Sep 20 02:15:05.270727 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:15:05.270693    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:15:05.290933 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:15:05.290901    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/796b838b-dd79-4953-bf6c-91e3326f3e4c-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-317780-8d7f49969-l2qbf\" (UID: \"796b838b-dd79-4953-bf6c-91e3326f3e4c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-317780-8d7f49969-l2qbf"
Sep 20 02:15:05.291121 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:15:05.290946    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jck2c\" (UniqueName: \"kubernetes.io/projected/796b838b-dd79-4953-bf6c-91e3326f3e4c-kube-api-access-jck2c\") pod \"maas-api-e2e-shared-b-w3-317780-8d7f49969-l2qbf\" (UID: \"796b838b-dd79-4953-bf6c-91e3326f3e4c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-317780-8d7f49969-l2qbf"
Sep 20 02:15:05.291121 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:15:05.290974    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/796b838b-dd79-4953-bf6c-91e3326f3e4c-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-317780-8d7f49969-l2qbf\" (UID: \"796b838b-dd79-4953-bf6c-91e3326f3e4c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-317780-8d7f49969-l2qbf"
Sep 20 02:15:05.291121 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:15:05.291091    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/796b838b-dd79-4953-bf6c-91e3326f3e4c-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-317780-8d7f49969-l2qbf\" (UID: \"796b838b-dd79-4953-bf6c-91e3326f3e4c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-317780-8d7f49969-l2qbf"
Sep 20 02:15:05.370087 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:15:05.370055    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:15:05.391999 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:15:05.391975    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jck2c\" (UniqueName: \"kubernetes.io/projected/796b838b-dd79-4953-bf6c-91e3326f3e4c-kube-api-access-jck2c\") pod \"maas-api-e2e-shared-b-w3-317780-8d7f49969-l2qbf\" (UID: \"796b838b-dd79-4953-bf6c-91e3326f3e4c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-317780-8d7f49969-l2qbf"
Sep 20 02:15:05.392446 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:15:05.392011    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/796b838b-dd79-4953-bf6c-91e3326f3e4c-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-317780-8d7f49969-l2qbf\" (UID: \"796b838b-dd79-4953-bf6c-91e3326f3e4c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-317780-8d7f49969-l2qbf"
Sep 20 02:15:05.392446 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:15:05.392071    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/796b838b-dd79-4953-bf6c-91e3326f3e4c-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-317780-8d7f49969-l2qbf\" (UID: \"796b838b-dd79-4953-bf6c-91e3326f3e4c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-317780-8d7f49969-l2qbf"
Sep 20 02:15:05.392446 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:15:05.392117    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/796b838b-dd79-4953-bf6c-91e3326f3e4c-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-317780-8d7f49969-l2qbf\" (UID: \"796b838b-dd79-4953-bf6c-91e3326f3e4c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-317780-8d7f49969-l2qbf"
Sep 20 02:15:05.392753 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:15:05.392728    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/796b838b-dd79-4953-bf6c-91e3326f3e4c-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-317780-8d7f49969-l2qbf\" (UID: \"796b838b-dd79-4953-bf6c-91e3326f3e4c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-317780-8d7f49969-l2qbf"
Sep 20 02:15:05.394634 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:15:05.394603    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/796b838b-dd79-4953-bf6c-91e3326f3e4c-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-317780-8d7f49969-l2qbf\" (UID: \"796b838b-dd79-4953-bf6c-91e3326f3e4c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-317780-8d7f49969-l2qbf"
Sep 20 02:15:05.395117 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:15:05.395093    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/796b838b-dd79-4953-bf6c-91e3326f3e4c-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-317780-8d7f49969-l2qbf\" (UID: \"796b838b-dd79-4953-bf6c-91e3326f3e4c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-317780-8d7f49969-l2qbf"
Sep 20 02:15:05.404879 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:15:05.404854    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jck2c\" (UniqueName: \"kubernetes.io/projected/796b838b-dd79-4953-bf6c-91e3326f3e4c-kube-api-access-jck2c\") pod \"maas-api-e2e-shared-b-w3-317780-8d7f49969-l2qbf\" (UID: \"796b838b-dd79-4953-bf6c-91e3326f3e4c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-317780-8d7f49969-l2qbf"
Sep 20 02:15:05.423857 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:15:05.423827    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:15:05.552726 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:15:05.552466    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-317780-8d7f49969-l2qbf"
Sep 20 02:15:05.732067 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:15:05.732038    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:15:05.837172 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:15:05.837147    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-317780-8d7f49969-l2qbf"]
Sep 20 02:15:05.838737 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:15:05.838708    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:15:05.839137 ip-10-0-130-159 kubenswrapper[2584]: W0920 02:15:05.839118    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod796b838b_dd79_4953_bf6c_91e3326f3e4c.slice/crio-56719b45818c62a668b8c9df82d85dcda8bf66a34a9bdf541bbec9844c52ae01 WatchSource:0}: Error finding container 56719b45818c62a668b8c9df82d85dcda8bf66a34a9bdf541bbec9844c52ae01: Status 404 returned error can't find the container with id 56719b45818c62a668b8c9df82d85dcda8bf66a34a9bdf541bbec9844c52ae01
Sep 20 02:15:06.273383 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:15:06.273349    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-317780-8d7f49969-l2qbf" event={"ID":"796b838b-dd79-4953-bf6c-91e3326f3e4c","Type":"ContainerStarted","Data":"4636529cb369b9fdd0314c8dde1f37c50cd646cb218b6f50c41f47a87355482e"}
Sep 20 02:15:06.273606 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:15:06.273410    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-317780-8d7f49969-l2qbf" event={"ID":"796b838b-dd79-4953-bf6c-91e3326f3e4c","Type":"ContainerStarted","Data":"56719b45818c62a668b8c9df82d85dcda8bf66a34a9bdf541bbec9844c52ae01"}
Sep 20 02:15:06.273606 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:15:06.273455    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-317780-8d7f49969-l2qbf"
Sep 20 02:15:06.290903 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:15:06.290850    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-317780-8d7f49969-l2qbf" podStartSLOduration=1.002092088 podStartE2EDuration="1.290834155s" totalImagesPullingTime="288.742067ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-20 02:15:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-20 02:15:06.28907964 +0000 UTC m=+1363.031758847" watchObservedRunningTime="2026-09-20 02:15:06.290834155 +0000 UTC m=+1363.033513407"
Sep 20 02:15:12.285276 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:15:12.285247    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-317780-8d7f49969-l2qbf"
Sep 20 02:15:22.218754 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:15:22.218712    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:15:28.246247 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:15:28.246212    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:15:28.924686 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:15:28.924630    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:15:29.604671 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:15:29.604635    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:15:30.449914 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:15:30.449881    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:15:30.795790 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:15:30.795705    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:15:31.173976 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:15:31.173892    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:15:31.963566 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:15:31.963534    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:15:32.637824 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:15:32.637790    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:15:33.446287 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:15:33.446253    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:15:38.411694 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:15:38.411652    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-8195ba/distinct-2-w0-8195ba-kserve-86f8d74d76-nh8n5"]
Sep 20 02:15:38.415563 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:15:38.415541    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-8195ba/distinct-2-w0-8195ba-kserve-86f8d74d76-nh8n5"
Sep 20 02:15:38.418419 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:15:38.418398    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-8195ba\"/\"kube-root-ca.crt\""
Sep 20 02:15:38.418622 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:15:38.418606    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-8195ba\"/\"default-dockercfg-s9bwr\""
Sep 20 02:15:38.419251 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:15:38.419231    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-8195ba\"/\"openshift-service-ca.crt\""
Sep 20 02:15:38.419358 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:15:38.419307    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-8195ba\"/\"distinct-2-w0-8195ba-kserve-self-signed-certs\""
Sep 20 02:15:38.427032 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:15:38.427010    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-8195ba/distinct-2-w0-8195ba-kserve-86f8d74d76-nh8n5"]
Sep 20 02:15:38.446643 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:15:38.446619    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:15:38.519659 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:15:38.519633    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/61a65af9-c40c-4f89-9d50-fbe0599e4f07-model-cache\") pod \"distinct-2-w0-8195ba-kserve-86f8d74d76-nh8n5\" (UID: \"61a65af9-c40c-4f89-9d50-fbe0599e4f07\") " pod="e2e-models-e2e-worker-w0-8195ba/distinct-2-w0-8195ba-kserve-86f8d74d76-nh8n5"
Sep 20 02:15:38.519828 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:15:38.519703    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r8vz6\" (UniqueName: \"kubernetes.io/projected/61a65af9-c40c-4f89-9d50-fbe0599e4f07-kube-api-access-r8vz6\") pod \"distinct-2-w0-8195ba-kserve-86f8d74d76-nh8n5\" (UID: \"61a65af9-c40c-4f89-9d50-fbe0599e4f07\") " pod="e2e-models-e2e-worker-w0-8195ba/distinct-2-w0-8195ba-kserve-86f8d74d76-nh8n5"
Sep 20 02:15:38.519828 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:15:38.519797    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/61a65af9-c40c-4f89-9d50-fbe0599e4f07-dshm\") pod \"distinct-2-w0-8195ba-kserve-86f8d74d76-nh8n5\" (UID: \"61a65af9-c40c-4f89-9d50-fbe0599e4f07\") " pod="e2e-models-e2e-worker-w0-8195ba/distinct-2-w0-8195ba-kserve-86f8d74d76-nh8n5"
Sep 20 02:15:38.519966 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:15:38.519838    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/61a65af9-c40c-4f89-9d50-fbe0599e4f07-tls-certs\") pod \"distinct-2-w0-8195ba-kserve-86f8d74d76-nh8n5\" (UID: \"61a65af9-c40c-4f89-9d50-fbe0599e4f07\") " pod="e2e-models-e2e-worker-w0-8195ba/distinct-2-w0-8195ba-kserve-86f8d74d76-nh8n5"
Sep 20 02:15:38.519966 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:15:38.519868    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/61a65af9-c40c-4f89-9d50-fbe0599e4f07-tmp-dir\") pod \"distinct-2-w0-8195ba-kserve-86f8d74d76-nh8n5\" (UID: \"61a65af9-c40c-4f89-9d50-fbe0599e4f07\") " pod="e2e-models-e2e-worker-w0-8195ba/distinct-2-w0-8195ba-kserve-86f8d74d76-nh8n5"
Sep 20 02:15:38.519966 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:15:38.519894    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/61a65af9-c40c-4f89-9d50-fbe0599e4f07-home\") pod \"distinct-2-w0-8195ba-kserve-86f8d74d76-nh8n5\" (UID: \"61a65af9-c40c-4f89-9d50-fbe0599e4f07\") " pod="e2e-models-e2e-worker-w0-8195ba/distinct-2-w0-8195ba-kserve-86f8d74d76-nh8n5"
Sep 20 02:15:38.621346 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:15:38.621317    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r8vz6\" (UniqueName: \"kubernetes.io/projected/61a65af9-c40c-4f89-9d50-fbe0599e4f07-kube-api-access-r8vz6\") pod \"distinct-2-w0-8195ba-kserve-86f8d74d76-nh8n5\" (UID: \"61a65af9-c40c-4f89-9d50-fbe0599e4f07\") " pod="e2e-models-e2e-worker-w0-8195ba/distinct-2-w0-8195ba-kserve-86f8d74d76-nh8n5"
Sep 20 02:15:38.621569 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:15:38.621369    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/61a65af9-c40c-4f89-9d50-fbe0599e4f07-dshm\") pod \"distinct-2-w0-8195ba-kserve-86f8d74d76-nh8n5\" (UID: \"61a65af9-c40c-4f89-9d50-fbe0599e4f07\") " pod="e2e-models-e2e-worker-w0-8195ba/distinct-2-w0-8195ba-kserve-86f8d74d76-nh8n5"
Sep 20 02:15:38.621569 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:15:38.621401    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/61a65af9-c40c-4f89-9d50-fbe0599e4f07-tls-certs\") pod \"distinct-2-w0-8195ba-kserve-86f8d74d76-nh8n5\" (UID: \"61a65af9-c40c-4f89-9d50-fbe0599e4f07\") " pod="e2e-models-e2e-worker-w0-8195ba/distinct-2-w0-8195ba-kserve-86f8d74d76-nh8n5"
Sep 20 02:15:38.621569 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:15:38.621420    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/61a65af9-c40c-4f89-9d50-fbe0599e4f07-tmp-dir\") pod \"distinct-2-w0-8195ba-kserve-86f8d74d76-nh8n5\" (UID: \"61a65af9-c40c-4f89-9d50-fbe0599e4f07\") " pod="e2e-models-e2e-worker-w0-8195ba/distinct-2-w0-8195ba-kserve-86f8d74d76-nh8n5"
Sep 20 02:15:38.621569 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:15:38.621437    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/61a65af9-c40c-4f89-9d50-fbe0599e4f07-home\") pod \"distinct-2-w0-8195ba-kserve-86f8d74d76-nh8n5\" (UID: \"61a65af9-c40c-4f89-9d50-fbe0599e4f07\") " pod="e2e-models-e2e-worker-w0-8195ba/distinct-2-w0-8195ba-kserve-86f8d74d76-nh8n5"
Sep 20 02:15:38.621569 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:15:38.621478    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/61a65af9-c40c-4f89-9d50-fbe0599e4f07-model-cache\") pod \"distinct-2-w0-8195ba-kserve-86f8d74d76-nh8n5\" (UID: \"61a65af9-c40c-4f89-9d50-fbe0599e4f07\") " pod="e2e-models-e2e-worker-w0-8195ba/distinct-2-w0-8195ba-kserve-86f8d74d76-nh8n5"
Sep 20 02:15:38.621852 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:15:38.621827    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/61a65af9-c40c-4f89-9d50-fbe0599e4f07-tmp-dir\") pod \"distinct-2-w0-8195ba-kserve-86f8d74d76-nh8n5\" (UID: \"61a65af9-c40c-4f89-9d50-fbe0599e4f07\") " pod="e2e-models-e2e-worker-w0-8195ba/distinct-2-w0-8195ba-kserve-86f8d74d76-nh8n5"
Sep 20 02:15:38.621912 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:15:38.621854    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/61a65af9-c40c-4f89-9d50-fbe0599e4f07-home\") pod \"distinct-2-w0-8195ba-kserve-86f8d74d76-nh8n5\" (UID: \"61a65af9-c40c-4f89-9d50-fbe0599e4f07\") " pod="e2e-models-e2e-worker-w0-8195ba/distinct-2-w0-8195ba-kserve-86f8d74d76-nh8n5"
Sep 20 02:15:38.621972 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:15:38.621955    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/61a65af9-c40c-4f89-9d50-fbe0599e4f07-model-cache\") pod \"distinct-2-w0-8195ba-kserve-86f8d74d76-nh8n5\" (UID: \"61a65af9-c40c-4f89-9d50-fbe0599e4f07\") " pod="e2e-models-e2e-worker-w0-8195ba/distinct-2-w0-8195ba-kserve-86f8d74d76-nh8n5"
Sep 20 02:15:38.623719 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:15:38.623698    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/61a65af9-c40c-4f89-9d50-fbe0599e4f07-dshm\") pod \"distinct-2-w0-8195ba-kserve-86f8d74d76-nh8n5\" (UID: \"61a65af9-c40c-4f89-9d50-fbe0599e4f07\") " pod="e2e-models-e2e-worker-w0-8195ba/distinct-2-w0-8195ba-kserve-86f8d74d76-nh8n5"
Sep 20 02:15:38.624049 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:15:38.624033    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/61a65af9-c40c-4f89-9d50-fbe0599e4f07-tls-certs\") pod \"distinct-2-w0-8195ba-kserve-86f8d74d76-nh8n5\" (UID: \"61a65af9-c40c-4f89-9d50-fbe0599e4f07\") " pod="e2e-models-e2e-worker-w0-8195ba/distinct-2-w0-8195ba-kserve-86f8d74d76-nh8n5"
Sep 20 02:15:38.629299 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:15:38.629280    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r8vz6\" (UniqueName: \"kubernetes.io/projected/61a65af9-c40c-4f89-9d50-fbe0599e4f07-kube-api-access-r8vz6\") pod \"distinct-2-w0-8195ba-kserve-86f8d74d76-nh8n5\" (UID: \"61a65af9-c40c-4f89-9d50-fbe0599e4f07\") " pod="e2e-models-e2e-worker-w0-8195ba/distinct-2-w0-8195ba-kserve-86f8d74d76-nh8n5"
Sep 20 02:15:38.728559 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:15:38.728511    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-8195ba/distinct-2-w0-8195ba-kserve-86f8d74d76-nh8n5"
Sep 20 02:15:39.001970 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:15:39.001947    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-8195ba/distinct-2-w0-8195ba-kserve-86f8d74d76-nh8n5"]
Sep 20 02:15:39.003637 ip-10-0-130-159 kubenswrapper[2584]: W0920 02:15:39.003607    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod61a65af9_c40c_4f89_9d50_fbe0599e4f07.slice/crio-e44402c4b6538960d5cf714b182afb8f29a497ae39e92c9009df4791fb8eb7f4 WatchSource:0}: Error finding container e44402c4b6538960d5cf714b182afb8f29a497ae39e92c9009df4791fb8eb7f4: Status 404 returned error can't find the container with id e44402c4b6538960d5cf714b182afb8f29a497ae39e92c9009df4791fb8eb7f4
Sep 20 02:15:39.462488 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:15:39.462449    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-8195ba/distinct-2-w0-8195ba-kserve-86f8d74d76-nh8n5" event={"ID":"61a65af9-c40c-4f89-9d50-fbe0599e4f07","Type":"ContainerStarted","Data":"4b3d3e4ee07423b379957d27f1bee462f89ccec4bd4d90bd53133246e7952c14"}
Sep 20 02:15:39.462997 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:15:39.462494    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-8195ba/distinct-2-w0-8195ba-kserve-86f8d74d76-nh8n5" event={"ID":"61a65af9-c40c-4f89-9d50-fbe0599e4f07","Type":"ContainerStarted","Data":"e44402c4b6538960d5cf714b182afb8f29a497ae39e92c9009df4791fb8eb7f4"}
Sep 20 02:15:39.479648 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:15:39.479598    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-8195ba/distinct-2-w0-8195ba-kserve-86f8d74d76-nh8n5" podStartSLOduration=1.47958185 podStartE2EDuration="1.47958185s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-20 02:15:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-20 02:15:39.477261178 +0000 UTC m=+1396.219940396" watchObservedRunningTime="2026-09-20 02:15:39.47958185 +0000 UTC m=+1396.222261063"
Sep 20 02:15:48.729398 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:15:48.729353    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-8195ba/distinct-2-w0-8195ba-kserve-86f8d74d76-nh8n5"
Sep 20 02:15:48.729937 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:15:48.729415    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-8195ba/distinct-2-w0-8195ba-kserve-86f8d74d76-nh8n5"
Sep 20 02:15:48.743448 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:15:48.743415    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-8195ba/distinct-2-w0-8195ba-kserve-86f8d74d76-nh8n5"
Sep 20 02:15:49.528264 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:15:49.528223    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-8195ba/distinct-2-w0-8195ba-kserve-86f8d74d76-nh8n5"
Sep 20 02:15:55.866987 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:15:55.866954    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:16:02.281536 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:02.281488    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:16:02.382836 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:02.382799    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:16:12.177818 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:12.177741    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:16:16.403212 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:16.403179    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-ed9f38/auth-test-model-ed9f38-kserve-5f7845bb7-gc7tn"]
Sep 20 02:16:16.407142 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:16.407120    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-ed9f38/auth-test-model-ed9f38-kserve-5f7845bb7-gc7tn"
Sep 20 02:16:16.409671 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:16.409644    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-ed9f38\"/\"openshift-service-ca.crt\""
Sep 20 02:16:16.410492 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:16.410467    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-ed9f38\"/\"auth-test-model-ed9f38-kserve-self-signed-certs\""
Sep 20 02:16:16.410645 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:16.410547    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-ed9f38\"/\"default-dockercfg-692d7\""
Sep 20 02:16:16.410645 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:16.410607    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-ed9f38\"/\"kube-root-ca.crt\""
Sep 20 02:16:16.417810 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:16.417776    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-ed9f38/auth-test-model-ed9f38-kserve-5f7845bb7-gc7tn"]
Sep 20 02:16:16.434839 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:16.434805    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:16:16.496364 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:16.496333    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d7e8345d-0ed1-4ede-bcaf-de80522b6bef-tmp-dir\") pod \"auth-test-model-ed9f38-kserve-5f7845bb7-gc7tn\" (UID: \"d7e8345d-0ed1-4ede-bcaf-de80522b6bef\") " pod="ai-tenant-e2e-shared-a-w3-ed9f38/auth-test-model-ed9f38-kserve-5f7845bb7-gc7tn"
Sep 20 02:16:16.496364 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:16.496366    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d7e8345d-0ed1-4ede-bcaf-de80522b6bef-tls-certs\") pod \"auth-test-model-ed9f38-kserve-5f7845bb7-gc7tn\" (UID: \"d7e8345d-0ed1-4ede-bcaf-de80522b6bef\") " pod="ai-tenant-e2e-shared-a-w3-ed9f38/auth-test-model-ed9f38-kserve-5f7845bb7-gc7tn"
Sep 20 02:16:16.496614 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:16.496393    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d7e8345d-0ed1-4ede-bcaf-de80522b6bef-model-cache\") pod \"auth-test-model-ed9f38-kserve-5f7845bb7-gc7tn\" (UID: \"d7e8345d-0ed1-4ede-bcaf-de80522b6bef\") " pod="ai-tenant-e2e-shared-a-w3-ed9f38/auth-test-model-ed9f38-kserve-5f7845bb7-gc7tn"
Sep 20 02:16:16.496614 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:16.496539    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d7e8345d-0ed1-4ede-bcaf-de80522b6bef-dshm\") pod \"auth-test-model-ed9f38-kserve-5f7845bb7-gc7tn\" (UID: \"d7e8345d-0ed1-4ede-bcaf-de80522b6bef\") " pod="ai-tenant-e2e-shared-a-w3-ed9f38/auth-test-model-ed9f38-kserve-5f7845bb7-gc7tn"
Sep 20 02:16:16.496614 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:16.496568    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8n8g2\" (UniqueName: \"kubernetes.io/projected/d7e8345d-0ed1-4ede-bcaf-de80522b6bef-kube-api-access-8n8g2\") pod \"auth-test-model-ed9f38-kserve-5f7845bb7-gc7tn\" (UID: \"d7e8345d-0ed1-4ede-bcaf-de80522b6bef\") " pod="ai-tenant-e2e-shared-a-w3-ed9f38/auth-test-model-ed9f38-kserve-5f7845bb7-gc7tn"
Sep 20 02:16:16.496718 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:16.496664    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d7e8345d-0ed1-4ede-bcaf-de80522b6bef-home\") pod \"auth-test-model-ed9f38-kserve-5f7845bb7-gc7tn\" (UID: \"d7e8345d-0ed1-4ede-bcaf-de80522b6bef\") " pod="ai-tenant-e2e-shared-a-w3-ed9f38/auth-test-model-ed9f38-kserve-5f7845bb7-gc7tn"
Sep 20 02:16:16.597456 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:16.597419    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d7e8345d-0ed1-4ede-bcaf-de80522b6bef-dshm\") pod \"auth-test-model-ed9f38-kserve-5f7845bb7-gc7tn\" (UID: \"d7e8345d-0ed1-4ede-bcaf-de80522b6bef\") " pod="ai-tenant-e2e-shared-a-w3-ed9f38/auth-test-model-ed9f38-kserve-5f7845bb7-gc7tn"
Sep 20 02:16:16.597456 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:16.597459    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8n8g2\" (UniqueName: \"kubernetes.io/projected/d7e8345d-0ed1-4ede-bcaf-de80522b6bef-kube-api-access-8n8g2\") pod \"auth-test-model-ed9f38-kserve-5f7845bb7-gc7tn\" (UID: \"d7e8345d-0ed1-4ede-bcaf-de80522b6bef\") " pod="ai-tenant-e2e-shared-a-w3-ed9f38/auth-test-model-ed9f38-kserve-5f7845bb7-gc7tn"
Sep 20 02:16:16.597743 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:16.597560    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d7e8345d-0ed1-4ede-bcaf-de80522b6bef-home\") pod \"auth-test-model-ed9f38-kserve-5f7845bb7-gc7tn\" (UID: \"d7e8345d-0ed1-4ede-bcaf-de80522b6bef\") " pod="ai-tenant-e2e-shared-a-w3-ed9f38/auth-test-model-ed9f38-kserve-5f7845bb7-gc7tn"
Sep 20 02:16:16.597743 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:16.597605    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d7e8345d-0ed1-4ede-bcaf-de80522b6bef-tmp-dir\") pod \"auth-test-model-ed9f38-kserve-5f7845bb7-gc7tn\" (UID: \"d7e8345d-0ed1-4ede-bcaf-de80522b6bef\") " pod="ai-tenant-e2e-shared-a-w3-ed9f38/auth-test-model-ed9f38-kserve-5f7845bb7-gc7tn"
Sep 20 02:16:16.597743 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:16.597633    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d7e8345d-0ed1-4ede-bcaf-de80522b6bef-tls-certs\") pod \"auth-test-model-ed9f38-kserve-5f7845bb7-gc7tn\" (UID: \"d7e8345d-0ed1-4ede-bcaf-de80522b6bef\") " pod="ai-tenant-e2e-shared-a-w3-ed9f38/auth-test-model-ed9f38-kserve-5f7845bb7-gc7tn"
Sep 20 02:16:16.597743 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:16.597670    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d7e8345d-0ed1-4ede-bcaf-de80522b6bef-model-cache\") pod \"auth-test-model-ed9f38-kserve-5f7845bb7-gc7tn\" (UID: \"d7e8345d-0ed1-4ede-bcaf-de80522b6bef\") " pod="ai-tenant-e2e-shared-a-w3-ed9f38/auth-test-model-ed9f38-kserve-5f7845bb7-gc7tn"
Sep 20 02:16:16.597961 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:16.597940    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d7e8345d-0ed1-4ede-bcaf-de80522b6bef-home\") pod \"auth-test-model-ed9f38-kserve-5f7845bb7-gc7tn\" (UID: \"d7e8345d-0ed1-4ede-bcaf-de80522b6bef\") " pod="ai-tenant-e2e-shared-a-w3-ed9f38/auth-test-model-ed9f38-kserve-5f7845bb7-gc7tn"
Sep 20 02:16:16.598021 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:16.597952    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d7e8345d-0ed1-4ede-bcaf-de80522b6bef-tmp-dir\") pod \"auth-test-model-ed9f38-kserve-5f7845bb7-gc7tn\" (UID: \"d7e8345d-0ed1-4ede-bcaf-de80522b6bef\") " pod="ai-tenant-e2e-shared-a-w3-ed9f38/auth-test-model-ed9f38-kserve-5f7845bb7-gc7tn"
Sep 20 02:16:16.598108 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:16.598089    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d7e8345d-0ed1-4ede-bcaf-de80522b6bef-model-cache\") pod \"auth-test-model-ed9f38-kserve-5f7845bb7-gc7tn\" (UID: \"d7e8345d-0ed1-4ede-bcaf-de80522b6bef\") " pod="ai-tenant-e2e-shared-a-w3-ed9f38/auth-test-model-ed9f38-kserve-5f7845bb7-gc7tn"
Sep 20 02:16:16.599791 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:16.599767    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d7e8345d-0ed1-4ede-bcaf-de80522b6bef-dshm\") pod \"auth-test-model-ed9f38-kserve-5f7845bb7-gc7tn\" (UID: \"d7e8345d-0ed1-4ede-bcaf-de80522b6bef\") " pod="ai-tenant-e2e-shared-a-w3-ed9f38/auth-test-model-ed9f38-kserve-5f7845bb7-gc7tn"
Sep 20 02:16:16.600040 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:16.600024    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d7e8345d-0ed1-4ede-bcaf-de80522b6bef-tls-certs\") pod \"auth-test-model-ed9f38-kserve-5f7845bb7-gc7tn\" (UID: \"d7e8345d-0ed1-4ede-bcaf-de80522b6bef\") " pod="ai-tenant-e2e-shared-a-w3-ed9f38/auth-test-model-ed9f38-kserve-5f7845bb7-gc7tn"
Sep 20 02:16:16.604925 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:16.604902    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8n8g2\" (UniqueName: \"kubernetes.io/projected/d7e8345d-0ed1-4ede-bcaf-de80522b6bef-kube-api-access-8n8g2\") pod \"auth-test-model-ed9f38-kserve-5f7845bb7-gc7tn\" (UID: \"d7e8345d-0ed1-4ede-bcaf-de80522b6bef\") " pod="ai-tenant-e2e-shared-a-w3-ed9f38/auth-test-model-ed9f38-kserve-5f7845bb7-gc7tn"
Sep 20 02:16:16.719967 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:16.719929    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-ed9f38/auth-test-model-ed9f38-kserve-5f7845bb7-gc7tn"
Sep 20 02:16:16.995488 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:16.995464    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-ed9f38/auth-test-model-ed9f38-kserve-5f7845bb7-gc7tn"]
Sep 20 02:16:17.000080 ip-10-0-130-159 kubenswrapper[2584]: W0920 02:16:17.000037    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd7e8345d_0ed1_4ede_bcaf_de80522b6bef.slice/crio-347cce1efe5c1471302b7ded0d07c6a736559a7887d07516c242afa22651f0db WatchSource:0}: Error finding container 347cce1efe5c1471302b7ded0d07c6a736559a7887d07516c242afa22651f0db: Status 404 returned error can't find the container with id 347cce1efe5c1471302b7ded0d07c6a736559a7887d07516c242afa22651f0db
Sep 20 02:16:17.674287 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:17.674243    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-ed9f38/auth-test-model-ed9f38-kserve-5f7845bb7-gc7tn" event={"ID":"d7e8345d-0ed1-4ede-bcaf-de80522b6bef","Type":"ContainerStarted","Data":"d7b9c498f712bf77890a04c1d18cdec173f4770a1b5fa06c19231cd1d439b2bb"}
Sep 20 02:16:17.674287 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:17.674293    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-ed9f38/auth-test-model-ed9f38-kserve-5f7845bb7-gc7tn" event={"ID":"d7e8345d-0ed1-4ede-bcaf-de80522b6bef","Type":"ContainerStarted","Data":"347cce1efe5c1471302b7ded0d07c6a736559a7887d07516c242afa22651f0db"}
Sep 20 02:16:17.691018 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:17.690961    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-ed9f38/auth-test-model-ed9f38-kserve-5f7845bb7-gc7tn" podStartSLOduration=1.690945326 podStartE2EDuration="1.690945326s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-20 02:16:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-20 02:16:17.689305278 +0000 UTC m=+1434.431984498" watchObservedRunningTime="2026-09-20 02:16:17.690945326 +0000 UTC m=+1434.433624530"
Sep 20 02:16:21.033486 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:21.033450    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:16:21.836214 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:21.836182    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:16:22.471281 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:22.471248    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:16:23.234649 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:23.234618    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:16:24.150986 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:24.150958    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:16:24.301285 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:24.301253    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-b15051/e2e-conflict-b-62c06db8-kserve-c7f7dbf8b-bskwp"]
Sep 20 02:16:24.807114 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:24.807073    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:16:24.948124 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:24.948092    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:16:25.641841 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:25.641804    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:16:26.457043 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:26.457008    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:16:26.721063 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:26.720985    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-ed9f38/auth-test-model-ed9f38-kserve-5f7845bb7-gc7tn"
Sep 20 02:16:26.721063 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:26.721045    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-ed9f38/auth-test-model-ed9f38-kserve-5f7845bb7-gc7tn"
Sep 20 02:16:26.736831 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:26.736800    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-ed9f38/auth-test-model-ed9f38-kserve-5f7845bb7-gc7tn"
Sep 20 02:16:27.281999 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:27.281966    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:16:27.742957 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:27.742922    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-ed9f38/auth-test-model-ed9f38-kserve-5f7845bb7-gc7tn"
Sep 20 02:16:36.359565 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:36.359131    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f5813346-7ff9bcd677-zf6tr"]
Sep 20 02:16:36.361018 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:36.360983    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f5813346-7ff9bcd677-zf6tr" podUID="99a6dc33-0436-461b-b728-44523b5811b2" containerName="maas-api" containerID="cri-o://359e66a2f2fe5659acc006650895b2a16aa7ca43594053f16aa9359af3212cf2" gracePeriod=30
Sep 20 02:16:36.392571 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:36.392543    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:16:36.418434 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:36.418400    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-f5813346-5897d44497-fc64d"]
Sep 20 02:16:36.418753 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:36.418722    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-f5813346-5897d44497-fc64d" podUID="65d3c0b7-83df-4dfb-abc3-8e4bb010709b" containerName="payload-processing" containerID="cri-o://010a404b3048546ccf8855b21065874e02538b5326c998ca4face7f16bf41822" gracePeriod=30
Sep 20 02:16:36.491457 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:36.491424    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:16:36.595339 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:36.595297    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:16:36.693885 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:36.693804    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f5813346-7ff9bcd677-zf6tr"
Sep 20 02:16:36.694301 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:36.694277    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:16:36.742694 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:36.742673    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-f5813346-5897d44497-fc64d"
Sep 20 02:16:36.785139 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:36.785108    2584 generic.go:356] "Generic (PLEG): container finished" podID="99a6dc33-0436-461b-b728-44523b5811b2" containerID="359e66a2f2fe5659acc006650895b2a16aa7ca43594053f16aa9359af3212cf2" exitCode=0
Sep 20 02:16:36.785316 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:36.785150    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f5813346-7ff9bcd677-zf6tr"
Sep 20 02:16:36.785316 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:36.785199    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f5813346-7ff9bcd677-zf6tr" event={"ID":"99a6dc33-0436-461b-b728-44523b5811b2","Type":"ContainerDied","Data":"359e66a2f2fe5659acc006650895b2a16aa7ca43594053f16aa9359af3212cf2"}
Sep 20 02:16:36.785316 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:36.785260    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f5813346-7ff9bcd677-zf6tr" event={"ID":"99a6dc33-0436-461b-b728-44523b5811b2","Type":"ContainerDied","Data":"82c2128e04a73039c9799251ff6f2ed21e9104e41625d01ba9bafe254582a1a4"}
Sep 20 02:16:36.785316 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:36.785298    2584 scope.go:117] "RemoveContainer" containerID="359e66a2f2fe5659acc006650895b2a16aa7ca43594053f16aa9359af3212cf2"
Sep 20 02:16:36.786840 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:36.786814    2584 generic.go:356] "Generic (PLEG): container finished" podID="65d3c0b7-83df-4dfb-abc3-8e4bb010709b" containerID="010a404b3048546ccf8855b21065874e02538b5326c998ca4face7f16bf41822" exitCode=0
Sep 20 02:16:36.786953 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:36.786868    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-f5813346-5897d44497-fc64d"
Sep 20 02:16:36.786953 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:36.786876    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-f5813346-5897d44497-fc64d" event={"ID":"65d3c0b7-83df-4dfb-abc3-8e4bb010709b","Type":"ContainerDied","Data":"010a404b3048546ccf8855b21065874e02538b5326c998ca4face7f16bf41822"}
Sep 20 02:16:36.786953 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:36.786917    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-f5813346-5897d44497-fc64d" event={"ID":"65d3c0b7-83df-4dfb-abc3-8e4bb010709b","Type":"ContainerDied","Data":"5259f6b59fe1a49bc011339b85f61fccf978f235a74dbf9eda7c8adfe4d8e829"}
Sep 20 02:16:36.794795 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:36.794771    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hx42x\" (UniqueName: \"kubernetes.io/projected/99a6dc33-0436-461b-b728-44523b5811b2-kube-api-access-hx42x\") pod \"99a6dc33-0436-461b-b728-44523b5811b2\" (UID: \"99a6dc33-0436-461b-b728-44523b5811b2\") "
Sep 20 02:16:36.794870 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:36.794797    2584 scope.go:117] "RemoveContainer" containerID="359e66a2f2fe5659acc006650895b2a16aa7ca43594053f16aa9359af3212cf2"
Sep 20 02:16:36.794870 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:36.794828    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/99a6dc33-0436-461b-b728-44523b5811b2-maas-api-tls\") pod \"99a6dc33-0436-461b-b728-44523b5811b2\" (UID: \"99a6dc33-0436-461b-b728-44523b5811b2\") "
Sep 20 02:16:36.794978 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:36.794900    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/99a6dc33-0436-461b-b728-44523b5811b2-metrics-certs\") pod \"99a6dc33-0436-461b-b728-44523b5811b2\" (UID: \"99a6dc33-0436-461b-b728-44523b5811b2\") "
Sep 20 02:16:36.794978 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:36.794965    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/99a6dc33-0436-461b-b728-44523b5811b2-openshift-service-ca\") pod \"99a6dc33-0436-461b-b728-44523b5811b2\" (UID: \"99a6dc33-0436-461b-b728-44523b5811b2\") "
Sep 20 02:16:36.795133 ip-10-0-130-159 kubenswrapper[2584]: E0920 02:16:36.795106    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"359e66a2f2fe5659acc006650895b2a16aa7ca43594053f16aa9359af3212cf2\": container with ID starting with 359e66a2f2fe5659acc006650895b2a16aa7ca43594053f16aa9359af3212cf2 not found: ID does not exist" containerID="359e66a2f2fe5659acc006650895b2a16aa7ca43594053f16aa9359af3212cf2"
Sep 20 02:16:36.795192 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:36.795145    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"359e66a2f2fe5659acc006650895b2a16aa7ca43594053f16aa9359af3212cf2"} err="failed to get container status \"359e66a2f2fe5659acc006650895b2a16aa7ca43594053f16aa9359af3212cf2\": rpc error: code = NotFound desc = could not find container \"359e66a2f2fe5659acc006650895b2a16aa7ca43594053f16aa9359af3212cf2\": container with ID starting with 359e66a2f2fe5659acc006650895b2a16aa7ca43594053f16aa9359af3212cf2 not found: ID does not exist"
Sep 20 02:16:36.795192 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:36.795169    2584 scope.go:117] "RemoveContainer" containerID="010a404b3048546ccf8855b21065874e02538b5326c998ca4face7f16bf41822"
Sep 20 02:16:36.795788 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:36.795765    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/99a6dc33-0436-461b-b728-44523b5811b2-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "99a6dc33-0436-461b-b728-44523b5811b2" (UID: "99a6dc33-0436-461b-b728-44523b5811b2"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 20 02:16:36.797672 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:36.797649    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/99a6dc33-0436-461b-b728-44523b5811b2-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "99a6dc33-0436-461b-b728-44523b5811b2" (UID: "99a6dc33-0436-461b-b728-44523b5811b2"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 20 02:16:36.797765 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:36.797750    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/99a6dc33-0436-461b-b728-44523b5811b2-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "99a6dc33-0436-461b-b728-44523b5811b2" (UID: "99a6dc33-0436-461b-b728-44523b5811b2"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 20 02:16:36.797815 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:36.797781    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/99a6dc33-0436-461b-b728-44523b5811b2-kube-api-access-hx42x" (OuterVolumeSpecName: "kube-api-access-hx42x") pod "99a6dc33-0436-461b-b728-44523b5811b2" (UID: "99a6dc33-0436-461b-b728-44523b5811b2"). InnerVolumeSpecName "kube-api-access-hx42x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 20 02:16:36.803603 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:36.803572    2584 scope.go:117] "RemoveContainer" containerID="010a404b3048546ccf8855b21065874e02538b5326c998ca4face7f16bf41822"
Sep 20 02:16:36.803825 ip-10-0-130-159 kubenswrapper[2584]: E0920 02:16:36.803810    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"010a404b3048546ccf8855b21065874e02538b5326c998ca4face7f16bf41822\": container with ID starting with 010a404b3048546ccf8855b21065874e02538b5326c998ca4face7f16bf41822 not found: ID does not exist" containerID="010a404b3048546ccf8855b21065874e02538b5326c998ca4face7f16bf41822"
Sep 20 02:16:36.803875 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:36.803831    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"010a404b3048546ccf8855b21065874e02538b5326c998ca4face7f16bf41822"} err="failed to get container status \"010a404b3048546ccf8855b21065874e02538b5326c998ca4face7f16bf41822\": rpc error: code = NotFound desc = could not find container \"010a404b3048546ccf8855b21065874e02538b5326c998ca4face7f16bf41822\": container with ID starting with 010a404b3048546ccf8855b21065874e02538b5326c998ca4face7f16bf41822 not found: ID does not exist"
Sep 20 02:16:36.896176 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:36.896145    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/65d3c0b7-83df-4dfb-abc3-8e4bb010709b-plugins-config-volume\") pod \"65d3c0b7-83df-4dfb-abc3-8e4bb010709b\" (UID: \"65d3c0b7-83df-4dfb-abc3-8e4bb010709b\") "
Sep 20 02:16:36.896351 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:36.896189    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s9dz5\" (UniqueName: \"kubernetes.io/projected/65d3c0b7-83df-4dfb-abc3-8e4bb010709b-kube-api-access-s9dz5\") pod \"65d3c0b7-83df-4dfb-abc3-8e4bb010709b\" (UID: \"65d3c0b7-83df-4dfb-abc3-8e4bb010709b\") "
Sep 20 02:16:36.896496 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:36.896482    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hx42x\" (UniqueName: \"kubernetes.io/projected/99a6dc33-0436-461b-b728-44523b5811b2-kube-api-access-hx42x\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:16:36.896567 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:36.896498    2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/99a6dc33-0436-461b-b728-44523b5811b2-maas-api-tls\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:16:36.896567 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:36.896507    2584 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/99a6dc33-0436-461b-b728-44523b5811b2-metrics-certs\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:16:36.896567 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:36.896515    2584 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/99a6dc33-0436-461b-b728-44523b5811b2-openshift-service-ca\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:16:36.896567 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:36.896541    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/65d3c0b7-83df-4dfb-abc3-8e4bb010709b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "65d3c0b7-83df-4dfb-abc3-8e4bb010709b" (UID: "65d3c0b7-83df-4dfb-abc3-8e4bb010709b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 20 02:16:36.898375 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:36.898353    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/65d3c0b7-83df-4dfb-abc3-8e4bb010709b-kube-api-access-s9dz5" (OuterVolumeSpecName: "kube-api-access-s9dz5") pod "65d3c0b7-83df-4dfb-abc3-8e4bb010709b" (UID: "65d3c0b7-83df-4dfb-abc3-8e4bb010709b"). InnerVolumeSpecName "kube-api-access-s9dz5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 20 02:16:36.997550 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:36.997492    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s9dz5\" (UniqueName: \"kubernetes.io/projected/65d3c0b7-83df-4dfb-abc3-8e4bb010709b-kube-api-access-s9dz5\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:16:36.997550 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:36.997542    2584 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/65d3c0b7-83df-4dfb-abc3-8e4bb010709b-plugins-config-volume\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:16:37.067594 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:37.067561    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:16:37.112613 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:37.112583    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f5813346-7ff9bcd677-zf6tr"]
Sep 20 02:16:37.114790 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:37.114767    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f5813346-7ff9bcd677-zf6tr"]
Sep 20 02:16:37.124062 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:37.124037    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-f5813346-5897d44497-fc64d"]
Sep 20 02:16:37.128209 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:37.128186    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-f5813346-5897d44497-fc64d"]
Sep 20 02:16:37.788731 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:37.788693    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:16:38.566393 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:38.566361    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:16:39.094103 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:39.094074    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="65d3c0b7-83df-4dfb-abc3-8e4bb010709b" path="/var/lib/kubelet/pods/65d3c0b7-83df-4dfb-abc3-8e4bb010709b/volumes"
Sep 20 02:16:39.094651 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:39.094632    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="99a6dc33-0436-461b-b728-44523b5811b2" path="/var/lib/kubelet/pods/99a6dc33-0436-461b-b728-44523b5811b2/volumes"
Sep 20 02:16:39.322508 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:39.322462    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-b15051/e2e-conflict-b-62c06db8-kserve-c7f7dbf8b-bskwp" podUID="a8313df0-7262-44a2-b0c1-7d2302d419f8" containerName="main" containerID="cri-o://011ea31ecc88652a0db13fcd66dca0e6078aca0ddc4a0fa0fffabe7805a33799" gracePeriod=45
Sep 20 02:16:39.340829 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:39.340801    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:16:39.583961 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:39.583937    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-b15051/e2e-conflict-b-62c06db8-kserve-c7f7dbf8b-bskwp"
Sep 20 02:16:39.728019 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:39.727991    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a8313df0-7262-44a2-b0c1-7d2302d419f8-home\") pod \"a8313df0-7262-44a2-b0c1-7d2302d419f8\" (UID: \"a8313df0-7262-44a2-b0c1-7d2302d419f8\") "
Sep 20 02:16:39.728223 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:39.728033    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a8313df0-7262-44a2-b0c1-7d2302d419f8-model-cache\") pod \"a8313df0-7262-44a2-b0c1-7d2302d419f8\" (UID: \"a8313df0-7262-44a2-b0c1-7d2302d419f8\") "
Sep 20 02:16:39.728223 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:39.728058    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lb8c4\" (UniqueName: \"kubernetes.io/projected/a8313df0-7262-44a2-b0c1-7d2302d419f8-kube-api-access-lb8c4\") pod \"a8313df0-7262-44a2-b0c1-7d2302d419f8\" (UID: \"a8313df0-7262-44a2-b0c1-7d2302d419f8\") "
Sep 20 02:16:39.728223 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:39.728133    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a8313df0-7262-44a2-b0c1-7d2302d419f8-dshm\") pod \"a8313df0-7262-44a2-b0c1-7d2302d419f8\" (UID: \"a8313df0-7262-44a2-b0c1-7d2302d419f8\") "
Sep 20 02:16:39.728223 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:39.728167    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a8313df0-7262-44a2-b0c1-7d2302d419f8-tmp-dir\") pod \"a8313df0-7262-44a2-b0c1-7d2302d419f8\" (UID: \"a8313df0-7262-44a2-b0c1-7d2302d419f8\") "
Sep 20 02:16:39.728223 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:39.728219    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a8313df0-7262-44a2-b0c1-7d2302d419f8-tls-certs\") pod \"a8313df0-7262-44a2-b0c1-7d2302d419f8\" (UID: \"a8313df0-7262-44a2-b0c1-7d2302d419f8\") "
Sep 20 02:16:39.728505 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:39.728335    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8313df0-7262-44a2-b0c1-7d2302d419f8-home" (OuterVolumeSpecName: "home") pod "a8313df0-7262-44a2-b0c1-7d2302d419f8" (UID: "a8313df0-7262-44a2-b0c1-7d2302d419f8"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 20 02:16:39.728505 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:39.728350    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8313df0-7262-44a2-b0c1-7d2302d419f8-model-cache" (OuterVolumeSpecName: "model-cache") pod "a8313df0-7262-44a2-b0c1-7d2302d419f8" (UID: "a8313df0-7262-44a2-b0c1-7d2302d419f8"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 20 02:16:39.728505 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:39.728442    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8313df0-7262-44a2-b0c1-7d2302d419f8-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a8313df0-7262-44a2-b0c1-7d2302d419f8" (UID: "a8313df0-7262-44a2-b0c1-7d2302d419f8"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 20 02:16:39.728688 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:39.728600    2584 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a8313df0-7262-44a2-b0c1-7d2302d419f8-home\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:16:39.728688 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:39.728616    2584 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a8313df0-7262-44a2-b0c1-7d2302d419f8-model-cache\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:16:39.728688 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:39.728626    2584 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a8313df0-7262-44a2-b0c1-7d2302d419f8-tmp-dir\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:16:39.730448 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:39.730426    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8313df0-7262-44a2-b0c1-7d2302d419f8-dshm" (OuterVolumeSpecName: "dshm") pod "a8313df0-7262-44a2-b0c1-7d2302d419f8" (UID: "a8313df0-7262-44a2-b0c1-7d2302d419f8"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 20 02:16:39.730555 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:39.730431    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a8313df0-7262-44a2-b0c1-7d2302d419f8-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a8313df0-7262-44a2-b0c1-7d2302d419f8" (UID: "a8313df0-7262-44a2-b0c1-7d2302d419f8"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 20 02:16:39.730555 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:39.730493    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a8313df0-7262-44a2-b0c1-7d2302d419f8-kube-api-access-lb8c4" (OuterVolumeSpecName: "kube-api-access-lb8c4") pod "a8313df0-7262-44a2-b0c1-7d2302d419f8" (UID: "a8313df0-7262-44a2-b0c1-7d2302d419f8"). InnerVolumeSpecName "kube-api-access-lb8c4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 20 02:16:39.806945 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:39.806917    2584 generic.go:356] "Generic (PLEG): container finished" podID="a8313df0-7262-44a2-b0c1-7d2302d419f8" containerID="011ea31ecc88652a0db13fcd66dca0e6078aca0ddc4a0fa0fffabe7805a33799" exitCode=0
Sep 20 02:16:39.807085 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:39.806957    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-b15051/e2e-conflict-b-62c06db8-kserve-c7f7dbf8b-bskwp"
Sep 20 02:16:39.807085 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:39.807002    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-b15051/e2e-conflict-b-62c06db8-kserve-c7f7dbf8b-bskwp" event={"ID":"a8313df0-7262-44a2-b0c1-7d2302d419f8","Type":"ContainerDied","Data":"011ea31ecc88652a0db13fcd66dca0e6078aca0ddc4a0fa0fffabe7805a33799"}
Sep 20 02:16:39.807085 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:39.807048    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-b15051/e2e-conflict-b-62c06db8-kserve-c7f7dbf8b-bskwp" event={"ID":"a8313df0-7262-44a2-b0c1-7d2302d419f8","Type":"ContainerDied","Data":"8175be3c47ffd6e7b316c272ac963f456e319601aacfbab6df289406ff4fc63b"}
Sep 20 02:16:39.807085 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:39.807068    2584 scope.go:117] "RemoveContainer" containerID="011ea31ecc88652a0db13fcd66dca0e6078aca0ddc4a0fa0fffabe7805a33799"
Sep 20 02:16:39.817293 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:39.817274    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:16:39.817746 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:39.817732    2584 scope.go:117] "RemoveContainer" containerID="011ea31ecc88652a0db13fcd66dca0e6078aca0ddc4a0fa0fffabe7805a33799"
Sep 20 02:16:39.817991 ip-10-0-130-159 kubenswrapper[2584]: E0920 02:16:39.817975    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"011ea31ecc88652a0db13fcd66dca0e6078aca0ddc4a0fa0fffabe7805a33799\": container with ID starting with 011ea31ecc88652a0db13fcd66dca0e6078aca0ddc4a0fa0fffabe7805a33799 not found: ID does not exist" containerID="011ea31ecc88652a0db13fcd66dca0e6078aca0ddc4a0fa0fffabe7805a33799"
Sep 20 02:16:39.818036 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:39.817999    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"011ea31ecc88652a0db13fcd66dca0e6078aca0ddc4a0fa0fffabe7805a33799"} err="failed to get container status \"011ea31ecc88652a0db13fcd66dca0e6078aca0ddc4a0fa0fffabe7805a33799\": rpc error: code = NotFound desc = could not find container \"011ea31ecc88652a0db13fcd66dca0e6078aca0ddc4a0fa0fffabe7805a33799\": container with ID starting with 011ea31ecc88652a0db13fcd66dca0e6078aca0ddc4a0fa0fffabe7805a33799 not found: ID does not exist"
Sep 20 02:16:39.829137 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:39.829112    2584 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a8313df0-7262-44a2-b0c1-7d2302d419f8-dshm\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:16:39.829137 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:39.829133    2584 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a8313df0-7262-44a2-b0c1-7d2302d419f8-tls-certs\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:16:39.829264 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:39.829145    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lb8c4\" (UniqueName: \"kubernetes.io/projected/a8313df0-7262-44a2-b0c1-7d2302d419f8-kube-api-access-lb8c4\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:16:39.829264 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:39.829222    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-b15051/e2e-conflict-b-62c06db8-kserve-c7f7dbf8b-bskwp"]
Sep 20 02:16:39.835723 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:39.835703    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-b15051/e2e-conflict-b-62c06db8-kserve-c7f7dbf8b-bskwp"]
Sep 20 02:16:40.123164 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:40.123073    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:16:40.824907 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:40.824877    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:16:40.929118 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:40.929084    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-b15051/e2e-conflict-a-62c06db8-kserve-bf8cfc568-wzjn7"]
Sep 20 02:16:40.947804 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:40.947775    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:16:41.095555 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:41.095439    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a8313df0-7262-44a2-b0c1-7d2302d419f8" path="/var/lib/kubelet/pods/a8313df0-7262-44a2-b0c1-7d2302d419f8/volumes"
Sep 20 02:16:41.510985 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:41.510948    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:16:44.837126 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:44.837098    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:16:53.819375 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:53.819335    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-b15051/distinct-w4-b15051-kserve-98f67785-dskgq"]
Sep 20 02:16:53.820888 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:53.820863    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="99a6dc33-0436-461b-b728-44523b5811b2" containerName="maas-api"
Sep 20 02:16:53.821132 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:53.821110    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="99a6dc33-0436-461b-b728-44523b5811b2" containerName="maas-api"
Sep 20 02:16:53.821206 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:53.821144    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a8313df0-7262-44a2-b0c1-7d2302d419f8" containerName="main"
Sep 20 02:16:53.821206 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:53.821156    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="a8313df0-7262-44a2-b0c1-7d2302d419f8" containerName="main"
Sep 20 02:16:53.821206 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:53.821193    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="65d3c0b7-83df-4dfb-abc3-8e4bb010709b" containerName="payload-processing"
Sep 20 02:16:53.821206 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:53.821203    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="65d3c0b7-83df-4dfb-abc3-8e4bb010709b" containerName="payload-processing"
Sep 20 02:16:53.821593 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:53.821571    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="65d3c0b7-83df-4dfb-abc3-8e4bb010709b" containerName="payload-processing"
Sep 20 02:16:53.821687 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:53.821598    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="a8313df0-7262-44a2-b0c1-7d2302d419f8" containerName="main"
Sep 20 02:16:53.821687 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:53.821625    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="99a6dc33-0436-461b-b728-44523b5811b2" containerName="maas-api"
Sep 20 02:16:53.826194 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:53.826176    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-b15051/distinct-w4-b15051-kserve-98f67785-dskgq"
Sep 20 02:16:53.828968 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:53.828947    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-b15051\"/\"distinct-w4-b15051-kserve-self-signed-certs\""
Sep 20 02:16:53.829990 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:53.829968    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-b15051/distinct-w4-b15051-kserve-98f67785-dskgq"]
Sep 20 02:16:53.845798 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:53.845772    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:16:53.865435 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:53.865410    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c0f8e7fe-c5c2-4a8a-b0e9-5db728be091e-dshm\") pod \"distinct-w4-b15051-kserve-98f67785-dskgq\" (UID: \"c0f8e7fe-c5c2-4a8a-b0e9-5db728be091e\") " pod="e2e-models-e2e-worker-w4-b15051/distinct-w4-b15051-kserve-98f67785-dskgq"
Sep 20 02:16:53.865596 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:53.865571    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c0f8e7fe-c5c2-4a8a-b0e9-5db728be091e-model-cache\") pod \"distinct-w4-b15051-kserve-98f67785-dskgq\" (UID: \"c0f8e7fe-c5c2-4a8a-b0e9-5db728be091e\") " pod="e2e-models-e2e-worker-w4-b15051/distinct-w4-b15051-kserve-98f67785-dskgq"
Sep 20 02:16:53.865674 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:53.865626    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c0f8e7fe-c5c2-4a8a-b0e9-5db728be091e-tmp-dir\") pod \"distinct-w4-b15051-kserve-98f67785-dskgq\" (UID: \"c0f8e7fe-c5c2-4a8a-b0e9-5db728be091e\") " pod="e2e-models-e2e-worker-w4-b15051/distinct-w4-b15051-kserve-98f67785-dskgq"
Sep 20 02:16:53.865674 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:53.865663    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c0f8e7fe-c5c2-4a8a-b0e9-5db728be091e-home\") pod \"distinct-w4-b15051-kserve-98f67785-dskgq\" (UID: \"c0f8e7fe-c5c2-4a8a-b0e9-5db728be091e\") " pod="e2e-models-e2e-worker-w4-b15051/distinct-w4-b15051-kserve-98f67785-dskgq"
Sep 20 02:16:53.865759 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:53.865691    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-szjt7\" (UniqueName: \"kubernetes.io/projected/c0f8e7fe-c5c2-4a8a-b0e9-5db728be091e-kube-api-access-szjt7\") pod \"distinct-w4-b15051-kserve-98f67785-dskgq\" (UID: \"c0f8e7fe-c5c2-4a8a-b0e9-5db728be091e\") " pod="e2e-models-e2e-worker-w4-b15051/distinct-w4-b15051-kserve-98f67785-dskgq"
Sep 20 02:16:53.865820 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:53.865780    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c0f8e7fe-c5c2-4a8a-b0e9-5db728be091e-tls-certs\") pod \"distinct-w4-b15051-kserve-98f67785-dskgq\" (UID: \"c0f8e7fe-c5c2-4a8a-b0e9-5db728be091e\") " pod="e2e-models-e2e-worker-w4-b15051/distinct-w4-b15051-kserve-98f67785-dskgq"
Sep 20 02:16:53.966953 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:53.966906    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c0f8e7fe-c5c2-4a8a-b0e9-5db728be091e-tmp-dir\") pod \"distinct-w4-b15051-kserve-98f67785-dskgq\" (UID: \"c0f8e7fe-c5c2-4a8a-b0e9-5db728be091e\") " pod="e2e-models-e2e-worker-w4-b15051/distinct-w4-b15051-kserve-98f67785-dskgq"
Sep 20 02:16:53.966953 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:53.966957    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c0f8e7fe-c5c2-4a8a-b0e9-5db728be091e-home\") pod \"distinct-w4-b15051-kserve-98f67785-dskgq\" (UID: \"c0f8e7fe-c5c2-4a8a-b0e9-5db728be091e\") " pod="e2e-models-e2e-worker-w4-b15051/distinct-w4-b15051-kserve-98f67785-dskgq"
Sep 20 02:16:53.967239 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:53.966984    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-szjt7\" (UniqueName: \"kubernetes.io/projected/c0f8e7fe-c5c2-4a8a-b0e9-5db728be091e-kube-api-access-szjt7\") pod \"distinct-w4-b15051-kserve-98f67785-dskgq\" (UID: \"c0f8e7fe-c5c2-4a8a-b0e9-5db728be091e\") " pod="e2e-models-e2e-worker-w4-b15051/distinct-w4-b15051-kserve-98f67785-dskgq"
Sep 20 02:16:53.967239 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:53.967029    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c0f8e7fe-c5c2-4a8a-b0e9-5db728be091e-tls-certs\") pod \"distinct-w4-b15051-kserve-98f67785-dskgq\" (UID: \"c0f8e7fe-c5c2-4a8a-b0e9-5db728be091e\") " pod="e2e-models-e2e-worker-w4-b15051/distinct-w4-b15051-kserve-98f67785-dskgq"
Sep 20 02:16:53.967239 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:53.967091    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c0f8e7fe-c5c2-4a8a-b0e9-5db728be091e-dshm\") pod \"distinct-w4-b15051-kserve-98f67785-dskgq\" (UID: \"c0f8e7fe-c5c2-4a8a-b0e9-5db728be091e\") " pod="e2e-models-e2e-worker-w4-b15051/distinct-w4-b15051-kserve-98f67785-dskgq"
Sep 20 02:16:53.967239 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:53.967232    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c0f8e7fe-c5c2-4a8a-b0e9-5db728be091e-model-cache\") pod \"distinct-w4-b15051-kserve-98f67785-dskgq\" (UID: \"c0f8e7fe-c5c2-4a8a-b0e9-5db728be091e\") " pod="e2e-models-e2e-worker-w4-b15051/distinct-w4-b15051-kserve-98f67785-dskgq"
Sep 20 02:16:53.967446 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:53.967420    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c0f8e7fe-c5c2-4a8a-b0e9-5db728be091e-tmp-dir\") pod \"distinct-w4-b15051-kserve-98f67785-dskgq\" (UID: \"c0f8e7fe-c5c2-4a8a-b0e9-5db728be091e\") " pod="e2e-models-e2e-worker-w4-b15051/distinct-w4-b15051-kserve-98f67785-dskgq"
Sep 20 02:16:53.967499 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:53.967452    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c0f8e7fe-c5c2-4a8a-b0e9-5db728be091e-home\") pod \"distinct-w4-b15051-kserve-98f67785-dskgq\" (UID: \"c0f8e7fe-c5c2-4a8a-b0e9-5db728be091e\") " pod="e2e-models-e2e-worker-w4-b15051/distinct-w4-b15051-kserve-98f67785-dskgq"
Sep 20 02:16:53.967594 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:53.967540    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c0f8e7fe-c5c2-4a8a-b0e9-5db728be091e-model-cache\") pod \"distinct-w4-b15051-kserve-98f67785-dskgq\" (UID: \"c0f8e7fe-c5c2-4a8a-b0e9-5db728be091e\") " pod="e2e-models-e2e-worker-w4-b15051/distinct-w4-b15051-kserve-98f67785-dskgq"
Sep 20 02:16:53.969689 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:53.969654    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c0f8e7fe-c5c2-4a8a-b0e9-5db728be091e-dshm\") pod \"distinct-w4-b15051-kserve-98f67785-dskgq\" (UID: \"c0f8e7fe-c5c2-4a8a-b0e9-5db728be091e\") " pod="e2e-models-e2e-worker-w4-b15051/distinct-w4-b15051-kserve-98f67785-dskgq"
Sep 20 02:16:53.969945 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:53.969923    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c0f8e7fe-c5c2-4a8a-b0e9-5db728be091e-tls-certs\") pod \"distinct-w4-b15051-kserve-98f67785-dskgq\" (UID: \"c0f8e7fe-c5c2-4a8a-b0e9-5db728be091e\") " pod="e2e-models-e2e-worker-w4-b15051/distinct-w4-b15051-kserve-98f67785-dskgq"
Sep 20 02:16:53.975122 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:53.975089    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-szjt7\" (UniqueName: \"kubernetes.io/projected/c0f8e7fe-c5c2-4a8a-b0e9-5db728be091e-kube-api-access-szjt7\") pod \"distinct-w4-b15051-kserve-98f67785-dskgq\" (UID: \"c0f8e7fe-c5c2-4a8a-b0e9-5db728be091e\") " pod="e2e-models-e2e-worker-w4-b15051/distinct-w4-b15051-kserve-98f67785-dskgq"
Sep 20 02:16:54.137240 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:54.137152    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-b15051/distinct-w4-b15051-kserve-98f67785-dskgq"
Sep 20 02:16:54.412486 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:54.412460    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-b15051/distinct-w4-b15051-kserve-98f67785-dskgq"]
Sep 20 02:16:54.414378 ip-10-0-130-159 kubenswrapper[2584]: W0920 02:16:54.414348    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc0f8e7fe_c5c2_4a8a_b0e9_5db728be091e.slice/crio-44320a545d7a1b8cc8f079f41d0c1d60ec456e7cdb32d48d34fa67152ded881d WatchSource:0}: Error finding container 44320a545d7a1b8cc8f079f41d0c1d60ec456e7cdb32d48d34fa67152ded881d: Status 404 returned error can't find the container with id 44320a545d7a1b8cc8f079f41d0c1d60ec456e7cdb32d48d34fa67152ded881d
Sep 20 02:16:54.692398 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:54.692363    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:16:54.893644 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:54.893607    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-b15051/distinct-w4-b15051-kserve-98f67785-dskgq" event={"ID":"c0f8e7fe-c5c2-4a8a-b0e9-5db728be091e","Type":"ContainerStarted","Data":"f0e1bfeb802bcd7e1e6dc21b3458986767de1ad3e0f5424a3806e7c9e7daea5b"}
Sep 20 02:16:54.893644 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:54.893648    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-b15051/distinct-w4-b15051-kserve-98f67785-dskgq" event={"ID":"c0f8e7fe-c5c2-4a8a-b0e9-5db728be091e","Type":"ContainerStarted","Data":"44320a545d7a1b8cc8f079f41d0c1d60ec456e7cdb32d48d34fa67152ded881d"}
Sep 20 02:16:54.911401 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:54.911345    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-b15051/distinct-w4-b15051-kserve-98f67785-dskgq" podStartSLOduration=1.911328855 podStartE2EDuration="1.911328855s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-20 02:16:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-20 02:16:54.909677143 +0000 UTC m=+1471.652356354" watchObservedRunningTime="2026-09-20 02:16:54.911328855 +0000 UTC m=+1471.654008061"
Sep 20 02:16:55.949223 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:55.949186    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-b15051/e2e-conflict-a-62c06db8-kserve-bf8cfc568-wzjn7" podUID="79805ee4-493b-4936-a8c9-ae0044f2f215" containerName="main" containerID="cri-o://528f92f9dfad55c6ebf66b683c9205c85c5da428d36c3301ace699afb08297b3" gracePeriod=45
Sep 20 02:16:56.211391 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:56.211332    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-b15051/e2e-conflict-a-62c06db8-kserve-bf8cfc568-wzjn7"
Sep 20 02:16:56.289581 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:56.289554    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/79805ee4-493b-4936-a8c9-ae0044f2f215-home\") pod \"79805ee4-493b-4936-a8c9-ae0044f2f215\" (UID: \"79805ee4-493b-4936-a8c9-ae0044f2f215\") "
Sep 20 02:16:56.289760 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:56.289612    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/79805ee4-493b-4936-a8c9-ae0044f2f215-model-cache\") pod \"79805ee4-493b-4936-a8c9-ae0044f2f215\" (UID: \"79805ee4-493b-4936-a8c9-ae0044f2f215\") "
Sep 20 02:16:56.289760 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:56.289660    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/79805ee4-493b-4936-a8c9-ae0044f2f215-dshm\") pod \"79805ee4-493b-4936-a8c9-ae0044f2f215\" (UID: \"79805ee4-493b-4936-a8c9-ae0044f2f215\") "
Sep 20 02:16:56.289760 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:56.289715    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5xq67\" (UniqueName: \"kubernetes.io/projected/79805ee4-493b-4936-a8c9-ae0044f2f215-kube-api-access-5xq67\") pod \"79805ee4-493b-4936-a8c9-ae0044f2f215\" (UID: \"79805ee4-493b-4936-a8c9-ae0044f2f215\") "
Sep 20 02:16:56.289760 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:56.289742    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/79805ee4-493b-4936-a8c9-ae0044f2f215-tls-certs\") pod \"79805ee4-493b-4936-a8c9-ae0044f2f215\" (UID: \"79805ee4-493b-4936-a8c9-ae0044f2f215\") "
Sep 20 02:16:56.289941 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:56.289768    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/79805ee4-493b-4936-a8c9-ae0044f2f215-tmp-dir\") pod \"79805ee4-493b-4936-a8c9-ae0044f2f215\" (UID: \"79805ee4-493b-4936-a8c9-ae0044f2f215\") "
Sep 20 02:16:56.289941 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:56.289866    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79805ee4-493b-4936-a8c9-ae0044f2f215-home" (OuterVolumeSpecName: "home") pod "79805ee4-493b-4936-a8c9-ae0044f2f215" (UID: "79805ee4-493b-4936-a8c9-ae0044f2f215"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 20 02:16:56.289941 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:56.289880    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79805ee4-493b-4936-a8c9-ae0044f2f215-model-cache" (OuterVolumeSpecName: "model-cache") pod "79805ee4-493b-4936-a8c9-ae0044f2f215" (UID: "79805ee4-493b-4936-a8c9-ae0044f2f215"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 20 02:16:56.290173 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:56.290128    2584 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/79805ee4-493b-4936-a8c9-ae0044f2f215-home\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:16:56.290173 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:56.290151    2584 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/79805ee4-493b-4936-a8c9-ae0044f2f215-model-cache\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:16:56.290288 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:56.290170    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79805ee4-493b-4936-a8c9-ae0044f2f215-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "79805ee4-493b-4936-a8c9-ae0044f2f215" (UID: "79805ee4-493b-4936-a8c9-ae0044f2f215"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 20 02:16:56.291967 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:56.291944    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79805ee4-493b-4936-a8c9-ae0044f2f215-dshm" (OuterVolumeSpecName: "dshm") pod "79805ee4-493b-4936-a8c9-ae0044f2f215" (UID: "79805ee4-493b-4936-a8c9-ae0044f2f215"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 20 02:16:56.292092 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:56.291982    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/79805ee4-493b-4936-a8c9-ae0044f2f215-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "79805ee4-493b-4936-a8c9-ae0044f2f215" (UID: "79805ee4-493b-4936-a8c9-ae0044f2f215"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 20 02:16:56.292092 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:56.292010    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/79805ee4-493b-4936-a8c9-ae0044f2f215-kube-api-access-5xq67" (OuterVolumeSpecName: "kube-api-access-5xq67") pod "79805ee4-493b-4936-a8c9-ae0044f2f215" (UID: "79805ee4-493b-4936-a8c9-ae0044f2f215"). InnerVolumeSpecName "kube-api-access-5xq67". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 20 02:16:56.391432 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:56.391391    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5xq67\" (UniqueName: \"kubernetes.io/projected/79805ee4-493b-4936-a8c9-ae0044f2f215-kube-api-access-5xq67\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:16:56.391432 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:56.391423    2584 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/79805ee4-493b-4936-a8c9-ae0044f2f215-tls-certs\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:16:56.391432 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:56.391432    2584 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/79805ee4-493b-4936-a8c9-ae0044f2f215-tmp-dir\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:16:56.391432 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:56.391440    2584 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/79805ee4-493b-4936-a8c9-ae0044f2f215-dshm\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:16:56.905902 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:56.905866    2584 generic.go:356] "Generic (PLEG): container finished" podID="79805ee4-493b-4936-a8c9-ae0044f2f215" containerID="528f92f9dfad55c6ebf66b683c9205c85c5da428d36c3301ace699afb08297b3" exitCode=0
Sep 20 02:16:56.906124 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:56.905914    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-b15051/e2e-conflict-a-62c06db8-kserve-bf8cfc568-wzjn7"
Sep 20 02:16:56.906124 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:56.905958    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-b15051/e2e-conflict-a-62c06db8-kserve-bf8cfc568-wzjn7" event={"ID":"79805ee4-493b-4936-a8c9-ae0044f2f215","Type":"ContainerDied","Data":"528f92f9dfad55c6ebf66b683c9205c85c5da428d36c3301ace699afb08297b3"}
Sep 20 02:16:56.906124 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:56.906009    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-b15051/e2e-conflict-a-62c06db8-kserve-bf8cfc568-wzjn7" event={"ID":"79805ee4-493b-4936-a8c9-ae0044f2f215","Type":"ContainerDied","Data":"18c54230ad3ea54572cbdda62ee07d4a210fa1ad4a2eb84abc0a8a090b2ea131"}
Sep 20 02:16:56.906124 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:56.906028    2584 scope.go:117] "RemoveContainer" containerID="528f92f9dfad55c6ebf66b683c9205c85c5da428d36c3301ace699afb08297b3"
Sep 20 02:16:56.915630 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:56.915613    2584 scope.go:117] "RemoveContainer" containerID="528f92f9dfad55c6ebf66b683c9205c85c5da428d36c3301ace699afb08297b3"
Sep 20 02:16:56.915895 ip-10-0-130-159 kubenswrapper[2584]: E0920 02:16:56.915878    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"528f92f9dfad55c6ebf66b683c9205c85c5da428d36c3301ace699afb08297b3\": container with ID starting with 528f92f9dfad55c6ebf66b683c9205c85c5da428d36c3301ace699afb08297b3 not found: ID does not exist" containerID="528f92f9dfad55c6ebf66b683c9205c85c5da428d36c3301ace699afb08297b3"
Sep 20 02:16:56.915947 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:56.915903    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"528f92f9dfad55c6ebf66b683c9205c85c5da428d36c3301ace699afb08297b3"} err="failed to get container status \"528f92f9dfad55c6ebf66b683c9205c85c5da428d36c3301ace699afb08297b3\": rpc error: code = NotFound desc = could not find container \"528f92f9dfad55c6ebf66b683c9205c85c5da428d36c3301ace699afb08297b3\": container with ID starting with 528f92f9dfad55c6ebf66b683c9205c85c5da428d36c3301ace699afb08297b3 not found: ID does not exist"
Sep 20 02:16:56.927430 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:56.927409    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-b15051/e2e-conflict-a-62c06db8-kserve-bf8cfc568-wzjn7"]
Sep 20 02:16:56.931750 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:56.931729    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-b15051/e2e-conflict-a-62c06db8-kserve-bf8cfc568-wzjn7"]
Sep 20 02:16:57.093935 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:57.093903    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="79805ee4-493b-4936-a8c9-ae0044f2f215" path="/var/lib/kubelet/pods/79805ee4-493b-4936-a8c9-ae0044f2f215/volumes"
Sep 20 02:16:57.721508 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:16:57.721474    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:17:04.138132 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:17:04.138094    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-b15051/distinct-w4-b15051-kserve-98f67785-dskgq"
Sep 20 02:17:04.138132 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:17:04.138141    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-b15051/distinct-w4-b15051-kserve-98f67785-dskgq"
Sep 20 02:17:04.150828 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:17:04.150794    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-b15051/distinct-w4-b15051-kserve-98f67785-dskgq"
Sep 20 02:17:04.965290 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:17:04.965261    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-b15051/distinct-w4-b15051-kserve-98f67785-dskgq"
Sep 20 02:17:07.188232 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:17:07.188197    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:17:21.635156 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:17:21.635122    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:17:22.580693 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:17:22.580659    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:17:25.194591 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:17:25.194562    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-ztdt2_ece82f19-68b5-4810-be50-d8678cfdee02/console-operator/1.log"
Sep 20 02:17:25.199889 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:17:25.199865    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-ztdt2_ece82f19-68b5-4810-be50-d8678cfdee02/console-operator/1.log"
Sep 20 02:17:26.352903 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:17:26.352862    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:17:28.241033 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:17:28.240986    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:17:28.497897 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:17:28.497806    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:17:41.034893 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:17:41.034813    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:17:41.092223 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:17:41.092189    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:17:42.666003 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:17:42.665964    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:17:42.726140 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:17:42.726104    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:17:42.909239 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:17:42.909205    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-f5813346-openshift-default-dcd586ff6-cvqn5"]
Sep 20 02:17:42.909786 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:17:42.909750    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-f5813346-openshift-default-dcd586ff6-cvqn5" podUID="2ce47e33-9674-4f3e-956b-d0083215f70f" containerName="istio-proxy" containerID="cri-o://a53a4e1965e80180d9e01965a3fc86c2699c827300230335979c48becc473101" gracePeriod=30
Sep 20 02:17:42.923559 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:17:42.923452    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:17:44.177924 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:17:44.177894    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:17:44.278059 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:17:44.278016    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:17:44.379108 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:17:44.379070    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:17:44.996576 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:17:44.996549    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:17:45.096672 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:17:45.096627    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:17:45.197438 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:17:45.197405    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:17:46.472676 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:17:46.472646    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:17:48.182095 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:17:48.182031    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-f5813346-openshift-default-dcd586ff6-cvqn5"
Sep 20 02:17:48.191678 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:17:48.191653    2584 generic.go:356] "Generic (PLEG): container finished" podID="2ce47e33-9674-4f3e-956b-d0083215f70f" containerID="a53a4e1965e80180d9e01965a3fc86c2699c827300230335979c48becc473101" exitCode=0
Sep 20 02:17:48.191824 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:17:48.191690    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f5813346-openshift-default-dcd586ff6-cvqn5" event={"ID":"2ce47e33-9674-4f3e-956b-d0083215f70f","Type":"ContainerDied","Data":"a53a4e1965e80180d9e01965a3fc86c2699c827300230335979c48becc473101"}
Sep 20 02:17:48.191824 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:17:48.191702    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-f5813346-openshift-default-dcd586ff6-cvqn5"
Sep 20 02:17:48.191824 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:17:48.191757    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f5813346-openshift-default-dcd586ff6-cvqn5" event={"ID":"2ce47e33-9674-4f3e-956b-d0083215f70f","Type":"ContainerDied","Data":"6e56d904eac1b315d2fe9693fce4ef0094788640086d032c993d18907b74eee0"}
Sep 20 02:17:48.191824 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:17:48.191789    2584 scope.go:117] "RemoveContainer" containerID="a53a4e1965e80180d9e01965a3fc86c2699c827300230335979c48becc473101"
Sep 20 02:17:48.200935 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:17:48.200913    2584 scope.go:117] "RemoveContainer" containerID="bca3d5c8741a70b6c889361d94901593472e7a9704e4445def400e8b08425ec6"
Sep 20 02:17:48.210234 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:17:48.210213    2584 scope.go:117] "RemoveContainer" containerID="a53a4e1965e80180d9e01965a3fc86c2699c827300230335979c48becc473101"
Sep 20 02:17:48.210623 ip-10-0-130-159 kubenswrapper[2584]: E0920 02:17:48.210592    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a53a4e1965e80180d9e01965a3fc86c2699c827300230335979c48becc473101\": container with ID starting with a53a4e1965e80180d9e01965a3fc86c2699c827300230335979c48becc473101 not found: ID does not exist" containerID="a53a4e1965e80180d9e01965a3fc86c2699c827300230335979c48becc473101"
Sep 20 02:17:48.210751 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:17:48.210629    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a53a4e1965e80180d9e01965a3fc86c2699c827300230335979c48becc473101"} err="failed to get container status \"a53a4e1965e80180d9e01965a3fc86c2699c827300230335979c48becc473101\": rpc error: code = NotFound desc = could not find container \"a53a4e1965e80180d9e01965a3fc86c2699c827300230335979c48becc473101\": container with ID starting with a53a4e1965e80180d9e01965a3fc86c2699c827300230335979c48becc473101 not found: ID does not exist"
Sep 20 02:17:48.210751 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:17:48.210647    2584 scope.go:117] "RemoveContainer" containerID="bca3d5c8741a70b6c889361d94901593472e7a9704e4445def400e8b08425ec6"
Sep 20 02:17:48.210952 ip-10-0-130-159 kubenswrapper[2584]: E0920 02:17:48.210930    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bca3d5c8741a70b6c889361d94901593472e7a9704e4445def400e8b08425ec6\": container with ID starting with bca3d5c8741a70b6c889361d94901593472e7a9704e4445def400e8b08425ec6 not found: ID does not exist" containerID="bca3d5c8741a70b6c889361d94901593472e7a9704e4445def400e8b08425ec6"
Sep 20 02:17:48.211021 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:17:48.210958    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bca3d5c8741a70b6c889361d94901593472e7a9704e4445def400e8b08425ec6"} err="failed to get container status \"bca3d5c8741a70b6c889361d94901593472e7a9704e4445def400e8b08425ec6\": rpc error: code = NotFound desc = could not find container \"bca3d5c8741a70b6c889361d94901593472e7a9704e4445def400e8b08425ec6\": container with ID starting with bca3d5c8741a70b6c889361d94901593472e7a9704e4445def400e8b08425ec6 not found: ID does not exist"
Sep 20 02:17:48.311118 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:17:48.311091    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2ce47e33-9674-4f3e-956b-d0083215f70f-istio-data\") pod \"2ce47e33-9674-4f3e-956b-d0083215f70f\" (UID: \"2ce47e33-9674-4f3e-956b-d0083215f70f\") "
Sep 20 02:17:48.311296 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:17:48.311136    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2ce47e33-9674-4f3e-956b-d0083215f70f-istio-podinfo\") pod \"2ce47e33-9674-4f3e-956b-d0083215f70f\" (UID: \"2ce47e33-9674-4f3e-956b-d0083215f70f\") "
Sep 20 02:17:48.311296 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:17:48.311173    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mzl5k\" (UniqueName: \"kubernetes.io/projected/2ce47e33-9674-4f3e-956b-d0083215f70f-kube-api-access-mzl5k\") pod \"2ce47e33-9674-4f3e-956b-d0083215f70f\" (UID: \"2ce47e33-9674-4f3e-956b-d0083215f70f\") "
Sep 20 02:17:48.311296 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:17:48.311243    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2ce47e33-9674-4f3e-956b-d0083215f70f-workload-certs\") pod \"2ce47e33-9674-4f3e-956b-d0083215f70f\" (UID: \"2ce47e33-9674-4f3e-956b-d0083215f70f\") "
Sep 20 02:17:48.311296 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:17:48.311266    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2ce47e33-9674-4f3e-956b-d0083215f70f-istiod-ca-cert\") pod \"2ce47e33-9674-4f3e-956b-d0083215f70f\" (UID: \"2ce47e33-9674-4f3e-956b-d0083215f70f\") "
Sep 20 02:17:48.311296 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:17:48.311286    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2ce47e33-9674-4f3e-956b-d0083215f70f-istio-token\") pod \"2ce47e33-9674-4f3e-956b-d0083215f70f\" (UID: \"2ce47e33-9674-4f3e-956b-d0083215f70f\") "
Sep 20 02:17:48.311622 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:17:48.311333    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2ce47e33-9674-4f3e-956b-d0083215f70f-istio-envoy\") pod \"2ce47e33-9674-4f3e-956b-d0083215f70f\" (UID: \"2ce47e33-9674-4f3e-956b-d0083215f70f\") "
Sep 20 02:17:48.311622 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:17:48.311338    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ce47e33-9674-4f3e-956b-d0083215f70f-istio-data" (OuterVolumeSpecName: "istio-data") pod "2ce47e33-9674-4f3e-956b-d0083215f70f" (UID: "2ce47e33-9674-4f3e-956b-d0083215f70f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 20 02:17:48.311622 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:17:48.311360    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ce47e33-9674-4f3e-956b-d0083215f70f-workload-socket\") pod \"2ce47e33-9674-4f3e-956b-d0083215f70f\" (UID: \"2ce47e33-9674-4f3e-956b-d0083215f70f\") "
Sep 20 02:17:48.311622 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:17:48.311390    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ce47e33-9674-4f3e-956b-d0083215f70f-credential-socket\") pod \"2ce47e33-9674-4f3e-956b-d0083215f70f\" (UID: \"2ce47e33-9674-4f3e-956b-d0083215f70f\") "
Sep 20 02:17:48.311844 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:17:48.311734    2584 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2ce47e33-9674-4f3e-956b-d0083215f70f-istio-data\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:17:48.311844 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:17:48.311747    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ce47e33-9674-4f3e-956b-d0083215f70f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "2ce47e33-9674-4f3e-956b-d0083215f70f" (UID: "2ce47e33-9674-4f3e-956b-d0083215f70f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 20 02:17:48.311844 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:17:48.311770    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ce47e33-9674-4f3e-956b-d0083215f70f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "2ce47e33-9674-4f3e-956b-d0083215f70f" (UID: "2ce47e33-9674-4f3e-956b-d0083215f70f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 20 02:17:48.311844 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:17:48.311779    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2ce47e33-9674-4f3e-956b-d0083215f70f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "2ce47e33-9674-4f3e-956b-d0083215f70f" (UID: "2ce47e33-9674-4f3e-956b-d0083215f70f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 20 02:17:48.311844 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:17:48.311804    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ce47e33-9674-4f3e-956b-d0083215f70f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "2ce47e33-9674-4f3e-956b-d0083215f70f" (UID: "2ce47e33-9674-4f3e-956b-d0083215f70f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 20 02:17:48.313790 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:17:48.313762    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2ce47e33-9674-4f3e-956b-d0083215f70f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "2ce47e33-9674-4f3e-956b-d0083215f70f" (UID: "2ce47e33-9674-4f3e-956b-d0083215f70f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 20 02:17:48.313904 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:17:48.313865    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ce47e33-9674-4f3e-956b-d0083215f70f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "2ce47e33-9674-4f3e-956b-d0083215f70f" (UID: "2ce47e33-9674-4f3e-956b-d0083215f70f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 20 02:17:48.314097 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:17:48.314076    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2ce47e33-9674-4f3e-956b-d0083215f70f-kube-api-access-mzl5k" (OuterVolumeSpecName: "kube-api-access-mzl5k") pod "2ce47e33-9674-4f3e-956b-d0083215f70f" (UID: "2ce47e33-9674-4f3e-956b-d0083215f70f"). InnerVolumeSpecName "kube-api-access-mzl5k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 20 02:17:48.314097 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:17:48.314086    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2ce47e33-9674-4f3e-956b-d0083215f70f-istio-token" (OuterVolumeSpecName: "istio-token") pod "2ce47e33-9674-4f3e-956b-d0083215f70f" (UID: "2ce47e33-9674-4f3e-956b-d0083215f70f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 20 02:17:48.412648 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:17:48.412615    2584 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2ce47e33-9674-4f3e-956b-d0083215f70f-workload-certs\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:17:48.412648 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:17:48.412641    2584 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2ce47e33-9674-4f3e-956b-d0083215f70f-istiod-ca-cert\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:17:48.412648 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:17:48.412650    2584 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2ce47e33-9674-4f3e-956b-d0083215f70f-istio-token\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:17:48.412648 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:17:48.412657    2584 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2ce47e33-9674-4f3e-956b-d0083215f70f-istio-envoy\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:17:48.412912 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:17:48.412665    2584 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ce47e33-9674-4f3e-956b-d0083215f70f-workload-socket\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:17:48.412912 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:17:48.412673    2584 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ce47e33-9674-4f3e-956b-d0083215f70f-credential-socket\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:17:48.412912 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:17:48.412681    2584 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2ce47e33-9674-4f3e-956b-d0083215f70f-istio-podinfo\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:17:48.412912 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:17:48.412689    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mzl5k\" (UniqueName: \"kubernetes.io/projected/2ce47e33-9674-4f3e-956b-d0083215f70f-kube-api-access-mzl5k\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:17:48.515077 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:17:48.515043    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-f5813346-openshift-default-dcd586ff6-cvqn5"]
Sep 20 02:17:48.521582 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:17:48.521514    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-f5813346-openshift-default-dcd586ff6-cvqn5"]
Sep 20 02:17:49.094120 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:17:49.094086    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2ce47e33-9674-4f3e-956b-d0083215f70f" path="/var/lib/kubelet/pods/2ce47e33-9674-4f3e-956b-d0083215f70f/volumes"
Sep 20 02:17:52.332950 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:17:52.332893    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:17:52.397410 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:17:52.397372    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:17:53.000159 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:17:53.000121    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:17:56.720253 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:17:56.720212    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-b15051/distinct-2-w4-b15051-kserve-6d8c96c4b-c69xj"]
Sep 20 02:17:56.720891 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:17:56.720872    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="79805ee4-493b-4936-a8c9-ae0044f2f215" containerName="main"
Sep 20 02:17:56.720966 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:17:56.720894    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="79805ee4-493b-4936-a8c9-ae0044f2f215" containerName="main"
Sep 20 02:17:56.720966 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:17:56.720934    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ce47e33-9674-4f3e-956b-d0083215f70f" containerName="istio-proxy"
Sep 20 02:17:56.720966 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:17:56.720943    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ce47e33-9674-4f3e-956b-d0083215f70f" containerName="istio-proxy"
Sep 20 02:17:56.720966 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:17:56.720956    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ce47e33-9674-4f3e-956b-d0083215f70f" containerName="istio-proxy"
Sep 20 02:17:56.720966 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:17:56.720964    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ce47e33-9674-4f3e-956b-d0083215f70f" containerName="istio-proxy"
Sep 20 02:17:56.721239 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:17:56.721062    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="79805ee4-493b-4936-a8c9-ae0044f2f215" containerName="main"
Sep 20 02:17:56.721239 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:17:56.721069    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="2ce47e33-9674-4f3e-956b-d0083215f70f" containerName="istio-proxy"
Sep 20 02:17:56.721239 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:17:56.721075    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="2ce47e33-9674-4f3e-956b-d0083215f70f" containerName="istio-proxy"
Sep 20 02:17:56.724230 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:17:56.724212    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-b15051/distinct-2-w4-b15051-kserve-6d8c96c4b-c69xj"
Sep 20 02:17:56.726561 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:17:56.726508    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-b15051\"/\"distinct-2-w4-b15051-kserve-self-signed-certs\""
Sep 20 02:17:56.732992 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:17:56.732971    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-b15051/distinct-2-w4-b15051-kserve-6d8c96c4b-c69xj"]
Sep 20 02:17:56.746076 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:17:56.746050    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:17:56.895905 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:17:56.895875    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ddf2a155-fe4b-4676-94b0-e2e88f5bfb07-home\") pod \"distinct-2-w4-b15051-kserve-6d8c96c4b-c69xj\" (UID: \"ddf2a155-fe4b-4676-94b0-e2e88f5bfb07\") " pod="e2e-models-e2e-worker-w4-b15051/distinct-2-w4-b15051-kserve-6d8c96c4b-c69xj"
Sep 20 02:17:56.896088 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:17:56.895916    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ddf2a155-fe4b-4676-94b0-e2e88f5bfb07-model-cache\") pod \"distinct-2-w4-b15051-kserve-6d8c96c4b-c69xj\" (UID: \"ddf2a155-fe4b-4676-94b0-e2e88f5bfb07\") " pod="e2e-models-e2e-worker-w4-b15051/distinct-2-w4-b15051-kserve-6d8c96c4b-c69xj"
Sep 20 02:17:56.896088 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:17:56.896005    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ddf2a155-fe4b-4676-94b0-e2e88f5bfb07-dshm\") pod \"distinct-2-w4-b15051-kserve-6d8c96c4b-c69xj\" (UID: \"ddf2a155-fe4b-4676-94b0-e2e88f5bfb07\") " pod="e2e-models-e2e-worker-w4-b15051/distinct-2-w4-b15051-kserve-6d8c96c4b-c69xj"
Sep 20 02:17:56.896088 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:17:56.896081    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ddf2a155-fe4b-4676-94b0-e2e88f5bfb07-tls-certs\") pod \"distinct-2-w4-b15051-kserve-6d8c96c4b-c69xj\" (UID: \"ddf2a155-fe4b-4676-94b0-e2e88f5bfb07\") " pod="e2e-models-e2e-worker-w4-b15051/distinct-2-w4-b15051-kserve-6d8c96c4b-c69xj"
Sep 20 02:17:56.896198 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:17:56.896127    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kkxg9\" (UniqueName: \"kubernetes.io/projected/ddf2a155-fe4b-4676-94b0-e2e88f5bfb07-kube-api-access-kkxg9\") pod \"distinct-2-w4-b15051-kserve-6d8c96c4b-c69xj\" (UID: \"ddf2a155-fe4b-4676-94b0-e2e88f5bfb07\") " pod="e2e-models-e2e-worker-w4-b15051/distinct-2-w4-b15051-kserve-6d8c96c4b-c69xj"
Sep 20 02:17:56.896198 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:17:56.896156    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ddf2a155-fe4b-4676-94b0-e2e88f5bfb07-tmp-dir\") pod \"distinct-2-w4-b15051-kserve-6d8c96c4b-c69xj\" (UID: \"ddf2a155-fe4b-4676-94b0-e2e88f5bfb07\") " pod="e2e-models-e2e-worker-w4-b15051/distinct-2-w4-b15051-kserve-6d8c96c4b-c69xj"
Sep 20 02:17:56.997376 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:17:56.997297    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ddf2a155-fe4b-4676-94b0-e2e88f5bfb07-home\") pod \"distinct-2-w4-b15051-kserve-6d8c96c4b-c69xj\" (UID: \"ddf2a155-fe4b-4676-94b0-e2e88f5bfb07\") " pod="e2e-models-e2e-worker-w4-b15051/distinct-2-w4-b15051-kserve-6d8c96c4b-c69xj"
Sep 20 02:17:56.997376 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:17:56.997340    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ddf2a155-fe4b-4676-94b0-e2e88f5bfb07-model-cache\") pod \"distinct-2-w4-b15051-kserve-6d8c96c4b-c69xj\" (UID: \"ddf2a155-fe4b-4676-94b0-e2e88f5bfb07\") " pod="e2e-models-e2e-worker-w4-b15051/distinct-2-w4-b15051-kserve-6d8c96c4b-c69xj"
Sep 20 02:17:56.997376 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:17:56.997370    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ddf2a155-fe4b-4676-94b0-e2e88f5bfb07-dshm\") pod \"distinct-2-w4-b15051-kserve-6d8c96c4b-c69xj\" (UID: \"ddf2a155-fe4b-4676-94b0-e2e88f5bfb07\") " pod="e2e-models-e2e-worker-w4-b15051/distinct-2-w4-b15051-kserve-6d8c96c4b-c69xj"
Sep 20 02:17:56.997673 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:17:56.997418    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ddf2a155-fe4b-4676-94b0-e2e88f5bfb07-tls-certs\") pod \"distinct-2-w4-b15051-kserve-6d8c96c4b-c69xj\" (UID: \"ddf2a155-fe4b-4676-94b0-e2e88f5bfb07\") " pod="e2e-models-e2e-worker-w4-b15051/distinct-2-w4-b15051-kserve-6d8c96c4b-c69xj"
Sep 20 02:17:56.997673 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:17:56.997453    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kkxg9\" (UniqueName: \"kubernetes.io/projected/ddf2a155-fe4b-4676-94b0-e2e88f5bfb07-kube-api-access-kkxg9\") pod \"distinct-2-w4-b15051-kserve-6d8c96c4b-c69xj\" (UID: \"ddf2a155-fe4b-4676-94b0-e2e88f5bfb07\") " pod="e2e-models-e2e-worker-w4-b15051/distinct-2-w4-b15051-kserve-6d8c96c4b-c69xj"
Sep 20 02:17:56.997673 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:17:56.997479    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ddf2a155-fe4b-4676-94b0-e2e88f5bfb07-tmp-dir\") pod \"distinct-2-w4-b15051-kserve-6d8c96c4b-c69xj\" (UID: \"ddf2a155-fe4b-4676-94b0-e2e88f5bfb07\") " pod="e2e-models-e2e-worker-w4-b15051/distinct-2-w4-b15051-kserve-6d8c96c4b-c69xj"
Sep 20 02:17:56.997838 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:17:56.997780    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ddf2a155-fe4b-4676-94b0-e2e88f5bfb07-home\") pod \"distinct-2-w4-b15051-kserve-6d8c96c4b-c69xj\" (UID: \"ddf2a155-fe4b-4676-94b0-e2e88f5bfb07\") " pod="e2e-models-e2e-worker-w4-b15051/distinct-2-w4-b15051-kserve-6d8c96c4b-c69xj"
Sep 20 02:17:56.997838 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:17:56.997793    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ddf2a155-fe4b-4676-94b0-e2e88f5bfb07-model-cache\") pod \"distinct-2-w4-b15051-kserve-6d8c96c4b-c69xj\" (UID: \"ddf2a155-fe4b-4676-94b0-e2e88f5bfb07\") " pod="e2e-models-e2e-worker-w4-b15051/distinct-2-w4-b15051-kserve-6d8c96c4b-c69xj"
Sep 20 02:17:56.997931 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:17:56.997865    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ddf2a155-fe4b-4676-94b0-e2e88f5bfb07-tmp-dir\") pod \"distinct-2-w4-b15051-kserve-6d8c96c4b-c69xj\" (UID: \"ddf2a155-fe4b-4676-94b0-e2e88f5bfb07\") " pod="e2e-models-e2e-worker-w4-b15051/distinct-2-w4-b15051-kserve-6d8c96c4b-c69xj"
Sep 20 02:17:56.999685 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:17:56.999662    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ddf2a155-fe4b-4676-94b0-e2e88f5bfb07-dshm\") pod \"distinct-2-w4-b15051-kserve-6d8c96c4b-c69xj\" (UID: \"ddf2a155-fe4b-4676-94b0-e2e88f5bfb07\") " pod="e2e-models-e2e-worker-w4-b15051/distinct-2-w4-b15051-kserve-6d8c96c4b-c69xj"
Sep 20 02:17:56.999949 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:17:56.999927    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ddf2a155-fe4b-4676-94b0-e2e88f5bfb07-tls-certs\") pod \"distinct-2-w4-b15051-kserve-6d8c96c4b-c69xj\" (UID: \"ddf2a155-fe4b-4676-94b0-e2e88f5bfb07\") " pod="e2e-models-e2e-worker-w4-b15051/distinct-2-w4-b15051-kserve-6d8c96c4b-c69xj"
Sep 20 02:17:57.005583 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:17:57.005554    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kkxg9\" (UniqueName: \"kubernetes.io/projected/ddf2a155-fe4b-4676-94b0-e2e88f5bfb07-kube-api-access-kkxg9\") pod \"distinct-2-w4-b15051-kserve-6d8c96c4b-c69xj\" (UID: \"ddf2a155-fe4b-4676-94b0-e2e88f5bfb07\") " pod="e2e-models-e2e-worker-w4-b15051/distinct-2-w4-b15051-kserve-6d8c96c4b-c69xj"
Sep 20 02:17:57.035596 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:17:57.035573    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-b15051/distinct-2-w4-b15051-kserve-6d8c96c4b-c69xj"
Sep 20 02:17:57.310793 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:17:57.310769    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-b15051/distinct-2-w4-b15051-kserve-6d8c96c4b-c69xj"]
Sep 20 02:17:57.312608 ip-10-0-130-159 kubenswrapper[2584]: W0920 02:17:57.312574    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podddf2a155_fe4b_4676_94b0_e2e88f5bfb07.slice/crio-aad80fb89b76f28494b049816bd06875a7533aac5eb99fe7c41fa2a06d203c7a WatchSource:0}: Error finding container aad80fb89b76f28494b049816bd06875a7533aac5eb99fe7c41fa2a06d203c7a: Status 404 returned error can't find the container with id aad80fb89b76f28494b049816bd06875a7533aac5eb99fe7c41fa2a06d203c7a
Sep 20 02:17:58.246343 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:17:58.246309    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-b15051/distinct-2-w4-b15051-kserve-6d8c96c4b-c69xj" event={"ID":"ddf2a155-fe4b-4676-94b0-e2e88f5bfb07","Type":"ContainerStarted","Data":"129d7e5a4d33b6e01503832cceb83656f4e72c805b0e4222e727561014200d4a"}
Sep 20 02:17:58.246343 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:17:58.246350    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-b15051/distinct-2-w4-b15051-kserve-6d8c96c4b-c69xj" event={"ID":"ddf2a155-fe4b-4676-94b0-e2e88f5bfb07","Type":"ContainerStarted","Data":"aad80fb89b76f28494b049816bd06875a7533aac5eb99fe7c41fa2a06d203c7a"}
Sep 20 02:17:58.264772 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:17:58.264720    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-b15051/distinct-2-w4-b15051-kserve-6d8c96c4b-c69xj" podStartSLOduration=2.2647053919999998 podStartE2EDuration="2.264705392s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-20 02:17:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-20 02:17:58.262560319 +0000 UTC m=+1535.005239605" watchObservedRunningTime="2026-09-20 02:17:58.264705392 +0000 UTC m=+1535.007384596"
Sep 20 02:17:58.859994 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:17:58.859950    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:17:59.544611 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:17:59.544578    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:17:59.643334 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:17:59.643294    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:18:07.035983 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:18:07.035933    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-b15051/distinct-2-w4-b15051-kserve-6d8c96c4b-c69xj"
Sep 20 02:18:07.036430 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:18:07.036192    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-b15051/distinct-2-w4-b15051-kserve-6d8c96c4b-c69xj"
Sep 20 02:18:07.048741 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:18:07.048718    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-b15051/distinct-2-w4-b15051-kserve-6d8c96c4b-c69xj"
Sep 20 02:18:07.309279 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:18:07.309189    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-b15051/distinct-2-w4-b15051-kserve-6d8c96c4b-c69xj"
Sep 20 02:18:09.548657 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:18:09.548597    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:18:09.759700 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:18:09.759664    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:18:20.211812 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:18:20.211776    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-8195ba/embedding-w0-8195ba-kserve-96fcd5dbd-dz54v"]
Sep 20 02:18:20.215599 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:18:20.215582    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-8195ba/embedding-w0-8195ba-kserve-96fcd5dbd-dz54v"
Sep 20 02:18:20.218008 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:18:20.217988    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-8195ba\"/\"embedding-w0-8195ba-kserve-self-signed-certs\""
Sep 20 02:18:20.224484 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:18:20.224454    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-8195ba/embedding-w0-8195ba-kserve-96fcd5dbd-dz54v"]
Sep 20 02:18:20.320909 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:18:20.320870    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:18:20.329430 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:18:20.329393    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b04dfd7a-50b4-47b1-b248-6f0949b48c52-tls-certs\") pod \"embedding-w0-8195ba-kserve-96fcd5dbd-dz54v\" (UID: \"b04dfd7a-50b4-47b1-b248-6f0949b48c52\") " pod="e2e-models-e2e-worker-w0-8195ba/embedding-w0-8195ba-kserve-96fcd5dbd-dz54v"
Sep 20 02:18:20.329666 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:18:20.329501    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nccch\" (UniqueName: \"kubernetes.io/projected/b04dfd7a-50b4-47b1-b248-6f0949b48c52-kube-api-access-nccch\") pod \"embedding-w0-8195ba-kserve-96fcd5dbd-dz54v\" (UID: \"b04dfd7a-50b4-47b1-b248-6f0949b48c52\") " pod="e2e-models-e2e-worker-w0-8195ba/embedding-w0-8195ba-kserve-96fcd5dbd-dz54v"
Sep 20 02:18:20.329666 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:18:20.329569    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b04dfd7a-50b4-47b1-b248-6f0949b48c52-tmp-dir\") pod \"embedding-w0-8195ba-kserve-96fcd5dbd-dz54v\" (UID: \"b04dfd7a-50b4-47b1-b248-6f0949b48c52\") " pod="e2e-models-e2e-worker-w0-8195ba/embedding-w0-8195ba-kserve-96fcd5dbd-dz54v"
Sep 20 02:18:20.329666 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:18:20.329599    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b04dfd7a-50b4-47b1-b248-6f0949b48c52-dshm\") pod \"embedding-w0-8195ba-kserve-96fcd5dbd-dz54v\" (UID: \"b04dfd7a-50b4-47b1-b248-6f0949b48c52\") " pod="e2e-models-e2e-worker-w0-8195ba/embedding-w0-8195ba-kserve-96fcd5dbd-dz54v"
Sep 20 02:18:20.329666 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:18:20.329650    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b04dfd7a-50b4-47b1-b248-6f0949b48c52-model-cache\") pod \"embedding-w0-8195ba-kserve-96fcd5dbd-dz54v\" (UID: \"b04dfd7a-50b4-47b1-b248-6f0949b48c52\") " pod="e2e-models-e2e-worker-w0-8195ba/embedding-w0-8195ba-kserve-96fcd5dbd-dz54v"
Sep 20 02:18:20.329666 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:18:20.329666    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b04dfd7a-50b4-47b1-b248-6f0949b48c52-home\") pod \"embedding-w0-8195ba-kserve-96fcd5dbd-dz54v\" (UID: \"b04dfd7a-50b4-47b1-b248-6f0949b48c52\") " pod="e2e-models-e2e-worker-w0-8195ba/embedding-w0-8195ba-kserve-96fcd5dbd-dz54v"
Sep 20 02:18:20.430967 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:18:20.430933    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b04dfd7a-50b4-47b1-b248-6f0949b48c52-tmp-dir\") pod \"embedding-w0-8195ba-kserve-96fcd5dbd-dz54v\" (UID: \"b04dfd7a-50b4-47b1-b248-6f0949b48c52\") " pod="e2e-models-e2e-worker-w0-8195ba/embedding-w0-8195ba-kserve-96fcd5dbd-dz54v"
Sep 20 02:18:20.430967 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:18:20.430976    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b04dfd7a-50b4-47b1-b248-6f0949b48c52-dshm\") pod \"embedding-w0-8195ba-kserve-96fcd5dbd-dz54v\" (UID: \"b04dfd7a-50b4-47b1-b248-6f0949b48c52\") " pod="e2e-models-e2e-worker-w0-8195ba/embedding-w0-8195ba-kserve-96fcd5dbd-dz54v"
Sep 20 02:18:20.431256 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:18:20.431116    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b04dfd7a-50b4-47b1-b248-6f0949b48c52-model-cache\") pod \"embedding-w0-8195ba-kserve-96fcd5dbd-dz54v\" (UID: \"b04dfd7a-50b4-47b1-b248-6f0949b48c52\") " pod="e2e-models-e2e-worker-w0-8195ba/embedding-w0-8195ba-kserve-96fcd5dbd-dz54v"
Sep 20 02:18:20.431256 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:18:20.431158    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b04dfd7a-50b4-47b1-b248-6f0949b48c52-home\") pod \"embedding-w0-8195ba-kserve-96fcd5dbd-dz54v\" (UID: \"b04dfd7a-50b4-47b1-b248-6f0949b48c52\") " pod="e2e-models-e2e-worker-w0-8195ba/embedding-w0-8195ba-kserve-96fcd5dbd-dz54v"
Sep 20 02:18:20.431256 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:18:20.431195    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b04dfd7a-50b4-47b1-b248-6f0949b48c52-tls-certs\") pod \"embedding-w0-8195ba-kserve-96fcd5dbd-dz54v\" (UID: \"b04dfd7a-50b4-47b1-b248-6f0949b48c52\") " pod="e2e-models-e2e-worker-w0-8195ba/embedding-w0-8195ba-kserve-96fcd5dbd-dz54v"
Sep 20 02:18:20.431416 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:18:20.431311    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nccch\" (UniqueName: \"kubernetes.io/projected/b04dfd7a-50b4-47b1-b248-6f0949b48c52-kube-api-access-nccch\") pod \"embedding-w0-8195ba-kserve-96fcd5dbd-dz54v\" (UID: \"b04dfd7a-50b4-47b1-b248-6f0949b48c52\") " pod="e2e-models-e2e-worker-w0-8195ba/embedding-w0-8195ba-kserve-96fcd5dbd-dz54v"
Sep 20 02:18:20.431470 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:18:20.431412    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b04dfd7a-50b4-47b1-b248-6f0949b48c52-tmp-dir\") pod \"embedding-w0-8195ba-kserve-96fcd5dbd-dz54v\" (UID: \"b04dfd7a-50b4-47b1-b248-6f0949b48c52\") " pod="e2e-models-e2e-worker-w0-8195ba/embedding-w0-8195ba-kserve-96fcd5dbd-dz54v"
Sep 20 02:18:20.431561 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:18:20.431467    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b04dfd7a-50b4-47b1-b248-6f0949b48c52-model-cache\") pod \"embedding-w0-8195ba-kserve-96fcd5dbd-dz54v\" (UID: \"b04dfd7a-50b4-47b1-b248-6f0949b48c52\") " pod="e2e-models-e2e-worker-w0-8195ba/embedding-w0-8195ba-kserve-96fcd5dbd-dz54v"
Sep 20 02:18:20.431561 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:18:20.431513    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b04dfd7a-50b4-47b1-b248-6f0949b48c52-home\") pod \"embedding-w0-8195ba-kserve-96fcd5dbd-dz54v\" (UID: \"b04dfd7a-50b4-47b1-b248-6f0949b48c52\") " pod="e2e-models-e2e-worker-w0-8195ba/embedding-w0-8195ba-kserve-96fcd5dbd-dz54v"
Sep 20 02:18:20.433257 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:18:20.433239    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b04dfd7a-50b4-47b1-b248-6f0949b48c52-dshm\") pod \"embedding-w0-8195ba-kserve-96fcd5dbd-dz54v\" (UID: \"b04dfd7a-50b4-47b1-b248-6f0949b48c52\") " pod="e2e-models-e2e-worker-w0-8195ba/embedding-w0-8195ba-kserve-96fcd5dbd-dz54v"
Sep 20 02:18:20.433865 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:18:20.433842    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b04dfd7a-50b4-47b1-b248-6f0949b48c52-tls-certs\") pod \"embedding-w0-8195ba-kserve-96fcd5dbd-dz54v\" (UID: \"b04dfd7a-50b4-47b1-b248-6f0949b48c52\") " pod="e2e-models-e2e-worker-w0-8195ba/embedding-w0-8195ba-kserve-96fcd5dbd-dz54v"
Sep 20 02:18:20.438876 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:18:20.438852    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nccch\" (UniqueName: \"kubernetes.io/projected/b04dfd7a-50b4-47b1-b248-6f0949b48c52-kube-api-access-nccch\") pod \"embedding-w0-8195ba-kserve-96fcd5dbd-dz54v\" (UID: \"b04dfd7a-50b4-47b1-b248-6f0949b48c52\") " pod="e2e-models-e2e-worker-w0-8195ba/embedding-w0-8195ba-kserve-96fcd5dbd-dz54v"
Sep 20 02:18:20.526618 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:18:20.526484    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-8195ba/embedding-w0-8195ba-kserve-96fcd5dbd-dz54v"
Sep 20 02:18:20.802469 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:18:20.802434    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-8195ba/embedding-w0-8195ba-kserve-96fcd5dbd-dz54v"]
Sep 20 02:18:20.804702 ip-10-0-130-159 kubenswrapper[2584]: W0920 02:18:20.804670    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb04dfd7a_50b4_47b1_b248_6f0949b48c52.slice/crio-eed82d3377159d00a9b6defe0a7611f605d9a14e89f63573cef373cb64430cf5 WatchSource:0}: Error finding container eed82d3377159d00a9b6defe0a7611f605d9a14e89f63573cef373cb64430cf5: Status 404 returned error can't find the container with id eed82d3377159d00a9b6defe0a7611f605d9a14e89f63573cef373cb64430cf5
Sep 20 02:18:21.375215 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:18:21.375178    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-8195ba/embedding-w0-8195ba-kserve-96fcd5dbd-dz54v" event={"ID":"b04dfd7a-50b4-47b1-b248-6f0949b48c52","Type":"ContainerStarted","Data":"4906b441599b6106601517e1eedb7fd19b35389c2b310066506ecc53220f519f"}
Sep 20 02:18:21.375215 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:18:21.375223    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-8195ba/embedding-w0-8195ba-kserve-96fcd5dbd-dz54v" event={"ID":"b04dfd7a-50b4-47b1-b248-6f0949b48c52","Type":"ContainerStarted","Data":"eed82d3377159d00a9b6defe0a7611f605d9a14e89f63573cef373cb64430cf5"}
Sep 20 02:18:21.393799 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:18:21.393747    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-8195ba/embedding-w0-8195ba-kserve-96fcd5dbd-dz54v" podStartSLOduration=1.393721736 podStartE2EDuration="1.393721736s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-20 02:18:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-20 02:18:21.391585045 +0000 UTC m=+1558.134264250" watchObservedRunningTime="2026-09-20 02:18:21.393721736 +0000 UTC m=+1558.136400941"
Sep 20 02:18:21.789083 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:18:21.789048    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:18:30.527407 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:18:30.527366    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-8195ba/embedding-w0-8195ba-kserve-96fcd5dbd-dz54v"
Sep 20 02:18:30.528067 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:18:30.527693    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-8195ba/embedding-w0-8195ba-kserve-96fcd5dbd-dz54v"
Sep 20 02:18:30.540215 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:18:30.540187    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-8195ba/embedding-w0-8195ba-kserve-96fcd5dbd-dz54v"
Sep 20 02:18:30.728790 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:18:30.728756    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:18:31.441976 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:18:31.441940    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-8195ba/embedding-w0-8195ba-kserve-96fcd5dbd-dz54v"
Sep 20 02:18:36.509366 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:18:36.509333    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:18:36.576395 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:18:36.576361    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:18:36.845009 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:18:36.844928    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:18:37.135002 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:18:37.134899    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-a66f8f-openshift-default-59d849dbcd-wggf6"]
Sep 20 02:18:37.139311 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:18:37.139288    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-a66f8f-openshift-default-59d849dbcd-wggf6"
Sep 20 02:18:37.141873 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:18:37.141843    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 20 02:18:37.142078 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:18:37.142048    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-a66f8f-openshift-default-dockercfg-lp589\""
Sep 20 02:18:37.152041 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:18:37.151975    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-a66f8f-openshift-default-59d849dbcd-wggf6"]
Sep 20 02:18:37.164319 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:18:37.164279    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:18:37.304291 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:18:37.304244    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ab219a0c-a537-49f7-a4e8-6dd1ec16f650-istio-token\") pod \"e2e-api-a-a66f8f-openshift-default-59d849dbcd-wggf6\" (UID: \"ab219a0c-a537-49f7-a4e8-6dd1ec16f650\") " pod="openshift-ingress/e2e-api-a-a66f8f-openshift-default-59d849dbcd-wggf6"
Sep 20 02:18:37.304481 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:18:37.304310    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ab219a0c-a537-49f7-a4e8-6dd1ec16f650-workload-certs\") pod \"e2e-api-a-a66f8f-openshift-default-59d849dbcd-wggf6\" (UID: \"ab219a0c-a537-49f7-a4e8-6dd1ec16f650\") " pod="openshift-ingress/e2e-api-a-a66f8f-openshift-default-59d849dbcd-wggf6"
Sep 20 02:18:37.304481 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:18:37.304340    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ab219a0c-a537-49f7-a4e8-6dd1ec16f650-istiod-ca-cert\") pod \"e2e-api-a-a66f8f-openshift-default-59d849dbcd-wggf6\" (UID: \"ab219a0c-a537-49f7-a4e8-6dd1ec16f650\") " pod="openshift-ingress/e2e-api-a-a66f8f-openshift-default-59d849dbcd-wggf6"
Sep 20 02:18:37.304481 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:18:37.304379    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ab219a0c-a537-49f7-a4e8-6dd1ec16f650-istio-envoy\") pod \"e2e-api-a-a66f8f-openshift-default-59d849dbcd-wggf6\" (UID: \"ab219a0c-a537-49f7-a4e8-6dd1ec16f650\") " pod="openshift-ingress/e2e-api-a-a66f8f-openshift-default-59d849dbcd-wggf6"
Sep 20 02:18:37.304481 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:18:37.304426    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-thw65\" (UniqueName: \"kubernetes.io/projected/ab219a0c-a537-49f7-a4e8-6dd1ec16f650-kube-api-access-thw65\") pod \"e2e-api-a-a66f8f-openshift-default-59d849dbcd-wggf6\" (UID: \"ab219a0c-a537-49f7-a4e8-6dd1ec16f650\") " pod="openshift-ingress/e2e-api-a-a66f8f-openshift-default-59d849dbcd-wggf6"
Sep 20 02:18:37.304481 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:18:37.304478    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ab219a0c-a537-49f7-a4e8-6dd1ec16f650-istio-data\") pod \"e2e-api-a-a66f8f-openshift-default-59d849dbcd-wggf6\" (UID: \"ab219a0c-a537-49f7-a4e8-6dd1ec16f650\") " pod="openshift-ingress/e2e-api-a-a66f8f-openshift-default-59d849dbcd-wggf6"
Sep 20 02:18:37.304716 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:18:37.304505    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab219a0c-a537-49f7-a4e8-6dd1ec16f650-workload-socket\") pod \"e2e-api-a-a66f8f-openshift-default-59d849dbcd-wggf6\" (UID: \"ab219a0c-a537-49f7-a4e8-6dd1ec16f650\") " pod="openshift-ingress/e2e-api-a-a66f8f-openshift-default-59d849dbcd-wggf6"
Sep 20 02:18:37.304716 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:18:37.304568    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab219a0c-a537-49f7-a4e8-6dd1ec16f650-credential-socket\") pod \"e2e-api-a-a66f8f-openshift-default-59d849dbcd-wggf6\" (UID: \"ab219a0c-a537-49f7-a4e8-6dd1ec16f650\") " pod="openshift-ingress/e2e-api-a-a66f8f-openshift-default-59d849dbcd-wggf6"
Sep 20 02:18:37.304716 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:18:37.304644    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ab219a0c-a537-49f7-a4e8-6dd1ec16f650-istio-podinfo\") pod \"e2e-api-a-a66f8f-openshift-default-59d849dbcd-wggf6\" (UID: \"ab219a0c-a537-49f7-a4e8-6dd1ec16f650\") " pod="openshift-ingress/e2e-api-a-a66f8f-openshift-default-59d849dbcd-wggf6"
Sep 20 02:18:37.405687 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:18:37.405599    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab219a0c-a537-49f7-a4e8-6dd1ec16f650-credential-socket\") pod \"e2e-api-a-a66f8f-openshift-default-59d849dbcd-wggf6\" (UID: \"ab219a0c-a537-49f7-a4e8-6dd1ec16f650\") " pod="openshift-ingress/e2e-api-a-a66f8f-openshift-default-59d849dbcd-wggf6"
Sep 20 02:18:37.405687 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:18:37.405659    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ab219a0c-a537-49f7-a4e8-6dd1ec16f650-istio-podinfo\") pod \"e2e-api-a-a66f8f-openshift-default-59d849dbcd-wggf6\" (UID: \"ab219a0c-a537-49f7-a4e8-6dd1ec16f650\") " pod="openshift-ingress/e2e-api-a-a66f8f-openshift-default-59d849dbcd-wggf6"
Sep 20 02:18:37.405909 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:18:37.405718    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ab219a0c-a537-49f7-a4e8-6dd1ec16f650-istio-token\") pod \"e2e-api-a-a66f8f-openshift-default-59d849dbcd-wggf6\" (UID: \"ab219a0c-a537-49f7-a4e8-6dd1ec16f650\") " pod="openshift-ingress/e2e-api-a-a66f8f-openshift-default-59d849dbcd-wggf6"
Sep 20 02:18:37.405909 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:18:37.405757    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ab219a0c-a537-49f7-a4e8-6dd1ec16f650-workload-certs\") pod \"e2e-api-a-a66f8f-openshift-default-59d849dbcd-wggf6\" (UID: \"ab219a0c-a537-49f7-a4e8-6dd1ec16f650\") " pod="openshift-ingress/e2e-api-a-a66f8f-openshift-default-59d849dbcd-wggf6"
Sep 20 02:18:37.405909 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:18:37.405879    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ab219a0c-a537-49f7-a4e8-6dd1ec16f650-istiod-ca-cert\") pod \"e2e-api-a-a66f8f-openshift-default-59d849dbcd-wggf6\" (UID: \"ab219a0c-a537-49f7-a4e8-6dd1ec16f650\") " pod="openshift-ingress/e2e-api-a-a66f8f-openshift-default-59d849dbcd-wggf6"
Sep 20 02:18:37.406080 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:18:37.405962    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ab219a0c-a537-49f7-a4e8-6dd1ec16f650-istio-envoy\") pod \"e2e-api-a-a66f8f-openshift-default-59d849dbcd-wggf6\" (UID: \"ab219a0c-a537-49f7-a4e8-6dd1ec16f650\") " pod="openshift-ingress/e2e-api-a-a66f8f-openshift-default-59d849dbcd-wggf6"
Sep 20 02:18:37.406080 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:18:37.406071    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab219a0c-a537-49f7-a4e8-6dd1ec16f650-credential-socket\") pod \"e2e-api-a-a66f8f-openshift-default-59d849dbcd-wggf6\" (UID: \"ab219a0c-a537-49f7-a4e8-6dd1ec16f650\") " pod="openshift-ingress/e2e-api-a-a66f8f-openshift-default-59d849dbcd-wggf6"
Sep 20 02:18:37.406202 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:18:37.406082    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-thw65\" (UniqueName: \"kubernetes.io/projected/ab219a0c-a537-49f7-a4e8-6dd1ec16f650-kube-api-access-thw65\") pod \"e2e-api-a-a66f8f-openshift-default-59d849dbcd-wggf6\" (UID: \"ab219a0c-a537-49f7-a4e8-6dd1ec16f650\") " pod="openshift-ingress/e2e-api-a-a66f8f-openshift-default-59d849dbcd-wggf6"
Sep 20 02:18:37.406202 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:18:37.406169    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ab219a0c-a537-49f7-a4e8-6dd1ec16f650-istio-data\") pod \"e2e-api-a-a66f8f-openshift-default-59d849dbcd-wggf6\" (UID: \"ab219a0c-a537-49f7-a4e8-6dd1ec16f650\") " pod="openshift-ingress/e2e-api-a-a66f8f-openshift-default-59d849dbcd-wggf6"
Sep 20 02:18:37.406321 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:18:37.406208    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab219a0c-a537-49f7-a4e8-6dd1ec16f650-workload-socket\") pod \"e2e-api-a-a66f8f-openshift-default-59d849dbcd-wggf6\" (UID: \"ab219a0c-a537-49f7-a4e8-6dd1ec16f650\") " pod="openshift-ingress/e2e-api-a-a66f8f-openshift-default-59d849dbcd-wggf6"
Sep 20 02:18:37.406321 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:18:37.406239    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ab219a0c-a537-49f7-a4e8-6dd1ec16f650-workload-certs\") pod \"e2e-api-a-a66f8f-openshift-default-59d849dbcd-wggf6\" (UID: \"ab219a0c-a537-49f7-a4e8-6dd1ec16f650\") " pod="openshift-ingress/e2e-api-a-a66f8f-openshift-default-59d849dbcd-wggf6"
Sep 20 02:18:37.406593 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:18:37.406569    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab219a0c-a537-49f7-a4e8-6dd1ec16f650-workload-socket\") pod \"e2e-api-a-a66f8f-openshift-default-59d849dbcd-wggf6\" (UID: \"ab219a0c-a537-49f7-a4e8-6dd1ec16f650\") " pod="openshift-ingress/e2e-api-a-a66f8f-openshift-default-59d849dbcd-wggf6"
Sep 20 02:18:37.406593 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:18:37.406585    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ab219a0c-a537-49f7-a4e8-6dd1ec16f650-istio-data\") pod \"e2e-api-a-a66f8f-openshift-default-59d849dbcd-wggf6\" (UID: \"ab219a0c-a537-49f7-a4e8-6dd1ec16f650\") " pod="openshift-ingress/e2e-api-a-a66f8f-openshift-default-59d849dbcd-wggf6"
Sep 20 02:18:37.406755 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:18:37.406714    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ab219a0c-a537-49f7-a4e8-6dd1ec16f650-istiod-ca-cert\") pod \"e2e-api-a-a66f8f-openshift-default-59d849dbcd-wggf6\" (UID: \"ab219a0c-a537-49f7-a4e8-6dd1ec16f650\") " pod="openshift-ingress/e2e-api-a-a66f8f-openshift-default-59d849dbcd-wggf6"
Sep 20 02:18:37.408164 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:18:37.408144    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ab219a0c-a537-49f7-a4e8-6dd1ec16f650-istio-envoy\") pod \"e2e-api-a-a66f8f-openshift-default-59d849dbcd-wggf6\" (UID: \"ab219a0c-a537-49f7-a4e8-6dd1ec16f650\") " pod="openshift-ingress/e2e-api-a-a66f8f-openshift-default-59d849dbcd-wggf6"
Sep 20 02:18:37.408443 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:18:37.408423    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ab219a0c-a537-49f7-a4e8-6dd1ec16f650-istio-podinfo\") pod \"e2e-api-a-a66f8f-openshift-default-59d849dbcd-wggf6\" (UID: \"ab219a0c-a537-49f7-a4e8-6dd1ec16f650\") " pod="openshift-ingress/e2e-api-a-a66f8f-openshift-default-59d849dbcd-wggf6"
Sep 20 02:18:37.413176 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:18:37.413151    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ab219a0c-a537-49f7-a4e8-6dd1ec16f650-istio-token\") pod \"e2e-api-a-a66f8f-openshift-default-59d849dbcd-wggf6\" (UID: \"ab219a0c-a537-49f7-a4e8-6dd1ec16f650\") " pod="openshift-ingress/e2e-api-a-a66f8f-openshift-default-59d849dbcd-wggf6"
Sep 20 02:18:37.413632 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:18:37.413593    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-thw65\" (UniqueName: \"kubernetes.io/projected/ab219a0c-a537-49f7-a4e8-6dd1ec16f650-kube-api-access-thw65\") pod \"e2e-api-a-a66f8f-openshift-default-59d849dbcd-wggf6\" (UID: \"ab219a0c-a537-49f7-a4e8-6dd1ec16f650\") " pod="openshift-ingress/e2e-api-a-a66f8f-openshift-default-59d849dbcd-wggf6"
Sep 20 02:18:37.453836 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:18:37.453802    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-a66f8f-openshift-default-59d849dbcd-wggf6"
Sep 20 02:18:37.727594 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:18:37.727506    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:18:37.736435 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:18:37.736403    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-a66f8f-openshift-default-59d849dbcd-wggf6"]
Sep 20 02:18:37.736435 ip-10-0-130-159 kubenswrapper[2584]: W0920 02:18:37.736424    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab219a0c_a537_49f7_a4e8_6dd1ec16f650.slice/crio-8a0747de19b955ce2f357709e5521c5f8818d7f92feb4d5c68e354968302c7e0 WatchSource:0}: Error finding container 8a0747de19b955ce2f357709e5521c5f8818d7f92feb4d5c68e354968302c7e0: Status 404 returned error can't find the container with id 8a0747de19b955ce2f357709e5521c5f8818d7f92feb4d5c68e354968302c7e0
Sep 20 02:18:37.738804 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:18:37.738770    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 20 02:18:37.738908 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:18:37.738839    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 20 02:18:37.738908 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:18:37.738868    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 20 02:18:38.470438 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:18:38.470393    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-a66f8f-openshift-default-59d849dbcd-wggf6" event={"ID":"ab219a0c-a537-49f7-a4e8-6dd1ec16f650","Type":"ContainerStarted","Data":"b9b93c5550454dc9c39a7fca79232b19d7fad9dbf59e6002a8d9e007bdad499a"}
Sep 20 02:18:38.470720 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:18:38.470462    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-a66f8f-openshift-default-59d849dbcd-wggf6" event={"ID":"ab219a0c-a537-49f7-a4e8-6dd1ec16f650","Type":"ContainerStarted","Data":"8a0747de19b955ce2f357709e5521c5f8818d7f92feb4d5c68e354968302c7e0"}
Sep 20 02:18:38.491512 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:18:38.491459    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-a66f8f-openshift-default-59d849dbcd-wggf6" podStartSLOduration=1.491443705 podStartE2EDuration="1.491443705s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-20 02:18:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-20 02:18:38.490543013 +0000 UTC m=+1575.233222218" watchObservedRunningTime="2026-09-20 02:18:38.491443705 +0000 UTC m=+1575.234122910"
Sep 20 02:18:39.454856 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:18:39.454823    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-a66f8f-openshift-default-59d849dbcd-wggf6"
Sep 20 02:18:39.461496 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:18:39.461465    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-a66f8f-openshift-default-59d849dbcd-wggf6"
Sep 20 02:18:39.478591 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:18:39.478561    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-a66f8f-openshift-default-59d849dbcd-wggf6"
Sep 20 02:18:39.479509 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:18:39.479481    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-a66f8f-openshift-default-59d849dbcd-wggf6"
Sep 20 02:18:39.748458 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:18:39.748318    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:18:40.435438 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:18:40.435404    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:18:40.523131 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:18:40.523093    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:18:41.251251 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:18:41.251211    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:18:41.950255 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:18:41.950217    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:18:42.746253 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:18:42.746220    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:18:43.397219 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:18:43.397187    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:18:44.159660 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:18:44.159580    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:18:44.921133 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:18:44.921089    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-a66f8f-6cf67d97df-pkxn8"]
Sep 20 02:18:44.925985 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:18:44.925954    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a66f8f-6cf67d97df-pkxn8"
Sep 20 02:18:44.928371 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:18:44.928338    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-a66f8f-serving-cert\""
Sep 20 02:18:44.934723 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:18:44.934693    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-a66f8f-6cf67d97df-pkxn8"]
Sep 20 02:18:44.952502 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:18:44.952459    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:18:44.987173 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:18:44.987138    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8a5cc051-2a9b-4b26-8edf-51bb5ed106bc-metrics-certs\") pod \"maas-api-e2e-api-a-a66f8f-6cf67d97df-pkxn8\" (UID: \"8a5cc051-2a9b-4b26-8edf-51bb5ed106bc\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a66f8f-6cf67d97df-pkxn8"
Sep 20 02:18:44.987354 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:18:44.987187    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8a5cc051-2a9b-4b26-8edf-51bb5ed106bc-maas-api-tls\") pod \"maas-api-e2e-api-a-a66f8f-6cf67d97df-pkxn8\" (UID: \"8a5cc051-2a9b-4b26-8edf-51bb5ed106bc\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a66f8f-6cf67d97df-pkxn8"
Sep 20 02:18:44.987354 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:18:44.987247    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8a5cc051-2a9b-4b26-8edf-51bb5ed106bc-openshift-service-ca\") pod \"maas-api-e2e-api-a-a66f8f-6cf67d97df-pkxn8\" (UID: \"8a5cc051-2a9b-4b26-8edf-51bb5ed106bc\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a66f8f-6cf67d97df-pkxn8"
Sep 20 02:18:44.987441 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:18:44.987376    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pjb9p\" (UniqueName: \"kubernetes.io/projected/8a5cc051-2a9b-4b26-8edf-51bb5ed106bc-kube-api-access-pjb9p\") pod \"maas-api-e2e-api-a-a66f8f-6cf67d97df-pkxn8\" (UID: \"8a5cc051-2a9b-4b26-8edf-51bb5ed106bc\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a66f8f-6cf67d97df-pkxn8"
Sep 20 02:18:45.005646 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:18:45.005612    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:18:45.088450 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:18:45.088413    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pjb9p\" (UniqueName: \"kubernetes.io/projected/8a5cc051-2a9b-4b26-8edf-51bb5ed106bc-kube-api-access-pjb9p\") pod \"maas-api-e2e-api-a-a66f8f-6cf67d97df-pkxn8\" (UID: \"8a5cc051-2a9b-4b26-8edf-51bb5ed106bc\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a66f8f-6cf67d97df-pkxn8"
Sep 20 02:18:45.088690 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:18:45.088471    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8a5cc051-2a9b-4b26-8edf-51bb5ed106bc-metrics-certs\") pod \"maas-api-e2e-api-a-a66f8f-6cf67d97df-pkxn8\" (UID: \"8a5cc051-2a9b-4b26-8edf-51bb5ed106bc\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a66f8f-6cf67d97df-pkxn8"
Sep 20 02:18:45.088690 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:18:45.088584    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8a5cc051-2a9b-4b26-8edf-51bb5ed106bc-maas-api-tls\") pod \"maas-api-e2e-api-a-a66f8f-6cf67d97df-pkxn8\" (UID: \"8a5cc051-2a9b-4b26-8edf-51bb5ed106bc\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a66f8f-6cf67d97df-pkxn8"
Sep 20 02:18:45.088690 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:18:45.088673    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8a5cc051-2a9b-4b26-8edf-51bb5ed106bc-openshift-service-ca\") pod \"maas-api-e2e-api-a-a66f8f-6cf67d97df-pkxn8\" (UID: \"8a5cc051-2a9b-4b26-8edf-51bb5ed106bc\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a66f8f-6cf67d97df-pkxn8"
Sep 20 02:18:45.088883 ip-10-0-130-159 kubenswrapper[2584]: E0920 02:18:45.088788    2584 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-a66f8f-serving-cert: secret "maas-api-e2e-api-a-a66f8f-serving-cert" not found
Sep 20 02:18:45.088883 ip-10-0-130-159 kubenswrapper[2584]: E0920 02:18:45.088873    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8a5cc051-2a9b-4b26-8edf-51bb5ed106bc-maas-api-tls podName:8a5cc051-2a9b-4b26-8edf-51bb5ed106bc nodeName:}" failed. No retries permitted until 2026-09-20 02:18:45.588853105 +0000 UTC m=+1582.331532291 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/8a5cc051-2a9b-4b26-8edf-51bb5ed106bc-maas-api-tls") pod "maas-api-e2e-api-a-a66f8f-6cf67d97df-pkxn8" (UID: "8a5cc051-2a9b-4b26-8edf-51bb5ed106bc") : secret "maas-api-e2e-api-a-a66f8f-serving-cert" not found
Sep 20 02:18:45.089472 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:18:45.089448    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8a5cc051-2a9b-4b26-8edf-51bb5ed106bc-openshift-service-ca\") pod \"maas-api-e2e-api-a-a66f8f-6cf67d97df-pkxn8\" (UID: \"8a5cc051-2a9b-4b26-8edf-51bb5ed106bc\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a66f8f-6cf67d97df-pkxn8"
Sep 20 02:18:45.091346 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:18:45.091325    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8a5cc051-2a9b-4b26-8edf-51bb5ed106bc-metrics-certs\") pod \"maas-api-e2e-api-a-a66f8f-6cf67d97df-pkxn8\" (UID: \"8a5cc051-2a9b-4b26-8edf-51bb5ed106bc\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a66f8f-6cf67d97df-pkxn8"
Sep 20 02:18:45.097739 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:18:45.097711    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pjb9p\" (UniqueName: \"kubernetes.io/projected/8a5cc051-2a9b-4b26-8edf-51bb5ed106bc-kube-api-access-pjb9p\") pod \"maas-api-e2e-api-a-a66f8f-6cf67d97df-pkxn8\" (UID: \"8a5cc051-2a9b-4b26-8edf-51bb5ed106bc\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a66f8f-6cf67d97df-pkxn8"
Sep 20 02:18:45.130641 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:18:45.130606    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:18:45.407065 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:18:45.407034    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:18:45.503467 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:18:45.503410    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:18:45.593422 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:18:45.593392    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8a5cc051-2a9b-4b26-8edf-51bb5ed106bc-maas-api-tls\") pod \"maas-api-e2e-api-a-a66f8f-6cf67d97df-pkxn8\" (UID: \"8a5cc051-2a9b-4b26-8edf-51bb5ed106bc\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a66f8f-6cf67d97df-pkxn8"
Sep 20 02:18:45.595840 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:18:45.595797    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8a5cc051-2a9b-4b26-8edf-51bb5ed106bc-maas-api-tls\") pod \"maas-api-e2e-api-a-a66f8f-6cf67d97df-pkxn8\" (UID: \"8a5cc051-2a9b-4b26-8edf-51bb5ed106bc\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a66f8f-6cf67d97df-pkxn8"
Sep 20 02:18:45.841454 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:18:45.841417    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a66f8f-6cf67d97df-pkxn8"
Sep 20 02:18:45.946702 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:18:45.946667    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:18:46.121698 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:18:46.121667    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-a66f8f-6cf67d97df-pkxn8"]
Sep 20 02:18:46.122913 ip-10-0-130-159 kubenswrapper[2584]: W0920 02:18:46.122882    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a5cc051_2a9b_4b26_8edf_51bb5ed106bc.slice/crio-3b457e597eb2b4cd17d2e8f8855a6a72b7c69f3a8893dcb23a4e6a3a2e685f57 WatchSource:0}: Error finding container 3b457e597eb2b4cd17d2e8f8855a6a72b7c69f3a8893dcb23a4e6a3a2e685f57: Status 404 returned error can't find the container with id 3b457e597eb2b4cd17d2e8f8855a6a72b7c69f3a8893dcb23a4e6a3a2e685f57
Sep 20 02:18:46.124376 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:18:46.124355    2584 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 20 02:18:46.518602 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:18:46.518567    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a66f8f-6cf67d97df-pkxn8" event={"ID":"8a5cc051-2a9b-4b26-8edf-51bb5ed106bc","Type":"ContainerStarted","Data":"3b457e597eb2b4cd17d2e8f8855a6a72b7c69f3a8893dcb23a4e6a3a2e685f57"}
Sep 20 02:18:46.715141 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:18:46.715112    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:18:47.526807 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:18:47.526771    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a66f8f-6cf67d97df-pkxn8" event={"ID":"8a5cc051-2a9b-4b26-8edf-51bb5ed106bc","Type":"ContainerStarted","Data":"c943b3c07ebe40caaaeffa50d839ffe8d87f1de066495499c6eda6d725ae5e4e"}
Sep 20 02:18:47.527180 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:18:47.526928    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a66f8f-6cf67d97df-pkxn8"
Sep 20 02:18:47.546041 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:18:47.545986    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a66f8f-6cf67d97df-pkxn8" podStartSLOduration=3.035719195 podStartE2EDuration="3.545969541s" totalImagesPullingTime="510.250346ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-20 02:18:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-20 02:18:47.543106718 +0000 UTC m=+1584.285785937" watchObservedRunningTime="2026-09-20 02:18:47.545969541 +0000 UTC m=+1584.288648745"
Sep 20 02:18:53.538193 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:18:53.538161    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a66f8f-6cf67d97df-pkxn8"
Sep 20 02:18:59.699692 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:18:59.699657    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:19:07.062760 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:07.062710    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:19:07.782920 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:07.782887    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:19:08.776970 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:08.776851    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:19:09.482130 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:09.482097    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:19:10.247955 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:10.247920    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:19:11.775934 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:11.775896    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:19:11.959490 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:11.959453    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:19:12.760408 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:12.760361    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:19:13.730225 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:13.730187    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:19:14.311269 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:14.311224    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:19:14.400494 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:14.400454    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:19:14.610421 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:14.610335    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:19:15.444860 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:15.444833    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:19:15.703418 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:15.703386    2584 generic.go:356] "Generic (PLEG): container finished" podID="ab219a0c-a537-49f7-a4e8-6dd1ec16f650" containerID="b9b93c5550454dc9c39a7fca79232b19d7fad9dbf59e6002a8d9e007bdad499a" exitCode=0
Sep 20 02:19:15.703598 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:15.703438    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-a66f8f-openshift-default-59d849dbcd-wggf6" event={"ID":"ab219a0c-a537-49f7-a4e8-6dd1ec16f650","Type":"ContainerDied","Data":"b9b93c5550454dc9c39a7fca79232b19d7fad9dbf59e6002a8d9e007bdad499a"}
Sep 20 02:19:15.703900 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:15.703885    2584 scope.go:117] "RemoveContainer" containerID="b9b93c5550454dc9c39a7fca79232b19d7fad9dbf59e6002a8d9e007bdad499a"
Sep 20 02:19:15.705213 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:15.705179    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 20 02:19:15.705294 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:15.705257    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 20 02:19:15.705332 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:15.705292    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 20 02:19:16.156802 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:16.156709    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:19:16.711408 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:16.711364    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-a66f8f-openshift-default-59d849dbcd-wggf6" event={"ID":"ab219a0c-a537-49f7-a4e8-6dd1ec16f650","Type":"ContainerStarted","Data":"f3b4c355cc4b69cd8c117abb68c5c0dcba59b42ca75acbcd98d3832469f30865"}
Sep 20 02:19:16.854253 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:16.854220    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:19:17.454205 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:17.454161    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-a66f8f-openshift-default-59d849dbcd-wggf6"
Sep 20 02:19:17.454452 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:17.454427    2584 patch_prober.go:28] interesting pod/e2e-api-a-a66f8f-openshift-default-59d849dbcd-wggf6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Sep 20 02:19:17.454504 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:17.454483    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a66f8f-openshift-default-59d849dbcd-wggf6" podUID="ab219a0c-a537-49f7-a4e8-6dd1ec16f650" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Sep 20 02:19:17.752427 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:17.752337    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:19:18.454327 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:18.454290    2584 patch_prober.go:28] interesting pod/e2e-api-a-a66f8f-openshift-default-59d849dbcd-wggf6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Sep 20 02:19:18.454582 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:18.454352    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a66f8f-openshift-default-59d849dbcd-wggf6" podUID="ab219a0c-a537-49f7-a4e8-6dd1ec16f650" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Sep 20 02:19:18.454923 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:18.454902    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:19:19.176915 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:19.176829    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:19:19.455150 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:19.455118    2584 patch_prober.go:28] interesting pod/e2e-api-a-a66f8f-openshift-default-59d849dbcd-wggf6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Sep 20 02:19:19.455338 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:19.455186    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a66f8f-openshift-default-59d849dbcd-wggf6" podUID="ab219a0c-a537-49f7-a4e8-6dd1ec16f650" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Sep 20 02:19:20.058926 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:20.058888    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-dd1246-85c859d9b4-9vqpv"]
Sep 20 02:19:20.083573 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:20.082764    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-dd1246-85c859d9b4-9vqpv"]
Sep 20 02:19:20.083573 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:20.082897    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-dd1246-85c859d9b4-9vqpv"
Sep 20 02:19:20.085858 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:20.085830    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-dd1246-serving-cert\""
Sep 20 02:19:20.088947 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:20.088922    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:19:20.182546 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:20.179071    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0b4d954b-058d-4680-a83e-a072c6a0e230-openshift-service-ca\") pod \"maas-api-e2e-api-b-dd1246-85c859d9b4-9vqpv\" (UID: \"0b4d954b-058d-4680-a83e-a072c6a0e230\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-dd1246-85c859d9b4-9vqpv"
Sep 20 02:19:20.182546 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:20.179145    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0b4d954b-058d-4680-a83e-a072c6a0e230-maas-api-tls\") pod \"maas-api-e2e-api-b-dd1246-85c859d9b4-9vqpv\" (UID: \"0b4d954b-058d-4680-a83e-a072c6a0e230\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-dd1246-85c859d9b4-9vqpv"
Sep 20 02:19:20.182546 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:20.179210    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r4pvs\" (UniqueName: \"kubernetes.io/projected/0b4d954b-058d-4680-a83e-a072c6a0e230-kube-api-access-r4pvs\") pod \"maas-api-e2e-api-b-dd1246-85c859d9b4-9vqpv\" (UID: \"0b4d954b-058d-4680-a83e-a072c6a0e230\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-dd1246-85c859d9b4-9vqpv"
Sep 20 02:19:20.182546 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:20.179365    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0b4d954b-058d-4680-a83e-a072c6a0e230-metrics-certs\") pod \"maas-api-e2e-api-b-dd1246-85c859d9b4-9vqpv\" (UID: \"0b4d954b-058d-4680-a83e-a072c6a0e230\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-dd1246-85c859d9b4-9vqpv"
Sep 20 02:19:20.190210 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:20.190181    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:19:20.281124 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:20.281079    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r4pvs\" (UniqueName: \"kubernetes.io/projected/0b4d954b-058d-4680-a83e-a072c6a0e230-kube-api-access-r4pvs\") pod \"maas-api-e2e-api-b-dd1246-85c859d9b4-9vqpv\" (UID: \"0b4d954b-058d-4680-a83e-a072c6a0e230\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-dd1246-85c859d9b4-9vqpv"
Sep 20 02:19:20.281513 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:20.281490    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0b4d954b-058d-4680-a83e-a072c6a0e230-metrics-certs\") pod \"maas-api-e2e-api-b-dd1246-85c859d9b4-9vqpv\" (UID: \"0b4d954b-058d-4680-a83e-a072c6a0e230\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-dd1246-85c859d9b4-9vqpv"
Sep 20 02:19:20.281751 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:20.281732    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0b4d954b-058d-4680-a83e-a072c6a0e230-openshift-service-ca\") pod \"maas-api-e2e-api-b-dd1246-85c859d9b4-9vqpv\" (UID: \"0b4d954b-058d-4680-a83e-a072c6a0e230\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-dd1246-85c859d9b4-9vqpv"
Sep 20 02:19:20.281890 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:20.281874    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0b4d954b-058d-4680-a83e-a072c6a0e230-maas-api-tls\") pod \"maas-api-e2e-api-b-dd1246-85c859d9b4-9vqpv\" (UID: \"0b4d954b-058d-4680-a83e-a072c6a0e230\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-dd1246-85c859d9b4-9vqpv"
Sep 20 02:19:20.282058 ip-10-0-130-159 kubenswrapper[2584]: E0920 02:19:20.282028    2584 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-dd1246-serving-cert: secret "maas-api-e2e-api-b-dd1246-serving-cert" not found
Sep 20 02:19:20.282277 ip-10-0-130-159 kubenswrapper[2584]: E0920 02:19:20.282261    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0b4d954b-058d-4680-a83e-a072c6a0e230-maas-api-tls podName:0b4d954b-058d-4680-a83e-a072c6a0e230 nodeName:}" failed. No retries permitted until 2026-09-20 02:19:20.782235208 +0000 UTC m=+1617.524914401 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/0b4d954b-058d-4680-a83e-a072c6a0e230-maas-api-tls") pod "maas-api-e2e-api-b-dd1246-85c859d9b4-9vqpv" (UID: "0b4d954b-058d-4680-a83e-a072c6a0e230") : secret "maas-api-e2e-api-b-dd1246-serving-cert" not found
Sep 20 02:19:20.282662 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:20.282629    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0b4d954b-058d-4680-a83e-a072c6a0e230-openshift-service-ca\") pod \"maas-api-e2e-api-b-dd1246-85c859d9b4-9vqpv\" (UID: \"0b4d954b-058d-4680-a83e-a072c6a0e230\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-dd1246-85c859d9b4-9vqpv"
Sep 20 02:19:20.284871 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:20.284843    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0b4d954b-058d-4680-a83e-a072c6a0e230-metrics-certs\") pod \"maas-api-e2e-api-b-dd1246-85c859d9b4-9vqpv\" (UID: \"0b4d954b-058d-4680-a83e-a072c6a0e230\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-dd1246-85c859d9b4-9vqpv"
Sep 20 02:19:20.289125 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:20.289099    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r4pvs\" (UniqueName: \"kubernetes.io/projected/0b4d954b-058d-4680-a83e-a072c6a0e230-kube-api-access-r4pvs\") pod \"maas-api-e2e-api-b-dd1246-85c859d9b4-9vqpv\" (UID: \"0b4d954b-058d-4680-a83e-a072c6a0e230\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-dd1246-85c859d9b4-9vqpv"
Sep 20 02:19:20.454442 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:20.454405    2584 patch_prober.go:28] interesting pod/e2e-api-a-a66f8f-openshift-default-59d849dbcd-wggf6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Sep 20 02:19:20.454622 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:20.454468    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a66f8f-openshift-default-59d849dbcd-wggf6" podUID="ab219a0c-a537-49f7-a4e8-6dd1ec16f650" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Sep 20 02:19:20.579274 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:20.579238    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:19:20.681544 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:20.681434    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:19:20.770445 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:20.770324    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:19:20.789666 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:20.789620    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0b4d954b-058d-4680-a83e-a072c6a0e230-maas-api-tls\") pod \"maas-api-e2e-api-b-dd1246-85c859d9b4-9vqpv\" (UID: \"0b4d954b-058d-4680-a83e-a072c6a0e230\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-dd1246-85c859d9b4-9vqpv"
Sep 20 02:19:20.792267 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:20.792237    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0b4d954b-058d-4680-a83e-a072c6a0e230-maas-api-tls\") pod \"maas-api-e2e-api-b-dd1246-85c859d9b4-9vqpv\" (UID: \"0b4d954b-058d-4680-a83e-a072c6a0e230\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-dd1246-85c859d9b4-9vqpv"
Sep 20 02:19:21.009941 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:21.009900    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-dd1246-85c859d9b4-9vqpv"
Sep 20 02:19:21.309629 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:21.309598    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-dd1246-85c859d9b4-9vqpv"]
Sep 20 02:19:21.311400 ip-10-0-130-159 kubenswrapper[2584]: W0920 02:19:21.311365    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b4d954b_058d_4680_a83e_a072c6a0e230.slice/crio-8e18bb7ec41edcc2c718791ead09128ed8a647de6956c5ebcd576c58d897af27 WatchSource:0}: Error finding container 8e18bb7ec41edcc2c718791ead09128ed8a647de6956c5ebcd576c58d897af27: Status 404 returned error can't find the container with id 8e18bb7ec41edcc2c718791ead09128ed8a647de6956c5ebcd576c58d897af27
Sep 20 02:19:21.454484 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:21.454449    2584 patch_prober.go:28] interesting pod/e2e-api-a-a66f8f-openshift-default-59d849dbcd-wggf6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Sep 20 02:19:21.454729 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:21.454510    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a66f8f-openshift-default-59d849dbcd-wggf6" podUID="ab219a0c-a537-49f7-a4e8-6dd1ec16f650" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Sep 20 02:19:21.741075 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:21.741034    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-dd1246-85c859d9b4-9vqpv" event={"ID":"0b4d954b-058d-4680-a83e-a072c6a0e230","Type":"ContainerStarted","Data":"8e18bb7ec41edcc2c718791ead09128ed8a647de6956c5ebcd576c58d897af27"}
Sep 20 02:19:22.459644 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:22.459611    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-a66f8f-openshift-default-59d849dbcd-wggf6"
Sep 20 02:19:22.460146 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:22.459962    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-a66f8f-openshift-default-59d849dbcd-wggf6"
Sep 20 02:19:22.460918 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:22.460898    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-a66f8f-openshift-default-59d849dbcd-wggf6"
Sep 20 02:19:22.749788 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:22.749686    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-dd1246-85c859d9b4-9vqpv" event={"ID":"0b4d954b-058d-4680-a83e-a072c6a0e230","Type":"ContainerStarted","Data":"48afbb3368a92fa5e2aee4b8dab55fba58e74fa7917b8a14dfd20d1db60c1a1f"}
Sep 20 02:19:22.749984 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:22.749946    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-dd1246-85c859d9b4-9vqpv"
Sep 20 02:19:22.766820 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:22.766770    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-dd1246-85c859d9b4-9vqpv" podStartSLOduration=2.41849287 podStartE2EDuration="2.766752623s" totalImagesPullingTime="348.259753ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-20 02:19:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-20 02:19:22.764842659 +0000 UTC m=+1619.507521863" watchObservedRunningTime="2026-09-20 02:19:22.766752623 +0000 UTC m=+1619.509431884"
Sep 20 02:19:28.761972 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:28.761935    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-dd1246-85c859d9b4-9vqpv"
Sep 20 02:19:32.566923 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:32.566881    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:19:33.147815 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:33.147763    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:19:33.940614 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:33.940581    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:19:34.678048 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:34.678012    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:19:35.437355 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:35.437320    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:19:36.135843 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:36.135812    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:19:37.034367 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:37.034309    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:19:37.548700 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:37.548660    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-8195ba/distinct-2-w0-8195ba-kserve-86f8d74d76-nh8n5"]
Sep 20 02:19:37.563080 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:37.563044    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-8195ba/embedding-w0-8195ba-kserve-96fcd5dbd-dz54v"]
Sep 20 02:19:37.643927 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:37.643890    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:19:37.834047 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:37.833949    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:19:38.632141 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:38.632112    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:19:39.290327 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:39.290272    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:19:40.200849 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:40.200802    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:19:40.867029 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:40.866994    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:19:41.743781 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:41.743747    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:19:42.379672 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:42.379613    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:19:43.043139 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:43.043102    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:19:43.676203 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:43.676168    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:19:44.335933 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:44.335854    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:19:44.436456 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:44.436427    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:19:44.812994 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:44.812956    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-gxh8t" podUID="af876e7a-2edc-43c3-a936-0bc828f859b9" containerName="manager" probeResult="failure" output="Get \"http://10.132.0.36:8081/readyz\": dial tcp 10.132.0.36:8081: connect: connection refused"
Sep 20 02:19:44.880013 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:44.879980    2584 generic.go:356] "Generic (PLEG): container finished" podID="af876e7a-2edc-43c3-a936-0bc828f859b9" containerID="ade5cebfb9803ec0ea1070d615995ba2f73e2cc6333818c99ff51daa60905785" exitCode=137
Sep 20 02:19:44.880194 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:44.880058    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-gxh8t" event={"ID":"af876e7a-2edc-43c3-a936-0bc828f859b9","Type":"ContainerDied","Data":"ade5cebfb9803ec0ea1070d615995ba2f73e2cc6333818c99ff51daa60905785"}
Sep 20 02:19:44.880469 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:44.880448    2584 scope.go:117] "RemoveContainer" containerID="ade5cebfb9803ec0ea1070d615995ba2f73e2cc6333818c99ff51daa60905785"
Sep 20 02:19:45.163947 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:45.163835    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:19:45.888854 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:45.888825    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-gxh8t" event={"ID":"af876e7a-2edc-43c3-a936-0bc828f859b9","Type":"ContainerStarted","Data":"d9510916438794c59f789dc7a50c3602e1dd4068787b53c41650a12bd34aece6"}
Sep 20 02:19:45.889341 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:45.889112    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-gxh8t"
Sep 20 02:19:46.031203 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:46.031173    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:19:46.974994 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:46.974952    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:19:48.048104 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:48.048068    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:19:48.836875 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:48.836842    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:19:49.607561 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:49.607504    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:19:50.240583 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:50.240508    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:19:50.926283 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:50.926254    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:19:51.657172 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:51.657109    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:19:52.195260 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:52.195211    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-8195ba-9b58d5bd6-5m5b8"]
Sep 20 02:19:52.196468 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:52.196431    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-8195ba-9b58d5bd6-5m5b8" podUID="634c1084-aa3b-4904-b893-1281899f03c3" containerName="maas-api" containerID="cri-o://3e9dbe0e237e1fc9ba94d91d040ad22b99a55cfd90fa6f3aa9d0dc304dff47a5" gracePeriod=30
Sep 20 02:19:52.218936 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:52.218901    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:19:52.290805 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:52.290769    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:19:52.347233 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:52.347197    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:19:52.423302 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:52.423266    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:19:52.474498 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:52.474467    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-8195ba-9b58d5bd6-5m5b8"
Sep 20 02:19:52.480989 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:52.480961    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:19:52.533491 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:52.533458    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/634c1084-aa3b-4904-b893-1281899f03c3-metrics-certs\") pod \"634c1084-aa3b-4904-b893-1281899f03c3\" (UID: \"634c1084-aa3b-4904-b893-1281899f03c3\") "
Sep 20 02:19:52.533711 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:52.533526    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-km77p\" (UniqueName: \"kubernetes.io/projected/634c1084-aa3b-4904-b893-1281899f03c3-kube-api-access-km77p\") pod \"634c1084-aa3b-4904-b893-1281899f03c3\" (UID: \"634c1084-aa3b-4904-b893-1281899f03c3\") "
Sep 20 02:19:52.533711 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:52.533626    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/634c1084-aa3b-4904-b893-1281899f03c3-maas-api-tls\") pod \"634c1084-aa3b-4904-b893-1281899f03c3\" (UID: \"634c1084-aa3b-4904-b893-1281899f03c3\") "
Sep 20 02:19:52.533711 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:52.533703    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/634c1084-aa3b-4904-b893-1281899f03c3-openshift-service-ca\") pod \"634c1084-aa3b-4904-b893-1281899f03c3\" (UID: \"634c1084-aa3b-4904-b893-1281899f03c3\") "
Sep 20 02:19:52.534173 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:52.534137    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/634c1084-aa3b-4904-b893-1281899f03c3-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "634c1084-aa3b-4904-b893-1281899f03c3" (UID: "634c1084-aa3b-4904-b893-1281899f03c3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 20 02:19:52.535616 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:52.535591    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/634c1084-aa3b-4904-b893-1281899f03c3-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "634c1084-aa3b-4904-b893-1281899f03c3" (UID: "634c1084-aa3b-4904-b893-1281899f03c3"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 20 02:19:52.535821 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:52.535807    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/634c1084-aa3b-4904-b893-1281899f03c3-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "634c1084-aa3b-4904-b893-1281899f03c3" (UID: "634c1084-aa3b-4904-b893-1281899f03c3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 20 02:19:52.535870 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:52.535815    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/634c1084-aa3b-4904-b893-1281899f03c3-kube-api-access-km77p" (OuterVolumeSpecName: "kube-api-access-km77p") pod "634c1084-aa3b-4904-b893-1281899f03c3" (UID: "634c1084-aa3b-4904-b893-1281899f03c3"). InnerVolumeSpecName "kube-api-access-km77p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 20 02:19:52.573757 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:52.573714    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-8195ba/distinct-2-w0-8195ba-kserve-86f8d74d76-nh8n5" podUID="61a65af9-c40c-4f89-9d50-fbe0599e4f07" containerName="main" containerID="cri-o://4b3d3e4ee07423b379957d27f1bee462f89ccec4bd4d90bd53133246e7952c14" gracePeriod=45
Sep 20 02:19:52.603925 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:52.603894    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:19:52.635423 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:52.635376    2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/634c1084-aa3b-4904-b893-1281899f03c3-maas-api-tls\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:19:52.635423 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:52.635417    2584 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/634c1084-aa3b-4904-b893-1281899f03c3-openshift-service-ca\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:19:52.635423 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:52.635427    2584 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/634c1084-aa3b-4904-b893-1281899f03c3-metrics-certs\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:19:52.635729 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:52.635435    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-km77p\" (UniqueName: \"kubernetes.io/projected/634c1084-aa3b-4904-b893-1281899f03c3-kube-api-access-km77p\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:19:52.768790 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:52.768758    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:19:52.824203 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:52.824179    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-8195ba/distinct-2-w0-8195ba-kserve-86f8d74d76-nh8n5"
Sep 20 02:19:52.860401 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:52.860347    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-8195ba/embedding-w0-8195ba-kserve-96fcd5dbd-dz54v" podUID="b04dfd7a-50b4-47b1-b248-6f0949b48c52" containerName="main" containerID="cri-o://4906b441599b6106601517e1eedb7fd19b35389c2b310066506ecc53220f519f" gracePeriod=45
Sep 20 02:19:52.929659 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:52.929624    2584 generic.go:356] "Generic (PLEG): container finished" podID="634c1084-aa3b-4904-b893-1281899f03c3" containerID="3e9dbe0e237e1fc9ba94d91d040ad22b99a55cfd90fa6f3aa9d0dc304dff47a5" exitCode=0
Sep 20 02:19:52.929851 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:52.929675    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-8195ba-9b58d5bd6-5m5b8"
Sep 20 02:19:52.929851 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:52.929710    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-8195ba-9b58d5bd6-5m5b8" event={"ID":"634c1084-aa3b-4904-b893-1281899f03c3","Type":"ContainerDied","Data":"3e9dbe0e237e1fc9ba94d91d040ad22b99a55cfd90fa6f3aa9d0dc304dff47a5"}
Sep 20 02:19:52.929851 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:52.929769    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-8195ba-9b58d5bd6-5m5b8" event={"ID":"634c1084-aa3b-4904-b893-1281899f03c3","Type":"ContainerDied","Data":"678aeb132f526073fda84385cf31f9e1a7c16754fc3a54ce37a904d3a90b9489"}
Sep 20 02:19:52.929851 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:52.929783    2584 scope.go:117] "RemoveContainer" containerID="3e9dbe0e237e1fc9ba94d91d040ad22b99a55cfd90fa6f3aa9d0dc304dff47a5"
Sep 20 02:19:52.931325 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:52.931300    2584 generic.go:356] "Generic (PLEG): container finished" podID="61a65af9-c40c-4f89-9d50-fbe0599e4f07" containerID="4b3d3e4ee07423b379957d27f1bee462f89ccec4bd4d90bd53133246e7952c14" exitCode=0
Sep 20 02:19:52.931466 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:52.931358    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-8195ba/distinct-2-w0-8195ba-kserve-86f8d74d76-nh8n5"
Sep 20 02:19:52.931466 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:52.931359    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-8195ba/distinct-2-w0-8195ba-kserve-86f8d74d76-nh8n5" event={"ID":"61a65af9-c40c-4f89-9d50-fbe0599e4f07","Type":"ContainerDied","Data":"4b3d3e4ee07423b379957d27f1bee462f89ccec4bd4d90bd53133246e7952c14"}
Sep 20 02:19:52.931585 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:52.931469    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-8195ba/distinct-2-w0-8195ba-kserve-86f8d74d76-nh8n5" event={"ID":"61a65af9-c40c-4f89-9d50-fbe0599e4f07","Type":"ContainerDied","Data":"e44402c4b6538960d5cf714b182afb8f29a497ae39e92c9009df4791fb8eb7f4"}
Sep 20 02:19:52.937883 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:52.937815    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/61a65af9-c40c-4f89-9d50-fbe0599e4f07-model-cache\") pod \"61a65af9-c40c-4f89-9d50-fbe0599e4f07\" (UID: \"61a65af9-c40c-4f89-9d50-fbe0599e4f07\") "
Sep 20 02:19:52.937959 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:52.937902    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/61a65af9-c40c-4f89-9d50-fbe0599e4f07-tls-certs\") pod \"61a65af9-c40c-4f89-9d50-fbe0599e4f07\" (UID: \"61a65af9-c40c-4f89-9d50-fbe0599e4f07\") "
Sep 20 02:19:52.937959 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:52.937932    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r8vz6\" (UniqueName: \"kubernetes.io/projected/61a65af9-c40c-4f89-9d50-fbe0599e4f07-kube-api-access-r8vz6\") pod \"61a65af9-c40c-4f89-9d50-fbe0599e4f07\" (UID: \"61a65af9-c40c-4f89-9d50-fbe0599e4f07\") "
Sep 20 02:19:52.938156 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:52.937960    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/61a65af9-c40c-4f89-9d50-fbe0599e4f07-home\") pod \"61a65af9-c40c-4f89-9d50-fbe0599e4f07\" (UID: \"61a65af9-c40c-4f89-9d50-fbe0599e4f07\") "
Sep 20 02:19:52.938156 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:52.937991    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/61a65af9-c40c-4f89-9d50-fbe0599e4f07-dshm\") pod \"61a65af9-c40c-4f89-9d50-fbe0599e4f07\" (UID: \"61a65af9-c40c-4f89-9d50-fbe0599e4f07\") "
Sep 20 02:19:52.938156 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:52.938099    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61a65af9-c40c-4f89-9d50-fbe0599e4f07-model-cache" (OuterVolumeSpecName: "model-cache") pod "61a65af9-c40c-4f89-9d50-fbe0599e4f07" (UID: "61a65af9-c40c-4f89-9d50-fbe0599e4f07"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 20 02:19:52.938319 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:52.938206    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/61a65af9-c40c-4f89-9d50-fbe0599e4f07-tmp-dir\") pod \"61a65af9-c40c-4f89-9d50-fbe0599e4f07\" (UID: \"61a65af9-c40c-4f89-9d50-fbe0599e4f07\") "
Sep 20 02:19:52.938319 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:52.938305    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61a65af9-c40c-4f89-9d50-fbe0599e4f07-home" (OuterVolumeSpecName: "home") pod "61a65af9-c40c-4f89-9d50-fbe0599e4f07" (UID: "61a65af9-c40c-4f89-9d50-fbe0599e4f07"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 20 02:19:52.938466 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:52.938430    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61a65af9-c40c-4f89-9d50-fbe0599e4f07-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "61a65af9-c40c-4f89-9d50-fbe0599e4f07" (UID: "61a65af9-c40c-4f89-9d50-fbe0599e4f07"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 20 02:19:52.938729 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:52.938664    2584 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/61a65af9-c40c-4f89-9d50-fbe0599e4f07-home\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:19:52.938729 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:52.938684    2584 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/61a65af9-c40c-4f89-9d50-fbe0599e4f07-tmp-dir\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:19:52.938729 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:52.938700    2584 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/61a65af9-c40c-4f89-9d50-fbe0599e4f07-model-cache\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:19:52.940572 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:52.940538    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61a65af9-c40c-4f89-9d50-fbe0599e4f07-dshm" (OuterVolumeSpecName: "dshm") pod "61a65af9-c40c-4f89-9d50-fbe0599e4f07" (UID: "61a65af9-c40c-4f89-9d50-fbe0599e4f07"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 20 02:19:52.940656 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:52.940603    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/61a65af9-c40c-4f89-9d50-fbe0599e4f07-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "61a65af9-c40c-4f89-9d50-fbe0599e4f07" (UID: "61a65af9-c40c-4f89-9d50-fbe0599e4f07"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 20 02:19:52.940656 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:52.940615    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/61a65af9-c40c-4f89-9d50-fbe0599e4f07-kube-api-access-r8vz6" (OuterVolumeSpecName: "kube-api-access-r8vz6") pod "61a65af9-c40c-4f89-9d50-fbe0599e4f07" (UID: "61a65af9-c40c-4f89-9d50-fbe0599e4f07"). InnerVolumeSpecName "kube-api-access-r8vz6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 20 02:19:52.957450 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:52.957409    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-8195ba-9b58d5bd6-5m5b8"]
Sep 20 02:19:52.962389 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:52.962355    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-8195ba-9b58d5bd6-5m5b8"]
Sep 20 02:19:52.975256 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:52.975202    2584 scope.go:117] "RemoveContainer" containerID="3e9dbe0e237e1fc9ba94d91d040ad22b99a55cfd90fa6f3aa9d0dc304dff47a5"
Sep 20 02:19:52.975726 ip-10-0-130-159 kubenswrapper[2584]: E0920 02:19:52.975702    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3e9dbe0e237e1fc9ba94d91d040ad22b99a55cfd90fa6f3aa9d0dc304dff47a5\": container with ID starting with 3e9dbe0e237e1fc9ba94d91d040ad22b99a55cfd90fa6f3aa9d0dc304dff47a5 not found: ID does not exist" containerID="3e9dbe0e237e1fc9ba94d91d040ad22b99a55cfd90fa6f3aa9d0dc304dff47a5"
Sep 20 02:19:52.975847 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:52.975740    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3e9dbe0e237e1fc9ba94d91d040ad22b99a55cfd90fa6f3aa9d0dc304dff47a5"} err="failed to get container status \"3e9dbe0e237e1fc9ba94d91d040ad22b99a55cfd90fa6f3aa9d0dc304dff47a5\": rpc error: code = NotFound desc = could not find container \"3e9dbe0e237e1fc9ba94d91d040ad22b99a55cfd90fa6f3aa9d0dc304dff47a5\": container with ID starting with 3e9dbe0e237e1fc9ba94d91d040ad22b99a55cfd90fa6f3aa9d0dc304dff47a5 not found: ID does not exist"
Sep 20 02:19:52.975847 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:52.975769    2584 scope.go:117] "RemoveContainer" containerID="4b3d3e4ee07423b379957d27f1bee462f89ccec4bd4d90bd53133246e7952c14"
Sep 20 02:19:52.986118 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:52.986077    2584 scope.go:117] "RemoveContainer" containerID="4b3d3e4ee07423b379957d27f1bee462f89ccec4bd4d90bd53133246e7952c14"
Sep 20 02:19:52.986465 ip-10-0-130-159 kubenswrapper[2584]: E0920 02:19:52.986438    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4b3d3e4ee07423b379957d27f1bee462f89ccec4bd4d90bd53133246e7952c14\": container with ID starting with 4b3d3e4ee07423b379957d27f1bee462f89ccec4bd4d90bd53133246e7952c14 not found: ID does not exist" containerID="4b3d3e4ee07423b379957d27f1bee462f89ccec4bd4d90bd53133246e7952c14"
Sep 20 02:19:52.986580 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:52.986475    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b3d3e4ee07423b379957d27f1bee462f89ccec4bd4d90bd53133246e7952c14"} err="failed to get container status \"4b3d3e4ee07423b379957d27f1bee462f89ccec4bd4d90bd53133246e7952c14\": rpc error: code = NotFound desc = could not find container \"4b3d3e4ee07423b379957d27f1bee462f89ccec4bd4d90bd53133246e7952c14\": container with ID starting with 4b3d3e4ee07423b379957d27f1bee462f89ccec4bd4d90bd53133246e7952c14 not found: ID does not exist"
Sep 20 02:19:53.040506 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:53.040478    2584 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/61a65af9-c40c-4f89-9d50-fbe0599e4f07-tls-certs\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:19:53.040506 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:53.040506    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r8vz6\" (UniqueName: \"kubernetes.io/projected/61a65af9-c40c-4f89-9d50-fbe0599e4f07-kube-api-access-r8vz6\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:19:53.040690 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:53.040540    2584 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/61a65af9-c40c-4f89-9d50-fbe0599e4f07-dshm\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:19:53.094717 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:53.094692    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="634c1084-aa3b-4904-b893-1281899f03c3" path="/var/lib/kubelet/pods/634c1084-aa3b-4904-b893-1281899f03c3/volumes"
Sep 20 02:19:53.102294 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:53.102272    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-8195ba/embedding-w0-8195ba-kserve-96fcd5dbd-dz54v"
Sep 20 02:19:53.141112 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:53.141084    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b04dfd7a-50b4-47b1-b248-6f0949b48c52-model-cache\") pod \"b04dfd7a-50b4-47b1-b248-6f0949b48c52\" (UID: \"b04dfd7a-50b4-47b1-b248-6f0949b48c52\") "
Sep 20 02:19:53.141291 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:53.141123    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b04dfd7a-50b4-47b1-b248-6f0949b48c52-tmp-dir\") pod \"b04dfd7a-50b4-47b1-b248-6f0949b48c52\" (UID: \"b04dfd7a-50b4-47b1-b248-6f0949b48c52\") "
Sep 20 02:19:53.141291 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:53.141154    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b04dfd7a-50b4-47b1-b248-6f0949b48c52-home\") pod \"b04dfd7a-50b4-47b1-b248-6f0949b48c52\" (UID: \"b04dfd7a-50b4-47b1-b248-6f0949b48c52\") "
Sep 20 02:19:53.141291 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:53.141192    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b04dfd7a-50b4-47b1-b248-6f0949b48c52-dshm\") pod \"b04dfd7a-50b4-47b1-b248-6f0949b48c52\" (UID: \"b04dfd7a-50b4-47b1-b248-6f0949b48c52\") "
Sep 20 02:19:53.141291 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:53.141260    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nccch\" (UniqueName: \"kubernetes.io/projected/b04dfd7a-50b4-47b1-b248-6f0949b48c52-kube-api-access-nccch\") pod \"b04dfd7a-50b4-47b1-b248-6f0949b48c52\" (UID: \"b04dfd7a-50b4-47b1-b248-6f0949b48c52\") "
Sep 20 02:19:53.141561 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:53.141297    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b04dfd7a-50b4-47b1-b248-6f0949b48c52-tls-certs\") pod \"b04dfd7a-50b4-47b1-b248-6f0949b48c52\" (UID: \"b04dfd7a-50b4-47b1-b248-6f0949b48c52\") "
Sep 20 02:19:53.141561 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:53.141364    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b04dfd7a-50b4-47b1-b248-6f0949b48c52-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b04dfd7a-50b4-47b1-b248-6f0949b48c52" (UID: "b04dfd7a-50b4-47b1-b248-6f0949b48c52"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 20 02:19:53.141679 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:53.141584    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b04dfd7a-50b4-47b1-b248-6f0949b48c52-home" (OuterVolumeSpecName: "home") pod "b04dfd7a-50b4-47b1-b248-6f0949b48c52" (UID: "b04dfd7a-50b4-47b1-b248-6f0949b48c52"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 20 02:19:53.141741 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:53.141724    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b04dfd7a-50b4-47b1-b248-6f0949b48c52-model-cache" (OuterVolumeSpecName: "model-cache") pod "b04dfd7a-50b4-47b1-b248-6f0949b48c52" (UID: "b04dfd7a-50b4-47b1-b248-6f0949b48c52"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 20 02:19:53.141813 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:53.141729    2584 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b04dfd7a-50b4-47b1-b248-6f0949b48c52-tmp-dir\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:19:53.141813 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:53.141757    2584 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b04dfd7a-50b4-47b1-b248-6f0949b48c52-home\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:19:53.143573 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:53.143548    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b04dfd7a-50b4-47b1-b248-6f0949b48c52-kube-api-access-nccch" (OuterVolumeSpecName: "kube-api-access-nccch") pod "b04dfd7a-50b4-47b1-b248-6f0949b48c52" (UID: "b04dfd7a-50b4-47b1-b248-6f0949b48c52"). InnerVolumeSpecName "kube-api-access-nccch". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 20 02:19:53.143694 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:53.143641    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b04dfd7a-50b4-47b1-b248-6f0949b48c52-dshm" (OuterVolumeSpecName: "dshm") pod "b04dfd7a-50b4-47b1-b248-6f0949b48c52" (UID: "b04dfd7a-50b4-47b1-b248-6f0949b48c52"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 20 02:19:53.143694 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:53.143654    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b04dfd7a-50b4-47b1-b248-6f0949b48c52-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b04dfd7a-50b4-47b1-b248-6f0949b48c52" (UID: "b04dfd7a-50b4-47b1-b248-6f0949b48c52"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 20 02:19:53.243036 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:53.243008    2584 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b04dfd7a-50b4-47b1-b248-6f0949b48c52-model-cache\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:19:53.243036 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:53.243032    2584 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b04dfd7a-50b4-47b1-b248-6f0949b48c52-dshm\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:19:53.243036 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:53.243041    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nccch\" (UniqueName: \"kubernetes.io/projected/b04dfd7a-50b4-47b1-b248-6f0949b48c52-kube-api-access-nccch\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:19:53.243462 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:53.243052    2584 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b04dfd7a-50b4-47b1-b248-6f0949b48c52-tls-certs\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:19:53.246794 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:53.246767    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-8195ba/distinct-2-w0-8195ba-kserve-86f8d74d76-nh8n5"]
Sep 20 02:19:53.250093 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:53.250072    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-8195ba/distinct-2-w0-8195ba-kserve-86f8d74d76-nh8n5"]
Sep 20 02:19:53.457802 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:53.457759    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:19:53.939568 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:53.939454    2584 generic.go:356] "Generic (PLEG): container finished" podID="b04dfd7a-50b4-47b1-b248-6f0949b48c52" containerID="4906b441599b6106601517e1eedb7fd19b35389c2b310066506ecc53220f519f" exitCode=0
Sep 20 02:19:53.939568 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:53.939543    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-8195ba/embedding-w0-8195ba-kserve-96fcd5dbd-dz54v"
Sep 20 02:19:53.939568 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:53.939550    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-8195ba/embedding-w0-8195ba-kserve-96fcd5dbd-dz54v" event={"ID":"b04dfd7a-50b4-47b1-b248-6f0949b48c52","Type":"ContainerDied","Data":"4906b441599b6106601517e1eedb7fd19b35389c2b310066506ecc53220f519f"}
Sep 20 02:19:53.939822 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:53.939594    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-8195ba/embedding-w0-8195ba-kserve-96fcd5dbd-dz54v" event={"ID":"b04dfd7a-50b4-47b1-b248-6f0949b48c52","Type":"ContainerDied","Data":"eed82d3377159d00a9b6defe0a7611f605d9a14e89f63573cef373cb64430cf5"}
Sep 20 02:19:53.939822 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:53.939609    2584 scope.go:117] "RemoveContainer" containerID="4906b441599b6106601517e1eedb7fd19b35389c2b310066506ecc53220f519f"
Sep 20 02:19:53.949477 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:53.949459    2584 scope.go:117] "RemoveContainer" containerID="4906b441599b6106601517e1eedb7fd19b35389c2b310066506ecc53220f519f"
Sep 20 02:19:53.949766 ip-10-0-130-159 kubenswrapper[2584]: E0920 02:19:53.949749    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4906b441599b6106601517e1eedb7fd19b35389c2b310066506ecc53220f519f\": container with ID starting with 4906b441599b6106601517e1eedb7fd19b35389c2b310066506ecc53220f519f not found: ID does not exist" containerID="4906b441599b6106601517e1eedb7fd19b35389c2b310066506ecc53220f519f"
Sep 20 02:19:53.949827 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:53.949775    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4906b441599b6106601517e1eedb7fd19b35389c2b310066506ecc53220f519f"} err="failed to get container status \"4906b441599b6106601517e1eedb7fd19b35389c2b310066506ecc53220f519f\": rpc error: code = NotFound desc = could not find container \"4906b441599b6106601517e1eedb7fd19b35389c2b310066506ecc53220f519f\": container with ID starting with 4906b441599b6106601517e1eedb7fd19b35389c2b310066506ecc53220f519f not found: ID does not exist"
Sep 20 02:19:53.960783 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:53.960753    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-8195ba/embedding-w0-8195ba-kserve-96fcd5dbd-dz54v"]
Sep 20 02:19:53.963876 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:53.963853    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-8195ba/embedding-w0-8195ba-kserve-96fcd5dbd-dz54v"]
Sep 20 02:19:54.351110 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:54.351065    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:19:55.075317 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:55.075285    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:19:55.093170 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:55.093132    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="61a65af9-c40c-4f89-9d50-fbe0599e4f07" path="/var/lib/kubelet/pods/61a65af9-c40c-4f89-9d50-fbe0599e4f07/volumes"
Sep 20 02:19:55.093650 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:55.093631    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b04dfd7a-50b4-47b1-b248-6f0949b48c52" path="/var/lib/kubelet/pods/b04dfd7a-50b4-47b1-b248-6f0949b48c52/volumes"
Sep 20 02:19:55.762469 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:55.762440    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:19:56.545924 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:56.545890    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:19:56.897488 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:56.897390    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-gxh8t"
Sep 20 02:19:57.291216 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:57.291179    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:19:58.147481 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:58.147447    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:19:59.117999 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:59.117947    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:19:59.180549 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:59.180486    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:19:59.245482 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:59.245423    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:19:59.384569 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:19:59.384440    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:20:04.780586 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:04.780554    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-a66f8f-6cf67d97df-pkxn8"]
Sep 20 02:20:04.781062 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:04.780841    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a66f8f-6cf67d97df-pkxn8" podUID="8a5cc051-2a9b-4b26-8edf-51bb5ed106bc" containerName="maas-api" containerID="cri-o://c943b3c07ebe40caaaeffa50d839ffe8d87f1de066495499c6eda6d725ae5e4e" gracePeriod=30
Sep 20 02:20:04.804179 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:04.804154    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:20:04.871173 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:04.871127    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:20:04.927054 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:04.927024    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:20:04.996240 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:04.996204    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:20:05.009860 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:05.009827    2584 generic.go:356] "Generic (PLEG): container finished" podID="8a5cc051-2a9b-4b26-8edf-51bb5ed106bc" containerID="c943b3c07ebe40caaaeffa50d839ffe8d87f1de066495499c6eda6d725ae5e4e" exitCode=0
Sep 20 02:20:05.010045 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:05.009897    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a66f8f-6cf67d97df-pkxn8" event={"ID":"8a5cc051-2a9b-4b26-8edf-51bb5ed106bc","Type":"ContainerDied","Data":"c943b3c07ebe40caaaeffa50d839ffe8d87f1de066495499c6eda6d725ae5e4e"}
Sep 20 02:20:05.044741 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:05.044669    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a66f8f-6cf67d97df-pkxn8"
Sep 20 02:20:05.054593 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:05.054558    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:20:05.171147 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:05.171108    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8a5cc051-2a9b-4b26-8edf-51bb5ed106bc-metrics-certs\") pod \"8a5cc051-2a9b-4b26-8edf-51bb5ed106bc\" (UID: \"8a5cc051-2a9b-4b26-8edf-51bb5ed106bc\") "
Sep 20 02:20:05.171355 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:05.171199    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8a5cc051-2a9b-4b26-8edf-51bb5ed106bc-openshift-service-ca\") pod \"8a5cc051-2a9b-4b26-8edf-51bb5ed106bc\" (UID: \"8a5cc051-2a9b-4b26-8edf-51bb5ed106bc\") "
Sep 20 02:20:05.171355 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:05.171293    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pjb9p\" (UniqueName: \"kubernetes.io/projected/8a5cc051-2a9b-4b26-8edf-51bb5ed106bc-kube-api-access-pjb9p\") pod \"8a5cc051-2a9b-4b26-8edf-51bb5ed106bc\" (UID: \"8a5cc051-2a9b-4b26-8edf-51bb5ed106bc\") "
Sep 20 02:20:05.171355 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:05.171336    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8a5cc051-2a9b-4b26-8edf-51bb5ed106bc-maas-api-tls\") pod \"8a5cc051-2a9b-4b26-8edf-51bb5ed106bc\" (UID: \"8a5cc051-2a9b-4b26-8edf-51bb5ed106bc\") "
Sep 20 02:20:05.172096 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:05.172049    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8a5cc051-2a9b-4b26-8edf-51bb5ed106bc-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "8a5cc051-2a9b-4b26-8edf-51bb5ed106bc" (UID: "8a5cc051-2a9b-4b26-8edf-51bb5ed106bc"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 20 02:20:05.175475 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:05.175429    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8a5cc051-2a9b-4b26-8edf-51bb5ed106bc-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "8a5cc051-2a9b-4b26-8edf-51bb5ed106bc" (UID: "8a5cc051-2a9b-4b26-8edf-51bb5ed106bc"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 20 02:20:05.175640 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:05.175609    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8a5cc051-2a9b-4b26-8edf-51bb5ed106bc-kube-api-access-pjb9p" (OuterVolumeSpecName: "kube-api-access-pjb9p") pod "8a5cc051-2a9b-4b26-8edf-51bb5ed106bc" (UID: "8a5cc051-2a9b-4b26-8edf-51bb5ed106bc"). InnerVolumeSpecName "kube-api-access-pjb9p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 20 02:20:05.175925 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:05.175901    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8a5cc051-2a9b-4b26-8edf-51bb5ed106bc-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "8a5cc051-2a9b-4b26-8edf-51bb5ed106bc" (UID: "8a5cc051-2a9b-4b26-8edf-51bb5ed106bc"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 20 02:20:05.272713 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:05.272664    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pjb9p\" (UniqueName: \"kubernetes.io/projected/8a5cc051-2a9b-4b26-8edf-51bb5ed106bc-kube-api-access-pjb9p\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:20:05.272713 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:05.272709    2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8a5cc051-2a9b-4b26-8edf-51bb5ed106bc-maas-api-tls\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:20:05.272713 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:05.272722    2584 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8a5cc051-2a9b-4b26-8edf-51bb5ed106bc-metrics-certs\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:20:05.272988 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:05.272734    2584 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8a5cc051-2a9b-4b26-8edf-51bb5ed106bc-openshift-service-ca\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:20:05.352431 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:05.352322    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:20:05.456072 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:05.456039    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:20:06.017176 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:06.017134    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a66f8f-6cf67d97df-pkxn8" event={"ID":"8a5cc051-2a9b-4b26-8edf-51bb5ed106bc","Type":"ContainerDied","Data":"3b457e597eb2b4cd17d2e8f8855a6a72b7c69f3a8893dcb23a4e6a3a2e685f57"}
Sep 20 02:20:06.017696 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:06.017182    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a66f8f-6cf67d97df-pkxn8"
Sep 20 02:20:06.017696 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:06.017213    2584 scope.go:117] "RemoveContainer" containerID="c943b3c07ebe40caaaeffa50d839ffe8d87f1de066495499c6eda6d725ae5e4e"
Sep 20 02:20:06.038352 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:06.038323    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-a66f8f-6cf67d97df-pkxn8"]
Sep 20 02:20:06.041923 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:06.041898    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-a66f8f-6cf67d97df-pkxn8"]
Sep 20 02:20:06.139992 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:06.139960    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:20:06.941446 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:06.941417    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:20:07.093452 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:07.093421    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8a5cc051-2a9b-4b26-8edf-51bb5ed106bc" path="/var/lib/kubelet/pods/8a5cc051-2a9b-4b26-8edf-51bb5ed106bc/volumes"
Sep 20 02:20:07.572107 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:07.572074    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:20:08.207549 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:08.207503    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:20:09.010368 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:09.010330    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:20:09.665389 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:09.665354    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:20:11.290195 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:11.290158    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:20:11.351650 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:11.351621    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:20:11.522947 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:11.522910    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-a66f8f-openshift-default-59d849dbcd-wggf6"]
Sep 20 02:20:11.523213 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:11.523183    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-a66f8f-openshift-default-59d849dbcd-wggf6" podUID="ab219a0c-a537-49f7-a4e8-6dd1ec16f650" containerName="istio-proxy" containerID="cri-o://f3b4c355cc4b69cd8c117abb68c5c0dcba59b42ca75acbcd98d3832469f30865" gracePeriod=30
Sep 20 02:20:11.541830 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:11.541744    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:20:12.152172 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:12.152129    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:20:12.954719 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:12.954684    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:20:13.730105 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:13.730060    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:20:13.834199 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:13.834166    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:20:13.930950 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:13.930911    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:20:14.418291 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:14.418257    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:20:15.047829 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:15.047777    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:20:15.410581 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:15.410478    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:20:15.847103 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:15.847066    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:20:16.632196 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:16.632167    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:20:16.798546 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:16.798458    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-a66f8f-openshift-default-59d849dbcd-wggf6"
Sep 20 02:20:16.890807 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:16.890778    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-thw65\" (UniqueName: \"kubernetes.io/projected/ab219a0c-a537-49f7-a4e8-6dd1ec16f650-kube-api-access-thw65\") pod \"ab219a0c-a537-49f7-a4e8-6dd1ec16f650\" (UID: \"ab219a0c-a537-49f7-a4e8-6dd1ec16f650\") "
Sep 20 02:20:16.891281 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:16.890822    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ab219a0c-a537-49f7-a4e8-6dd1ec16f650-istio-token\") pod \"ab219a0c-a537-49f7-a4e8-6dd1ec16f650\" (UID: \"ab219a0c-a537-49f7-a4e8-6dd1ec16f650\") "
Sep 20 02:20:16.891281 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:16.890867    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab219a0c-a537-49f7-a4e8-6dd1ec16f650-workload-socket\") pod \"ab219a0c-a537-49f7-a4e8-6dd1ec16f650\" (UID: \"ab219a0c-a537-49f7-a4e8-6dd1ec16f650\") "
Sep 20 02:20:16.891281 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:16.890906    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ab219a0c-a537-49f7-a4e8-6dd1ec16f650-workload-certs\") pod \"ab219a0c-a537-49f7-a4e8-6dd1ec16f650\" (UID: \"ab219a0c-a537-49f7-a4e8-6dd1ec16f650\") "
Sep 20 02:20:16.891281 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:16.890940    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab219a0c-a537-49f7-a4e8-6dd1ec16f650-credential-socket\") pod \"ab219a0c-a537-49f7-a4e8-6dd1ec16f650\" (UID: \"ab219a0c-a537-49f7-a4e8-6dd1ec16f650\") "
Sep 20 02:20:16.891281 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:16.890965    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ab219a0c-a537-49f7-a4e8-6dd1ec16f650-istiod-ca-cert\") pod \"ab219a0c-a537-49f7-a4e8-6dd1ec16f650\" (UID: \"ab219a0c-a537-49f7-a4e8-6dd1ec16f650\") "
Sep 20 02:20:16.891281 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:16.891008    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ab219a0c-a537-49f7-a4e8-6dd1ec16f650-istio-envoy\") pod \"ab219a0c-a537-49f7-a4e8-6dd1ec16f650\" (UID: \"ab219a0c-a537-49f7-a4e8-6dd1ec16f650\") "
Sep 20 02:20:16.891281 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:16.891192    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab219a0c-a537-49f7-a4e8-6dd1ec16f650-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ab219a0c-a537-49f7-a4e8-6dd1ec16f650" (UID: "ab219a0c-a537-49f7-a4e8-6dd1ec16f650"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 20 02:20:16.891281 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:16.891237    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab219a0c-a537-49f7-a4e8-6dd1ec16f650-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ab219a0c-a537-49f7-a4e8-6dd1ec16f650" (UID: "ab219a0c-a537-49f7-a4e8-6dd1ec16f650"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 20 02:20:16.891281 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:16.891256    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab219a0c-a537-49f7-a4e8-6dd1ec16f650-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ab219a0c-a537-49f7-a4e8-6dd1ec16f650" (UID: "ab219a0c-a537-49f7-a4e8-6dd1ec16f650"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 20 02:20:16.891281 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:16.891261    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ab219a0c-a537-49f7-a4e8-6dd1ec16f650-istio-podinfo\") pod \"ab219a0c-a537-49f7-a4e8-6dd1ec16f650\" (UID: \"ab219a0c-a537-49f7-a4e8-6dd1ec16f650\") "
Sep 20 02:20:16.891887 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:16.891388    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ab219a0c-a537-49f7-a4e8-6dd1ec16f650-istio-data\") pod \"ab219a0c-a537-49f7-a4e8-6dd1ec16f650\" (UID: \"ab219a0c-a537-49f7-a4e8-6dd1ec16f650\") "
Sep 20 02:20:16.891887 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:16.891502    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ab219a0c-a537-49f7-a4e8-6dd1ec16f650-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ab219a0c-a537-49f7-a4e8-6dd1ec16f650" (UID: "ab219a0c-a537-49f7-a4e8-6dd1ec16f650"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 20 02:20:16.891887 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:16.891649    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab219a0c-a537-49f7-a4e8-6dd1ec16f650-istio-data" (OuterVolumeSpecName: "istio-data") pod "ab219a0c-a537-49f7-a4e8-6dd1ec16f650" (UID: "ab219a0c-a537-49f7-a4e8-6dd1ec16f650"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 20 02:20:16.891887 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:16.891865    2584 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ab219a0c-a537-49f7-a4e8-6dd1ec16f650-istio-data\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:20:16.891887 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:16.891885    2584 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab219a0c-a537-49f7-a4e8-6dd1ec16f650-workload-socket\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:20:16.892050 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:16.891900    2584 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ab219a0c-a537-49f7-a4e8-6dd1ec16f650-workload-certs\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:20:16.892050 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:16.891914    2584 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab219a0c-a537-49f7-a4e8-6dd1ec16f650-credential-socket\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:20:16.892050 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:16.891928    2584 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ab219a0c-a537-49f7-a4e8-6dd1ec16f650-istiod-ca-cert\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:20:16.893261 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:16.893235    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab219a0c-a537-49f7-a4e8-6dd1ec16f650-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ab219a0c-a537-49f7-a4e8-6dd1ec16f650" (UID: "ab219a0c-a537-49f7-a4e8-6dd1ec16f650"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 20 02:20:16.893387 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:16.893258    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ab219a0c-a537-49f7-a4e8-6dd1ec16f650-istio-token" (OuterVolumeSpecName: "istio-token") pod "ab219a0c-a537-49f7-a4e8-6dd1ec16f650" (UID: "ab219a0c-a537-49f7-a4e8-6dd1ec16f650"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 20 02:20:16.893482 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:16.893466    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ab219a0c-a537-49f7-a4e8-6dd1ec16f650-kube-api-access-thw65" (OuterVolumeSpecName: "kube-api-access-thw65") pod "ab219a0c-a537-49f7-a4e8-6dd1ec16f650" (UID: "ab219a0c-a537-49f7-a4e8-6dd1ec16f650"). InnerVolumeSpecName "kube-api-access-thw65". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 20 02:20:16.893555 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:16.893500    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ab219a0c-a537-49f7-a4e8-6dd1ec16f650-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ab219a0c-a537-49f7-a4e8-6dd1ec16f650" (UID: "ab219a0c-a537-49f7-a4e8-6dd1ec16f650"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 20 02:20:16.992891 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:16.992856    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-thw65\" (UniqueName: \"kubernetes.io/projected/ab219a0c-a537-49f7-a4e8-6dd1ec16f650-kube-api-access-thw65\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:20:16.992891 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:16.992883    2584 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ab219a0c-a537-49f7-a4e8-6dd1ec16f650-istio-token\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:20:16.992891 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:16.992895    2584 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ab219a0c-a537-49f7-a4e8-6dd1ec16f650-istio-envoy\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:20:16.992891 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:16.992904    2584 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ab219a0c-a537-49f7-a4e8-6dd1ec16f650-istio-podinfo\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:20:17.087369 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:17.087277    2584 generic.go:356] "Generic (PLEG): container finished" podID="ab219a0c-a537-49f7-a4e8-6dd1ec16f650" containerID="f3b4c355cc4b69cd8c117abb68c5c0dcba59b42ca75acbcd98d3832469f30865" exitCode=0
Sep 20 02:20:17.087369 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:17.087362    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-a66f8f-openshift-default-59d849dbcd-wggf6"
Sep 20 02:20:17.093804 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:17.093775    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-a66f8f-openshift-default-59d849dbcd-wggf6" event={"ID":"ab219a0c-a537-49f7-a4e8-6dd1ec16f650","Type":"ContainerDied","Data":"f3b4c355cc4b69cd8c117abb68c5c0dcba59b42ca75acbcd98d3832469f30865"}
Sep 20 02:20:17.093992 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:17.093831    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-a66f8f-openshift-default-59d849dbcd-wggf6" event={"ID":"ab219a0c-a537-49f7-a4e8-6dd1ec16f650","Type":"ContainerDied","Data":"8a0747de19b955ce2f357709e5521c5f8818d7f92feb4d5c68e354968302c7e0"}
Sep 20 02:20:17.093992 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:17.093863    2584 scope.go:117] "RemoveContainer" containerID="f3b4c355cc4b69cd8c117abb68c5c0dcba59b42ca75acbcd98d3832469f30865"
Sep 20 02:20:17.106790 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:17.106767    2584 scope.go:117] "RemoveContainer" containerID="b9b93c5550454dc9c39a7fca79232b19d7fad9dbf59e6002a8d9e007bdad499a"
Sep 20 02:20:17.115780 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:17.115759    2584 scope.go:117] "RemoveContainer" containerID="f3b4c355cc4b69cd8c117abb68c5c0dcba59b42ca75acbcd98d3832469f30865"
Sep 20 02:20:17.116106 ip-10-0-130-159 kubenswrapper[2584]: E0920 02:20:17.116084    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f3b4c355cc4b69cd8c117abb68c5c0dcba59b42ca75acbcd98d3832469f30865\": container with ID starting with f3b4c355cc4b69cd8c117abb68c5c0dcba59b42ca75acbcd98d3832469f30865 not found: ID does not exist" containerID="f3b4c355cc4b69cd8c117abb68c5c0dcba59b42ca75acbcd98d3832469f30865"
Sep 20 02:20:17.116181 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:17.116114    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f3b4c355cc4b69cd8c117abb68c5c0dcba59b42ca75acbcd98d3832469f30865"} err="failed to get container status \"f3b4c355cc4b69cd8c117abb68c5c0dcba59b42ca75acbcd98d3832469f30865\": rpc error: code = NotFound desc = could not find container \"f3b4c355cc4b69cd8c117abb68c5c0dcba59b42ca75acbcd98d3832469f30865\": container with ID starting with f3b4c355cc4b69cd8c117abb68c5c0dcba59b42ca75acbcd98d3832469f30865 not found: ID does not exist"
Sep 20 02:20:17.116181 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:17.116135    2584 scope.go:117] "RemoveContainer" containerID="b9b93c5550454dc9c39a7fca79232b19d7fad9dbf59e6002a8d9e007bdad499a"
Sep 20 02:20:17.116412 ip-10-0-130-159 kubenswrapper[2584]: E0920 02:20:17.116384    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b9b93c5550454dc9c39a7fca79232b19d7fad9dbf59e6002a8d9e007bdad499a\": container with ID starting with b9b93c5550454dc9c39a7fca79232b19d7fad9dbf59e6002a8d9e007bdad499a not found: ID does not exist" containerID="b9b93c5550454dc9c39a7fca79232b19d7fad9dbf59e6002a8d9e007bdad499a"
Sep 20 02:20:17.116459 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:17.116421    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b9b93c5550454dc9c39a7fca79232b19d7fad9dbf59e6002a8d9e007bdad499a"} err="failed to get container status \"b9b93c5550454dc9c39a7fca79232b19d7fad9dbf59e6002a8d9e007bdad499a\": rpc error: code = NotFound desc = could not find container \"b9b93c5550454dc9c39a7fca79232b19d7fad9dbf59e6002a8d9e007bdad499a\": container with ID starting with b9b93c5550454dc9c39a7fca79232b19d7fad9dbf59e6002a8d9e007bdad499a not found: ID does not exist"
Sep 20 02:20:17.122042 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:17.122018    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-a66f8f-openshift-default-59d849dbcd-wggf6"]
Sep 20 02:20:17.130342 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:17.130315    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-a66f8f-openshift-default-59d849dbcd-wggf6"]
Sep 20 02:20:17.369659 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:17.369553    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:20:19.093388 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:19.093342    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ab219a0c-a537-49f7-a4e8-6dd1ec16f650" path="/var/lib/kubelet/pods/ab219a0c-a537-49f7-a4e8-6dd1ec16f650/volumes"
Sep 20 02:20:21.025450 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:21.025416    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:20:27.335019 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:27.334981    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-dd1246-85c859d9b4-9vqpv"]
Sep 20 02:20:27.335480 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:27.335254    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-dd1246-85c859d9b4-9vqpv" podUID="0b4d954b-058d-4680-a83e-a072c6a0e230" containerName="maas-api" containerID="cri-o://48afbb3368a92fa5e2aee4b8dab55fba58e74fa7917b8a14dfd20d1db60c1a1f" gracePeriod=30
Sep 20 02:20:27.360894 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:27.359380    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:20:27.417845 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:27.417805    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:20:27.474107 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:27.474063    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:20:27.574356 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:27.574326    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:20:27.601681 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:27.601618    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-dd1246-85c859d9b4-9vqpv"
Sep 20 02:20:27.675194 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:27.675160    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:20:27.694326 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:27.694294    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0b4d954b-058d-4680-a83e-a072c6a0e230-openshift-service-ca\") pod \"0b4d954b-058d-4680-a83e-a072c6a0e230\" (UID: \"0b4d954b-058d-4680-a83e-a072c6a0e230\") "
Sep 20 02:20:27.694555 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:27.694349    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0b4d954b-058d-4680-a83e-a072c6a0e230-maas-api-tls\") pod \"0b4d954b-058d-4680-a83e-a072c6a0e230\" (UID: \"0b4d954b-058d-4680-a83e-a072c6a0e230\") "
Sep 20 02:20:27.694555 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:27.694411    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0b4d954b-058d-4680-a83e-a072c6a0e230-metrics-certs\") pod \"0b4d954b-058d-4680-a83e-a072c6a0e230\" (UID: \"0b4d954b-058d-4680-a83e-a072c6a0e230\") "
Sep 20 02:20:27.694555 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:27.694433    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r4pvs\" (UniqueName: \"kubernetes.io/projected/0b4d954b-058d-4680-a83e-a072c6a0e230-kube-api-access-r4pvs\") pod \"0b4d954b-058d-4680-a83e-a072c6a0e230\" (UID: \"0b4d954b-058d-4680-a83e-a072c6a0e230\") "
Sep 20 02:20:27.694800 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:27.694772    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0b4d954b-058d-4680-a83e-a072c6a0e230-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "0b4d954b-058d-4680-a83e-a072c6a0e230" (UID: "0b4d954b-058d-4680-a83e-a072c6a0e230"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 20 02:20:27.697371 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:27.697339    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0b4d954b-058d-4680-a83e-a072c6a0e230-kube-api-access-r4pvs" (OuterVolumeSpecName: "kube-api-access-r4pvs") pod "0b4d954b-058d-4680-a83e-a072c6a0e230" (UID: "0b4d954b-058d-4680-a83e-a072c6a0e230"). InnerVolumeSpecName "kube-api-access-r4pvs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 20 02:20:27.697515 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:27.697465    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0b4d954b-058d-4680-a83e-a072c6a0e230-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "0b4d954b-058d-4680-a83e-a072c6a0e230" (UID: "0b4d954b-058d-4680-a83e-a072c6a0e230"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 20 02:20:27.697515 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:27.697487    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0b4d954b-058d-4680-a83e-a072c6a0e230-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "0b4d954b-058d-4680-a83e-a072c6a0e230" (UID: "0b4d954b-058d-4680-a83e-a072c6a0e230"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 20 02:20:27.795689 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:27.795659    2584 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0b4d954b-058d-4680-a83e-a072c6a0e230-openshift-service-ca\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:20:27.795689 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:27.795689    2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0b4d954b-058d-4680-a83e-a072c6a0e230-maas-api-tls\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:20:27.795689 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:27.795698    2584 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0b4d954b-058d-4680-a83e-a072c6a0e230-metrics-certs\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:20:27.795934 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:27.795706    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r4pvs\" (UniqueName: \"kubernetes.io/projected/0b4d954b-058d-4680-a83e-a072c6a0e230-kube-api-access-r4pvs\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:20:27.805173 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:27.805142    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:20:28.058796 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:28.058753    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:20:28.149533 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:28.149471    2584 generic.go:356] "Generic (PLEG): container finished" podID="0b4d954b-058d-4680-a83e-a072c6a0e230" containerID="48afbb3368a92fa5e2aee4b8dab55fba58e74fa7917b8a14dfd20d1db60c1a1f" exitCode=0
Sep 20 02:20:28.149715 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:28.149541    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-dd1246-85c859d9b4-9vqpv"
Sep 20 02:20:28.149715 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:28.149564    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-dd1246-85c859d9b4-9vqpv" event={"ID":"0b4d954b-058d-4680-a83e-a072c6a0e230","Type":"ContainerDied","Data":"48afbb3368a92fa5e2aee4b8dab55fba58e74fa7917b8a14dfd20d1db60c1a1f"}
Sep 20 02:20:28.149715 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:28.149627    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-dd1246-85c859d9b4-9vqpv" event={"ID":"0b4d954b-058d-4680-a83e-a072c6a0e230","Type":"ContainerDied","Data":"8e18bb7ec41edcc2c718791ead09128ed8a647de6956c5ebcd576c58d897af27"}
Sep 20 02:20:28.149715 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:28.149658    2584 scope.go:117] "RemoveContainer" containerID="48afbb3368a92fa5e2aee4b8dab55fba58e74fa7917b8a14dfd20d1db60c1a1f"
Sep 20 02:20:28.160325 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:28.160304    2584 scope.go:117] "RemoveContainer" containerID="48afbb3368a92fa5e2aee4b8dab55fba58e74fa7917b8a14dfd20d1db60c1a1f"
Sep 20 02:20:28.160720 ip-10-0-130-159 kubenswrapper[2584]: E0920 02:20:28.160694    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"48afbb3368a92fa5e2aee4b8dab55fba58e74fa7917b8a14dfd20d1db60c1a1f\": container with ID starting with 48afbb3368a92fa5e2aee4b8dab55fba58e74fa7917b8a14dfd20d1db60c1a1f not found: ID does not exist" containerID="48afbb3368a92fa5e2aee4b8dab55fba58e74fa7917b8a14dfd20d1db60c1a1f"
Sep 20 02:20:28.160811 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:28.160732    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"48afbb3368a92fa5e2aee4b8dab55fba58e74fa7917b8a14dfd20d1db60c1a1f"} err="failed to get container status \"48afbb3368a92fa5e2aee4b8dab55fba58e74fa7917b8a14dfd20d1db60c1a1f\": rpc error: code = NotFound desc = could not find container \"48afbb3368a92fa5e2aee4b8dab55fba58e74fa7917b8a14dfd20d1db60c1a1f\": container with ID starting with 48afbb3368a92fa5e2aee4b8dab55fba58e74fa7917b8a14dfd20d1db60c1a1f not found: ID does not exist"
Sep 20 02:20:28.172443 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:28.172413    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-dd1246-85c859d9b4-9vqpv"]
Sep 20 02:20:28.177222 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:28.177199    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-dd1246-85c859d9b4-9vqpv"]
Sep 20 02:20:28.653665 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:28.653631    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:20:29.094794 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:29.094760    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0b4d954b-058d-4680-a83e-a072c6a0e230" path="/var/lib/kubelet/pods/0b4d954b-058d-4680-a83e-a072c6a0e230/volumes"
Sep 20 02:20:29.307792 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:29.307759    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:20:30.113783 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:30.113749    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:20:30.762450 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:30.762417    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:20:31.437566 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:31.437509    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:20:32.261792 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:32.261759    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:20:33.782950 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:33.782914    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:20:33.883509 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:33.883469    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:20:34.048384 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:34.048292    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:20:34.459706 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:34.459676    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:20:35.435585 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:35.435551    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:20:36.238391 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:36.238357    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:20:36.945615 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:36.945579    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:20:37.736936 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:37.736903    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:20:38.534029 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:38.533944    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:20:39.317353 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:39.317321    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:20:44.855181 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:44.855145    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:20:44.917890 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:44.917862    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:20:44.983577 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:44.983500    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:20:45.048801 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:45.048770    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:20:45.105918 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:45.105824    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:20:45.398586 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:45.398476    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:20:45.498762 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:45.498726    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:20:46.146770 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:46.146735    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:20:47.044948 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:47.044907    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:20:47.803701 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:47.803664    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:20:48.446461 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:48.446419    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:20:51.191007 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:51.190970    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:20:51.291223 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:51.291189    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:20:51.449503 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:51.449473    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:20:51.926657 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:51.926570    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:20:52.768887 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:52.768836    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:20:53.632883 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:53.632850    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:20:54.428145 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:54.428112    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:20:55.253443 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:55.253412    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:20:55.909356 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:20:55.909321    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:21:02.371959 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:02.371921    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-90811b-66749f8cf9-svlvk"]
Sep 20 02:21:02.372813 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:02.372770    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-90811b-66749f8cf9-svlvk" podUID="e906d174-1565-4ce9-819b-df16a4977b15" containerName="maas-api" containerID="cri-o://4b63083e7969e1f4c689c8b6c2de092272e1724bb7127fe77f8ab438cac311ea" gracePeriod=30
Sep 20 02:21:02.376877 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:02.376833    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:21:02.456565 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:02.456324    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:21:02.519737 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:02.519664    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:21:02.580551 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:02.580431    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:21:02.640276 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:02.640239    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:21:02.656462 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:02.656427    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-90811b-66749f8cf9-svlvk"
Sep 20 02:21:02.739995 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:02.739966    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e906d174-1565-4ce9-819b-df16a4977b15-openshift-service-ca\") pod \"e906d174-1565-4ce9-819b-df16a4977b15\" (UID: \"e906d174-1565-4ce9-819b-df16a4977b15\") "
Sep 20 02:21:02.740181 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:02.740026    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k7jqw\" (UniqueName: \"kubernetes.io/projected/e906d174-1565-4ce9-819b-df16a4977b15-kube-api-access-k7jqw\") pod \"e906d174-1565-4ce9-819b-df16a4977b15\" (UID: \"e906d174-1565-4ce9-819b-df16a4977b15\") "
Sep 20 02:21:02.740181 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:02.740059    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e906d174-1565-4ce9-819b-df16a4977b15-metrics-certs\") pod \"e906d174-1565-4ce9-819b-df16a4977b15\" (UID: \"e906d174-1565-4ce9-819b-df16a4977b15\") "
Sep 20 02:21:02.740280 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:02.740176    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e906d174-1565-4ce9-819b-df16a4977b15-maas-api-tls\") pod \"e906d174-1565-4ce9-819b-df16a4977b15\" (UID: \"e906d174-1565-4ce9-819b-df16a4977b15\") "
Sep 20 02:21:02.740335 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:02.740311    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e906d174-1565-4ce9-819b-df16a4977b15-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e906d174-1565-4ce9-819b-df16a4977b15" (UID: "e906d174-1565-4ce9-819b-df16a4977b15"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 20 02:21:02.740541 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:02.740501    2584 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e906d174-1565-4ce9-819b-df16a4977b15-openshift-service-ca\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:21:02.742252 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:02.742229    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e906d174-1565-4ce9-819b-df16a4977b15-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "e906d174-1565-4ce9-819b-df16a4977b15" (UID: "e906d174-1565-4ce9-819b-df16a4977b15"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 20 02:21:02.742339 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:02.742277    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e906d174-1565-4ce9-819b-df16a4977b15-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e906d174-1565-4ce9-819b-df16a4977b15" (UID: "e906d174-1565-4ce9-819b-df16a4977b15"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 20 02:21:02.742382 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:02.742347    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e906d174-1565-4ce9-819b-df16a4977b15-kube-api-access-k7jqw" (OuterVolumeSpecName: "kube-api-access-k7jqw") pod "e906d174-1565-4ce9-819b-df16a4977b15" (UID: "e906d174-1565-4ce9-819b-df16a4977b15"). InnerVolumeSpecName "kube-api-access-k7jqw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 20 02:21:02.841011 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:02.840971    2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e906d174-1565-4ce9-819b-df16a4977b15-maas-api-tls\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:21:02.841011 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:02.841002    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k7jqw\" (UniqueName: \"kubernetes.io/projected/e906d174-1565-4ce9-819b-df16a4977b15-kube-api-access-k7jqw\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:21:02.841011 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:02.841012    2584 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e906d174-1565-4ce9-819b-df16a4977b15-metrics-certs\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:21:02.939807 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:02.939712    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:21:03.047481 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:03.047426    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:21:03.371600 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:03.371567    2584 generic.go:356] "Generic (PLEG): container finished" podID="e906d174-1565-4ce9-819b-df16a4977b15" containerID="4b63083e7969e1f4c689c8b6c2de092272e1724bb7127fe77f8ab438cac311ea" exitCode=0
Sep 20 02:21:03.371775 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:03.371616    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-90811b-66749f8cf9-svlvk"
Sep 20 02:21:03.371775 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:03.371651    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-90811b-66749f8cf9-svlvk" event={"ID":"e906d174-1565-4ce9-819b-df16a4977b15","Type":"ContainerDied","Data":"4b63083e7969e1f4c689c8b6c2de092272e1724bb7127fe77f8ab438cac311ea"}
Sep 20 02:21:03.371775 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:03.371708    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-90811b-66749f8cf9-svlvk" event={"ID":"e906d174-1565-4ce9-819b-df16a4977b15","Type":"ContainerDied","Data":"7ee3aa233ef044aafc59762642f80c06af8f9ca6ab391addab2ced7aff8d4efd"}
Sep 20 02:21:03.371775 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:03.371731    2584 scope.go:117] "RemoveContainer" containerID="4b63083e7969e1f4c689c8b6c2de092272e1724bb7127fe77f8ab438cac311ea"
Sep 20 02:21:03.381377 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:03.381183    2584 scope.go:117] "RemoveContainer" containerID="4b63083e7969e1f4c689c8b6c2de092272e1724bb7127fe77f8ab438cac311ea"
Sep 20 02:21:03.381645 ip-10-0-130-159 kubenswrapper[2584]: E0920 02:21:03.381466    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4b63083e7969e1f4c689c8b6c2de092272e1724bb7127fe77f8ab438cac311ea\": container with ID starting with 4b63083e7969e1f4c689c8b6c2de092272e1724bb7127fe77f8ab438cac311ea not found: ID does not exist" containerID="4b63083e7969e1f4c689c8b6c2de092272e1724bb7127fe77f8ab438cac311ea"
Sep 20 02:21:03.381645 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:03.381491    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b63083e7969e1f4c689c8b6c2de092272e1724bb7127fe77f8ab438cac311ea"} err="failed to get container status \"4b63083e7969e1f4c689c8b6c2de092272e1724bb7127fe77f8ab438cac311ea\": rpc error: code = NotFound desc = could not find container \"4b63083e7969e1f4c689c8b6c2de092272e1724bb7127fe77f8ab438cac311ea\": container with ID starting with 4b63083e7969e1f4c689c8b6c2de092272e1724bb7127fe77f8ab438cac311ea not found: ID does not exist"
Sep 20 02:21:03.390543 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:03.390494    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-90811b-66749f8cf9-svlvk"]
Sep 20 02:21:03.392431 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:03.392410    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-90811b-66749f8cf9-svlvk"]
Sep 20 02:21:03.742020 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:03.741987    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:21:04.581704 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:04.581667    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:21:05.092988 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:05.092955    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e906d174-1565-4ce9-819b-df16a4977b15" path="/var/lib/kubelet/pods/e906d174-1565-4ce9-819b-df16a4977b15/volumes"
Sep 20 02:21:05.225465 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:05.225422    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:21:08.668512 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:08.668477    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:21:08.768749 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:08.768704    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:21:08.931950 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:08.931854    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:21:25.849929 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:25.849892    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:21:26.849883 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:26.849847    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:21:27.545424 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:27.545384    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:21:28.243812 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:28.243779    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:21:29.116634 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:29.116602    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:21:30.823678 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:30.823643    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-b15051/distinct-2-w4-b15051-kserve-6d8c96c4b-c69xj"]
Sep 20 02:21:30.828753 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:30.828728    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-b15051/distinct-w4-b15051-kserve-98f67785-dskgq"]
Sep 20 02:21:30.919467 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:30.919431    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:21:41.607597 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:41.607567    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-b15051-f8f665d5b-694xh"]
Sep 20 02:21:41.608143 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:41.607761    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-b15051-f8f665d5b-694xh" podUID="9f64dd1e-7983-4545-b7ab-f8d684791433" containerName="maas-api" containerID="cri-o://17f45bdd4bc3d3ec0cea272b545396bdb7cacc8213cca9bf7e979aa1fac4eb84" gracePeriod=30
Sep 20 02:21:41.637100 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:41.637072    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:21:41.665338 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:41.665299    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-b15051-554564495-2c89t"]
Sep 20 02:21:41.666163 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:41.666124    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w4-b15051-554564495-2c89t" podUID="2dd7d85a-5316-4cc9-ba92-1bea129040c1" containerName="payload-processing" containerID="cri-o://d785b644368da37dbf0e699928d2217898918d2a6b8fdbb1430ffbd870099a61" gracePeriod=30
Sep 20 02:21:41.669618 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:41.669585    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-b15051-75f77884c4-klpfp"]
Sep 20 02:21:41.669847 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:41.669820    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-b15051-75f77884c4-klpfp" podUID="883d6aff-0186-4fdc-84da-6c9a122abe0c" containerName="payload-pre-processing" containerID="cri-o://5f14d2c14cfbed6ffcc443bf7d96ea66e1da3dc7c495c710ac47f45cae1f1d05" gracePeriod=30
Sep 20 02:21:41.710884 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:41.710849    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:21:41.773071 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:41.773041    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:21:41.841218 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:41.841148    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:21:41.907939 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:41.907855    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:21:41.968375 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:41.968339    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-b15051-75f77884c4-klpfp"
Sep 20 02:21:41.972252 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:41.972223    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-b15051-554564495-2c89t"
Sep 20 02:21:41.989831 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:41.989799    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-b15051-f8f665d5b-694xh"
Sep 20 02:21:42.027282 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:42.027249    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-84dvl\" (UniqueName: \"kubernetes.io/projected/883d6aff-0186-4fdc-84da-6c9a122abe0c-kube-api-access-84dvl\") pod \"883d6aff-0186-4fdc-84da-6c9a122abe0c\" (UID: \"883d6aff-0186-4fdc-84da-6c9a122abe0c\") "
Sep 20 02:21:42.027471 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:42.027344    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rfl45\" (UniqueName: \"kubernetes.io/projected/9f64dd1e-7983-4545-b7ab-f8d684791433-kube-api-access-rfl45\") pod \"9f64dd1e-7983-4545-b7ab-f8d684791433\" (UID: \"9f64dd1e-7983-4545-b7ab-f8d684791433\") "
Sep 20 02:21:42.027471 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:42.027376    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2dd7d85a-5316-4cc9-ba92-1bea129040c1-plugins-config-volume\") pod \"2dd7d85a-5316-4cc9-ba92-1bea129040c1\" (UID: \"2dd7d85a-5316-4cc9-ba92-1bea129040c1\") "
Sep 20 02:21:42.027471 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:42.027405    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9f64dd1e-7983-4545-b7ab-f8d684791433-metrics-certs\") pod \"9f64dd1e-7983-4545-b7ab-f8d684791433\" (UID: \"9f64dd1e-7983-4545-b7ab-f8d684791433\") "
Sep 20 02:21:42.027471 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:42.027458    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9f64dd1e-7983-4545-b7ab-f8d684791433-maas-api-tls\") pod \"9f64dd1e-7983-4545-b7ab-f8d684791433\" (UID: \"9f64dd1e-7983-4545-b7ab-f8d684791433\") "
Sep 20 02:21:42.027729 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:42.027483    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/883d6aff-0186-4fdc-84da-6c9a122abe0c-plugins-config-volume\") pod \"883d6aff-0186-4fdc-84da-6c9a122abe0c\" (UID: \"883d6aff-0186-4fdc-84da-6c9a122abe0c\") "
Sep 20 02:21:42.027729 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:42.027506    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zrbtm\" (UniqueName: \"kubernetes.io/projected/2dd7d85a-5316-4cc9-ba92-1bea129040c1-kube-api-access-zrbtm\") pod \"2dd7d85a-5316-4cc9-ba92-1bea129040c1\" (UID: \"2dd7d85a-5316-4cc9-ba92-1bea129040c1\") "
Sep 20 02:21:42.027729 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:42.027566    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9f64dd1e-7983-4545-b7ab-f8d684791433-openshift-service-ca\") pod \"9f64dd1e-7983-4545-b7ab-f8d684791433\" (UID: \"9f64dd1e-7983-4545-b7ab-f8d684791433\") "
Sep 20 02:21:42.028329 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:42.028301    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9f64dd1e-7983-4545-b7ab-f8d684791433-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "9f64dd1e-7983-4545-b7ab-f8d684791433" (UID: "9f64dd1e-7983-4545-b7ab-f8d684791433"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 20 02:21:42.028689 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:42.028637    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/883d6aff-0186-4fdc-84da-6c9a122abe0c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "883d6aff-0186-4fdc-84da-6c9a122abe0c" (UID: "883d6aff-0186-4fdc-84da-6c9a122abe0c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 20 02:21:42.029091 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:42.028985    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2dd7d85a-5316-4cc9-ba92-1bea129040c1-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2dd7d85a-5316-4cc9-ba92-1bea129040c1" (UID: "2dd7d85a-5316-4cc9-ba92-1bea129040c1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 20 02:21:42.030074 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:42.030037    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/883d6aff-0186-4fdc-84da-6c9a122abe0c-kube-api-access-84dvl" (OuterVolumeSpecName: "kube-api-access-84dvl") pod "883d6aff-0186-4fdc-84da-6c9a122abe0c" (UID: "883d6aff-0186-4fdc-84da-6c9a122abe0c"). InnerVolumeSpecName "kube-api-access-84dvl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 20 02:21:42.030728 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:42.030706    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2dd7d85a-5316-4cc9-ba92-1bea129040c1-kube-api-access-zrbtm" (OuterVolumeSpecName: "kube-api-access-zrbtm") pod "2dd7d85a-5316-4cc9-ba92-1bea129040c1" (UID: "2dd7d85a-5316-4cc9-ba92-1bea129040c1"). InnerVolumeSpecName "kube-api-access-zrbtm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 20 02:21:42.030941 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:42.030914    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9f64dd1e-7983-4545-b7ab-f8d684791433-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "9f64dd1e-7983-4545-b7ab-f8d684791433" (UID: "9f64dd1e-7983-4545-b7ab-f8d684791433"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 20 02:21:42.031434 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:42.031405    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9f64dd1e-7983-4545-b7ab-f8d684791433-kube-api-access-rfl45" (OuterVolumeSpecName: "kube-api-access-rfl45") pod "9f64dd1e-7983-4545-b7ab-f8d684791433" (UID: "9f64dd1e-7983-4545-b7ab-f8d684791433"). InnerVolumeSpecName "kube-api-access-rfl45". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 20 02:21:42.031973 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:42.031950    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9f64dd1e-7983-4545-b7ab-f8d684791433-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "9f64dd1e-7983-4545-b7ab-f8d684791433" (UID: "9f64dd1e-7983-4545-b7ab-f8d684791433"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 20 02:21:42.128458 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:42.128424    2584 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2dd7d85a-5316-4cc9-ba92-1bea129040c1-plugins-config-volume\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:21:42.128458 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:42.128456    2584 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9f64dd1e-7983-4545-b7ab-f8d684791433-metrics-certs\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:21:42.128458 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:42.128466    2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9f64dd1e-7983-4545-b7ab-f8d684791433-maas-api-tls\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:21:42.128716 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:42.128475    2584 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/883d6aff-0186-4fdc-84da-6c9a122abe0c-plugins-config-volume\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:21:42.128716 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:42.128487    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zrbtm\" (UniqueName: \"kubernetes.io/projected/2dd7d85a-5316-4cc9-ba92-1bea129040c1-kube-api-access-zrbtm\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:21:42.128716 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:42.128498    2584 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9f64dd1e-7983-4545-b7ab-f8d684791433-openshift-service-ca\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:21:42.128716 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:42.128508    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-84dvl\" (UniqueName: \"kubernetes.io/projected/883d6aff-0186-4fdc-84da-6c9a122abe0c-kube-api-access-84dvl\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:21:42.128716 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:42.128547    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rfl45\" (UniqueName: \"kubernetes.io/projected/9f64dd1e-7983-4545-b7ab-f8d684791433-kube-api-access-rfl45\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:21:42.201014 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:42.200981    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:21:42.301217 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:42.301179    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:21:42.597576 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:42.597442    2584 generic.go:356] "Generic (PLEG): container finished" podID="9f64dd1e-7983-4545-b7ab-f8d684791433" containerID="17f45bdd4bc3d3ec0cea272b545396bdb7cacc8213cca9bf7e979aa1fac4eb84" exitCode=0
Sep 20 02:21:42.597576 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:42.597550    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-b15051-f8f665d5b-694xh"
Sep 20 02:21:42.597576 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:42.597558    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-b15051-f8f665d5b-694xh" event={"ID":"9f64dd1e-7983-4545-b7ab-f8d684791433","Type":"ContainerDied","Data":"17f45bdd4bc3d3ec0cea272b545396bdb7cacc8213cca9bf7e979aa1fac4eb84"}
Sep 20 02:21:42.597847 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:42.597623    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-b15051-f8f665d5b-694xh" event={"ID":"9f64dd1e-7983-4545-b7ab-f8d684791433","Type":"ContainerDied","Data":"32aa72bd7bcfdb3bf9855c5fef8080aa6d96fe488fe81edf07f0c044a8c6d38e"}
Sep 20 02:21:42.597847 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:42.597643    2584 scope.go:117] "RemoveContainer" containerID="17f45bdd4bc3d3ec0cea272b545396bdb7cacc8213cca9bf7e979aa1fac4eb84"
Sep 20 02:21:42.599223 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:42.599194    2584 generic.go:356] "Generic (PLEG): container finished" podID="2dd7d85a-5316-4cc9-ba92-1bea129040c1" containerID="d785b644368da37dbf0e699928d2217898918d2a6b8fdbb1430ffbd870099a61" exitCode=0
Sep 20 02:21:42.599386 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:42.599261    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-b15051-554564495-2c89t"
Sep 20 02:21:42.599386 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:42.599289    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-b15051-554564495-2c89t" event={"ID":"2dd7d85a-5316-4cc9-ba92-1bea129040c1","Type":"ContainerDied","Data":"d785b644368da37dbf0e699928d2217898918d2a6b8fdbb1430ffbd870099a61"}
Sep 20 02:21:42.599386 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:42.599353    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-b15051-554564495-2c89t" event={"ID":"2dd7d85a-5316-4cc9-ba92-1bea129040c1","Type":"ContainerDied","Data":"58beb0cb5630d4c2ed71af7b85fb96987e3958730203c2513065f4eaf034fadc"}
Sep 20 02:21:42.601163 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:42.601137    2584 generic.go:356] "Generic (PLEG): container finished" podID="883d6aff-0186-4fdc-84da-6c9a122abe0c" containerID="5f14d2c14cfbed6ffcc443bf7d96ea66e1da3dc7c495c710ac47f45cae1f1d05" exitCode=0
Sep 20 02:21:42.601302 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:42.601200    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-b15051-75f77884c4-klpfp" event={"ID":"883d6aff-0186-4fdc-84da-6c9a122abe0c","Type":"ContainerDied","Data":"5f14d2c14cfbed6ffcc443bf7d96ea66e1da3dc7c495c710ac47f45cae1f1d05"}
Sep 20 02:21:42.601302 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:42.601213    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-b15051-75f77884c4-klpfp"
Sep 20 02:21:42.601302 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:42.601250    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-b15051-75f77884c4-klpfp" event={"ID":"883d6aff-0186-4fdc-84da-6c9a122abe0c","Type":"ContainerDied","Data":"020c7abbff65ffae946d28bf7caf25a9c76ca5b0de98092f93a995d9134d27bd"}
Sep 20 02:21:42.609270 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:42.609137    2584 scope.go:117] "RemoveContainer" containerID="17f45bdd4bc3d3ec0cea272b545396bdb7cacc8213cca9bf7e979aa1fac4eb84"
Sep 20 02:21:42.609555 ip-10-0-130-159 kubenswrapper[2584]: E0920 02:21:42.609536    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"17f45bdd4bc3d3ec0cea272b545396bdb7cacc8213cca9bf7e979aa1fac4eb84\": container with ID starting with 17f45bdd4bc3d3ec0cea272b545396bdb7cacc8213cca9bf7e979aa1fac4eb84 not found: ID does not exist" containerID="17f45bdd4bc3d3ec0cea272b545396bdb7cacc8213cca9bf7e979aa1fac4eb84"
Sep 20 02:21:42.609623 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:42.609564    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"17f45bdd4bc3d3ec0cea272b545396bdb7cacc8213cca9bf7e979aa1fac4eb84"} err="failed to get container status \"17f45bdd4bc3d3ec0cea272b545396bdb7cacc8213cca9bf7e979aa1fac4eb84\": rpc error: code = NotFound desc = could not find container \"17f45bdd4bc3d3ec0cea272b545396bdb7cacc8213cca9bf7e979aa1fac4eb84\": container with ID starting with 17f45bdd4bc3d3ec0cea272b545396bdb7cacc8213cca9bf7e979aa1fac4eb84 not found: ID does not exist"
Sep 20 02:21:42.609623 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:42.609584    2584 scope.go:117] "RemoveContainer" containerID="d785b644368da37dbf0e699928d2217898918d2a6b8fdbb1430ffbd870099a61"
Sep 20 02:21:42.620408 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:42.620385    2584 scope.go:117] "RemoveContainer" containerID="d785b644368da37dbf0e699928d2217898918d2a6b8fdbb1430ffbd870099a61"
Sep 20 02:21:42.620943 ip-10-0-130-159 kubenswrapper[2584]: E0920 02:21:42.620912    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d785b644368da37dbf0e699928d2217898918d2a6b8fdbb1430ffbd870099a61\": container with ID starting with d785b644368da37dbf0e699928d2217898918d2a6b8fdbb1430ffbd870099a61 not found: ID does not exist" containerID="d785b644368da37dbf0e699928d2217898918d2a6b8fdbb1430ffbd870099a61"
Sep 20 02:21:42.621064 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:42.620954    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d785b644368da37dbf0e699928d2217898918d2a6b8fdbb1430ffbd870099a61"} err="failed to get container status \"d785b644368da37dbf0e699928d2217898918d2a6b8fdbb1430ffbd870099a61\": rpc error: code = NotFound desc = could not find container \"d785b644368da37dbf0e699928d2217898918d2a6b8fdbb1430ffbd870099a61\": container with ID starting with d785b644368da37dbf0e699928d2217898918d2a6b8fdbb1430ffbd870099a61 not found: ID does not exist"
Sep 20 02:21:42.621064 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:42.620982    2584 scope.go:117] "RemoveContainer" containerID="5f14d2c14cfbed6ffcc443bf7d96ea66e1da3dc7c495c710ac47f45cae1f1d05"
Sep 20 02:21:42.630835 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:42.630807    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-b15051-f8f665d5b-694xh"]
Sep 20 02:21:42.633574 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:42.633553    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-b15051-f8f665d5b-694xh"]
Sep 20 02:21:42.641687 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:42.641659    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-b15051-75f77884c4-klpfp"]
Sep 20 02:21:42.642692 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:42.642673    2584 scope.go:117] "RemoveContainer" containerID="5f14d2c14cfbed6ffcc443bf7d96ea66e1da3dc7c495c710ac47f45cae1f1d05"
Sep 20 02:21:42.643047 ip-10-0-130-159 kubenswrapper[2584]: E0920 02:21:42.643027    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5f14d2c14cfbed6ffcc443bf7d96ea66e1da3dc7c495c710ac47f45cae1f1d05\": container with ID starting with 5f14d2c14cfbed6ffcc443bf7d96ea66e1da3dc7c495c710ac47f45cae1f1d05 not found: ID does not exist" containerID="5f14d2c14cfbed6ffcc443bf7d96ea66e1da3dc7c495c710ac47f45cae1f1d05"
Sep 20 02:21:42.643122 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:42.643058    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5f14d2c14cfbed6ffcc443bf7d96ea66e1da3dc7c495c710ac47f45cae1f1d05"} err="failed to get container status \"5f14d2c14cfbed6ffcc443bf7d96ea66e1da3dc7c495c710ac47f45cae1f1d05\": rpc error: code = NotFound desc = could not find container \"5f14d2c14cfbed6ffcc443bf7d96ea66e1da3dc7c495c710ac47f45cae1f1d05\": container with ID starting with 5f14d2c14cfbed6ffcc443bf7d96ea66e1da3dc7c495c710ac47f45cae1f1d05 not found: ID does not exist"
Sep 20 02:21:42.647666 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:42.647643    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-b15051-75f77884c4-klpfp"]
Sep 20 02:21:42.658183 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:42.658154    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-b15051-554564495-2c89t"]
Sep 20 02:21:42.660121 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:42.660095    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-b15051-554564495-2c89t"]
Sep 20 02:21:43.002018 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:43.001972    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:21:43.093567 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:43.093535    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2dd7d85a-5316-4cc9-ba92-1bea129040c1" path="/var/lib/kubelet/pods/2dd7d85a-5316-4cc9-ba92-1bea129040c1/volumes"
Sep 20 02:21:43.093929 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:43.093916    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="883d6aff-0186-4fdc-84da-6c9a122abe0c" path="/var/lib/kubelet/pods/883d6aff-0186-4fdc-84da-6c9a122abe0c/volumes"
Sep 20 02:21:43.094268 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:43.094257    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9f64dd1e-7983-4545-b7ab-f8d684791433" path="/var/lib/kubelet/pods/9f64dd1e-7983-4545-b7ab-f8d684791433/volumes"
Sep 20 02:21:43.853374 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:43.853342    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:21:45.843922 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:45.843880    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-b15051/distinct-2-w4-b15051-kserve-6d8c96c4b-c69xj" podUID="ddf2a155-fe4b-4676-94b0-e2e88f5bfb07" containerName="main" containerID="cri-o://129d7e5a4d33b6e01503832cceb83656f4e72c805b0e4222e727561014200d4a" gracePeriod=45
Sep 20 02:21:45.849233 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:45.849198    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-b15051/distinct-w4-b15051-kserve-98f67785-dskgq" podUID="c0f8e7fe-c5c2-4a8a-b0e9-5db728be091e" containerName="main" containerID="cri-o://f0e1bfeb802bcd7e1e6dc21b3458986767de1ad3e0f5424a3806e7c9e7daea5b" gracePeriod=45
Sep 20 02:21:46.129470 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:46.129443    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-b15051/distinct-2-w4-b15051-kserve-6d8c96c4b-c69xj"
Sep 20 02:21:46.167687 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:46.167656    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-b15051/distinct-w4-b15051-kserve-98f67785-dskgq"
Sep 20 02:21:46.169512 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:46.169489    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ddf2a155-fe4b-4676-94b0-e2e88f5bfb07-tmp-dir\") pod \"ddf2a155-fe4b-4676-94b0-e2e88f5bfb07\" (UID: \"ddf2a155-fe4b-4676-94b0-e2e88f5bfb07\") "
Sep 20 02:21:46.169671 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:46.169607    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ddf2a155-fe4b-4676-94b0-e2e88f5bfb07-dshm\") pod \"ddf2a155-fe4b-4676-94b0-e2e88f5bfb07\" (UID: \"ddf2a155-fe4b-4676-94b0-e2e88f5bfb07\") "
Sep 20 02:21:46.169671 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:46.169658    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ddf2a155-fe4b-4676-94b0-e2e88f5bfb07-home\") pod \"ddf2a155-fe4b-4676-94b0-e2e88f5bfb07\" (UID: \"ddf2a155-fe4b-4676-94b0-e2e88f5bfb07\") "
Sep 20 02:21:46.169796 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:46.169678    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ddf2a155-fe4b-4676-94b0-e2e88f5bfb07-model-cache\") pod \"ddf2a155-fe4b-4676-94b0-e2e88f5bfb07\" (UID: \"ddf2a155-fe4b-4676-94b0-e2e88f5bfb07\") "
Sep 20 02:21:46.169796 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:46.169702    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kkxg9\" (UniqueName: \"kubernetes.io/projected/ddf2a155-fe4b-4676-94b0-e2e88f5bfb07-kube-api-access-kkxg9\") pod \"ddf2a155-fe4b-4676-94b0-e2e88f5bfb07\" (UID: \"ddf2a155-fe4b-4676-94b0-e2e88f5bfb07\") "
Sep 20 02:21:46.169796 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:46.169721    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ddf2a155-fe4b-4676-94b0-e2e88f5bfb07-tls-certs\") pod \"ddf2a155-fe4b-4676-94b0-e2e88f5bfb07\" (UID: \"ddf2a155-fe4b-4676-94b0-e2e88f5bfb07\") "
Sep 20 02:21:46.169957 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:46.169787    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ddf2a155-fe4b-4676-94b0-e2e88f5bfb07-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ddf2a155-fe4b-4676-94b0-e2e88f5bfb07" (UID: "ddf2a155-fe4b-4676-94b0-e2e88f5bfb07"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 20 02:21:46.170075 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:46.170053    2584 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ddf2a155-fe4b-4676-94b0-e2e88f5bfb07-tmp-dir\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:21:46.170282 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:46.170238    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ddf2a155-fe4b-4676-94b0-e2e88f5bfb07-home" (OuterVolumeSpecName: "home") pod "ddf2a155-fe4b-4676-94b0-e2e88f5bfb07" (UID: "ddf2a155-fe4b-4676-94b0-e2e88f5bfb07"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 20 02:21:46.170365 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:46.170284    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ddf2a155-fe4b-4676-94b0-e2e88f5bfb07-model-cache" (OuterVolumeSpecName: "model-cache") pod "ddf2a155-fe4b-4676-94b0-e2e88f5bfb07" (UID: "ddf2a155-fe4b-4676-94b0-e2e88f5bfb07"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 20 02:21:46.172245 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:46.172214    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ddf2a155-fe4b-4676-94b0-e2e88f5bfb07-kube-api-access-kkxg9" (OuterVolumeSpecName: "kube-api-access-kkxg9") pod "ddf2a155-fe4b-4676-94b0-e2e88f5bfb07" (UID: "ddf2a155-fe4b-4676-94b0-e2e88f5bfb07"). InnerVolumeSpecName "kube-api-access-kkxg9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 20 02:21:46.172360 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:46.172300    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ddf2a155-fe4b-4676-94b0-e2e88f5bfb07-dshm" (OuterVolumeSpecName: "dshm") pod "ddf2a155-fe4b-4676-94b0-e2e88f5bfb07" (UID: "ddf2a155-fe4b-4676-94b0-e2e88f5bfb07"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 20 02:21:46.172360 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:46.172305    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ddf2a155-fe4b-4676-94b0-e2e88f5bfb07-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ddf2a155-fe4b-4676-94b0-e2e88f5bfb07" (UID: "ddf2a155-fe4b-4676-94b0-e2e88f5bfb07"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 20 02:21:46.270931 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:46.270899    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-szjt7\" (UniqueName: \"kubernetes.io/projected/c0f8e7fe-c5c2-4a8a-b0e9-5db728be091e-kube-api-access-szjt7\") pod \"c0f8e7fe-c5c2-4a8a-b0e9-5db728be091e\" (UID: \"c0f8e7fe-c5c2-4a8a-b0e9-5db728be091e\") "
Sep 20 02:21:46.270931 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:46.270937    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c0f8e7fe-c5c2-4a8a-b0e9-5db728be091e-tmp-dir\") pod \"c0f8e7fe-c5c2-4a8a-b0e9-5db728be091e\" (UID: \"c0f8e7fe-c5c2-4a8a-b0e9-5db728be091e\") "
Sep 20 02:21:46.271204 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:46.270981    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c0f8e7fe-c5c2-4a8a-b0e9-5db728be091e-model-cache\") pod \"c0f8e7fe-c5c2-4a8a-b0e9-5db728be091e\" (UID: \"c0f8e7fe-c5c2-4a8a-b0e9-5db728be091e\") "
Sep 20 02:21:46.271204 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:46.271020    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c0f8e7fe-c5c2-4a8a-b0e9-5db728be091e-dshm\") pod \"c0f8e7fe-c5c2-4a8a-b0e9-5db728be091e\" (UID: \"c0f8e7fe-c5c2-4a8a-b0e9-5db728be091e\") "
Sep 20 02:21:46.271204 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:46.271053    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c0f8e7fe-c5c2-4a8a-b0e9-5db728be091e-home\") pod \"c0f8e7fe-c5c2-4a8a-b0e9-5db728be091e\" (UID: \"c0f8e7fe-c5c2-4a8a-b0e9-5db728be091e\") "
Sep 20 02:21:46.271204 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:46.271098    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c0f8e7fe-c5c2-4a8a-b0e9-5db728be091e-tls-certs\") pod \"c0f8e7fe-c5c2-4a8a-b0e9-5db728be091e\" (UID: \"c0f8e7fe-c5c2-4a8a-b0e9-5db728be091e\") "
Sep 20 02:21:46.271410 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:46.271286    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0f8e7fe-c5c2-4a8a-b0e9-5db728be091e-model-cache" (OuterVolumeSpecName: "model-cache") pod "c0f8e7fe-c5c2-4a8a-b0e9-5db728be091e" (UID: "c0f8e7fe-c5c2-4a8a-b0e9-5db728be091e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 20 02:21:46.271410 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:46.271331    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0f8e7fe-c5c2-4a8a-b0e9-5db728be091e-home" (OuterVolumeSpecName: "home") pod "c0f8e7fe-c5c2-4a8a-b0e9-5db728be091e" (UID: "c0f8e7fe-c5c2-4a8a-b0e9-5db728be091e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 20 02:21:46.271410 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:46.271303    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0f8e7fe-c5c2-4a8a-b0e9-5db728be091e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "c0f8e7fe-c5c2-4a8a-b0e9-5db728be091e" (UID: "c0f8e7fe-c5c2-4a8a-b0e9-5db728be091e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 20 02:21:46.271531 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:46.271420    2584 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ddf2a155-fe4b-4676-94b0-e2e88f5bfb07-dshm\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:21:46.271531 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:46.271438    2584 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c0f8e7fe-c5c2-4a8a-b0e9-5db728be091e-home\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:21:46.271531 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:46.271450    2584 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ddf2a155-fe4b-4676-94b0-e2e88f5bfb07-home\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:21:46.271531 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:46.271462    2584 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ddf2a155-fe4b-4676-94b0-e2e88f5bfb07-model-cache\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:21:46.271531 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:46.271475    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kkxg9\" (UniqueName: \"kubernetes.io/projected/ddf2a155-fe4b-4676-94b0-e2e88f5bfb07-kube-api-access-kkxg9\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:21:46.271531 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:46.271489    2584 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ddf2a155-fe4b-4676-94b0-e2e88f5bfb07-tls-certs\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:21:46.271531 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:46.271502    2584 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c0f8e7fe-c5c2-4a8a-b0e9-5db728be091e-tmp-dir\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:21:46.271531 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:46.271514    2584 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c0f8e7fe-c5c2-4a8a-b0e9-5db728be091e-model-cache\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:21:46.273316 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:46.273284    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c0f8e7fe-c5c2-4a8a-b0e9-5db728be091e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c0f8e7fe-c5c2-4a8a-b0e9-5db728be091e" (UID: "c0f8e7fe-c5c2-4a8a-b0e9-5db728be091e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 20 02:21:46.273645 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:46.273627    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0f8e7fe-c5c2-4a8a-b0e9-5db728be091e-dshm" (OuterVolumeSpecName: "dshm") pod "c0f8e7fe-c5c2-4a8a-b0e9-5db728be091e" (UID: "c0f8e7fe-c5c2-4a8a-b0e9-5db728be091e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 20 02:21:46.273724 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:46.273666    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c0f8e7fe-c5c2-4a8a-b0e9-5db728be091e-kube-api-access-szjt7" (OuterVolumeSpecName: "kube-api-access-szjt7") pod "c0f8e7fe-c5c2-4a8a-b0e9-5db728be091e" (UID: "c0f8e7fe-c5c2-4a8a-b0e9-5db728be091e"). InnerVolumeSpecName "kube-api-access-szjt7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 20 02:21:46.372393 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:46.372297    2584 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c0f8e7fe-c5c2-4a8a-b0e9-5db728be091e-tls-certs\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:21:46.372393 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:46.372334    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-szjt7\" (UniqueName: \"kubernetes.io/projected/c0f8e7fe-c5c2-4a8a-b0e9-5db728be091e-kube-api-access-szjt7\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:21:46.372393 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:46.372344    2584 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c0f8e7fe-c5c2-4a8a-b0e9-5db728be091e-dshm\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:21:46.625480 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:46.625390    2584 generic.go:356] "Generic (PLEG): container finished" podID="ddf2a155-fe4b-4676-94b0-e2e88f5bfb07" containerID="129d7e5a4d33b6e01503832cceb83656f4e72c805b0e4222e727561014200d4a" exitCode=0
Sep 20 02:21:46.625480 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:46.625431    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-b15051/distinct-2-w4-b15051-kserve-6d8c96c4b-c69xj"
Sep 20 02:21:46.625724 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:46.625480    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-b15051/distinct-2-w4-b15051-kserve-6d8c96c4b-c69xj" event={"ID":"ddf2a155-fe4b-4676-94b0-e2e88f5bfb07","Type":"ContainerDied","Data":"129d7e5a4d33b6e01503832cceb83656f4e72c805b0e4222e727561014200d4a"}
Sep 20 02:21:46.625724 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:46.625533    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-b15051/distinct-2-w4-b15051-kserve-6d8c96c4b-c69xj" event={"ID":"ddf2a155-fe4b-4676-94b0-e2e88f5bfb07","Type":"ContainerDied","Data":"aad80fb89b76f28494b049816bd06875a7533aac5eb99fe7c41fa2a06d203c7a"}
Sep 20 02:21:46.625724 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:46.625558    2584 scope.go:117] "RemoveContainer" containerID="129d7e5a4d33b6e01503832cceb83656f4e72c805b0e4222e727561014200d4a"
Sep 20 02:21:46.627138 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:46.627111    2584 generic.go:356] "Generic (PLEG): container finished" podID="c0f8e7fe-c5c2-4a8a-b0e9-5db728be091e" containerID="f0e1bfeb802bcd7e1e6dc21b3458986767de1ad3e0f5424a3806e7c9e7daea5b" exitCode=0
Sep 20 02:21:46.627352 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:46.627147    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-b15051/distinct-w4-b15051-kserve-98f67785-dskgq"
Sep 20 02:21:46.627352 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:46.627140    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-b15051/distinct-w4-b15051-kserve-98f67785-dskgq" event={"ID":"c0f8e7fe-c5c2-4a8a-b0e9-5db728be091e","Type":"ContainerDied","Data":"f0e1bfeb802bcd7e1e6dc21b3458986767de1ad3e0f5424a3806e7c9e7daea5b"}
Sep 20 02:21:46.627352 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:46.627247    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-b15051/distinct-w4-b15051-kserve-98f67785-dskgq" event={"ID":"c0f8e7fe-c5c2-4a8a-b0e9-5db728be091e","Type":"ContainerDied","Data":"44320a545d7a1b8cc8f079f41d0c1d60ec456e7cdb32d48d34fa67152ded881d"}
Sep 20 02:21:46.635297 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:46.635188    2584 scope.go:117] "RemoveContainer" containerID="129d7e5a4d33b6e01503832cceb83656f4e72c805b0e4222e727561014200d4a"
Sep 20 02:21:46.635689 ip-10-0-130-159 kubenswrapper[2584]: E0920 02:21:46.635662    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"129d7e5a4d33b6e01503832cceb83656f4e72c805b0e4222e727561014200d4a\": container with ID starting with 129d7e5a4d33b6e01503832cceb83656f4e72c805b0e4222e727561014200d4a not found: ID does not exist" containerID="129d7e5a4d33b6e01503832cceb83656f4e72c805b0e4222e727561014200d4a"
Sep 20 02:21:46.635814 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:46.635699    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"129d7e5a4d33b6e01503832cceb83656f4e72c805b0e4222e727561014200d4a"} err="failed to get container status \"129d7e5a4d33b6e01503832cceb83656f4e72c805b0e4222e727561014200d4a\": rpc error: code = NotFound desc = could not find container \"129d7e5a4d33b6e01503832cceb83656f4e72c805b0e4222e727561014200d4a\": container with ID starting with 129d7e5a4d33b6e01503832cceb83656f4e72c805b0e4222e727561014200d4a not found: ID does not exist"
Sep 20 02:21:46.635814 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:46.635726    2584 scope.go:117] "RemoveContainer" containerID="f0e1bfeb802bcd7e1e6dc21b3458986767de1ad3e0f5424a3806e7c9e7daea5b"
Sep 20 02:21:46.644789 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:46.644766    2584 scope.go:117] "RemoveContainer" containerID="f0e1bfeb802bcd7e1e6dc21b3458986767de1ad3e0f5424a3806e7c9e7daea5b"
Sep 20 02:21:46.645127 ip-10-0-130-159 kubenswrapper[2584]: E0920 02:21:46.645105    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f0e1bfeb802bcd7e1e6dc21b3458986767de1ad3e0f5424a3806e7c9e7daea5b\": container with ID starting with f0e1bfeb802bcd7e1e6dc21b3458986767de1ad3e0f5424a3806e7c9e7daea5b not found: ID does not exist" containerID="f0e1bfeb802bcd7e1e6dc21b3458986767de1ad3e0f5424a3806e7c9e7daea5b"
Sep 20 02:21:46.645179 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:46.645137    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f0e1bfeb802bcd7e1e6dc21b3458986767de1ad3e0f5424a3806e7c9e7daea5b"} err="failed to get container status \"f0e1bfeb802bcd7e1e6dc21b3458986767de1ad3e0f5424a3806e7c9e7daea5b\": rpc error: code = NotFound desc = could not find container \"f0e1bfeb802bcd7e1e6dc21b3458986767de1ad3e0f5424a3806e7c9e7daea5b\": container with ID starting with f0e1bfeb802bcd7e1e6dc21b3458986767de1ad3e0f5424a3806e7c9e7daea5b not found: ID does not exist"
Sep 20 02:21:46.650174 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:46.650146    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-b15051/distinct-2-w4-b15051-kserve-6d8c96c4b-c69xj"]
Sep 20 02:21:46.654090 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:46.654066    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-b15051/distinct-2-w4-b15051-kserve-6d8c96c4b-c69xj"]
Sep 20 02:21:46.664600 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:46.664563    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-b15051/distinct-w4-b15051-kserve-98f67785-dskgq"]
Sep 20 02:21:46.667163 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:46.667134    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-b15051/distinct-w4-b15051-kserve-98f67785-dskgq"]
Sep 20 02:21:47.100489 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:47.100459    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c0f8e7fe-c5c2-4a8a-b0e9-5db728be091e" path="/var/lib/kubelet/pods/c0f8e7fe-c5c2-4a8a-b0e9-5db728be091e/volumes"
Sep 20 02:21:47.100994 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:47.100977    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ddf2a155-fe4b-4676-94b0-e2e88f5bfb07" path="/var/lib/kubelet/pods/ddf2a155-fe4b-4676-94b0-e2e88f5bfb07/volumes"
Sep 20 02:21:47.994111 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:47.994073    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:21:48.062874 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:48.062840    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:21:48.274321 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:48.274218    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:21:56.331793 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:56.331752    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7534-481430"]
Sep 20 02:21:56.332242 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:56.332163    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="61a65af9-c40c-4f89-9d50-fbe0599e4f07" containerName="main"
Sep 20 02:21:56.332242 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:56.332176    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="61a65af9-c40c-4f89-9d50-fbe0599e4f07" containerName="main"
Sep 20 02:21:56.332242 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:56.332186    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ddf2a155-fe4b-4676-94b0-e2e88f5bfb07" containerName="main"
Sep 20 02:21:56.332242 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:56.332192    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="ddf2a155-fe4b-4676-94b0-e2e88f5bfb07" containerName="main"
Sep 20 02:21:56.332242 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:56.332204    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab219a0c-a537-49f7-a4e8-6dd1ec16f650" containerName="istio-proxy"
Sep 20 02:21:56.332242 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:56.332210    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab219a0c-a537-49f7-a4e8-6dd1ec16f650" containerName="istio-proxy"
Sep 20 02:21:56.332242 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:56.332216    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab219a0c-a537-49f7-a4e8-6dd1ec16f650" containerName="istio-proxy"
Sep 20 02:21:56.332242 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:56.332222    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab219a0c-a537-49f7-a4e8-6dd1ec16f650" containerName="istio-proxy"
Sep 20 02:21:56.332242 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:56.332230    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="634c1084-aa3b-4904-b893-1281899f03c3" containerName="maas-api"
Sep 20 02:21:56.332242 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:56.332236    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="634c1084-aa3b-4904-b893-1281899f03c3" containerName="maas-api"
Sep 20 02:21:56.332625 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:56.332249    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e906d174-1565-4ce9-819b-df16a4977b15" containerName="maas-api"
Sep 20 02:21:56.332625 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:56.332255    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="e906d174-1565-4ce9-819b-df16a4977b15" containerName="maas-api"
Sep 20 02:21:56.332625 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:56.332261    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8a5cc051-2a9b-4b26-8edf-51bb5ed106bc" containerName="maas-api"
Sep 20 02:21:56.332625 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:56.332266    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a5cc051-2a9b-4b26-8edf-51bb5ed106bc" containerName="maas-api"
Sep 20 02:21:56.332625 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:56.332274    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2dd7d85a-5316-4cc9-ba92-1bea129040c1" containerName="payload-processing"
Sep 20 02:21:56.332625 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:56.332279    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="2dd7d85a-5316-4cc9-ba92-1bea129040c1" containerName="payload-processing"
Sep 20 02:21:56.332625 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:56.332285    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b04dfd7a-50b4-47b1-b248-6f0949b48c52" containerName="main"
Sep 20 02:21:56.332625 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:56.332290    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="b04dfd7a-50b4-47b1-b248-6f0949b48c52" containerName="main"
Sep 20 02:21:56.332625 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:56.332296    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0b4d954b-058d-4680-a83e-a072c6a0e230" containerName="maas-api"
Sep 20 02:21:56.332625 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:56.332301    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b4d954b-058d-4680-a83e-a072c6a0e230" containerName="maas-api"
Sep 20 02:21:56.332625 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:56.332308    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="883d6aff-0186-4fdc-84da-6c9a122abe0c" containerName="payload-pre-processing"
Sep 20 02:21:56.332625 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:56.332313    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="883d6aff-0186-4fdc-84da-6c9a122abe0c" containerName="payload-pre-processing"
Sep 20 02:21:56.332625 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:56.332319    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0f8e7fe-c5c2-4a8a-b0e9-5db728be091e" containerName="main"
Sep 20 02:21:56.332625 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:56.332323    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0f8e7fe-c5c2-4a8a-b0e9-5db728be091e" containerName="main"
Sep 20 02:21:56.332625 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:56.332334    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f64dd1e-7983-4545-b7ab-f8d684791433" containerName="maas-api"
Sep 20 02:21:56.332625 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:56.332339    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f64dd1e-7983-4545-b7ab-f8d684791433" containerName="maas-api"
Sep 20 02:21:56.332625 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:56.332406    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="61a65af9-c40c-4f89-9d50-fbe0599e4f07" containerName="main"
Sep 20 02:21:56.332625 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:56.332415    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="2dd7d85a-5316-4cc9-ba92-1bea129040c1" containerName="payload-processing"
Sep 20 02:21:56.332625 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:56.332421    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="8a5cc051-2a9b-4b26-8edf-51bb5ed106bc" containerName="maas-api"
Sep 20 02:21:56.332625 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:56.332428    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="c0f8e7fe-c5c2-4a8a-b0e9-5db728be091e" containerName="main"
Sep 20 02:21:56.332625 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:56.332434    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="e906d174-1565-4ce9-819b-df16a4977b15" containerName="maas-api"
Sep 20 02:21:56.332625 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:56.332440    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="883d6aff-0186-4fdc-84da-6c9a122abe0c" containerName="payload-pre-processing"
Sep 20 02:21:56.332625 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:56.332445    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="634c1084-aa3b-4904-b893-1281899f03c3" containerName="maas-api"
Sep 20 02:21:56.332625 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:56.332451    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="ab219a0c-a537-49f7-a4e8-6dd1ec16f650" containerName="istio-proxy"
Sep 20 02:21:56.332625 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:56.332456    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="0b4d954b-058d-4680-a83e-a072c6a0e230" containerName="maas-api"
Sep 20 02:21:56.332625 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:56.332462    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="9f64dd1e-7983-4545-b7ab-f8d684791433" containerName="maas-api"
Sep 20 02:21:56.332625 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:56.332467    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="b04dfd7a-50b4-47b1-b248-6f0949b48c52" containerName="main"
Sep 20 02:21:56.332625 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:56.332473    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="ddf2a155-fe4b-4676-94b0-e2e88f5bfb07" containerName="main"
Sep 20 02:21:56.335778 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:56.335756    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7534-481430"
Sep 20 02:21:56.338083 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:56.338058    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-7l75z\""
Sep 20 02:21:56.342394 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:56.342358    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7534-481430"]
Sep 20 02:21:56.462128 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:56.462091    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nvz7v\" (UniqueName: \"kubernetes.io/projected/c83c2385-6f57-42ec-9e9b-d7db940b8a3b-kube-api-access-nvz7v\") pod \"test-curl-7534-481430\" (UID: \"c83c2385-6f57-42ec-9e9b-d7db940b8a3b\") " pod="opendatahub/test-curl-7534-481430"
Sep 20 02:21:56.563300 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:56.563253    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nvz7v\" (UniqueName: \"kubernetes.io/projected/c83c2385-6f57-42ec-9e9b-d7db940b8a3b-kube-api-access-nvz7v\") pod \"test-curl-7534-481430\" (UID: \"c83c2385-6f57-42ec-9e9b-d7db940b8a3b\") " pod="opendatahub/test-curl-7534-481430"
Sep 20 02:21:56.572172 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:56.572145    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nvz7v\" (UniqueName: \"kubernetes.io/projected/c83c2385-6f57-42ec-9e9b-d7db940b8a3b-kube-api-access-nvz7v\") pod \"test-curl-7534-481430\" (UID: \"c83c2385-6f57-42ec-9e9b-d7db940b8a3b\") " pod="opendatahub/test-curl-7534-481430"
Sep 20 02:21:56.647822 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:56.647738    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7534-481430"
Sep 20 02:21:56.924939 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:56.924911    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7534-481430"]
Sep 20 02:21:56.926926 ip-10-0-130-159 kubenswrapper[2584]: W0920 02:21:56.926898    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc83c2385_6f57_42ec_9e9b_d7db940b8a3b.slice/crio-9ab40c4de33df479381c4d32070dd959f234995d9e28dd1b701ade71e7fd3887 WatchSource:0}: Error finding container 9ab40c4de33df479381c4d32070dd959f234995d9e28dd1b701ade71e7fd3887: Status 404 returned error can't find the container with id 9ab40c4de33df479381c4d32070dd959f234995d9e28dd1b701ade71e7fd3887
Sep 20 02:21:57.690575 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:57.690513    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7534-481430" event={"ID":"c83c2385-6f57-42ec-9e9b-d7db940b8a3b","Type":"ContainerStarted","Data":"94978c7d7ba4f3306d87402d5284dd6a6cb8dcd6e9a1e34f42746c97ee951c24"}
Sep 20 02:21:57.690575 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:57.690583    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7534-481430" event={"ID":"c83c2385-6f57-42ec-9e9b-d7db940b8a3b","Type":"ContainerStarted","Data":"9ab40c4de33df479381c4d32070dd959f234995d9e28dd1b701ade71e7fd3887"}
Sep 20 02:21:57.705482 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:57.705426    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7534-481430" podStartSLOduration=1.70540615 podStartE2EDuration="1.70540615s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-20 02:21:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-20 02:21:57.705024685 +0000 UTC m=+1774.447703893" watchObservedRunningTime="2026-09-20 02:21:57.70540615 +0000 UTC m=+1774.448085355"
Sep 20 02:21:58.327249 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:58.327215    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7534-481430"]
Sep 20 02:21:58.331418 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:58.331392    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7534-481430"]
Sep 20 02:21:58.696025 ip-10-0-130-159 kubenswrapper[2584]: E0920 02:21:58.695995    2584 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7534-481430" container="test-curl-7534-481430"
Sep 20 02:21:59.138989 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:59.138898    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:21:59.700730 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:59.700686    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7534-481430" podUID="c83c2385-6f57-42ec-9e9b-d7db940b8a3b" containerName="test-curl-7534-481430" containerID="cri-o://94978c7d7ba4f3306d87402d5284dd6a6cb8dcd6e9a1e34f42746c97ee951c24" gracePeriod=2
Sep 20 02:21:59.803284 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:59.803243    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-ed9f38/auth-test-model-ed9f38-kserve-5f7845bb7-gc7tn"]
Sep 20 02:21:59.825565 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:21:59.825512    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:22:01.122419 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:22:01.122382    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:22:01.899223 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:22:01.899191    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7534-481430"
Sep 20 02:22:02.012327 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:22:02.012232    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nvz7v\" (UniqueName: \"kubernetes.io/projected/c83c2385-6f57-42ec-9e9b-d7db940b8a3b-kube-api-access-nvz7v\") pod \"c83c2385-6f57-42ec-9e9b-d7db940b8a3b\" (UID: \"c83c2385-6f57-42ec-9e9b-d7db940b8a3b\") "
Sep 20 02:22:02.014401 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:22:02.014375    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c83c2385-6f57-42ec-9e9b-d7db940b8a3b-kube-api-access-nvz7v" (OuterVolumeSpecName: "kube-api-access-nvz7v") pod "c83c2385-6f57-42ec-9e9b-d7db940b8a3b" (UID: "c83c2385-6f57-42ec-9e9b-d7db940b8a3b"). InnerVolumeSpecName "kube-api-access-nvz7v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 20 02:22:02.113895 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:22:02.113859    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nvz7v\" (UniqueName: \"kubernetes.io/projected/c83c2385-6f57-42ec-9e9b-d7db940b8a3b-kube-api-access-nvz7v\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:22:02.719308 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:22:02.719270    2584 generic.go:356] "Generic (PLEG): container finished" podID="c83c2385-6f57-42ec-9e9b-d7db940b8a3b" containerID="94978c7d7ba4f3306d87402d5284dd6a6cb8dcd6e9a1e34f42746c97ee951c24" exitCode=137
Sep 20 02:22:02.719765 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:22:02.719315    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7534-481430"
Sep 20 02:22:02.719765 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:22:02.719370    2584 scope.go:117] "RemoveContainer" containerID="94978c7d7ba4f3306d87402d5284dd6a6cb8dcd6e9a1e34f42746c97ee951c24"
Sep 20 02:22:02.729726 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:22:02.729706    2584 scope.go:117] "RemoveContainer" containerID="94978c7d7ba4f3306d87402d5284dd6a6cb8dcd6e9a1e34f42746c97ee951c24"
Sep 20 02:22:02.730101 ip-10-0-130-159 kubenswrapper[2584]: E0920 02:22:02.730082    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"94978c7d7ba4f3306d87402d5284dd6a6cb8dcd6e9a1e34f42746c97ee951c24\": container with ID starting with 94978c7d7ba4f3306d87402d5284dd6a6cb8dcd6e9a1e34f42746c97ee951c24 not found: ID does not exist" containerID="94978c7d7ba4f3306d87402d5284dd6a6cb8dcd6e9a1e34f42746c97ee951c24"
Sep 20 02:22:02.730175 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:22:02.730110    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"94978c7d7ba4f3306d87402d5284dd6a6cb8dcd6e9a1e34f42746c97ee951c24"} err="failed to get container status \"94978c7d7ba4f3306d87402d5284dd6a6cb8dcd6e9a1e34f42746c97ee951c24\": rpc error: code = NotFound desc = could not find container \"94978c7d7ba4f3306d87402d5284dd6a6cb8dcd6e9a1e34f42746c97ee951c24\": container with ID starting with 94978c7d7ba4f3306d87402d5284dd6a6cb8dcd6e9a1e34f42746c97ee951c24 not found: ID does not exist"
Sep 20 02:22:02.731443 ip-10-0-130-159 kubenswrapper[2584]: E0920 02:22:02.731419    2584 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7534-481430\" is forbidden: User \"system:node:ip-10-0-130-159.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-159.ec2.internal' and this object" podUID="c83c2385-6f57-42ec-9e9b-d7db940b8a3b" pod="opendatahub/test-curl-7534-481430"
Sep 20 02:22:03.096089 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:22:03.096008    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c83c2385-6f57-42ec-9e9b-d7db940b8a3b" path="/var/lib/kubelet/pods/c83c2385-6f57-42ec-9e9b-d7db940b8a3b/volumes"
Sep 20 02:22:08.149179 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:22:08.149147    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:22:08.246858 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:22:08.246779    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:22:08.989718 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:22:08.989685    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:22:14.822725 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:22:14.822677    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-ed9f38/auth-test-model-ed9f38-kserve-5f7845bb7-gc7tn" podUID="d7e8345d-0ed1-4ede-bcaf-de80522b6bef" containerName="main" containerID="cri-o://d7b9c498f712bf77890a04c1d18cdec173f4770a1b5fa06c19231cd1d439b2bb" gracePeriod=45
Sep 20 02:22:15.083774 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:22:15.083699    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-ed9f38/auth-test-model-ed9f38-kserve-5f7845bb7-gc7tn"
Sep 20 02:22:15.237654 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:22:15.237616    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d7e8345d-0ed1-4ede-bcaf-de80522b6bef-tmp-dir\") pod \"d7e8345d-0ed1-4ede-bcaf-de80522b6bef\" (UID: \"d7e8345d-0ed1-4ede-bcaf-de80522b6bef\") "
Sep 20 02:22:15.237654 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:22:15.237661    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d7e8345d-0ed1-4ede-bcaf-de80522b6bef-tls-certs\") pod \"d7e8345d-0ed1-4ede-bcaf-de80522b6bef\" (UID: \"d7e8345d-0ed1-4ede-bcaf-de80522b6bef\") "
Sep 20 02:22:15.237922 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:22:15.237694    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d7e8345d-0ed1-4ede-bcaf-de80522b6bef-model-cache\") pod \"d7e8345d-0ed1-4ede-bcaf-de80522b6bef\" (UID: \"d7e8345d-0ed1-4ede-bcaf-de80522b6bef\") "
Sep 20 02:22:15.237922 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:22:15.237805    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d7e8345d-0ed1-4ede-bcaf-de80522b6bef-home\") pod \"d7e8345d-0ed1-4ede-bcaf-de80522b6bef\" (UID: \"d7e8345d-0ed1-4ede-bcaf-de80522b6bef\") "
Sep 20 02:22:15.237922 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:22:15.237844    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8n8g2\" (UniqueName: \"kubernetes.io/projected/d7e8345d-0ed1-4ede-bcaf-de80522b6bef-kube-api-access-8n8g2\") pod \"d7e8345d-0ed1-4ede-bcaf-de80522b6bef\" (UID: \"d7e8345d-0ed1-4ede-bcaf-de80522b6bef\") "
Sep 20 02:22:15.237922 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:22:15.237869    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d7e8345d-0ed1-4ede-bcaf-de80522b6bef-dshm\") pod \"d7e8345d-0ed1-4ede-bcaf-de80522b6bef\" (UID: \"d7e8345d-0ed1-4ede-bcaf-de80522b6bef\") "
Sep 20 02:22:15.238130 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:22:15.237983    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7e8345d-0ed1-4ede-bcaf-de80522b6bef-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d7e8345d-0ed1-4ede-bcaf-de80522b6bef" (UID: "d7e8345d-0ed1-4ede-bcaf-de80522b6bef"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 20 02:22:15.238130 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:22:15.238065    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7e8345d-0ed1-4ede-bcaf-de80522b6bef-model-cache" (OuterVolumeSpecName: "model-cache") pod "d7e8345d-0ed1-4ede-bcaf-de80522b6bef" (UID: "d7e8345d-0ed1-4ede-bcaf-de80522b6bef"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 20 02:22:15.238130 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:22:15.238106    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7e8345d-0ed1-4ede-bcaf-de80522b6bef-home" (OuterVolumeSpecName: "home") pod "d7e8345d-0ed1-4ede-bcaf-de80522b6bef" (UID: "d7e8345d-0ed1-4ede-bcaf-de80522b6bef"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 20 02:22:15.238274 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:22:15.238197    2584 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d7e8345d-0ed1-4ede-bcaf-de80522b6bef-model-cache\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:22:15.238274 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:22:15.238217    2584 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d7e8345d-0ed1-4ede-bcaf-de80522b6bef-home\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:22:15.238274 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:22:15.238231    2584 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d7e8345d-0ed1-4ede-bcaf-de80522b6bef-tmp-dir\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:22:15.240090 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:22:15.240060    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7e8345d-0ed1-4ede-bcaf-de80522b6bef-dshm" (OuterVolumeSpecName: "dshm") pod "d7e8345d-0ed1-4ede-bcaf-de80522b6bef" (UID: "d7e8345d-0ed1-4ede-bcaf-de80522b6bef"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 20 02:22:15.240202 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:22:15.240083    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7e8345d-0ed1-4ede-bcaf-de80522b6bef-kube-api-access-8n8g2" (OuterVolumeSpecName: "kube-api-access-8n8g2") pod "d7e8345d-0ed1-4ede-bcaf-de80522b6bef" (UID: "d7e8345d-0ed1-4ede-bcaf-de80522b6bef"). InnerVolumeSpecName "kube-api-access-8n8g2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 20 02:22:15.240202 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:22:15.240118    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d7e8345d-0ed1-4ede-bcaf-de80522b6bef-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d7e8345d-0ed1-4ede-bcaf-de80522b6bef" (UID: "d7e8345d-0ed1-4ede-bcaf-de80522b6bef"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 20 02:22:15.338956 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:22:15.338862    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8n8g2\" (UniqueName: \"kubernetes.io/projected/d7e8345d-0ed1-4ede-bcaf-de80522b6bef-kube-api-access-8n8g2\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:22:15.338956 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:22:15.338891    2584 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d7e8345d-0ed1-4ede-bcaf-de80522b6bef-dshm\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:22:15.338956 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:22:15.338903    2584 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d7e8345d-0ed1-4ede-bcaf-de80522b6bef-tls-certs\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:22:15.789894 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:22:15.789858    2584 generic.go:356] "Generic (PLEG): container finished" podID="d7e8345d-0ed1-4ede-bcaf-de80522b6bef" containerID="d7b9c498f712bf77890a04c1d18cdec173f4770a1b5fa06c19231cd1d439b2bb" exitCode=0
Sep 20 02:22:15.790080 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:22:15.789904    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-ed9f38/auth-test-model-ed9f38-kserve-5f7845bb7-gc7tn"
Sep 20 02:22:15.790080 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:22:15.789902    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-ed9f38/auth-test-model-ed9f38-kserve-5f7845bb7-gc7tn" event={"ID":"d7e8345d-0ed1-4ede-bcaf-de80522b6bef","Type":"ContainerDied","Data":"d7b9c498f712bf77890a04c1d18cdec173f4770a1b5fa06c19231cd1d439b2bb"}
Sep 20 02:22:15.790080 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:22:15.790022    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-ed9f38/auth-test-model-ed9f38-kserve-5f7845bb7-gc7tn" event={"ID":"d7e8345d-0ed1-4ede-bcaf-de80522b6bef","Type":"ContainerDied","Data":"347cce1efe5c1471302b7ded0d07c6a736559a7887d07516c242afa22651f0db"}
Sep 20 02:22:15.790080 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:22:15.790049    2584 scope.go:117] "RemoveContainer" containerID="d7b9c498f712bf77890a04c1d18cdec173f4770a1b5fa06c19231cd1d439b2bb"
Sep 20 02:22:15.799381 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:22:15.799363    2584 scope.go:117] "RemoveContainer" containerID="d7b9c498f712bf77890a04c1d18cdec173f4770a1b5fa06c19231cd1d439b2bb"
Sep 20 02:22:15.799733 ip-10-0-130-159 kubenswrapper[2584]: E0920 02:22:15.799709    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d7b9c498f712bf77890a04c1d18cdec173f4770a1b5fa06c19231cd1d439b2bb\": container with ID starting with d7b9c498f712bf77890a04c1d18cdec173f4770a1b5fa06c19231cd1d439b2bb not found: ID does not exist" containerID="d7b9c498f712bf77890a04c1d18cdec173f4770a1b5fa06c19231cd1d439b2bb"
Sep 20 02:22:15.799799 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:22:15.799745    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7b9c498f712bf77890a04c1d18cdec173f4770a1b5fa06c19231cd1d439b2bb"} err="failed to get container status \"d7b9c498f712bf77890a04c1d18cdec173f4770a1b5fa06c19231cd1d439b2bb\": rpc error: code = NotFound desc = could not find container \"d7b9c498f712bf77890a04c1d18cdec173f4770a1b5fa06c19231cd1d439b2bb\": container with ID starting with d7b9c498f712bf77890a04c1d18cdec173f4770a1b5fa06c19231cd1d439b2bb not found: ID does not exist"
Sep 20 02:22:15.812867 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:22:15.812836    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-ed9f38/auth-test-model-ed9f38-kserve-5f7845bb7-gc7tn"]
Sep 20 02:22:15.816239 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:22:15.816213    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-ed9f38/auth-test-model-ed9f38-kserve-5f7845bb7-gc7tn"]
Sep 20 02:22:17.093568 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:22:17.093533    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d7e8345d-0ed1-4ede-bcaf-de80522b6bef" path="/var/lib/kubelet/pods/d7e8345d-0ed1-4ede-bcaf-de80522b6bef/volumes"
Sep 20 02:22:21.629495 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:22:21.629459    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-ed9f38/sub-test-model-ed9f38-kserve-75bb5795d5-82g9b"]
Sep 20 02:22:21.630032 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:22:21.630012    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c83c2385-6f57-42ec-9e9b-d7db940b8a3b" containerName="test-curl-7534-481430"
Sep 20 02:22:21.630121 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:22:21.630035    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="c83c2385-6f57-42ec-9e9b-d7db940b8a3b" containerName="test-curl-7534-481430"
Sep 20 02:22:21.630121 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:22:21.630056    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7e8345d-0ed1-4ede-bcaf-de80522b6bef" containerName="main"
Sep 20 02:22:21.630121 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:22:21.630065    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7e8345d-0ed1-4ede-bcaf-de80522b6bef" containerName="main"
Sep 20 02:22:21.630274 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:22:21.630190    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="c83c2385-6f57-42ec-9e9b-d7db940b8a3b" containerName="test-curl-7534-481430"
Sep 20 02:22:21.630274 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:22:21.630206    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="d7e8345d-0ed1-4ede-bcaf-de80522b6bef" containerName="main"
Sep 20 02:22:21.630274 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:22:21.630215    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="ab219a0c-a537-49f7-a4e8-6dd1ec16f650" containerName="istio-proxy"
Sep 20 02:22:21.633285 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:22:21.633263    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-ed9f38/sub-test-model-ed9f38-kserve-75bb5795d5-82g9b"
Sep 20 02:22:21.636921 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:22:21.636901    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-ed9f38\"/\"sub-test-model-ed9f38-kserve-self-signed-certs\""
Sep 20 02:22:21.636921 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:22:21.636908    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-ed9f38\"/\"default-dockercfg-692d7\""
Sep 20 02:22:21.637099 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:22:21.636906    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-ed9f38\"/\"openshift-service-ca.crt\""
Sep 20 02:22:21.637099 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:22:21.637065    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-ed9f38\"/\"kube-root-ca.crt\""
Sep 20 02:22:21.646499 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:22:21.646477    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-ed9f38/sub-test-model-ed9f38-kserve-75bb5795d5-82g9b"]
Sep 20 02:22:21.669582 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:22:21.669546    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:22:21.797228 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:22:21.797180    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5rsrr\" (UniqueName: \"kubernetes.io/projected/d739b158-4789-4467-9fce-ba506ca96b55-kube-api-access-5rsrr\") pod \"sub-test-model-ed9f38-kserve-75bb5795d5-82g9b\" (UID: \"d739b158-4789-4467-9fce-ba506ca96b55\") " pod="ai-tenant-e2e-shared-a-w3-ed9f38/sub-test-model-ed9f38-kserve-75bb5795d5-82g9b"
Sep 20 02:22:21.797228 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:22:21.797233    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d739b158-4789-4467-9fce-ba506ca96b55-tmp-dir\") pod \"sub-test-model-ed9f38-kserve-75bb5795d5-82g9b\" (UID: \"d739b158-4789-4467-9fce-ba506ca96b55\") " pod="ai-tenant-e2e-shared-a-w3-ed9f38/sub-test-model-ed9f38-kserve-75bb5795d5-82g9b"
Sep 20 02:22:21.797469 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:22:21.797305    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d739b158-4789-4467-9fce-ba506ca96b55-home\") pod \"sub-test-model-ed9f38-kserve-75bb5795d5-82g9b\" (UID: \"d739b158-4789-4467-9fce-ba506ca96b55\") " pod="ai-tenant-e2e-shared-a-w3-ed9f38/sub-test-model-ed9f38-kserve-75bb5795d5-82g9b"
Sep 20 02:22:21.797469 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:22:21.797339    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d739b158-4789-4467-9fce-ba506ca96b55-model-cache\") pod \"sub-test-model-ed9f38-kserve-75bb5795d5-82g9b\" (UID: \"d739b158-4789-4467-9fce-ba506ca96b55\") " pod="ai-tenant-e2e-shared-a-w3-ed9f38/sub-test-model-ed9f38-kserve-75bb5795d5-82g9b"
Sep 20 02:22:21.797469 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:22:21.797368    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d739b158-4789-4467-9fce-ba506ca96b55-tls-certs\") pod \"sub-test-model-ed9f38-kserve-75bb5795d5-82g9b\" (UID: \"d739b158-4789-4467-9fce-ba506ca96b55\") " pod="ai-tenant-e2e-shared-a-w3-ed9f38/sub-test-model-ed9f38-kserve-75bb5795d5-82g9b"
Sep 20 02:22:21.797469 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:22:21.797446    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d739b158-4789-4467-9fce-ba506ca96b55-dshm\") pod \"sub-test-model-ed9f38-kserve-75bb5795d5-82g9b\" (UID: \"d739b158-4789-4467-9fce-ba506ca96b55\") " pod="ai-tenant-e2e-shared-a-w3-ed9f38/sub-test-model-ed9f38-kserve-75bb5795d5-82g9b"
Sep 20 02:22:21.898753 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:22:21.898649    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d739b158-4789-4467-9fce-ba506ca96b55-dshm\") pod \"sub-test-model-ed9f38-kserve-75bb5795d5-82g9b\" (UID: \"d739b158-4789-4467-9fce-ba506ca96b55\") " pod="ai-tenant-e2e-shared-a-w3-ed9f38/sub-test-model-ed9f38-kserve-75bb5795d5-82g9b"
Sep 20 02:22:21.898943 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:22:21.898755    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5rsrr\" (UniqueName: \"kubernetes.io/projected/d739b158-4789-4467-9fce-ba506ca96b55-kube-api-access-5rsrr\") pod \"sub-test-model-ed9f38-kserve-75bb5795d5-82g9b\" (UID: \"d739b158-4789-4467-9fce-ba506ca96b55\") " pod="ai-tenant-e2e-shared-a-w3-ed9f38/sub-test-model-ed9f38-kserve-75bb5795d5-82g9b"
Sep 20 02:22:21.898943 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:22:21.898789    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d739b158-4789-4467-9fce-ba506ca96b55-tmp-dir\") pod \"sub-test-model-ed9f38-kserve-75bb5795d5-82g9b\" (UID: \"d739b158-4789-4467-9fce-ba506ca96b55\") " pod="ai-tenant-e2e-shared-a-w3-ed9f38/sub-test-model-ed9f38-kserve-75bb5795d5-82g9b"
Sep 20 02:22:21.898943 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:22:21.898824    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d739b158-4789-4467-9fce-ba506ca96b55-home\") pod \"sub-test-model-ed9f38-kserve-75bb5795d5-82g9b\" (UID: \"d739b158-4789-4467-9fce-ba506ca96b55\") " pod="ai-tenant-e2e-shared-a-w3-ed9f38/sub-test-model-ed9f38-kserve-75bb5795d5-82g9b"
Sep 20 02:22:21.898943 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:22:21.898876    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d739b158-4789-4467-9fce-ba506ca96b55-model-cache\") pod \"sub-test-model-ed9f38-kserve-75bb5795d5-82g9b\" (UID: \"d739b158-4789-4467-9fce-ba506ca96b55\") " pod="ai-tenant-e2e-shared-a-w3-ed9f38/sub-test-model-ed9f38-kserve-75bb5795d5-82g9b"
Sep 20 02:22:21.898943 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:22:21.898924    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d739b158-4789-4467-9fce-ba506ca96b55-tls-certs\") pod \"sub-test-model-ed9f38-kserve-75bb5795d5-82g9b\" (UID: \"d739b158-4789-4467-9fce-ba506ca96b55\") " pod="ai-tenant-e2e-shared-a-w3-ed9f38/sub-test-model-ed9f38-kserve-75bb5795d5-82g9b"
Sep 20 02:22:21.899234 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:22:21.899221    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d739b158-4789-4467-9fce-ba506ca96b55-home\") pod \"sub-test-model-ed9f38-kserve-75bb5795d5-82g9b\" (UID: \"d739b158-4789-4467-9fce-ba506ca96b55\") " pod="ai-tenant-e2e-shared-a-w3-ed9f38/sub-test-model-ed9f38-kserve-75bb5795d5-82g9b"
Sep 20 02:22:21.899289 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:22:21.899268    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d739b158-4789-4467-9fce-ba506ca96b55-model-cache\") pod \"sub-test-model-ed9f38-kserve-75bb5795d5-82g9b\" (UID: \"d739b158-4789-4467-9fce-ba506ca96b55\") " pod="ai-tenant-e2e-shared-a-w3-ed9f38/sub-test-model-ed9f38-kserve-75bb5795d5-82g9b"
Sep 20 02:22:21.899348 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:22:21.899324    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d739b158-4789-4467-9fce-ba506ca96b55-tmp-dir\") pod \"sub-test-model-ed9f38-kserve-75bb5795d5-82g9b\" (UID: \"d739b158-4789-4467-9fce-ba506ca96b55\") " pod="ai-tenant-e2e-shared-a-w3-ed9f38/sub-test-model-ed9f38-kserve-75bb5795d5-82g9b"
Sep 20 02:22:21.901067 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:22:21.901044    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d739b158-4789-4467-9fce-ba506ca96b55-dshm\") pod \"sub-test-model-ed9f38-kserve-75bb5795d5-82g9b\" (UID: \"d739b158-4789-4467-9fce-ba506ca96b55\") " pod="ai-tenant-e2e-shared-a-w3-ed9f38/sub-test-model-ed9f38-kserve-75bb5795d5-82g9b"
Sep 20 02:22:21.901342 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:22:21.901324    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d739b158-4789-4467-9fce-ba506ca96b55-tls-certs\") pod \"sub-test-model-ed9f38-kserve-75bb5795d5-82g9b\" (UID: \"d739b158-4789-4467-9fce-ba506ca96b55\") " pod="ai-tenant-e2e-shared-a-w3-ed9f38/sub-test-model-ed9f38-kserve-75bb5795d5-82g9b"
Sep 20 02:22:21.911373 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:22:21.911343    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5rsrr\" (UniqueName: \"kubernetes.io/projected/d739b158-4789-4467-9fce-ba506ca96b55-kube-api-access-5rsrr\") pod \"sub-test-model-ed9f38-kserve-75bb5795d5-82g9b\" (UID: \"d739b158-4789-4467-9fce-ba506ca96b55\") " pod="ai-tenant-e2e-shared-a-w3-ed9f38/sub-test-model-ed9f38-kserve-75bb5795d5-82g9b"
Sep 20 02:22:21.943395 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:22:21.943352    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-ed9f38/sub-test-model-ed9f38-kserve-75bb5795d5-82g9b"
Sep 20 02:22:22.223937 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:22:22.223905    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-ed9f38/sub-test-model-ed9f38-kserve-75bb5795d5-82g9b"]
Sep 20 02:22:22.225667 ip-10-0-130-159 kubenswrapper[2584]: W0920 02:22:22.225640    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd739b158_4789_4467_9fce_ba506ca96b55.slice/crio-5eba3ce9463bd0a494ef27878dcaa22684b4c38d2aac2dee40481c537c7f82b0 WatchSource:0}: Error finding container 5eba3ce9463bd0a494ef27878dcaa22684b4c38d2aac2dee40481c537c7f82b0: Status 404 returned error can't find the container with id 5eba3ce9463bd0a494ef27878dcaa22684b4c38d2aac2dee40481c537c7f82b0
Sep 20 02:22:22.830611 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:22:22.830569    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-ed9f38/sub-test-model-ed9f38-kserve-75bb5795d5-82g9b" event={"ID":"d739b158-4789-4467-9fce-ba506ca96b55","Type":"ContainerStarted","Data":"4ab3c732b51fbbcd836a8ae3d8b21f9853542ee86d727acc1b9548448275f08e"}
Sep 20 02:22:22.830611 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:22:22.830619    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-ed9f38/sub-test-model-ed9f38-kserve-75bb5795d5-82g9b" event={"ID":"d739b158-4789-4467-9fce-ba506ca96b55","Type":"ContainerStarted","Data":"5eba3ce9463bd0a494ef27878dcaa22684b4c38d2aac2dee40481c537c7f82b0"}
Sep 20 02:22:22.851308 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:22:22.851253    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-ed9f38/sub-test-model-ed9f38-kserve-75bb5795d5-82g9b" podStartSLOduration=1.851234971 podStartE2EDuration="1.851234971s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-20 02:22:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-20 02:22:22.84983808 +0000 UTC m=+1799.592517287" watchObservedRunningTime="2026-09-20 02:22:22.851234971 +0000 UTC m=+1799.593914565"
Sep 20 02:22:25.230466 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:22:25.230439    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-ztdt2_ece82f19-68b5-4810-be50-d8678cfdee02/console-operator/1.log"
Sep 20 02:22:25.235360 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:22:25.235336    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-ztdt2_ece82f19-68b5-4810-be50-d8678cfdee02/console-operator/1.log"
Sep 20 02:22:31.943896 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:22:31.943849    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-ed9f38/sub-test-model-ed9f38-kserve-75bb5795d5-82g9b"
Sep 20 02:22:31.944302 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:22:31.943916    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-ed9f38/sub-test-model-ed9f38-kserve-75bb5795d5-82g9b"
Sep 20 02:22:31.956585 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:22:31.956551    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-ed9f38/sub-test-model-ed9f38-kserve-75bb5795d5-82g9b"
Sep 20 02:22:32.591966 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:22:32.591916    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:22:32.896192 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:22:32.896105    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-ed9f38/sub-test-model-ed9f38-kserve-75bb5795d5-82g9b"
Sep 20 02:22:44.961220 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:22:44.961184    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-317780/sub-test-model-317780-kserve-84648cb75-5sp4q"]
Sep 20 02:22:44.966309 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:22:44.966288    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-317780/sub-test-model-317780-kserve-84648cb75-5sp4q"
Sep 20 02:22:44.970767 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:22:44.970744    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-317780\"/\"default-dockercfg-5nxwn\""
Sep 20 02:22:44.971493 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:22:44.971466    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-317780\"/\"sub-test-model-317780-kserve-self-signed-certs\""
Sep 20 02:22:44.971675 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:22:44.971505    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-317780\"/\"kube-root-ca.crt\""
Sep 20 02:22:44.973844 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:22:44.973814    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-317780\"/\"openshift-service-ca.crt\""
Sep 20 02:22:44.989051 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:22:44.989017    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-317780/sub-test-model-317780-kserve-84648cb75-5sp4q"]
Sep 20 02:22:44.998480 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:22:44.998443    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1ada1a62-1319-482a-8898-8f7251c9a59f-tmp-dir\") pod \"sub-test-model-317780-kserve-84648cb75-5sp4q\" (UID: \"1ada1a62-1319-482a-8898-8f7251c9a59f\") " pod="ai-tenant-e2e-shared-b-w3-317780/sub-test-model-317780-kserve-84648cb75-5sp4q"
Sep 20 02:22:44.998679 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:22:44.998497    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1ada1a62-1319-482a-8898-8f7251c9a59f-home\") pod \"sub-test-model-317780-kserve-84648cb75-5sp4q\" (UID: \"1ada1a62-1319-482a-8898-8f7251c9a59f\") " pod="ai-tenant-e2e-shared-b-w3-317780/sub-test-model-317780-kserve-84648cb75-5sp4q"
Sep 20 02:22:44.998679 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:22:44.998592    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1ada1a62-1319-482a-8898-8f7251c9a59f-dshm\") pod \"sub-test-model-317780-kserve-84648cb75-5sp4q\" (UID: \"1ada1a62-1319-482a-8898-8f7251c9a59f\") " pod="ai-tenant-e2e-shared-b-w3-317780/sub-test-model-317780-kserve-84648cb75-5sp4q"
Sep 20 02:22:44.998679 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:22:44.998645    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1ada1a62-1319-482a-8898-8f7251c9a59f-tls-certs\") pod \"sub-test-model-317780-kserve-84648cb75-5sp4q\" (UID: \"1ada1a62-1319-482a-8898-8f7251c9a59f\") " pod="ai-tenant-e2e-shared-b-w3-317780/sub-test-model-317780-kserve-84648cb75-5sp4q"
Sep 20 02:22:44.998860 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:22:44.998683    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1ada1a62-1319-482a-8898-8f7251c9a59f-model-cache\") pod \"sub-test-model-317780-kserve-84648cb75-5sp4q\" (UID: \"1ada1a62-1319-482a-8898-8f7251c9a59f\") " pod="ai-tenant-e2e-shared-b-w3-317780/sub-test-model-317780-kserve-84648cb75-5sp4q"
Sep 20 02:22:44.998860 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:22:44.998751    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gftv6\" (UniqueName: \"kubernetes.io/projected/1ada1a62-1319-482a-8898-8f7251c9a59f-kube-api-access-gftv6\") pod \"sub-test-model-317780-kserve-84648cb75-5sp4q\" (UID: \"1ada1a62-1319-482a-8898-8f7251c9a59f\") " pod="ai-tenant-e2e-shared-b-w3-317780/sub-test-model-317780-kserve-84648cb75-5sp4q"
Sep 20 02:22:45.044758 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:22:45.044726    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:22:45.099358 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:22:45.099313    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gftv6\" (UniqueName: \"kubernetes.io/projected/1ada1a62-1319-482a-8898-8f7251c9a59f-kube-api-access-gftv6\") pod \"sub-test-model-317780-kserve-84648cb75-5sp4q\" (UID: \"1ada1a62-1319-482a-8898-8f7251c9a59f\") " pod="ai-tenant-e2e-shared-b-w3-317780/sub-test-model-317780-kserve-84648cb75-5sp4q"
Sep 20 02:22:45.099665 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:22:45.099458    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1ada1a62-1319-482a-8898-8f7251c9a59f-tmp-dir\") pod \"sub-test-model-317780-kserve-84648cb75-5sp4q\" (UID: \"1ada1a62-1319-482a-8898-8f7251c9a59f\") " pod="ai-tenant-e2e-shared-b-w3-317780/sub-test-model-317780-kserve-84648cb75-5sp4q"
Sep 20 02:22:45.099665 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:22:45.099500    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1ada1a62-1319-482a-8898-8f7251c9a59f-home\") pod \"sub-test-model-317780-kserve-84648cb75-5sp4q\" (UID: \"1ada1a62-1319-482a-8898-8f7251c9a59f\") " pod="ai-tenant-e2e-shared-b-w3-317780/sub-test-model-317780-kserve-84648cb75-5sp4q"
Sep 20 02:22:45.099665 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:22:45.099544    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1ada1a62-1319-482a-8898-8f7251c9a59f-dshm\") pod \"sub-test-model-317780-kserve-84648cb75-5sp4q\" (UID: \"1ada1a62-1319-482a-8898-8f7251c9a59f\") " pod="ai-tenant-e2e-shared-b-w3-317780/sub-test-model-317780-kserve-84648cb75-5sp4q"
Sep 20 02:22:45.099665 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:22:45.099584    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1ada1a62-1319-482a-8898-8f7251c9a59f-tls-certs\") pod \"sub-test-model-317780-kserve-84648cb75-5sp4q\" (UID: \"1ada1a62-1319-482a-8898-8f7251c9a59f\") " pod="ai-tenant-e2e-shared-b-w3-317780/sub-test-model-317780-kserve-84648cb75-5sp4q"
Sep 20 02:22:45.099665 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:22:45.099623    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1ada1a62-1319-482a-8898-8f7251c9a59f-model-cache\") pod \"sub-test-model-317780-kserve-84648cb75-5sp4q\" (UID: \"1ada1a62-1319-482a-8898-8f7251c9a59f\") " pod="ai-tenant-e2e-shared-b-w3-317780/sub-test-model-317780-kserve-84648cb75-5sp4q"
Sep 20 02:22:45.099961 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:22:45.099946    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1ada1a62-1319-482a-8898-8f7251c9a59f-tmp-dir\") pod \"sub-test-model-317780-kserve-84648cb75-5sp4q\" (UID: \"1ada1a62-1319-482a-8898-8f7251c9a59f\") " pod="ai-tenant-e2e-shared-b-w3-317780/sub-test-model-317780-kserve-84648cb75-5sp4q"
Sep 20 02:22:45.099961 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:22:45.099958    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1ada1a62-1319-482a-8898-8f7251c9a59f-home\") pod \"sub-test-model-317780-kserve-84648cb75-5sp4q\" (UID: \"1ada1a62-1319-482a-8898-8f7251c9a59f\") " pod="ai-tenant-e2e-shared-b-w3-317780/sub-test-model-317780-kserve-84648cb75-5sp4q"
Sep 20 02:22:45.100058 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:22:45.099994    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1ada1a62-1319-482a-8898-8f7251c9a59f-model-cache\") pod \"sub-test-model-317780-kserve-84648cb75-5sp4q\" (UID: \"1ada1a62-1319-482a-8898-8f7251c9a59f\") " pod="ai-tenant-e2e-shared-b-w3-317780/sub-test-model-317780-kserve-84648cb75-5sp4q"
Sep 20 02:22:45.102169 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:22:45.102145    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1ada1a62-1319-482a-8898-8f7251c9a59f-tls-certs\") pod \"sub-test-model-317780-kserve-84648cb75-5sp4q\" (UID: \"1ada1a62-1319-482a-8898-8f7251c9a59f\") " pod="ai-tenant-e2e-shared-b-w3-317780/sub-test-model-317780-kserve-84648cb75-5sp4q"
Sep 20 02:22:45.102277 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:22:45.102197    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1ada1a62-1319-482a-8898-8f7251c9a59f-dshm\") pod \"sub-test-model-317780-kserve-84648cb75-5sp4q\" (UID: \"1ada1a62-1319-482a-8898-8f7251c9a59f\") " pod="ai-tenant-e2e-shared-b-w3-317780/sub-test-model-317780-kserve-84648cb75-5sp4q"
Sep 20 02:22:45.128141 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:22:45.128111    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gftv6\" (UniqueName: \"kubernetes.io/projected/1ada1a62-1319-482a-8898-8f7251c9a59f-kube-api-access-gftv6\") pod \"sub-test-model-317780-kserve-84648cb75-5sp4q\" (UID: \"1ada1a62-1319-482a-8898-8f7251c9a59f\") " pod="ai-tenant-e2e-shared-b-w3-317780/sub-test-model-317780-kserve-84648cb75-5sp4q"
Sep 20 02:22:45.276604 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:22:45.276481    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-317780/sub-test-model-317780-kserve-84648cb75-5sp4q"
Sep 20 02:22:45.567293 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:22:45.567198    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-317780/sub-test-model-317780-kserve-84648cb75-5sp4q"]
Sep 20 02:22:45.571255 ip-10-0-130-159 kubenswrapper[2584]: W0920 02:22:45.571220    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1ada1a62_1319_482a_8898_8f7251c9a59f.slice/crio-72abd1cc3ad4bc289a4a8a24a96fd4528f3f41d868e4ec77bf310d822ba3347b WatchSource:0}: Error finding container 72abd1cc3ad4bc289a4a8a24a96fd4528f3f41d868e4ec77bf310d822ba3347b: Status 404 returned error can't find the container with id 72abd1cc3ad4bc289a4a8a24a96fd4528f3f41d868e4ec77bf310d822ba3347b
Sep 20 02:22:45.962474 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:22:45.962437    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-317780/sub-test-model-317780-kserve-84648cb75-5sp4q" event={"ID":"1ada1a62-1319-482a-8898-8f7251c9a59f","Type":"ContainerStarted","Data":"afffa1ab191c845c99847acf46b0ca48c7d32f641904fa890eee333ff95af431"}
Sep 20 02:22:45.963035 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:22:45.962491    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-317780/sub-test-model-317780-kserve-84648cb75-5sp4q" event={"ID":"1ada1a62-1319-482a-8898-8f7251c9a59f","Type":"ContainerStarted","Data":"72abd1cc3ad4bc289a4a8a24a96fd4528f3f41d868e4ec77bf310d822ba3347b"}
Sep 20 02:22:45.983498 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:22:45.983432    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-317780/sub-test-model-317780-kserve-84648cb75-5sp4q" podStartSLOduration=1.983410618 podStartE2EDuration="1.983410618s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-20 02:22:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-20 02:22:45.982885957 +0000 UTC m=+1822.725565162" watchObservedRunningTime="2026-09-20 02:22:45.983410618 +0000 UTC m=+1822.726089824"
Sep 20 02:22:55.276967 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:22:55.276930    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-317780/sub-test-model-317780-kserve-84648cb75-5sp4q"
Sep 20 02:22:55.276967 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:22:55.276979    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-317780/sub-test-model-317780-kserve-84648cb75-5sp4q"
Sep 20 02:22:55.289903 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:22:55.289873    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-317780/sub-test-model-317780-kserve-84648cb75-5sp4q"
Sep 20 02:22:56.026386 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:22:56.026350    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-317780/sub-test-model-317780-kserve-84648cb75-5sp4q"
Sep 20 02:23:20.000194 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:23:20.000159    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7534-6998aa"]
Sep 20 02:23:20.003689 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:23:20.003672    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7534-6998aa"
Sep 20 02:23:20.011857 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:23:20.011836    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-7l75z\""
Sep 20 02:23:20.035117 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:23:20.035095    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7534-6998aa"]
Sep 20 02:23:20.043685 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:23:20.043645    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-klfpw\" (UniqueName: \"kubernetes.io/projected/9ad9f645-99d9-495a-b8b9-f93f4e7f1b2e-kube-api-access-klfpw\") pod \"test-curl-7534-6998aa\" (UID: \"9ad9f645-99d9-495a-b8b9-f93f4e7f1b2e\") " pod="opendatahub/test-curl-7534-6998aa"
Sep 20 02:23:20.145623 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:23:20.145584    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-klfpw\" (UniqueName: \"kubernetes.io/projected/9ad9f645-99d9-495a-b8b9-f93f4e7f1b2e-kube-api-access-klfpw\") pod \"test-curl-7534-6998aa\" (UID: \"9ad9f645-99d9-495a-b8b9-f93f4e7f1b2e\") " pod="opendatahub/test-curl-7534-6998aa"
Sep 20 02:23:20.157692 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:23:20.157664    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-klfpw\" (UniqueName: \"kubernetes.io/projected/9ad9f645-99d9-495a-b8b9-f93f4e7f1b2e-kube-api-access-klfpw\") pod \"test-curl-7534-6998aa\" (UID: \"9ad9f645-99d9-495a-b8b9-f93f4e7f1b2e\") " pod="opendatahub/test-curl-7534-6998aa"
Sep 20 02:23:20.313460 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:23:20.313381    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7534-6998aa"
Sep 20 02:23:20.605064 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:23:20.604984    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7534-6998aa"]
Sep 20 02:23:20.608268 ip-10-0-130-159 kubenswrapper[2584]: W0920 02:23:20.608240    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9ad9f645_99d9_495a_b8b9_f93f4e7f1b2e.slice/crio-6d6c6b0897a3f6b42f8ed57747e92f9966d7f79e3fc45ae88d600cddbd410ad0 WatchSource:0}: Error finding container 6d6c6b0897a3f6b42f8ed57747e92f9966d7f79e3fc45ae88d600cddbd410ad0: Status 404 returned error can't find the container with id 6d6c6b0897a3f6b42f8ed57747e92f9966d7f79e3fc45ae88d600cddbd410ad0
Sep 20 02:23:21.141663 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:23:21.141626    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7534-6998aa" event={"ID":"9ad9f645-99d9-495a-b8b9-f93f4e7f1b2e","Type":"ContainerStarted","Data":"37712d72f011d80ba3bac4835af457ae04ed551d15878fb6450f8e93e070dd3f"}
Sep 20 02:23:21.141663 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:23:21.141666    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7534-6998aa" event={"ID":"9ad9f645-99d9-495a-b8b9-f93f4e7f1b2e","Type":"ContainerStarted","Data":"6d6c6b0897a3f6b42f8ed57747e92f9966d7f79e3fc45ae88d600cddbd410ad0"}
Sep 20 02:23:21.163107 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:23:21.163050    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7534-6998aa" podStartSLOduration=2.163031359 podStartE2EDuration="2.163031359s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-20 02:23:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-20 02:23:21.162837588 +0000 UTC m=+1857.905516793" watchObservedRunningTime="2026-09-20 02:23:21.163031359 +0000 UTC m=+1857.905710565"
Sep 20 02:23:21.922830 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:23:21.922746    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7534-6998aa"]
Sep 20 02:23:21.937313 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:23:21.937286    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7534-6998aa"]
Sep 20 02:23:22.148321 ip-10-0-130-159 kubenswrapper[2584]: E0920 02:23:22.148280    2584 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7534-6998aa" container="test-curl-7534-6998aa"
Sep 20 02:23:23.152989 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:23:23.152946    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7534-6998aa" podUID="9ad9f645-99d9-495a-b8b9-f93f4e7f1b2e" containerName="test-curl-7534-6998aa" containerID="cri-o://37712d72f011d80ba3bac4835af457ae04ed551d15878fb6450f8e93e070dd3f" gracePeriod=2
Sep 20 02:23:25.332493 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:23:25.332470    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7534-6998aa"
Sep 20 02:23:25.395181 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:23:25.395153    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-klfpw\" (UniqueName: \"kubernetes.io/projected/9ad9f645-99d9-495a-b8b9-f93f4e7f1b2e-kube-api-access-klfpw\") pod \"9ad9f645-99d9-495a-b8b9-f93f4e7f1b2e\" (UID: \"9ad9f645-99d9-495a-b8b9-f93f4e7f1b2e\") "
Sep 20 02:23:25.397161 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:23:25.397131    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ad9f645-99d9-495a-b8b9-f93f4e7f1b2e-kube-api-access-klfpw" (OuterVolumeSpecName: "kube-api-access-klfpw") pod "9ad9f645-99d9-495a-b8b9-f93f4e7f1b2e" (UID: "9ad9f645-99d9-495a-b8b9-f93f4e7f1b2e"). InnerVolumeSpecName "kube-api-access-klfpw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 20 02:23:25.496589 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:23:25.496559    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-klfpw\" (UniqueName: \"kubernetes.io/projected/9ad9f645-99d9-495a-b8b9-f93f4e7f1b2e-kube-api-access-klfpw\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:23:26.151618 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:23:26.151580    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-ed9f38/sub-test-model-ed9f38-kserve-75bb5795d5-82g9b"]
Sep 20 02:23:26.170474 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:23:26.170442    2584 generic.go:356] "Generic (PLEG): container finished" podID="9ad9f645-99d9-495a-b8b9-f93f4e7f1b2e" containerID="37712d72f011d80ba3bac4835af457ae04ed551d15878fb6450f8e93e070dd3f" exitCode=137
Sep 20 02:23:26.170652 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:23:26.170501    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7534-6998aa"
Sep 20 02:23:26.170652 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:23:26.170513    2584 scope.go:117] "RemoveContainer" containerID="37712d72f011d80ba3bac4835af457ae04ed551d15878fb6450f8e93e070dd3f"
Sep 20 02:23:26.180660 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:23:26.180642    2584 scope.go:117] "RemoveContainer" containerID="37712d72f011d80ba3bac4835af457ae04ed551d15878fb6450f8e93e070dd3f"
Sep 20 02:23:26.180939 ip-10-0-130-159 kubenswrapper[2584]: E0920 02:23:26.180919    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"37712d72f011d80ba3bac4835af457ae04ed551d15878fb6450f8e93e070dd3f\": container with ID starting with 37712d72f011d80ba3bac4835af457ae04ed551d15878fb6450f8e93e070dd3f not found: ID does not exist" containerID="37712d72f011d80ba3bac4835af457ae04ed551d15878fb6450f8e93e070dd3f"
Sep 20 02:23:26.181043 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:23:26.180962    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"37712d72f011d80ba3bac4835af457ae04ed551d15878fb6450f8e93e070dd3f"} err="failed to get container status \"37712d72f011d80ba3bac4835af457ae04ed551d15878fb6450f8e93e070dd3f\": rpc error: code = NotFound desc = could not find container \"37712d72f011d80ba3bac4835af457ae04ed551d15878fb6450f8e93e070dd3f\": container with ID starting with 37712d72f011d80ba3bac4835af457ae04ed551d15878fb6450f8e93e070dd3f not found: ID does not exist"
Sep 20 02:23:26.185997 ip-10-0-130-159 kubenswrapper[2584]: E0920 02:23:26.185968    2584 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7534-6998aa\" is forbidden: User \"system:node:ip-10-0-130-159.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-159.ec2.internal' and this object" podUID="9ad9f645-99d9-495a-b8b9-f93f4e7f1b2e" pod="opendatahub/test-curl-7534-6998aa"
Sep 20 02:23:26.204002 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:23:26.203977    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:23:27.093275 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:23:27.093241    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9ad9f645-99d9-495a-b8b9-f93f4e7f1b2e" path="/var/lib/kubelet/pods/9ad9f645-99d9-495a-b8b9-f93f4e7f1b2e/volumes"
Sep 20 02:23:27.314234 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:23:27.314198    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-317780/sub-test-model-317780-kserve-84648cb75-5sp4q"]
Sep 20 02:23:27.373286 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:23:27.373201    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:23:38.604273 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:23:38.604188    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:23:41.174002 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:23:41.173956    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-ed9f38/sub-test-model-ed9f38-kserve-75bb5795d5-82g9b" podUID="d739b158-4789-4467-9fce-ba506ca96b55" containerName="main" containerID="cri-o://4ab3c732b51fbbcd836a8ae3d8b21f9853542ee86d727acc1b9548448275f08e" gracePeriod=45
Sep 20 02:23:41.430945 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:23:41.430886    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-ed9f38/sub-test-model-ed9f38-kserve-75bb5795d5-82g9b"
Sep 20 02:23:41.543834 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:23:41.543801    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d739b158-4789-4467-9fce-ba506ca96b55-model-cache\") pod \"d739b158-4789-4467-9fce-ba506ca96b55\" (UID: \"d739b158-4789-4467-9fce-ba506ca96b55\") "
Sep 20 02:23:41.544032 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:23:41.543877    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d739b158-4789-4467-9fce-ba506ca96b55-tmp-dir\") pod \"d739b158-4789-4467-9fce-ba506ca96b55\" (UID: \"d739b158-4789-4467-9fce-ba506ca96b55\") "
Sep 20 02:23:41.544032 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:23:41.543905    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5rsrr\" (UniqueName: \"kubernetes.io/projected/d739b158-4789-4467-9fce-ba506ca96b55-kube-api-access-5rsrr\") pod \"d739b158-4789-4467-9fce-ba506ca96b55\" (UID: \"d739b158-4789-4467-9fce-ba506ca96b55\") "
Sep 20 02:23:41.544032 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:23:41.543920    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d739b158-4789-4467-9fce-ba506ca96b55-home\") pod \"d739b158-4789-4467-9fce-ba506ca96b55\" (UID: \"d739b158-4789-4467-9fce-ba506ca96b55\") "
Sep 20 02:23:41.544032 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:23:41.543954    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d739b158-4789-4467-9fce-ba506ca96b55-dshm\") pod \"d739b158-4789-4467-9fce-ba506ca96b55\" (UID: \"d739b158-4789-4467-9fce-ba506ca96b55\") "
Sep 20 02:23:41.544032 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:23:41.544016    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d739b158-4789-4467-9fce-ba506ca96b55-tls-certs\") pod \"d739b158-4789-4467-9fce-ba506ca96b55\" (UID: \"d739b158-4789-4467-9fce-ba506ca96b55\") "
Sep 20 02:23:41.544381 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:23:41.544077    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d739b158-4789-4467-9fce-ba506ca96b55-model-cache" (OuterVolumeSpecName: "model-cache") pod "d739b158-4789-4467-9fce-ba506ca96b55" (UID: "d739b158-4789-4467-9fce-ba506ca96b55"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 20 02:23:41.544381 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:23:41.544152    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d739b158-4789-4467-9fce-ba506ca96b55-home" (OuterVolumeSpecName: "home") pod "d739b158-4789-4467-9fce-ba506ca96b55" (UID: "d739b158-4789-4467-9fce-ba506ca96b55"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 20 02:23:41.544381 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:23:41.544188    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d739b158-4789-4467-9fce-ba506ca96b55-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d739b158-4789-4467-9fce-ba506ca96b55" (UID: "d739b158-4789-4467-9fce-ba506ca96b55"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 20 02:23:41.544381 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:23:41.544319    2584 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d739b158-4789-4467-9fce-ba506ca96b55-model-cache\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:23:41.544381 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:23:41.544338    2584 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d739b158-4789-4467-9fce-ba506ca96b55-tmp-dir\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:23:41.544381 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:23:41.544352    2584 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d739b158-4789-4467-9fce-ba506ca96b55-home\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:23:41.546157 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:23:41.546134    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d739b158-4789-4467-9fce-ba506ca96b55-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d739b158-4789-4467-9fce-ba506ca96b55" (UID: "d739b158-4789-4467-9fce-ba506ca96b55"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 20 02:23:41.546240 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:23:41.546161    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d739b158-4789-4467-9fce-ba506ca96b55-dshm" (OuterVolumeSpecName: "dshm") pod "d739b158-4789-4467-9fce-ba506ca96b55" (UID: "d739b158-4789-4467-9fce-ba506ca96b55"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 20 02:23:41.546240 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:23:41.546222    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d739b158-4789-4467-9fce-ba506ca96b55-kube-api-access-5rsrr" (OuterVolumeSpecName: "kube-api-access-5rsrr") pod "d739b158-4789-4467-9fce-ba506ca96b55" (UID: "d739b158-4789-4467-9fce-ba506ca96b55"). InnerVolumeSpecName "kube-api-access-5rsrr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 20 02:23:41.644822 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:23:41.644793    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5rsrr\" (UniqueName: \"kubernetes.io/projected/d739b158-4789-4467-9fce-ba506ca96b55-kube-api-access-5rsrr\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:23:41.644822 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:23:41.644817    2584 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d739b158-4789-4467-9fce-ba506ca96b55-dshm\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:23:41.644822 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:23:41.644827    2584 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d739b158-4789-4467-9fce-ba506ca96b55-tls-certs\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:23:42.254870 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:23:42.254834    2584 generic.go:356] "Generic (PLEG): container finished" podID="d739b158-4789-4467-9fce-ba506ca96b55" containerID="4ab3c732b51fbbcd836a8ae3d8b21f9853542ee86d727acc1b9548448275f08e" exitCode=0
Sep 20 02:23:42.254870 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:23:42.254871    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-ed9f38/sub-test-model-ed9f38-kserve-75bb5795d5-82g9b"
Sep 20 02:23:42.255431 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:23:42.254921    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-ed9f38/sub-test-model-ed9f38-kserve-75bb5795d5-82g9b" event={"ID":"d739b158-4789-4467-9fce-ba506ca96b55","Type":"ContainerDied","Data":"4ab3c732b51fbbcd836a8ae3d8b21f9853542ee86d727acc1b9548448275f08e"}
Sep 20 02:23:42.255431 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:23:42.254968    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-ed9f38/sub-test-model-ed9f38-kserve-75bb5795d5-82g9b" event={"ID":"d739b158-4789-4467-9fce-ba506ca96b55","Type":"ContainerDied","Data":"5eba3ce9463bd0a494ef27878dcaa22684b4c38d2aac2dee40481c537c7f82b0"}
Sep 20 02:23:42.255431 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:23:42.254986    2584 scope.go:117] "RemoveContainer" containerID="4ab3c732b51fbbcd836a8ae3d8b21f9853542ee86d727acc1b9548448275f08e"
Sep 20 02:23:42.269838 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:23:42.269725    2584 scope.go:117] "RemoveContainer" containerID="4ab3c732b51fbbcd836a8ae3d8b21f9853542ee86d727acc1b9548448275f08e"
Sep 20 02:23:42.270162 ip-10-0-130-159 kubenswrapper[2584]: E0920 02:23:42.270133    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4ab3c732b51fbbcd836a8ae3d8b21f9853542ee86d727acc1b9548448275f08e\": container with ID starting with 4ab3c732b51fbbcd836a8ae3d8b21f9853542ee86d727acc1b9548448275f08e not found: ID does not exist" containerID="4ab3c732b51fbbcd836a8ae3d8b21f9853542ee86d727acc1b9548448275f08e"
Sep 20 02:23:42.270268 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:23:42.270171    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ab3c732b51fbbcd836a8ae3d8b21f9853542ee86d727acc1b9548448275f08e"} err="failed to get container status \"4ab3c732b51fbbcd836a8ae3d8b21f9853542ee86d727acc1b9548448275f08e\": rpc error: code = NotFound desc = could not find container \"4ab3c732b51fbbcd836a8ae3d8b21f9853542ee86d727acc1b9548448275f08e\": container with ID starting with 4ab3c732b51fbbcd836a8ae3d8b21f9853542ee86d727acc1b9548448275f08e not found: ID does not exist"
Sep 20 02:23:42.285547 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:23:42.285509    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-ed9f38/sub-test-model-ed9f38-kserve-75bb5795d5-82g9b"]
Sep 20 02:23:42.293531 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:23:42.293498    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-ed9f38/sub-test-model-ed9f38-kserve-75bb5795d5-82g9b"]
Sep 20 02:23:42.333048 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:23:42.333007    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-317780/sub-test-model-317780-kserve-84648cb75-5sp4q" podUID="1ada1a62-1319-482a-8898-8f7251c9a59f" containerName="main" containerID="cri-o://afffa1ab191c845c99847acf46b0ca48c7d32f641904fa890eee333ff95af431" gracePeriod=45
Sep 20 02:23:42.586248 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:23:42.586223    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-317780/sub-test-model-317780-kserve-84648cb75-5sp4q"
Sep 20 02:23:42.652180 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:23:42.652153    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1ada1a62-1319-482a-8898-8f7251c9a59f-dshm\") pod \"1ada1a62-1319-482a-8898-8f7251c9a59f\" (UID: \"1ada1a62-1319-482a-8898-8f7251c9a59f\") "
Sep 20 02:23:42.652324 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:23:42.652210    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1ada1a62-1319-482a-8898-8f7251c9a59f-tls-certs\") pod \"1ada1a62-1319-482a-8898-8f7251c9a59f\" (UID: \"1ada1a62-1319-482a-8898-8f7251c9a59f\") "
Sep 20 02:23:42.652324 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:23:42.652235    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1ada1a62-1319-482a-8898-8f7251c9a59f-tmp-dir\") pod \"1ada1a62-1319-482a-8898-8f7251c9a59f\" (UID: \"1ada1a62-1319-482a-8898-8f7251c9a59f\") "
Sep 20 02:23:42.652324 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:23:42.652277    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1ada1a62-1319-482a-8898-8f7251c9a59f-home\") pod \"1ada1a62-1319-482a-8898-8f7251c9a59f\" (UID: \"1ada1a62-1319-482a-8898-8f7251c9a59f\") "
Sep 20 02:23:42.652324 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:23:42.652309    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gftv6\" (UniqueName: \"kubernetes.io/projected/1ada1a62-1319-482a-8898-8f7251c9a59f-kube-api-access-gftv6\") pod \"1ada1a62-1319-482a-8898-8f7251c9a59f\" (UID: \"1ada1a62-1319-482a-8898-8f7251c9a59f\") "
Sep 20 02:23:42.652585 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:23:42.652331    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1ada1a62-1319-482a-8898-8f7251c9a59f-model-cache\") pod \"1ada1a62-1319-482a-8898-8f7251c9a59f\" (UID: \"1ada1a62-1319-482a-8898-8f7251c9a59f\") "
Sep 20 02:23:42.652585 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:23:42.652562    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ada1a62-1319-482a-8898-8f7251c9a59f-home" (OuterVolumeSpecName: "home") pod "1ada1a62-1319-482a-8898-8f7251c9a59f" (UID: "1ada1a62-1319-482a-8898-8f7251c9a59f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 20 02:23:42.652687 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:23:42.652580    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ada1a62-1319-482a-8898-8f7251c9a59f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "1ada1a62-1319-482a-8898-8f7251c9a59f" (UID: "1ada1a62-1319-482a-8898-8f7251c9a59f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 20 02:23:42.652726 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:23:42.652679    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ada1a62-1319-482a-8898-8f7251c9a59f-model-cache" (OuterVolumeSpecName: "model-cache") pod "1ada1a62-1319-482a-8898-8f7251c9a59f" (UID: "1ada1a62-1319-482a-8898-8f7251c9a59f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 20 02:23:42.654450 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:23:42.654427    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ada1a62-1319-482a-8898-8f7251c9a59f-dshm" (OuterVolumeSpecName: "dshm") pod "1ada1a62-1319-482a-8898-8f7251c9a59f" (UID: "1ada1a62-1319-482a-8898-8f7251c9a59f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 20 02:23:42.654577 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:23:42.654427    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1ada1a62-1319-482a-8898-8f7251c9a59f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "1ada1a62-1319-482a-8898-8f7251c9a59f" (UID: "1ada1a62-1319-482a-8898-8f7251c9a59f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 20 02:23:42.654577 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:23:42.654553    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1ada1a62-1319-482a-8898-8f7251c9a59f-kube-api-access-gftv6" (OuterVolumeSpecName: "kube-api-access-gftv6") pod "1ada1a62-1319-482a-8898-8f7251c9a59f" (UID: "1ada1a62-1319-482a-8898-8f7251c9a59f"). InnerVolumeSpecName "kube-api-access-gftv6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 20 02:23:42.753264 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:23:42.753219    2584 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1ada1a62-1319-482a-8898-8f7251c9a59f-home\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:23:42.753264 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:23:42.753261    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gftv6\" (UniqueName: \"kubernetes.io/projected/1ada1a62-1319-482a-8898-8f7251c9a59f-kube-api-access-gftv6\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:23:42.753264 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:23:42.753271    2584 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1ada1a62-1319-482a-8898-8f7251c9a59f-model-cache\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:23:42.753264 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:23:42.753280    2584 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1ada1a62-1319-482a-8898-8f7251c9a59f-dshm\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:23:42.753559 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:23:42.753288    2584 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1ada1a62-1319-482a-8898-8f7251c9a59f-tls-certs\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:23:42.753559 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:23:42.753295    2584 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1ada1a62-1319-482a-8898-8f7251c9a59f-tmp-dir\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:23:43.093398 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:23:43.093364    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d739b158-4789-4467-9fce-ba506ca96b55" path="/var/lib/kubelet/pods/d739b158-4789-4467-9fce-ba506ca96b55/volumes"
Sep 20 02:23:43.264718 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:23:43.264669    2584 generic.go:356] "Generic (PLEG): container finished" podID="1ada1a62-1319-482a-8898-8f7251c9a59f" containerID="afffa1ab191c845c99847acf46b0ca48c7d32f641904fa890eee333ff95af431" exitCode=0
Sep 20 02:23:43.265206 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:23:43.264774    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-317780/sub-test-model-317780-kserve-84648cb75-5sp4q"
Sep 20 02:23:43.265206 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:23:43.264807    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-317780/sub-test-model-317780-kserve-84648cb75-5sp4q" event={"ID":"1ada1a62-1319-482a-8898-8f7251c9a59f","Type":"ContainerDied","Data":"afffa1ab191c845c99847acf46b0ca48c7d32f641904fa890eee333ff95af431"}
Sep 20 02:23:43.265206 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:23:43.264856    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-317780/sub-test-model-317780-kserve-84648cb75-5sp4q" event={"ID":"1ada1a62-1319-482a-8898-8f7251c9a59f","Type":"ContainerDied","Data":"72abd1cc3ad4bc289a4a8a24a96fd4528f3f41d868e4ec77bf310d822ba3347b"}
Sep 20 02:23:43.265206 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:23:43.264879    2584 scope.go:117] "RemoveContainer" containerID="afffa1ab191c845c99847acf46b0ca48c7d32f641904fa890eee333ff95af431"
Sep 20 02:23:43.273892 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:23:43.273874    2584 scope.go:117] "RemoveContainer" containerID="afffa1ab191c845c99847acf46b0ca48c7d32f641904fa890eee333ff95af431"
Sep 20 02:23:43.274163 ip-10-0-130-159 kubenswrapper[2584]: E0920 02:23:43.274139    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"afffa1ab191c845c99847acf46b0ca48c7d32f641904fa890eee333ff95af431\": container with ID starting with afffa1ab191c845c99847acf46b0ca48c7d32f641904fa890eee333ff95af431 not found: ID does not exist" containerID="afffa1ab191c845c99847acf46b0ca48c7d32f641904fa890eee333ff95af431"
Sep 20 02:23:43.274242 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:23:43.274179    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"afffa1ab191c845c99847acf46b0ca48c7d32f641904fa890eee333ff95af431"} err="failed to get container status \"afffa1ab191c845c99847acf46b0ca48c7d32f641904fa890eee333ff95af431\": rpc error: code = NotFound desc = could not find container \"afffa1ab191c845c99847acf46b0ca48c7d32f641904fa890eee333ff95af431\": container with ID starting with afffa1ab191c845c99847acf46b0ca48c7d32f641904fa890eee333ff95af431 not found: ID does not exist"
Sep 20 02:23:43.283818 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:23:43.283790    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-317780/sub-test-model-317780-kserve-84648cb75-5sp4q"]
Sep 20 02:23:43.287122 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:23:43.287097    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-317780/sub-test-model-317780-kserve-84648cb75-5sp4q"]
Sep 20 02:23:45.093100 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:23:45.093069    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1ada1a62-1319-482a-8898-8f7251c9a59f" path="/var/lib/kubelet/pods/1ada1a62-1319-482a-8898-8f7251c9a59f/volumes"
Sep 20 02:24:07.654775 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:24:07.654738    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-317780/rate-test-model-317780-kserve-86f5589b59-4dtjr"]
Sep 20 02:24:07.655209 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:24:07.655109    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1ada1a62-1319-482a-8898-8f7251c9a59f" containerName="main"
Sep 20 02:24:07.655209 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:24:07.655119    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ada1a62-1319-482a-8898-8f7251c9a59f" containerName="main"
Sep 20 02:24:07.655209 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:24:07.655129    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d739b158-4789-4467-9fce-ba506ca96b55" containerName="main"
Sep 20 02:24:07.655209 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:24:07.655134    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="d739b158-4789-4467-9fce-ba506ca96b55" containerName="main"
Sep 20 02:24:07.655209 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:24:07.655156    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ad9f645-99d9-495a-b8b9-f93f4e7f1b2e" containerName="test-curl-7534-6998aa"
Sep 20 02:24:07.655209 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:24:07.655162    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ad9f645-99d9-495a-b8b9-f93f4e7f1b2e" containerName="test-curl-7534-6998aa"
Sep 20 02:24:07.655404 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:24:07.655216    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ad9f645-99d9-495a-b8b9-f93f4e7f1b2e" containerName="test-curl-7534-6998aa"
Sep 20 02:24:07.655404 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:24:07.655224    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="d739b158-4789-4467-9fce-ba506ca96b55" containerName="main"
Sep 20 02:24:07.655404 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:24:07.655232    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="1ada1a62-1319-482a-8898-8f7251c9a59f" containerName="main"
Sep 20 02:24:07.658411 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:24:07.658390    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-317780/rate-test-model-317780-kserve-86f5589b59-4dtjr"
Sep 20 02:24:07.661418 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:24:07.661400    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-317780\"/\"kube-root-ca.crt\""
Sep 20 02:24:07.662291 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:24:07.662269    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-317780\"/\"openshift-service-ca.crt\""
Sep 20 02:24:07.662291 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:24:07.662281    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-317780\"/\"rate-test-model-317780-kserve-self-signed-certs\""
Sep 20 02:24:07.662291 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:24:07.662291    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-317780\"/\"default-dockercfg-5nxwn\""
Sep 20 02:24:07.671359 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:24:07.671339    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/080142f0-79f1-4419-b2a9-ef917bc46884-dshm\") pod \"rate-test-model-317780-kserve-86f5589b59-4dtjr\" (UID: \"080142f0-79f1-4419-b2a9-ef917bc46884\") " pod="ai-tenant-e2e-shared-b-w3-317780/rate-test-model-317780-kserve-86f5589b59-4dtjr"
Sep 20 02:24:07.671475 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:24:07.671379    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/080142f0-79f1-4419-b2a9-ef917bc46884-model-cache\") pod \"rate-test-model-317780-kserve-86f5589b59-4dtjr\" (UID: \"080142f0-79f1-4419-b2a9-ef917bc46884\") " pod="ai-tenant-e2e-shared-b-w3-317780/rate-test-model-317780-kserve-86f5589b59-4dtjr"
Sep 20 02:24:07.671475 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:24:07.671411    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/080142f0-79f1-4419-b2a9-ef917bc46884-tmp-dir\") pod \"rate-test-model-317780-kserve-86f5589b59-4dtjr\" (UID: \"080142f0-79f1-4419-b2a9-ef917bc46884\") " pod="ai-tenant-e2e-shared-b-w3-317780/rate-test-model-317780-kserve-86f5589b59-4dtjr"
Sep 20 02:24:07.671616 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:24:07.671491    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/080142f0-79f1-4419-b2a9-ef917bc46884-home\") pod \"rate-test-model-317780-kserve-86f5589b59-4dtjr\" (UID: \"080142f0-79f1-4419-b2a9-ef917bc46884\") " pod="ai-tenant-e2e-shared-b-w3-317780/rate-test-model-317780-kserve-86f5589b59-4dtjr"
Sep 20 02:24:07.671616 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:24:07.671552    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/080142f0-79f1-4419-b2a9-ef917bc46884-tls-certs\") pod \"rate-test-model-317780-kserve-86f5589b59-4dtjr\" (UID: \"080142f0-79f1-4419-b2a9-ef917bc46884\") " pod="ai-tenant-e2e-shared-b-w3-317780/rate-test-model-317780-kserve-86f5589b59-4dtjr"
Sep 20 02:24:07.671616 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:24:07.671583    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z4hd4\" (UniqueName: \"kubernetes.io/projected/080142f0-79f1-4419-b2a9-ef917bc46884-kube-api-access-z4hd4\") pod \"rate-test-model-317780-kserve-86f5589b59-4dtjr\" (UID: \"080142f0-79f1-4419-b2a9-ef917bc46884\") " pod="ai-tenant-e2e-shared-b-w3-317780/rate-test-model-317780-kserve-86f5589b59-4dtjr"
Sep 20 02:24:07.698725 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:24:07.698700    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-317780/rate-test-model-317780-kserve-86f5589b59-4dtjr"]
Sep 20 02:24:07.772596 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:24:07.772563    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/080142f0-79f1-4419-b2a9-ef917bc46884-home\") pod \"rate-test-model-317780-kserve-86f5589b59-4dtjr\" (UID: \"080142f0-79f1-4419-b2a9-ef917bc46884\") " pod="ai-tenant-e2e-shared-b-w3-317780/rate-test-model-317780-kserve-86f5589b59-4dtjr"
Sep 20 02:24:07.772596 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:24:07.772600    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/080142f0-79f1-4419-b2a9-ef917bc46884-tls-certs\") pod \"rate-test-model-317780-kserve-86f5589b59-4dtjr\" (UID: \"080142f0-79f1-4419-b2a9-ef917bc46884\") " pod="ai-tenant-e2e-shared-b-w3-317780/rate-test-model-317780-kserve-86f5589b59-4dtjr"
Sep 20 02:24:07.772811 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:24:07.772622    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z4hd4\" (UniqueName: \"kubernetes.io/projected/080142f0-79f1-4419-b2a9-ef917bc46884-kube-api-access-z4hd4\") pod \"rate-test-model-317780-kserve-86f5589b59-4dtjr\" (UID: \"080142f0-79f1-4419-b2a9-ef917bc46884\") " pod="ai-tenant-e2e-shared-b-w3-317780/rate-test-model-317780-kserve-86f5589b59-4dtjr"
Sep 20 02:24:07.772811 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:24:07.772665    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/080142f0-79f1-4419-b2a9-ef917bc46884-dshm\") pod \"rate-test-model-317780-kserve-86f5589b59-4dtjr\" (UID: \"080142f0-79f1-4419-b2a9-ef917bc46884\") " pod="ai-tenant-e2e-shared-b-w3-317780/rate-test-model-317780-kserve-86f5589b59-4dtjr"
Sep 20 02:24:07.772811 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:24:07.772691    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/080142f0-79f1-4419-b2a9-ef917bc46884-model-cache\") pod \"rate-test-model-317780-kserve-86f5589b59-4dtjr\" (UID: \"080142f0-79f1-4419-b2a9-ef917bc46884\") " pod="ai-tenant-e2e-shared-b-w3-317780/rate-test-model-317780-kserve-86f5589b59-4dtjr"
Sep 20 02:24:07.772811 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:24:07.772718    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/080142f0-79f1-4419-b2a9-ef917bc46884-tmp-dir\") pod \"rate-test-model-317780-kserve-86f5589b59-4dtjr\" (UID: \"080142f0-79f1-4419-b2a9-ef917bc46884\") " pod="ai-tenant-e2e-shared-b-w3-317780/rate-test-model-317780-kserve-86f5589b59-4dtjr"
Sep 20 02:24:07.773092 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:24:07.773069    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/080142f0-79f1-4419-b2a9-ef917bc46884-home\") pod \"rate-test-model-317780-kserve-86f5589b59-4dtjr\" (UID: \"080142f0-79f1-4419-b2a9-ef917bc46884\") " pod="ai-tenant-e2e-shared-b-w3-317780/rate-test-model-317780-kserve-86f5589b59-4dtjr"
Sep 20 02:24:07.773092 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:24:07.773087    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/080142f0-79f1-4419-b2a9-ef917bc46884-tmp-dir\") pod \"rate-test-model-317780-kserve-86f5589b59-4dtjr\" (UID: \"080142f0-79f1-4419-b2a9-ef917bc46884\") " pod="ai-tenant-e2e-shared-b-w3-317780/rate-test-model-317780-kserve-86f5589b59-4dtjr"
Sep 20 02:24:07.773225 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:24:07.773069    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/080142f0-79f1-4419-b2a9-ef917bc46884-model-cache\") pod \"rate-test-model-317780-kserve-86f5589b59-4dtjr\" (UID: \"080142f0-79f1-4419-b2a9-ef917bc46884\") " pod="ai-tenant-e2e-shared-b-w3-317780/rate-test-model-317780-kserve-86f5589b59-4dtjr"
Sep 20 02:24:07.775010 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:24:07.774989    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/080142f0-79f1-4419-b2a9-ef917bc46884-dshm\") pod \"rate-test-model-317780-kserve-86f5589b59-4dtjr\" (UID: \"080142f0-79f1-4419-b2a9-ef917bc46884\") " pod="ai-tenant-e2e-shared-b-w3-317780/rate-test-model-317780-kserve-86f5589b59-4dtjr"
Sep 20 02:24:07.775124 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:24:07.775078    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/080142f0-79f1-4419-b2a9-ef917bc46884-tls-certs\") pod \"rate-test-model-317780-kserve-86f5589b59-4dtjr\" (UID: \"080142f0-79f1-4419-b2a9-ef917bc46884\") " pod="ai-tenant-e2e-shared-b-w3-317780/rate-test-model-317780-kserve-86f5589b59-4dtjr"
Sep 20 02:24:07.796079 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:24:07.796055    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z4hd4\" (UniqueName: \"kubernetes.io/projected/080142f0-79f1-4419-b2a9-ef917bc46884-kube-api-access-z4hd4\") pod \"rate-test-model-317780-kserve-86f5589b59-4dtjr\" (UID: \"080142f0-79f1-4419-b2a9-ef917bc46884\") " pod="ai-tenant-e2e-shared-b-w3-317780/rate-test-model-317780-kserve-86f5589b59-4dtjr"
Sep 20 02:24:07.865827 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:24:07.865796    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:24:07.967634 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:24:07.967601    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-317780/rate-test-model-317780-kserve-86f5589b59-4dtjr"
Sep 20 02:24:08.252877 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:24:08.252854    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-317780/rate-test-model-317780-kserve-86f5589b59-4dtjr"]
Sep 20 02:24:08.255307 ip-10-0-130-159 kubenswrapper[2584]: W0920 02:24:08.255271    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod080142f0_79f1_4419_b2a9_ef917bc46884.slice/crio-e6a5bbdada18ee76a3c7aa3fcbdace3d8098e9fdcb059f836fbd0940d646b536 WatchSource:0}: Error finding container e6a5bbdada18ee76a3c7aa3fcbdace3d8098e9fdcb059f836fbd0940d646b536: Status 404 returned error can't find the container with id e6a5bbdada18ee76a3c7aa3fcbdace3d8098e9fdcb059f836fbd0940d646b536
Sep 20 02:24:08.392334 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:24:08.392286    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-317780/rate-test-model-317780-kserve-86f5589b59-4dtjr" event={"ID":"080142f0-79f1-4419-b2a9-ef917bc46884","Type":"ContainerStarted","Data":"6f9f50a7187ebac18aed270817b150bea956e779d3bcafa11a9d1e954f3a01b1"}
Sep 20 02:24:08.392492 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:24:08.392344    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-317780/rate-test-model-317780-kserve-86f5589b59-4dtjr" event={"ID":"080142f0-79f1-4419-b2a9-ef917bc46884","Type":"ContainerStarted","Data":"e6a5bbdada18ee76a3c7aa3fcbdace3d8098e9fdcb059f836fbd0940d646b536"}
Sep 20 02:24:08.416579 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:24:08.416533    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-317780/rate-test-model-317780-kserve-86f5589b59-4dtjr" podStartSLOduration=1.416503488 podStartE2EDuration="1.416503488s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-20 02:24:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-20 02:24:08.416050863 +0000 UTC m=+1905.158730085" watchObservedRunningTime="2026-09-20 02:24:08.416503488 +0000 UTC m=+1905.159182693"
Sep 20 02:24:09.613141 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:24:09.613107    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:24:09.730757 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:24:09.730722    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:24:14.722668 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:24:14.722631    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:24:17.968092 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:24:17.968051    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-317780/rate-test-model-317780-kserve-86f5589b59-4dtjr"
Sep 20 02:24:17.970574 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:24:17.968106    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-317780/rate-test-model-317780-kserve-86f5589b59-4dtjr"
Sep 20 02:24:17.981236 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:24:17.981206    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-317780/rate-test-model-317780-kserve-86f5589b59-4dtjr"
Sep 20 02:24:18.455410 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:24:18.455384    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-317780/rate-test-model-317780-kserve-86f5589b59-4dtjr"
Sep 20 02:24:46.032599 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:24:46.032564    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:24:47.000574 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:24:47.000541    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-317780/rate-test-model-317780-kserve-86f5589b59-4dtjr"]
Sep 20 02:24:47.051639 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:24:47.051606    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:24:47.146252 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:24:47.146221    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:24:47.441946 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:24:47.441921    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:24:47.831547 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:24:47.831451    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-ec8b82-openshift-default-65cf789494-k4s5q"]
Sep 20 02:24:47.836146 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:24:47.836129    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-ec8b82-openshift-default-65cf789494-k4s5q"
Sep 20 02:24:47.843003 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:24:47.842980    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c016c6f5-99f6-400e-ac31-292ede5ae4dd-workload-socket\") pod \"e2e-ipp-a-ec8b82-openshift-default-65cf789494-k4s5q\" (UID: \"c016c6f5-99f6-400e-ac31-292ede5ae4dd\") " pod="openshift-ingress/e2e-ipp-a-ec8b82-openshift-default-65cf789494-k4s5q"
Sep 20 02:24:47.843110 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:24:47.843022    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c016c6f5-99f6-400e-ac31-292ede5ae4dd-credential-socket\") pod \"e2e-ipp-a-ec8b82-openshift-default-65cf789494-k4s5q\" (UID: \"c016c6f5-99f6-400e-ac31-292ede5ae4dd\") " pod="openshift-ingress/e2e-ipp-a-ec8b82-openshift-default-65cf789494-k4s5q"
Sep 20 02:24:47.843110 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:24:47.843048    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c016c6f5-99f6-400e-ac31-292ede5ae4dd-istiod-ca-cert\") pod \"e2e-ipp-a-ec8b82-openshift-default-65cf789494-k4s5q\" (UID: \"c016c6f5-99f6-400e-ac31-292ede5ae4dd\") " pod="openshift-ingress/e2e-ipp-a-ec8b82-openshift-default-65cf789494-k4s5q"
Sep 20 02:24:47.843110 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:24:47.843079    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c016c6f5-99f6-400e-ac31-292ede5ae4dd-istio-data\") pod \"e2e-ipp-a-ec8b82-openshift-default-65cf789494-k4s5q\" (UID: \"c016c6f5-99f6-400e-ac31-292ede5ae4dd\") " pod="openshift-ingress/e2e-ipp-a-ec8b82-openshift-default-65cf789494-k4s5q"
Sep 20 02:24:47.843239 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:24:47.843111    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5jqfp\" (UniqueName: \"kubernetes.io/projected/c016c6f5-99f6-400e-ac31-292ede5ae4dd-kube-api-access-5jqfp\") pod \"e2e-ipp-a-ec8b82-openshift-default-65cf789494-k4s5q\" (UID: \"c016c6f5-99f6-400e-ac31-292ede5ae4dd\") " pod="openshift-ingress/e2e-ipp-a-ec8b82-openshift-default-65cf789494-k4s5q"
Sep 20 02:24:47.843239 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:24:47.843165    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c016c6f5-99f6-400e-ac31-292ede5ae4dd-workload-certs\") pod \"e2e-ipp-a-ec8b82-openshift-default-65cf789494-k4s5q\" (UID: \"c016c6f5-99f6-400e-ac31-292ede5ae4dd\") " pod="openshift-ingress/e2e-ipp-a-ec8b82-openshift-default-65cf789494-k4s5q"
Sep 20 02:24:47.843239 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:24:47.843199    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c016c6f5-99f6-400e-ac31-292ede5ae4dd-istio-podinfo\") pod \"e2e-ipp-a-ec8b82-openshift-default-65cf789494-k4s5q\" (UID: \"c016c6f5-99f6-400e-ac31-292ede5ae4dd\") " pod="openshift-ingress/e2e-ipp-a-ec8b82-openshift-default-65cf789494-k4s5q"
Sep 20 02:24:47.843339 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:24:47.843262    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c016c6f5-99f6-400e-ac31-292ede5ae4dd-istio-token\") pod \"e2e-ipp-a-ec8b82-openshift-default-65cf789494-k4s5q\" (UID: \"c016c6f5-99f6-400e-ac31-292ede5ae4dd\") " pod="openshift-ingress/e2e-ipp-a-ec8b82-openshift-default-65cf789494-k4s5q"
Sep 20 02:24:47.843339 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:24:47.843330    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c016c6f5-99f6-400e-ac31-292ede5ae4dd-istio-envoy\") pod \"e2e-ipp-a-ec8b82-openshift-default-65cf789494-k4s5q\" (UID: \"c016c6f5-99f6-400e-ac31-292ede5ae4dd\") " pod="openshift-ingress/e2e-ipp-a-ec8b82-openshift-default-65cf789494-k4s5q"
Sep 20 02:24:47.843478 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:24:47.843464    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 20 02:24:47.844840 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:24:47.844825    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-ec8b82-openshift-default-dockercfg-4bchp\""
Sep 20 02:24:47.861991 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:24:47.861960    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-ec8b82-openshift-default-65cf789494-k4s5q"]
Sep 20 02:24:47.868433 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:24:47.868398    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:24:47.943943 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:24:47.943918    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5jqfp\" (UniqueName: \"kubernetes.io/projected/c016c6f5-99f6-400e-ac31-292ede5ae4dd-kube-api-access-5jqfp\") pod \"e2e-ipp-a-ec8b82-openshift-default-65cf789494-k4s5q\" (UID: \"c016c6f5-99f6-400e-ac31-292ede5ae4dd\") " pod="openshift-ingress/e2e-ipp-a-ec8b82-openshift-default-65cf789494-k4s5q"
Sep 20 02:24:47.944114 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:24:47.943950    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c016c6f5-99f6-400e-ac31-292ede5ae4dd-workload-certs\") pod \"e2e-ipp-a-ec8b82-openshift-default-65cf789494-k4s5q\" (UID: \"c016c6f5-99f6-400e-ac31-292ede5ae4dd\") " pod="openshift-ingress/e2e-ipp-a-ec8b82-openshift-default-65cf789494-k4s5q"
Sep 20 02:24:47.944114 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:24:47.943966    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c016c6f5-99f6-400e-ac31-292ede5ae4dd-istio-podinfo\") pod \"e2e-ipp-a-ec8b82-openshift-default-65cf789494-k4s5q\" (UID: \"c016c6f5-99f6-400e-ac31-292ede5ae4dd\") " pod="openshift-ingress/e2e-ipp-a-ec8b82-openshift-default-65cf789494-k4s5q"
Sep 20 02:24:47.944114 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:24:47.944001    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c016c6f5-99f6-400e-ac31-292ede5ae4dd-istio-token\") pod \"e2e-ipp-a-ec8b82-openshift-default-65cf789494-k4s5q\" (UID: \"c016c6f5-99f6-400e-ac31-292ede5ae4dd\") " pod="openshift-ingress/e2e-ipp-a-ec8b82-openshift-default-65cf789494-k4s5q"
Sep 20 02:24:47.944114 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:24:47.944043    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c016c6f5-99f6-400e-ac31-292ede5ae4dd-istio-envoy\") pod \"e2e-ipp-a-ec8b82-openshift-default-65cf789494-k4s5q\" (UID: \"c016c6f5-99f6-400e-ac31-292ede5ae4dd\") " pod="openshift-ingress/e2e-ipp-a-ec8b82-openshift-default-65cf789494-k4s5q"
Sep 20 02:24:47.944363 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:24:47.944196    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c016c6f5-99f6-400e-ac31-292ede5ae4dd-workload-socket\") pod \"e2e-ipp-a-ec8b82-openshift-default-65cf789494-k4s5q\" (UID: \"c016c6f5-99f6-400e-ac31-292ede5ae4dd\") " pod="openshift-ingress/e2e-ipp-a-ec8b82-openshift-default-65cf789494-k4s5q"
Sep 20 02:24:47.944363 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:24:47.944239    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c016c6f5-99f6-400e-ac31-292ede5ae4dd-credential-socket\") pod \"e2e-ipp-a-ec8b82-openshift-default-65cf789494-k4s5q\" (UID: \"c016c6f5-99f6-400e-ac31-292ede5ae4dd\") " pod="openshift-ingress/e2e-ipp-a-ec8b82-openshift-default-65cf789494-k4s5q"
Sep 20 02:24:47.944363 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:24:47.944267    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c016c6f5-99f6-400e-ac31-292ede5ae4dd-istiod-ca-cert\") pod \"e2e-ipp-a-ec8b82-openshift-default-65cf789494-k4s5q\" (UID: \"c016c6f5-99f6-400e-ac31-292ede5ae4dd\") " pod="openshift-ingress/e2e-ipp-a-ec8b82-openshift-default-65cf789494-k4s5q"
Sep 20 02:24:47.944363 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:24:47.944289    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c016c6f5-99f6-400e-ac31-292ede5ae4dd-istio-data\") pod \"e2e-ipp-a-ec8b82-openshift-default-65cf789494-k4s5q\" (UID: \"c016c6f5-99f6-400e-ac31-292ede5ae4dd\") " pod="openshift-ingress/e2e-ipp-a-ec8b82-openshift-default-65cf789494-k4s5q"
Sep 20 02:24:47.944625 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:24:47.944370    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c016c6f5-99f6-400e-ac31-292ede5ae4dd-workload-certs\") pod \"e2e-ipp-a-ec8b82-openshift-default-65cf789494-k4s5q\" (UID: \"c016c6f5-99f6-400e-ac31-292ede5ae4dd\") " pod="openshift-ingress/e2e-ipp-a-ec8b82-openshift-default-65cf789494-k4s5q"
Sep 20 02:24:47.944625 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:24:47.944599    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c016c6f5-99f6-400e-ac31-292ede5ae4dd-workload-socket\") pod \"e2e-ipp-a-ec8b82-openshift-default-65cf789494-k4s5q\" (UID: \"c016c6f5-99f6-400e-ac31-292ede5ae4dd\") " pod="openshift-ingress/e2e-ipp-a-ec8b82-openshift-default-65cf789494-k4s5q"
Sep 20 02:24:47.944808 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:24:47.944783    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c016c6f5-99f6-400e-ac31-292ede5ae4dd-istio-data\") pod \"e2e-ipp-a-ec8b82-openshift-default-65cf789494-k4s5q\" (UID: \"c016c6f5-99f6-400e-ac31-292ede5ae4dd\") " pod="openshift-ingress/e2e-ipp-a-ec8b82-openshift-default-65cf789494-k4s5q"
Sep 20 02:24:47.944883 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:24:47.944786    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c016c6f5-99f6-400e-ac31-292ede5ae4dd-credential-socket\") pod \"e2e-ipp-a-ec8b82-openshift-default-65cf789494-k4s5q\" (UID: \"c016c6f5-99f6-400e-ac31-292ede5ae4dd\") " pod="openshift-ingress/e2e-ipp-a-ec8b82-openshift-default-65cf789494-k4s5q"
Sep 20 02:24:47.945044 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:24:47.945024    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c016c6f5-99f6-400e-ac31-292ede5ae4dd-istiod-ca-cert\") pod \"e2e-ipp-a-ec8b82-openshift-default-65cf789494-k4s5q\" (UID: \"c016c6f5-99f6-400e-ac31-292ede5ae4dd\") " pod="openshift-ingress/e2e-ipp-a-ec8b82-openshift-default-65cf789494-k4s5q"
Sep 20 02:24:47.946428 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:24:47.946408    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c016c6f5-99f6-400e-ac31-292ede5ae4dd-istio-envoy\") pod \"e2e-ipp-a-ec8b82-openshift-default-65cf789494-k4s5q\" (UID: \"c016c6f5-99f6-400e-ac31-292ede5ae4dd\") " pod="openshift-ingress/e2e-ipp-a-ec8b82-openshift-default-65cf789494-k4s5q"
Sep 20 02:24:47.946674 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:24:47.946654    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c016c6f5-99f6-400e-ac31-292ede5ae4dd-istio-podinfo\") pod \"e2e-ipp-a-ec8b82-openshift-default-65cf789494-k4s5q\" (UID: \"c016c6f5-99f6-400e-ac31-292ede5ae4dd\") " pod="openshift-ingress/e2e-ipp-a-ec8b82-openshift-default-65cf789494-k4s5q"
Sep 20 02:24:47.959262 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:24:47.959236    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5jqfp\" (UniqueName: \"kubernetes.io/projected/c016c6f5-99f6-400e-ac31-292ede5ae4dd-kube-api-access-5jqfp\") pod \"e2e-ipp-a-ec8b82-openshift-default-65cf789494-k4s5q\" (UID: \"c016c6f5-99f6-400e-ac31-292ede5ae4dd\") " pod="openshift-ingress/e2e-ipp-a-ec8b82-openshift-default-65cf789494-k4s5q"
Sep 20 02:24:47.964989 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:24:47.964959    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c016c6f5-99f6-400e-ac31-292ede5ae4dd-istio-token\") pod \"e2e-ipp-a-ec8b82-openshift-default-65cf789494-k4s5q\" (UID: \"c016c6f5-99f6-400e-ac31-292ede5ae4dd\") " pod="openshift-ingress/e2e-ipp-a-ec8b82-openshift-default-65cf789494-k4s5q"
Sep 20 02:24:48.023078 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:24:48.023047    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:24:48.146123 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:24:48.146027    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-ec8b82-openshift-default-65cf789494-k4s5q"
Sep 20 02:24:48.434124 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:24:48.434050    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-ec8b82-openshift-default-65cf789494-k4s5q"]
Sep 20 02:24:48.437643 ip-10-0-130-159 kubenswrapper[2584]: W0920 02:24:48.437617    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc016c6f5_99f6_400e_ac31_292ede5ae4dd.slice/crio-3f21ba664888756572fa9ca2b0bca04f8ca6a461dee1d01f31a2d0ec3051ce6e WatchSource:0}: Error finding container 3f21ba664888756572fa9ca2b0bca04f8ca6a461dee1d01f31a2d0ec3051ce6e: Status 404 returned error can't find the container with id 3f21ba664888756572fa9ca2b0bca04f8ca6a461dee1d01f31a2d0ec3051ce6e
Sep 20 02:24:48.439767 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:24:48.439734    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 20 02:24:48.439855 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:24:48.439797    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 20 02:24:48.439855 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:24:48.439824    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 20 02:24:48.598582 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:24:48.598537    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-ec8b82-openshift-default-65cf789494-k4s5q" event={"ID":"c016c6f5-99f6-400e-ac31-292ede5ae4dd","Type":"ContainerStarted","Data":"b37a9fd4f2fa18ed669ec5c7715dd8057759eab59f3bb192844893bcd5560567"}
Sep 20 02:24:48.598741 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:24:48.598609    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-ec8b82-openshift-default-65cf789494-k4s5q" event={"ID":"c016c6f5-99f6-400e-ac31-292ede5ae4dd","Type":"ContainerStarted","Data":"3f21ba664888756572fa9ca2b0bca04f8ca6a461dee1d01f31a2d0ec3051ce6e"}
Sep 20 02:24:48.630344 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:24:48.630279    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-ec8b82-openshift-default-65cf789494-k4s5q" podStartSLOduration=1.6302601129999998 podStartE2EDuration="1.630260113s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-20 02:24:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-20 02:24:48.628397807 +0000 UTC m=+1945.371077014" watchObservedRunningTime="2026-09-20 02:24:48.630260113 +0000 UTC m=+1945.372939317"
Sep 20 02:24:49.146979 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:24:49.146946    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-ec8b82-openshift-default-65cf789494-k4s5q"
Sep 20 02:24:49.151907 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:24:49.151884    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-ec8b82-openshift-default-65cf789494-k4s5q"
Sep 20 02:24:49.603790 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:24:49.603752    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-ec8b82-openshift-default-65cf789494-k4s5q"
Sep 20 02:24:49.604679 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:24:49.604656    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-ec8b82-openshift-default-65cf789494-k4s5q"
Sep 20 02:24:50.850791 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:24:50.850760    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:24:51.028124 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:24:51.028090    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:24:51.369908 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:24:51.369879    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:24:53.497189 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:24:53.497152    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:24:54.578420 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:24:54.578385    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-ec8b82-59584946fc-5j586"]
Sep 20 02:24:54.581816 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:24:54.581800    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-ec8b82-59584946fc-5j586"
Sep 20 02:24:54.586720 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:24:54.586695    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-ec8b82-serving-cert\""
Sep 20 02:24:54.589261 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:24:54.589236    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7x724\" (UniqueName: \"kubernetes.io/projected/3020f8fa-d45a-48b9-8ce9-f9af5b0321dc-kube-api-access-7x724\") pod \"maas-api-e2e-ipp-a-ec8b82-59584946fc-5j586\" (UID: \"3020f8fa-d45a-48b9-8ce9-f9af5b0321dc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-ec8b82-59584946fc-5j586"
Sep 20 02:24:54.589389 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:24:54.589288    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3020f8fa-d45a-48b9-8ce9-f9af5b0321dc-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-ec8b82-59584946fc-5j586\" (UID: \"3020f8fa-d45a-48b9-8ce9-f9af5b0321dc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-ec8b82-59584946fc-5j586"
Sep 20 02:24:54.589389 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:24:54.589324    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3020f8fa-d45a-48b9-8ce9-f9af5b0321dc-maas-api-tls\") pod \"maas-api-e2e-ipp-a-ec8b82-59584946fc-5j586\" (UID: \"3020f8fa-d45a-48b9-8ce9-f9af5b0321dc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-ec8b82-59584946fc-5j586"
Sep 20 02:24:54.589389 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:24:54.589352    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3020f8fa-d45a-48b9-8ce9-f9af5b0321dc-metrics-certs\") pod \"maas-api-e2e-ipp-a-ec8b82-59584946fc-5j586\" (UID: \"3020f8fa-d45a-48b9-8ce9-f9af5b0321dc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-ec8b82-59584946fc-5j586"
Sep 20 02:24:54.651397 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:24:54.651357    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:24:54.689893 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:24:54.689863    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3020f8fa-d45a-48b9-8ce9-f9af5b0321dc-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-ec8b82-59584946fc-5j586\" (UID: \"3020f8fa-d45a-48b9-8ce9-f9af5b0321dc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-ec8b82-59584946fc-5j586"
Sep 20 02:24:54.690058 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:24:54.689903    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3020f8fa-d45a-48b9-8ce9-f9af5b0321dc-maas-api-tls\") pod \"maas-api-e2e-ipp-a-ec8b82-59584946fc-5j586\" (UID: \"3020f8fa-d45a-48b9-8ce9-f9af5b0321dc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-ec8b82-59584946fc-5j586"
Sep 20 02:24:54.690058 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:24:54.689931    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3020f8fa-d45a-48b9-8ce9-f9af5b0321dc-metrics-certs\") pod \"maas-api-e2e-ipp-a-ec8b82-59584946fc-5j586\" (UID: \"3020f8fa-d45a-48b9-8ce9-f9af5b0321dc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-ec8b82-59584946fc-5j586"
Sep 20 02:24:54.690058 ip-10-0-130-159 kubenswrapper[2584]: E0920 02:24:54.690027    2584 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-a-ec8b82-serving-cert: secret "maas-api-e2e-ipp-a-ec8b82-serving-cert" not found
Sep 20 02:24:54.690220 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:24:54.690076    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7x724\" (UniqueName: \"kubernetes.io/projected/3020f8fa-d45a-48b9-8ce9-f9af5b0321dc-kube-api-access-7x724\") pod \"maas-api-e2e-ipp-a-ec8b82-59584946fc-5j586\" (UID: \"3020f8fa-d45a-48b9-8ce9-f9af5b0321dc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-ec8b82-59584946fc-5j586"
Sep 20 02:24:54.690220 ip-10-0-130-159 kubenswrapper[2584]: E0920 02:24:54.690100    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3020f8fa-d45a-48b9-8ce9-f9af5b0321dc-maas-api-tls podName:3020f8fa-d45a-48b9-8ce9-f9af5b0321dc nodeName:}" failed. No retries permitted until 2026-09-20 02:24:55.190079989 +0000 UTC m=+1951.932759187 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/3020f8fa-d45a-48b9-8ce9-f9af5b0321dc-maas-api-tls") pod "maas-api-e2e-ipp-a-ec8b82-59584946fc-5j586" (UID: "3020f8fa-d45a-48b9-8ce9-f9af5b0321dc") : secret "maas-api-e2e-ipp-a-ec8b82-serving-cert" not found
Sep 20 02:24:54.690670 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:24:54.690640    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3020f8fa-d45a-48b9-8ce9-f9af5b0321dc-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-ec8b82-59584946fc-5j586\" (UID: \"3020f8fa-d45a-48b9-8ce9-f9af5b0321dc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-ec8b82-59584946fc-5j586"
Sep 20 02:24:54.693049 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:24:54.693028    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3020f8fa-d45a-48b9-8ce9-f9af5b0321dc-metrics-certs\") pod \"maas-api-e2e-ipp-a-ec8b82-59584946fc-5j586\" (UID: \"3020f8fa-d45a-48b9-8ce9-f9af5b0321dc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-ec8b82-59584946fc-5j586"
Sep 20 02:24:54.699431 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:24:54.699406    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-ec8b82-59584946fc-5j586"]
Sep 20 02:24:54.706956 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:24:54.706928    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7x724\" (UniqueName: \"kubernetes.io/projected/3020f8fa-d45a-48b9-8ce9-f9af5b0321dc-kube-api-access-7x724\") pod \"maas-api-e2e-ipp-a-ec8b82-59584946fc-5j586\" (UID: \"3020f8fa-d45a-48b9-8ce9-f9af5b0321dc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-ec8b82-59584946fc-5j586"
Sep 20 02:24:54.847254 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:24:54.847171    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:24:54.967533 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:24:54.967499    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:24:55.195881 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:24:55.195847    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3020f8fa-d45a-48b9-8ce9-f9af5b0321dc-maas-api-tls\") pod \"maas-api-e2e-ipp-a-ec8b82-59584946fc-5j586\" (UID: \"3020f8fa-d45a-48b9-8ce9-f9af5b0321dc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-ec8b82-59584946fc-5j586"
Sep 20 02:24:55.198295 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:24:55.198277    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3020f8fa-d45a-48b9-8ce9-f9af5b0321dc-maas-api-tls\") pod \"maas-api-e2e-ipp-a-ec8b82-59584946fc-5j586\" (UID: \"3020f8fa-d45a-48b9-8ce9-f9af5b0321dc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-ec8b82-59584946fc-5j586"
Sep 20 02:24:55.493793 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:24:55.493714    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-ec8b82-59584946fc-5j586"
Sep 20 02:24:55.676025 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:24:55.675988    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:24:55.831332 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:24:55.831301    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-ec8b82-59584946fc-5j586"]
Sep 20 02:24:55.834299 ip-10-0-130-159 kubenswrapper[2584]: W0920 02:24:55.834229    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3020f8fa_d45a_48b9_8ce9_f9af5b0321dc.slice/crio-55efcc3f6050605184c8dd7ba24d4d0b1454625a4765073d555d7b66d75115b9 WatchSource:0}: Error finding container 55efcc3f6050605184c8dd7ba24d4d0b1454625a4765073d555d7b66d75115b9: Status 404 returned error can't find the container with id 55efcc3f6050605184c8dd7ba24d4d0b1454625a4765073d555d7b66d75115b9
Sep 20 02:24:55.835788 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:24:55.835767    2584 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 20 02:24:55.855392 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:24:55.855353    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:24:56.640061 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:24:56.639968    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-ec8b82-59584946fc-5j586" event={"ID":"3020f8fa-d45a-48b9-8ce9-f9af5b0321dc","Type":"ContainerStarted","Data":"b08c52e3f3163fc042063e07c63c1485d948aa7a94ae0360052ab53762c1df78"}
Sep 20 02:24:56.640061 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:24:56.640022    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-ec8b82-59584946fc-5j586" event={"ID":"3020f8fa-d45a-48b9-8ce9-f9af5b0321dc","Type":"ContainerStarted","Data":"55efcc3f6050605184c8dd7ba24d4d0b1454625a4765073d555d7b66d75115b9"}
Sep 20 02:24:56.640061 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:24:56.640048    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-ec8b82-59584946fc-5j586"
Sep 20 02:24:56.689683 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:24:56.689631    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-ec8b82-59584946fc-5j586" podStartSLOduration=2.268489878 podStartE2EDuration="2.689614093s" totalImagesPullingTime="421.124215ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-20 02:24:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-20 02:24:56.689322614 +0000 UTC m=+1953.432001831" watchObservedRunningTime="2026-09-20 02:24:56.689614093 +0000 UTC m=+1953.432293296"
Sep 20 02:24:56.703490 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:24:56.703463    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:24:58.083393 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:24:58.083363    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:25:02.018949 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:02.018902    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-317780/rate-test-model-317780-kserve-86f5589b59-4dtjr" podUID="080142f0-79f1-4419-b2a9-ef917bc46884" containerName="main" containerID="cri-o://6f9f50a7187ebac18aed270817b150bea956e779d3bcafa11a9d1e954f3a01b1" gracePeriod=45
Sep 20 02:25:02.265652 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:02.265620    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-317780/rate-test-model-317780-kserve-86f5589b59-4dtjr"
Sep 20 02:25:02.357704 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:02.357626    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/080142f0-79f1-4419-b2a9-ef917bc46884-dshm\") pod \"080142f0-79f1-4419-b2a9-ef917bc46884\" (UID: \"080142f0-79f1-4419-b2a9-ef917bc46884\") "
Sep 20 02:25:02.357704 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:02.357662    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/080142f0-79f1-4419-b2a9-ef917bc46884-tls-certs\") pod \"080142f0-79f1-4419-b2a9-ef917bc46884\" (UID: \"080142f0-79f1-4419-b2a9-ef917bc46884\") "
Sep 20 02:25:02.357704 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:02.357684    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z4hd4\" (UniqueName: \"kubernetes.io/projected/080142f0-79f1-4419-b2a9-ef917bc46884-kube-api-access-z4hd4\") pod \"080142f0-79f1-4419-b2a9-ef917bc46884\" (UID: \"080142f0-79f1-4419-b2a9-ef917bc46884\") "
Sep 20 02:25:02.357987 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:02.357756    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/080142f0-79f1-4419-b2a9-ef917bc46884-model-cache\") pod \"080142f0-79f1-4419-b2a9-ef917bc46884\" (UID: \"080142f0-79f1-4419-b2a9-ef917bc46884\") "
Sep 20 02:25:02.357987 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:02.357782    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/080142f0-79f1-4419-b2a9-ef917bc46884-tmp-dir\") pod \"080142f0-79f1-4419-b2a9-ef917bc46884\" (UID: \"080142f0-79f1-4419-b2a9-ef917bc46884\") "
Sep 20 02:25:02.357987 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:02.357805    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/080142f0-79f1-4419-b2a9-ef917bc46884-home\") pod \"080142f0-79f1-4419-b2a9-ef917bc46884\" (UID: \"080142f0-79f1-4419-b2a9-ef917bc46884\") "
Sep 20 02:25:02.358151 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:02.358010    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/080142f0-79f1-4419-b2a9-ef917bc46884-model-cache" (OuterVolumeSpecName: "model-cache") pod "080142f0-79f1-4419-b2a9-ef917bc46884" (UID: "080142f0-79f1-4419-b2a9-ef917bc46884"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 20 02:25:02.358207 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:02.358156    2584 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/080142f0-79f1-4419-b2a9-ef917bc46884-model-cache\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:25:02.358260 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:02.358218    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/080142f0-79f1-4419-b2a9-ef917bc46884-home" (OuterVolumeSpecName: "home") pod "080142f0-79f1-4419-b2a9-ef917bc46884" (UID: "080142f0-79f1-4419-b2a9-ef917bc46884"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 20 02:25:02.358260 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:02.358232    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/080142f0-79f1-4419-b2a9-ef917bc46884-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "080142f0-79f1-4419-b2a9-ef917bc46884" (UID: "080142f0-79f1-4419-b2a9-ef917bc46884"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 20 02:25:02.359806 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:02.359780    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/080142f0-79f1-4419-b2a9-ef917bc46884-dshm" (OuterVolumeSpecName: "dshm") pod "080142f0-79f1-4419-b2a9-ef917bc46884" (UID: "080142f0-79f1-4419-b2a9-ef917bc46884"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 20 02:25:02.359932 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:02.359845    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/080142f0-79f1-4419-b2a9-ef917bc46884-kube-api-access-z4hd4" (OuterVolumeSpecName: "kube-api-access-z4hd4") pod "080142f0-79f1-4419-b2a9-ef917bc46884" (UID: "080142f0-79f1-4419-b2a9-ef917bc46884"). InnerVolumeSpecName "kube-api-access-z4hd4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 20 02:25:02.359932 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:02.359881    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/080142f0-79f1-4419-b2a9-ef917bc46884-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "080142f0-79f1-4419-b2a9-ef917bc46884" (UID: "080142f0-79f1-4419-b2a9-ef917bc46884"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 20 02:25:02.458969 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:02.458943    2584 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/080142f0-79f1-4419-b2a9-ef917bc46884-tmp-dir\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:25:02.458969 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:02.458967    2584 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/080142f0-79f1-4419-b2a9-ef917bc46884-home\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:25:02.458969 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:02.458976    2584 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/080142f0-79f1-4419-b2a9-ef917bc46884-dshm\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:25:02.459192 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:02.458984    2584 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/080142f0-79f1-4419-b2a9-ef917bc46884-tls-certs\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:25:02.459192 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:02.458992    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z4hd4\" (UniqueName: \"kubernetes.io/projected/080142f0-79f1-4419-b2a9-ef917bc46884-kube-api-access-z4hd4\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:25:02.650627 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:02.650548    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-ec8b82-59584946fc-5j586"
Sep 20 02:25:02.671314 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:02.671287    2584 generic.go:356] "Generic (PLEG): container finished" podID="080142f0-79f1-4419-b2a9-ef917bc46884" containerID="6f9f50a7187ebac18aed270817b150bea956e779d3bcafa11a9d1e954f3a01b1" exitCode=0
Sep 20 02:25:02.671467 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:02.671319    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-317780/rate-test-model-317780-kserve-86f5589b59-4dtjr"
Sep 20 02:25:02.671467 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:02.671360    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-317780/rate-test-model-317780-kserve-86f5589b59-4dtjr" event={"ID":"080142f0-79f1-4419-b2a9-ef917bc46884","Type":"ContainerDied","Data":"6f9f50a7187ebac18aed270817b150bea956e779d3bcafa11a9d1e954f3a01b1"}
Sep 20 02:25:02.671467 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:02.671402    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-317780/rate-test-model-317780-kserve-86f5589b59-4dtjr" event={"ID":"080142f0-79f1-4419-b2a9-ef917bc46884","Type":"ContainerDied","Data":"e6a5bbdada18ee76a3c7aa3fcbdace3d8098e9fdcb059f836fbd0940d646b536"}
Sep 20 02:25:02.671467 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:02.671418    2584 scope.go:117] "RemoveContainer" containerID="6f9f50a7187ebac18aed270817b150bea956e779d3bcafa11a9d1e954f3a01b1"
Sep 20 02:25:02.681023 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:02.681004    2584 scope.go:117] "RemoveContainer" containerID="6f9f50a7187ebac18aed270817b150bea956e779d3bcafa11a9d1e954f3a01b1"
Sep 20 02:25:02.681292 ip-10-0-130-159 kubenswrapper[2584]: E0920 02:25:02.681274    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6f9f50a7187ebac18aed270817b150bea956e779d3bcafa11a9d1e954f3a01b1\": container with ID starting with 6f9f50a7187ebac18aed270817b150bea956e779d3bcafa11a9d1e954f3a01b1 not found: ID does not exist" containerID="6f9f50a7187ebac18aed270817b150bea956e779d3bcafa11a9d1e954f3a01b1"
Sep 20 02:25:02.681348 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:02.681301    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f9f50a7187ebac18aed270817b150bea956e779d3bcafa11a9d1e954f3a01b1"} err="failed to get container status \"6f9f50a7187ebac18aed270817b150bea956e779d3bcafa11a9d1e954f3a01b1\": rpc error: code = NotFound desc = could not find container \"6f9f50a7187ebac18aed270817b150bea956e779d3bcafa11a9d1e954f3a01b1\": container with ID starting with 6f9f50a7187ebac18aed270817b150bea956e779d3bcafa11a9d1e954f3a01b1 not found: ID does not exist"
Sep 20 02:25:02.715010 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:02.714981    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-317780/rate-test-model-317780-kserve-86f5589b59-4dtjr"]
Sep 20 02:25:02.733074 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:02.733044    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-317780/rate-test-model-317780-kserve-86f5589b59-4dtjr"]
Sep 20 02:25:03.099646 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:03.099605    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="080142f0-79f1-4419-b2a9-ef917bc46884" path="/var/lib/kubelet/pods/080142f0-79f1-4419-b2a9-ef917bc46884/volumes"
Sep 20 02:25:13.731899 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:13.731860    2584 generic.go:356] "Generic (PLEG): container finished" podID="c016c6f5-99f6-400e-ac31-292ede5ae4dd" containerID="b37a9fd4f2fa18ed669ec5c7715dd8057759eab59f3bb192844893bcd5560567" exitCode=0
Sep 20 02:25:13.732347 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:13.731945    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-ec8b82-openshift-default-65cf789494-k4s5q" event={"ID":"c016c6f5-99f6-400e-ac31-292ede5ae4dd","Type":"ContainerDied","Data":"b37a9fd4f2fa18ed669ec5c7715dd8057759eab59f3bb192844893bcd5560567"}
Sep 20 02:25:13.732347 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:13.732343    2584 scope.go:117] "RemoveContainer" containerID="b37a9fd4f2fa18ed669ec5c7715dd8057759eab59f3bb192844893bcd5560567"
Sep 20 02:25:13.733411 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:13.733380    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 20 02:25:13.733539 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:13.733473    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 20 02:25:13.733630 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:13.733552    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 20 02:25:14.739110 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:14.739079    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-ec8b82-openshift-default-65cf789494-k4s5q" event={"ID":"c016c6f5-99f6-400e-ac31-292ede5ae4dd","Type":"ContainerStarted","Data":"058977c9b6a552f0538b7d7b11f5c6d5de24d571b5c228521458b3cec83d0e6f"}
Sep 20 02:25:15.146429 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:15.146343    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-ec8b82-openshift-default-65cf789494-k4s5q"
Sep 20 02:25:15.147621 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:15.147594    2584 patch_prober.go:28] interesting pod/e2e-ipp-a-ec8b82-openshift-default-65cf789494-k4s5q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body=
Sep 20 02:25:15.147744 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:15.147644    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-ec8b82-openshift-default-65cf789494-k4s5q" podUID="c016c6f5-99f6-400e-ac31-292ede5ae4dd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused"
Sep 20 02:25:16.147288 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:16.147252    2584 patch_prober.go:28] interesting pod/e2e-ipp-a-ec8b82-openshift-default-65cf789494-k4s5q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body=
Sep 20 02:25:16.147730 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:16.147307    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-ec8b82-openshift-default-65cf789494-k4s5q" podUID="c016c6f5-99f6-400e-ac31-292ede5ae4dd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused"
Sep 20 02:25:17.146370 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:17.146330    2584 patch_prober.go:28] interesting pod/e2e-ipp-a-ec8b82-openshift-default-65cf789494-k4s5q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body=
Sep 20 02:25:17.146689 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:17.146392    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-ec8b82-openshift-default-65cf789494-k4s5q" podUID="c016c6f5-99f6-400e-ac31-292ede5ae4dd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused"
Sep 20 02:25:17.864237 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:17.864194    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:25:18.020594 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:18.020562    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:25:18.146991 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:18.146892    2584 patch_prober.go:28] interesting pod/e2e-ipp-a-ec8b82-openshift-default-65cf789494-k4s5q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body=
Sep 20 02:25:18.146991 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:18.146952    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-ec8b82-openshift-default-65cf789494-k4s5q" podUID="c016c6f5-99f6-400e-ac31-292ede5ae4dd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused"
Sep 20 02:25:18.660881 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:18.660800    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:25:19.147067 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:19.147032    2584 patch_prober.go:28] interesting pod/e2e-ipp-a-ec8b82-openshift-default-65cf789494-k4s5q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body=
Sep 20 02:25:19.147568 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:19.147091    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-ec8b82-openshift-default-65cf789494-k4s5q" podUID="c016c6f5-99f6-400e-ac31-292ede5ae4dd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused"
Sep 20 02:25:19.498378 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:19.498343    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:25:19.603970 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:19.603933    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-ec8b82-openshift-default-65cf789494-k4s5q"
Sep 20 02:25:20.225157 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:20.225124    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-ec8b82-openshift-default-65cf789494-k4s5q"
Sep 20 02:25:20.226171 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:20.226152    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-ec8b82-openshift-default-65cf789494-k4s5q"
Sep 20 02:25:20.434215 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:20.434186    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:25:20.907708 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:20.907677    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:25:21.663249 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:21.663218    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:25:22.085275 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:22.085238    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-6be25a-openshift-default-6796dcdc8-zzwz6"]
Sep 20 02:25:22.085688 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:22.085675    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="080142f0-79f1-4419-b2a9-ef917bc46884" containerName="main"
Sep 20 02:25:22.085740 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:22.085690    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="080142f0-79f1-4419-b2a9-ef917bc46884" containerName="main"
Sep 20 02:25:22.085780 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:22.085759    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="080142f0-79f1-4419-b2a9-ef917bc46884" containerName="main"
Sep 20 02:25:22.088786 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:22.088769    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-6be25a-openshift-default-6796dcdc8-zzwz6"
Sep 20 02:25:22.093992 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:22.093974    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-6be25a-openshift-default-dockercfg-bttq5\""
Sep 20 02:25:22.114499 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:22.114470    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-6be25a-openshift-default-6796dcdc8-zzwz6"]
Sep 20 02:25:22.148352 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:22.148308    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:25:22.148352 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:22.148322    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cb81a34c-3882-46fc-8436-c5c94ad03045-istiod-ca-cert\") pod \"e2e-ipp-b-6be25a-openshift-default-6796dcdc8-zzwz6\" (UID: \"cb81a34c-3882-46fc-8436-c5c94ad03045\") " pod="openshift-ingress/e2e-ipp-b-6be25a-openshift-default-6796dcdc8-zzwz6"
Sep 20 02:25:22.148590 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:22.148400    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb81a34c-3882-46fc-8436-c5c94ad03045-credential-socket\") pod \"e2e-ipp-b-6be25a-openshift-default-6796dcdc8-zzwz6\" (UID: \"cb81a34c-3882-46fc-8436-c5c94ad03045\") " pod="openshift-ingress/e2e-ipp-b-6be25a-openshift-default-6796dcdc8-zzwz6"
Sep 20 02:25:22.148590 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:22.148448    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb81a34c-3882-46fc-8436-c5c94ad03045-workload-socket\") pod \"e2e-ipp-b-6be25a-openshift-default-6796dcdc8-zzwz6\" (UID: \"cb81a34c-3882-46fc-8436-c5c94ad03045\") " pod="openshift-ingress/e2e-ipp-b-6be25a-openshift-default-6796dcdc8-zzwz6"
Sep 20 02:25:22.148590 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:22.148473    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cb81a34c-3882-46fc-8436-c5c94ad03045-workload-certs\") pod \"e2e-ipp-b-6be25a-openshift-default-6796dcdc8-zzwz6\" (UID: \"cb81a34c-3882-46fc-8436-c5c94ad03045\") " pod="openshift-ingress/e2e-ipp-b-6be25a-openshift-default-6796dcdc8-zzwz6"
Sep 20 02:25:22.148590 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:22.148567    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cb81a34c-3882-46fc-8436-c5c94ad03045-istio-podinfo\") pod \"e2e-ipp-b-6be25a-openshift-default-6796dcdc8-zzwz6\" (UID: \"cb81a34c-3882-46fc-8436-c5c94ad03045\") " pod="openshift-ingress/e2e-ipp-b-6be25a-openshift-default-6796dcdc8-zzwz6"
Sep 20 02:25:22.148742 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:22.148593    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cb81a34c-3882-46fc-8436-c5c94ad03045-istio-data\") pod \"e2e-ipp-b-6be25a-openshift-default-6796dcdc8-zzwz6\" (UID: \"cb81a34c-3882-46fc-8436-c5c94ad03045\") " pod="openshift-ingress/e2e-ipp-b-6be25a-openshift-default-6796dcdc8-zzwz6"
Sep 20 02:25:22.148742 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:22.148613    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cb81a34c-3882-46fc-8436-c5c94ad03045-istio-token\") pod \"e2e-ipp-b-6be25a-openshift-default-6796dcdc8-zzwz6\" (UID: \"cb81a34c-3882-46fc-8436-c5c94ad03045\") " pod="openshift-ingress/e2e-ipp-b-6be25a-openshift-default-6796dcdc8-zzwz6"
Sep 20 02:25:22.148742 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:22.148686    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cb81a34c-3882-46fc-8436-c5c94ad03045-istio-envoy\") pod \"e2e-ipp-b-6be25a-openshift-default-6796dcdc8-zzwz6\" (UID: \"cb81a34c-3882-46fc-8436-c5c94ad03045\") " pod="openshift-ingress/e2e-ipp-b-6be25a-openshift-default-6796dcdc8-zzwz6"
Sep 20 02:25:22.148742 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:22.148707    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6qc6r\" (UniqueName: \"kubernetes.io/projected/cb81a34c-3882-46fc-8436-c5c94ad03045-kube-api-access-6qc6r\") pod \"e2e-ipp-b-6be25a-openshift-default-6796dcdc8-zzwz6\" (UID: \"cb81a34c-3882-46fc-8436-c5c94ad03045\") " pod="openshift-ingress/e2e-ipp-b-6be25a-openshift-default-6796dcdc8-zzwz6"
Sep 20 02:25:22.249739 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:22.249703    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cb81a34c-3882-46fc-8436-c5c94ad03045-istio-envoy\") pod \"e2e-ipp-b-6be25a-openshift-default-6796dcdc8-zzwz6\" (UID: \"cb81a34c-3882-46fc-8436-c5c94ad03045\") " pod="openshift-ingress/e2e-ipp-b-6be25a-openshift-default-6796dcdc8-zzwz6"
Sep 20 02:25:22.249931 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:22.249749    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6qc6r\" (UniqueName: \"kubernetes.io/projected/cb81a34c-3882-46fc-8436-c5c94ad03045-kube-api-access-6qc6r\") pod \"e2e-ipp-b-6be25a-openshift-default-6796dcdc8-zzwz6\" (UID: \"cb81a34c-3882-46fc-8436-c5c94ad03045\") " pod="openshift-ingress/e2e-ipp-b-6be25a-openshift-default-6796dcdc8-zzwz6"
Sep 20 02:25:22.249931 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:22.249793    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cb81a34c-3882-46fc-8436-c5c94ad03045-istiod-ca-cert\") pod \"e2e-ipp-b-6be25a-openshift-default-6796dcdc8-zzwz6\" (UID: \"cb81a34c-3882-46fc-8436-c5c94ad03045\") " pod="openshift-ingress/e2e-ipp-b-6be25a-openshift-default-6796dcdc8-zzwz6"
Sep 20 02:25:22.249931 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:22.249822    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb81a34c-3882-46fc-8436-c5c94ad03045-credential-socket\") pod \"e2e-ipp-b-6be25a-openshift-default-6796dcdc8-zzwz6\" (UID: \"cb81a34c-3882-46fc-8436-c5c94ad03045\") " pod="openshift-ingress/e2e-ipp-b-6be25a-openshift-default-6796dcdc8-zzwz6"
Sep 20 02:25:22.249931 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:22.249884    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb81a34c-3882-46fc-8436-c5c94ad03045-workload-socket\") pod \"e2e-ipp-b-6be25a-openshift-default-6796dcdc8-zzwz6\" (UID: \"cb81a34c-3882-46fc-8436-c5c94ad03045\") " pod="openshift-ingress/e2e-ipp-b-6be25a-openshift-default-6796dcdc8-zzwz6"
Sep 20 02:25:22.249931 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:22.249919    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cb81a34c-3882-46fc-8436-c5c94ad03045-workload-certs\") pod \"e2e-ipp-b-6be25a-openshift-default-6796dcdc8-zzwz6\" (UID: \"cb81a34c-3882-46fc-8436-c5c94ad03045\") " pod="openshift-ingress/e2e-ipp-b-6be25a-openshift-default-6796dcdc8-zzwz6"
Sep 20 02:25:22.250201 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:22.249944    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cb81a34c-3882-46fc-8436-c5c94ad03045-istio-podinfo\") pod \"e2e-ipp-b-6be25a-openshift-default-6796dcdc8-zzwz6\" (UID: \"cb81a34c-3882-46fc-8436-c5c94ad03045\") " pod="openshift-ingress/e2e-ipp-b-6be25a-openshift-default-6796dcdc8-zzwz6"
Sep 20 02:25:22.250201 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:22.249973    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cb81a34c-3882-46fc-8436-c5c94ad03045-istio-data\") pod \"e2e-ipp-b-6be25a-openshift-default-6796dcdc8-zzwz6\" (UID: \"cb81a34c-3882-46fc-8436-c5c94ad03045\") " pod="openshift-ingress/e2e-ipp-b-6be25a-openshift-default-6796dcdc8-zzwz6"
Sep 20 02:25:22.250201 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:22.250002    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cb81a34c-3882-46fc-8436-c5c94ad03045-istio-token\") pod \"e2e-ipp-b-6be25a-openshift-default-6796dcdc8-zzwz6\" (UID: \"cb81a34c-3882-46fc-8436-c5c94ad03045\") " pod="openshift-ingress/e2e-ipp-b-6be25a-openshift-default-6796dcdc8-zzwz6"
Sep 20 02:25:22.250354 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:22.250219    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb81a34c-3882-46fc-8436-c5c94ad03045-credential-socket\") pod \"e2e-ipp-b-6be25a-openshift-default-6796dcdc8-zzwz6\" (UID: \"cb81a34c-3882-46fc-8436-c5c94ad03045\") " pod="openshift-ingress/e2e-ipp-b-6be25a-openshift-default-6796dcdc8-zzwz6"
Sep 20 02:25:22.250354 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:22.250236    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb81a34c-3882-46fc-8436-c5c94ad03045-workload-socket\") pod \"e2e-ipp-b-6be25a-openshift-default-6796dcdc8-zzwz6\" (UID: \"cb81a34c-3882-46fc-8436-c5c94ad03045\") " pod="openshift-ingress/e2e-ipp-b-6be25a-openshift-default-6796dcdc8-zzwz6"
Sep 20 02:25:22.250516 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:22.250494    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cb81a34c-3882-46fc-8436-c5c94ad03045-workload-certs\") pod \"e2e-ipp-b-6be25a-openshift-default-6796dcdc8-zzwz6\" (UID: \"cb81a34c-3882-46fc-8436-c5c94ad03045\") " pod="openshift-ingress/e2e-ipp-b-6be25a-openshift-default-6796dcdc8-zzwz6"
Sep 20 02:25:22.250613 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:22.250582    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cb81a34c-3882-46fc-8436-c5c94ad03045-istiod-ca-cert\") pod \"e2e-ipp-b-6be25a-openshift-default-6796dcdc8-zzwz6\" (UID: \"cb81a34c-3882-46fc-8436-c5c94ad03045\") " pod="openshift-ingress/e2e-ipp-b-6be25a-openshift-default-6796dcdc8-zzwz6"
Sep 20 02:25:22.250613 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:22.250602    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cb81a34c-3882-46fc-8436-c5c94ad03045-istio-data\") pod \"e2e-ipp-b-6be25a-openshift-default-6796dcdc8-zzwz6\" (UID: \"cb81a34c-3882-46fc-8436-c5c94ad03045\") " pod="openshift-ingress/e2e-ipp-b-6be25a-openshift-default-6796dcdc8-zzwz6"
Sep 20 02:25:22.251971 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:22.251949    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cb81a34c-3882-46fc-8436-c5c94ad03045-istio-envoy\") pod \"e2e-ipp-b-6be25a-openshift-default-6796dcdc8-zzwz6\" (UID: \"cb81a34c-3882-46fc-8436-c5c94ad03045\") " pod="openshift-ingress/e2e-ipp-b-6be25a-openshift-default-6796dcdc8-zzwz6"
Sep 20 02:25:22.252226 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:22.252207    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cb81a34c-3882-46fc-8436-c5c94ad03045-istio-podinfo\") pod \"e2e-ipp-b-6be25a-openshift-default-6796dcdc8-zzwz6\" (UID: \"cb81a34c-3882-46fc-8436-c5c94ad03045\") " pod="openshift-ingress/e2e-ipp-b-6be25a-openshift-default-6796dcdc8-zzwz6"
Sep 20 02:25:22.257705 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:22.257680    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:25:22.267582 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:22.267559    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cb81a34c-3882-46fc-8436-c5c94ad03045-istio-token\") pod \"e2e-ipp-b-6be25a-openshift-default-6796dcdc8-zzwz6\" (UID: \"cb81a34c-3882-46fc-8436-c5c94ad03045\") " pod="openshift-ingress/e2e-ipp-b-6be25a-openshift-default-6796dcdc8-zzwz6"
Sep 20 02:25:22.269381 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:22.269357    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6qc6r\" (UniqueName: \"kubernetes.io/projected/cb81a34c-3882-46fc-8436-c5c94ad03045-kube-api-access-6qc6r\") pod \"e2e-ipp-b-6be25a-openshift-default-6796dcdc8-zzwz6\" (UID: \"cb81a34c-3882-46fc-8436-c5c94ad03045\") " pod="openshift-ingress/e2e-ipp-b-6be25a-openshift-default-6796dcdc8-zzwz6"
Sep 20 02:25:22.398946 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:22.398865    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-6be25a-openshift-default-6796dcdc8-zzwz6"
Sep 20 02:25:22.703944 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:22.703909    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-6be25a-openshift-default-6796dcdc8-zzwz6"]
Sep 20 02:25:22.726743 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:22.726711    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 20 02:25:22.726832 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:22.726778    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 20 02:25:22.726832 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:22.726807    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 20 02:25:22.800546 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:22.800486    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-6be25a-openshift-default-6796dcdc8-zzwz6" event={"ID":"cb81a34c-3882-46fc-8436-c5c94ad03045","Type":"ContainerStarted","Data":"8c3f2b4a85049fe3f8f9ec916a0bacb5fe09b2f9f4308af0420a13c36356b6e6"}
Sep 20 02:25:22.800859 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:22.800572    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-6be25a-openshift-default-6796dcdc8-zzwz6" event={"ID":"cb81a34c-3882-46fc-8436-c5c94ad03045","Type":"ContainerStarted","Data":"32ba8685659a0c4057b42226426209bfac086e9511dc975ac6ab4acef08e8438"}
Sep 20 02:25:23.399509 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:23.399473    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-6be25a-openshift-default-6796dcdc8-zzwz6"
Sep 20 02:25:23.404475 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:23.404451    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-6be25a-openshift-default-6796dcdc8-zzwz6"
Sep 20 02:25:23.505091 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:23.505036    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-6be25a-openshift-default-6796dcdc8-zzwz6" podStartSLOduration=1.5050222130000002 podStartE2EDuration="1.505022213s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-20 02:25:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-20 02:25:22.849822687 +0000 UTC m=+1979.592501904" watchObservedRunningTime="2026-09-20 02:25:23.505022213 +0000 UTC m=+1980.247701417"
Sep 20 02:25:23.805877 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:23.805847    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-6be25a-openshift-default-6796dcdc8-zzwz6"
Sep 20 02:25:23.806780 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:23.806763    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-6be25a-openshift-default-6796dcdc8-zzwz6"
Sep 20 02:25:24.595734 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:24.595701    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:25:25.311245 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:25.311210    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:25:25.471978 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:25.471944    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:25:26.163758 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:26.163726    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:25:27.226695 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:27.226635    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:25:28.577222 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:28.577186    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:25:29.326560 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:29.326504    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-6be25a-5ddb98f8cf-5hh4g"]
Sep 20 02:25:29.331240 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:29.331220    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6be25a-5ddb98f8cf-5hh4g"
Sep 20 02:25:29.334110 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:29.334089    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-6be25a-serving-cert\""
Sep 20 02:25:29.349858 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:29.349831    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-6be25a-5ddb98f8cf-5hh4g"]
Sep 20 02:25:29.381911 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:29.381875    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:25:29.418981 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:29.418947    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xmp4h\" (UniqueName: \"kubernetes.io/projected/a81869b4-741f-43d7-a18c-041836710946-kube-api-access-xmp4h\") pod \"maas-api-e2e-ipp-b-6be25a-5ddb98f8cf-5hh4g\" (UID: \"a81869b4-741f-43d7-a18c-041836710946\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6be25a-5ddb98f8cf-5hh4g"
Sep 20 02:25:29.419135 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:29.419015    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a81869b4-741f-43d7-a18c-041836710946-metrics-certs\") pod \"maas-api-e2e-ipp-b-6be25a-5ddb98f8cf-5hh4g\" (UID: \"a81869b4-741f-43d7-a18c-041836710946\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6be25a-5ddb98f8cf-5hh4g"
Sep 20 02:25:29.419135 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:29.419086    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a81869b4-741f-43d7-a18c-041836710946-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-6be25a-5ddb98f8cf-5hh4g\" (UID: \"a81869b4-741f-43d7-a18c-041836710946\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6be25a-5ddb98f8cf-5hh4g"
Sep 20 02:25:29.419135 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:29.419117    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a81869b4-741f-43d7-a18c-041836710946-maas-api-tls\") pod \"maas-api-e2e-ipp-b-6be25a-5ddb98f8cf-5hh4g\" (UID: \"a81869b4-741f-43d7-a18c-041836710946\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6be25a-5ddb98f8cf-5hh4g"
Sep 20 02:25:29.486370 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:29.486333    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:25:29.520078 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:29.520051    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xmp4h\" (UniqueName: \"kubernetes.io/projected/a81869b4-741f-43d7-a18c-041836710946-kube-api-access-xmp4h\") pod \"maas-api-e2e-ipp-b-6be25a-5ddb98f8cf-5hh4g\" (UID: \"a81869b4-741f-43d7-a18c-041836710946\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6be25a-5ddb98f8cf-5hh4g"
Sep 20 02:25:29.520223 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:29.520122    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a81869b4-741f-43d7-a18c-041836710946-metrics-certs\") pod \"maas-api-e2e-ipp-b-6be25a-5ddb98f8cf-5hh4g\" (UID: \"a81869b4-741f-43d7-a18c-041836710946\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6be25a-5ddb98f8cf-5hh4g"
Sep 20 02:25:29.520223 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:29.520156    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a81869b4-741f-43d7-a18c-041836710946-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-6be25a-5ddb98f8cf-5hh4g\" (UID: \"a81869b4-741f-43d7-a18c-041836710946\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6be25a-5ddb98f8cf-5hh4g"
Sep 20 02:25:29.520223 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:29.520172    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a81869b4-741f-43d7-a18c-041836710946-maas-api-tls\") pod \"maas-api-e2e-ipp-b-6be25a-5ddb98f8cf-5hh4g\" (UID: \"a81869b4-741f-43d7-a18c-041836710946\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6be25a-5ddb98f8cf-5hh4g"
Sep 20 02:25:29.520349 ip-10-0-130-159 kubenswrapper[2584]: E0920 02:25:29.520290    2584 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-b-6be25a-serving-cert: secret "maas-api-e2e-ipp-b-6be25a-serving-cert" not found
Sep 20 02:25:29.520403 ip-10-0-130-159 kubenswrapper[2584]: E0920 02:25:29.520363    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a81869b4-741f-43d7-a18c-041836710946-maas-api-tls podName:a81869b4-741f-43d7-a18c-041836710946 nodeName:}" failed. No retries permitted until 2026-09-20 02:25:30.020346212 +0000 UTC m=+1986.763025394 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a81869b4-741f-43d7-a18c-041836710946-maas-api-tls") pod "maas-api-e2e-ipp-b-6be25a-5ddb98f8cf-5hh4g" (UID: "a81869b4-741f-43d7-a18c-041836710946") : secret "maas-api-e2e-ipp-b-6be25a-serving-cert" not found
Sep 20 02:25:29.520971 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:29.520947    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a81869b4-741f-43d7-a18c-041836710946-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-6be25a-5ddb98f8cf-5hh4g\" (UID: \"a81869b4-741f-43d7-a18c-041836710946\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6be25a-5ddb98f8cf-5hh4g"
Sep 20 02:25:29.522625 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:29.522596    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a81869b4-741f-43d7-a18c-041836710946-metrics-certs\") pod \"maas-api-e2e-ipp-b-6be25a-5ddb98f8cf-5hh4g\" (UID: \"a81869b4-741f-43d7-a18c-041836710946\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6be25a-5ddb98f8cf-5hh4g"
Sep 20 02:25:29.537868 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:29.537837    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xmp4h\" (UniqueName: \"kubernetes.io/projected/a81869b4-741f-43d7-a18c-041836710946-kube-api-access-xmp4h\") pod \"maas-api-e2e-ipp-b-6be25a-5ddb98f8cf-5hh4g\" (UID: \"a81869b4-741f-43d7-a18c-041836710946\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6be25a-5ddb98f8cf-5hh4g"
Sep 20 02:25:29.774201 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:29.774152    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:25:29.958510 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:29.958474    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:25:30.024909 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:30.024823    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a81869b4-741f-43d7-a18c-041836710946-maas-api-tls\") pod \"maas-api-e2e-ipp-b-6be25a-5ddb98f8cf-5hh4g\" (UID: \"a81869b4-741f-43d7-a18c-041836710946\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6be25a-5ddb98f8cf-5hh4g"
Sep 20 02:25:30.027234 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:30.027212    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a81869b4-741f-43d7-a18c-041836710946-maas-api-tls\") pod \"maas-api-e2e-ipp-b-6be25a-5ddb98f8cf-5hh4g\" (UID: \"a81869b4-741f-43d7-a18c-041836710946\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6be25a-5ddb98f8cf-5hh4g"
Sep 20 02:25:30.058894 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:30.058864    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:25:30.241444 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:30.241409    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6be25a-5ddb98f8cf-5hh4g"
Sep 20 02:25:30.524335 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:30.524289    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-6be25a-5ddb98f8cf-5hh4g"]
Sep 20 02:25:30.526765 ip-10-0-130-159 kubenswrapper[2584]: W0920 02:25:30.526741    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda81869b4_741f_43d7_a18c_041836710946.slice/crio-af2fcda1f979648c67579d4581332ca9db7b78376d92cf8931af5e73eccca336 WatchSource:0}: Error finding container af2fcda1f979648c67579d4581332ca9db7b78376d92cf8931af5e73eccca336: Status 404 returned error can't find the container with id af2fcda1f979648c67579d4581332ca9db7b78376d92cf8931af5e73eccca336
Sep 20 02:25:30.531277 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:30.531156    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:25:30.842658 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:30.842622    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6be25a-5ddb98f8cf-5hh4g" event={"ID":"a81869b4-741f-43d7-a18c-041836710946","Type":"ContainerStarted","Data":"af2fcda1f979648c67579d4581332ca9db7b78376d92cf8931af5e73eccca336"}
Sep 20 02:25:31.381122 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:31.381089    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:25:31.849256 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:31.849218    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6be25a-5ddb98f8cf-5hh4g" event={"ID":"a81869b4-741f-43d7-a18c-041836710946","Type":"ContainerStarted","Data":"d5100088dce49111e396d679fbb5c6beac91c57113e44470c55a6a08e94ffbac"}
Sep 20 02:25:31.849672 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:31.849395    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6be25a-5ddb98f8cf-5hh4g"
Sep 20 02:25:31.868775 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:31.868726    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6be25a-5ddb98f8cf-5hh4g" podStartSLOduration=2.540127393 podStartE2EDuration="2.868708909s" totalImagesPullingTime="328.581516ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-20 02:25:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-20 02:25:31.866819936 +0000 UTC m=+1988.609499142" watchObservedRunningTime="2026-09-20 02:25:31.868708909 +0000 UTC m=+1988.611388155"
Sep 20 02:25:37.876860 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:37.876824    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6be25a-5ddb98f8cf-5hh4g"
Sep 20 02:25:43.921973 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:43.921881    2584 generic.go:356] "Generic (PLEG): container finished" podID="cb81a34c-3882-46fc-8436-c5c94ad03045" containerID="8c3f2b4a85049fe3f8f9ec916a0bacb5fe09b2f9f4308af0420a13c36356b6e6" exitCode=0
Sep 20 02:25:43.921973 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:43.921930    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-6be25a-openshift-default-6796dcdc8-zzwz6" event={"ID":"cb81a34c-3882-46fc-8436-c5c94ad03045","Type":"ContainerDied","Data":"8c3f2b4a85049fe3f8f9ec916a0bacb5fe09b2f9f4308af0420a13c36356b6e6"}
Sep 20 02:25:43.922426 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:43.922295    2584 scope.go:117] "RemoveContainer" containerID="8c3f2b4a85049fe3f8f9ec916a0bacb5fe09b2f9f4308af0420a13c36356b6e6"
Sep 20 02:25:43.923353 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:43.923322    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 20 02:25:43.923405 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:43.923392    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 20 02:25:43.923445 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:43.923423    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 20 02:25:44.929241 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:44.929202    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-6be25a-openshift-default-6796dcdc8-zzwz6" event={"ID":"cb81a34c-3882-46fc-8436-c5c94ad03045","Type":"ContainerStarted","Data":"e4fd210fee8d33651706ffb2ec01da9f33d4af1f0998729ec8bfebe874285dc1"}
Sep 20 02:25:45.399005 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:45.398966    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-6be25a-openshift-default-6796dcdc8-zzwz6"
Sep 20 02:25:45.399269 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:45.399245    2584 patch_prober.go:28] interesting pod/e2e-ipp-b-6be25a-openshift-default-6796dcdc8-zzwz6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body=
Sep 20 02:25:45.399317 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:45.399294    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-6be25a-openshift-default-6796dcdc8-zzwz6" podUID="cb81a34c-3882-46fc-8436-c5c94ad03045" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused"
Sep 20 02:25:46.400125 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:46.400086    2584 patch_prober.go:28] interesting pod/e2e-ipp-b-6be25a-openshift-default-6796dcdc8-zzwz6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body=
Sep 20 02:25:46.400727 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:46.400150    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-6be25a-openshift-default-6796dcdc8-zzwz6" podUID="cb81a34c-3882-46fc-8436-c5c94ad03045" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused"
Sep 20 02:25:47.400251 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:47.400213    2584 patch_prober.go:28] interesting pod/e2e-ipp-b-6be25a-openshift-default-6796dcdc8-zzwz6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body=
Sep 20 02:25:47.400817 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:47.400278    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-6be25a-openshift-default-6796dcdc8-zzwz6" podUID="cb81a34c-3882-46fc-8436-c5c94ad03045" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused"
Sep 20 02:25:48.399405 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:48.399370    2584 patch_prober.go:28] interesting pod/e2e-ipp-b-6be25a-openshift-default-6796dcdc8-zzwz6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body=
Sep 20 02:25:48.399740 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:48.399431    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-6be25a-openshift-default-6796dcdc8-zzwz6" podUID="cb81a34c-3882-46fc-8436-c5c94ad03045" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused"
Sep 20 02:25:49.400115 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:49.400076    2584 patch_prober.go:28] interesting pod/e2e-ipp-b-6be25a-openshift-default-6796dcdc8-zzwz6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body=
Sep 20 02:25:49.400587 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:49.400137    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-6be25a-openshift-default-6796dcdc8-zzwz6" podUID="cb81a34c-3882-46fc-8436-c5c94ad03045" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused"
Sep 20 02:25:50.403891 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:50.403859    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-6be25a-openshift-default-6796dcdc8-zzwz6"
Sep 20 02:25:50.404401 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:50.404253    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-6be25a-openshift-default-6796dcdc8-zzwz6"
Sep 20 02:25:50.405058 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:50.405035    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-6be25a-openshift-default-6796dcdc8-zzwz6"
Sep 20 02:25:51.473200 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:51.473164    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:25:52.252298 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:52.252265    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:25:53.054834 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:53.054798    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:25:53.737135 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:53.737098    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:25:54.444631 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:54.444602    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:25:55.155448 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:25:55.155416    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:26:04.750156 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:26:04.750122    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:26:23.874895 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:26:23.874858    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:26:31.914255 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:26:31.914220    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:26:32.197638 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:26:32.197606    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-d26a0a-openshift-default-65d567cc84-47lv9"]
Sep 20 02:26:32.200872 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:26:32.200851    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-d26a0a-openshift-default-65d567cc84-47lv9"
Sep 20 02:26:32.203821 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:26:32.203800    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-cleanup-d26a0a-openshift-default-dockercfg-fk8rn\""
Sep 20 02:26:32.217647 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:26:32.217623    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-d26a0a-openshift-default-65d567cc84-47lv9"]
Sep 20 02:26:32.238566 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:26:32.238501    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:26:32.320936 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:26:32.320898    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f0bec830-952f-41b2-9077-8fd4b8c0cd89-workload-certs\") pod \"e2e-ipp-cleanup-d26a0a-openshift-default-65d567cc84-47lv9\" (UID: \"f0bec830-952f-41b2-9077-8fd4b8c0cd89\") " pod="openshift-ingress/e2e-ipp-cleanup-d26a0a-openshift-default-65d567cc84-47lv9"
Sep 20 02:26:32.321139 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:26:32.320974    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f0bec830-952f-41b2-9077-8fd4b8c0cd89-istio-envoy\") pod \"e2e-ipp-cleanup-d26a0a-openshift-default-65d567cc84-47lv9\" (UID: \"f0bec830-952f-41b2-9077-8fd4b8c0cd89\") " pod="openshift-ingress/e2e-ipp-cleanup-d26a0a-openshift-default-65d567cc84-47lv9"
Sep 20 02:26:32.321139 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:26:32.321030    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f0bec830-952f-41b2-9077-8fd4b8c0cd89-istio-token\") pod \"e2e-ipp-cleanup-d26a0a-openshift-default-65d567cc84-47lv9\" (UID: \"f0bec830-952f-41b2-9077-8fd4b8c0cd89\") " pod="openshift-ingress/e2e-ipp-cleanup-d26a0a-openshift-default-65d567cc84-47lv9"
Sep 20 02:26:32.321139 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:26:32.321077    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f0bec830-952f-41b2-9077-8fd4b8c0cd89-istiod-ca-cert\") pod \"e2e-ipp-cleanup-d26a0a-openshift-default-65d567cc84-47lv9\" (UID: \"f0bec830-952f-41b2-9077-8fd4b8c0cd89\") " pod="openshift-ingress/e2e-ipp-cleanup-d26a0a-openshift-default-65d567cc84-47lv9"
Sep 20 02:26:32.321139 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:26:32.321129    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0bec830-952f-41b2-9077-8fd4b8c0cd89-credential-socket\") pod \"e2e-ipp-cleanup-d26a0a-openshift-default-65d567cc84-47lv9\" (UID: \"f0bec830-952f-41b2-9077-8fd4b8c0cd89\") " pod="openshift-ingress/e2e-ipp-cleanup-d26a0a-openshift-default-65d567cc84-47lv9"
Sep 20 02:26:32.321308 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:26:32.321148    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f0bec830-952f-41b2-9077-8fd4b8c0cd89-istio-podinfo\") pod \"e2e-ipp-cleanup-d26a0a-openshift-default-65d567cc84-47lv9\" (UID: \"f0bec830-952f-41b2-9077-8fd4b8c0cd89\") " pod="openshift-ingress/e2e-ipp-cleanup-d26a0a-openshift-default-65d567cc84-47lv9"
Sep 20 02:26:32.321308 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:26:32.321184    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f0bec830-952f-41b2-9077-8fd4b8c0cd89-istio-data\") pod \"e2e-ipp-cleanup-d26a0a-openshift-default-65d567cc84-47lv9\" (UID: \"f0bec830-952f-41b2-9077-8fd4b8c0cd89\") " pod="openshift-ingress/e2e-ipp-cleanup-d26a0a-openshift-default-65d567cc84-47lv9"
Sep 20 02:26:32.321308 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:26:32.321207    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-85qwf\" (UniqueName: \"kubernetes.io/projected/f0bec830-952f-41b2-9077-8fd4b8c0cd89-kube-api-access-85qwf\") pod \"e2e-ipp-cleanup-d26a0a-openshift-default-65d567cc84-47lv9\" (UID: \"f0bec830-952f-41b2-9077-8fd4b8c0cd89\") " pod="openshift-ingress/e2e-ipp-cleanup-d26a0a-openshift-default-65d567cc84-47lv9"
Sep 20 02:26:32.321308 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:26:32.321223    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0bec830-952f-41b2-9077-8fd4b8c0cd89-workload-socket\") pod \"e2e-ipp-cleanup-d26a0a-openshift-default-65d567cc84-47lv9\" (UID: \"f0bec830-952f-41b2-9077-8fd4b8c0cd89\") " pod="openshift-ingress/e2e-ipp-cleanup-d26a0a-openshift-default-65d567cc84-47lv9"
Sep 20 02:26:32.422648 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:26:32.422615    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f0bec830-952f-41b2-9077-8fd4b8c0cd89-istio-envoy\") pod \"e2e-ipp-cleanup-d26a0a-openshift-default-65d567cc84-47lv9\" (UID: \"f0bec830-952f-41b2-9077-8fd4b8c0cd89\") " pod="openshift-ingress/e2e-ipp-cleanup-d26a0a-openshift-default-65d567cc84-47lv9"
Sep 20 02:26:32.422852 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:26:32.422668    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f0bec830-952f-41b2-9077-8fd4b8c0cd89-istio-token\") pod \"e2e-ipp-cleanup-d26a0a-openshift-default-65d567cc84-47lv9\" (UID: \"f0bec830-952f-41b2-9077-8fd4b8c0cd89\") " pod="openshift-ingress/e2e-ipp-cleanup-d26a0a-openshift-default-65d567cc84-47lv9"
Sep 20 02:26:32.422852 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:26:32.422708    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f0bec830-952f-41b2-9077-8fd4b8c0cd89-istiod-ca-cert\") pod \"e2e-ipp-cleanup-d26a0a-openshift-default-65d567cc84-47lv9\" (UID: \"f0bec830-952f-41b2-9077-8fd4b8c0cd89\") " pod="openshift-ingress/e2e-ipp-cleanup-d26a0a-openshift-default-65d567cc84-47lv9"
Sep 20 02:26:32.422852 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:26:32.422763    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0bec830-952f-41b2-9077-8fd4b8c0cd89-credential-socket\") pod \"e2e-ipp-cleanup-d26a0a-openshift-default-65d567cc84-47lv9\" (UID: \"f0bec830-952f-41b2-9077-8fd4b8c0cd89\") " pod="openshift-ingress/e2e-ipp-cleanup-d26a0a-openshift-default-65d567cc84-47lv9"
Sep 20 02:26:32.422852 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:26:32.422782    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f0bec830-952f-41b2-9077-8fd4b8c0cd89-istio-podinfo\") pod \"e2e-ipp-cleanup-d26a0a-openshift-default-65d567cc84-47lv9\" (UID: \"f0bec830-952f-41b2-9077-8fd4b8c0cd89\") " pod="openshift-ingress/e2e-ipp-cleanup-d26a0a-openshift-default-65d567cc84-47lv9"
Sep 20 02:26:32.422852 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:26:32.422818    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f0bec830-952f-41b2-9077-8fd4b8c0cd89-istio-data\") pod \"e2e-ipp-cleanup-d26a0a-openshift-default-65d567cc84-47lv9\" (UID: \"f0bec830-952f-41b2-9077-8fd4b8c0cd89\") " pod="openshift-ingress/e2e-ipp-cleanup-d26a0a-openshift-default-65d567cc84-47lv9"
Sep 20 02:26:32.422852 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:26:32.422847    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-85qwf\" (UniqueName: \"kubernetes.io/projected/f0bec830-952f-41b2-9077-8fd4b8c0cd89-kube-api-access-85qwf\") pod \"e2e-ipp-cleanup-d26a0a-openshift-default-65d567cc84-47lv9\" (UID: \"f0bec830-952f-41b2-9077-8fd4b8c0cd89\") " pod="openshift-ingress/e2e-ipp-cleanup-d26a0a-openshift-default-65d567cc84-47lv9"
Sep 20 02:26:32.423175 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:26:32.422875    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0bec830-952f-41b2-9077-8fd4b8c0cd89-workload-socket\") pod \"e2e-ipp-cleanup-d26a0a-openshift-default-65d567cc84-47lv9\" (UID: \"f0bec830-952f-41b2-9077-8fd4b8c0cd89\") " pod="openshift-ingress/e2e-ipp-cleanup-d26a0a-openshift-default-65d567cc84-47lv9"
Sep 20 02:26:32.423175 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:26:32.422934    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f0bec830-952f-41b2-9077-8fd4b8c0cd89-workload-certs\") pod \"e2e-ipp-cleanup-d26a0a-openshift-default-65d567cc84-47lv9\" (UID: \"f0bec830-952f-41b2-9077-8fd4b8c0cd89\") " pod="openshift-ingress/e2e-ipp-cleanup-d26a0a-openshift-default-65d567cc84-47lv9"
Sep 20 02:26:32.423175 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:26:32.423138    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0bec830-952f-41b2-9077-8fd4b8c0cd89-credential-socket\") pod \"e2e-ipp-cleanup-d26a0a-openshift-default-65d567cc84-47lv9\" (UID: \"f0bec830-952f-41b2-9077-8fd4b8c0cd89\") " pod="openshift-ingress/e2e-ipp-cleanup-d26a0a-openshift-default-65d567cc84-47lv9"
Sep 20 02:26:32.423374 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:26:32.423356    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f0bec830-952f-41b2-9077-8fd4b8c0cd89-workload-certs\") pod \"e2e-ipp-cleanup-d26a0a-openshift-default-65d567cc84-47lv9\" (UID: \"f0bec830-952f-41b2-9077-8fd4b8c0cd89\") " pod="openshift-ingress/e2e-ipp-cleanup-d26a0a-openshift-default-65d567cc84-47lv9"
Sep 20 02:26:32.423437 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:26:32.423383    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0bec830-952f-41b2-9077-8fd4b8c0cd89-workload-socket\") pod \"e2e-ipp-cleanup-d26a0a-openshift-default-65d567cc84-47lv9\" (UID: \"f0bec830-952f-41b2-9077-8fd4b8c0cd89\") " pod="openshift-ingress/e2e-ipp-cleanup-d26a0a-openshift-default-65d567cc84-47lv9"
Sep 20 02:26:32.423437 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:26:32.423413    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f0bec830-952f-41b2-9077-8fd4b8c0cd89-istio-data\") pod \"e2e-ipp-cleanup-d26a0a-openshift-default-65d567cc84-47lv9\" (UID: \"f0bec830-952f-41b2-9077-8fd4b8c0cd89\") " pod="openshift-ingress/e2e-ipp-cleanup-d26a0a-openshift-default-65d567cc84-47lv9"
Sep 20 02:26:32.423552 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:26:32.423472    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f0bec830-952f-41b2-9077-8fd4b8c0cd89-istiod-ca-cert\") pod \"e2e-ipp-cleanup-d26a0a-openshift-default-65d567cc84-47lv9\" (UID: \"f0bec830-952f-41b2-9077-8fd4b8c0cd89\") " pod="openshift-ingress/e2e-ipp-cleanup-d26a0a-openshift-default-65d567cc84-47lv9"
Sep 20 02:26:32.425431 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:26:32.425405    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f0bec830-952f-41b2-9077-8fd4b8c0cd89-istio-envoy\") pod \"e2e-ipp-cleanup-d26a0a-openshift-default-65d567cc84-47lv9\" (UID: \"f0bec830-952f-41b2-9077-8fd4b8c0cd89\") " pod="openshift-ingress/e2e-ipp-cleanup-d26a0a-openshift-default-65d567cc84-47lv9"
Sep 20 02:26:32.425669 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:26:32.425651    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f0bec830-952f-41b2-9077-8fd4b8c0cd89-istio-podinfo\") pod \"e2e-ipp-cleanup-d26a0a-openshift-default-65d567cc84-47lv9\" (UID: \"f0bec830-952f-41b2-9077-8fd4b8c0cd89\") " pod="openshift-ingress/e2e-ipp-cleanup-d26a0a-openshift-default-65d567cc84-47lv9"
Sep 20 02:26:32.430704 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:26:32.430682    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f0bec830-952f-41b2-9077-8fd4b8c0cd89-istio-token\") pod \"e2e-ipp-cleanup-d26a0a-openshift-default-65d567cc84-47lv9\" (UID: \"f0bec830-952f-41b2-9077-8fd4b8c0cd89\") " pod="openshift-ingress/e2e-ipp-cleanup-d26a0a-openshift-default-65d567cc84-47lv9"
Sep 20 02:26:32.430894 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:26:32.430877    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-85qwf\" (UniqueName: \"kubernetes.io/projected/f0bec830-952f-41b2-9077-8fd4b8c0cd89-kube-api-access-85qwf\") pod \"e2e-ipp-cleanup-d26a0a-openshift-default-65d567cc84-47lv9\" (UID: \"f0bec830-952f-41b2-9077-8fd4b8c0cd89\") " pod="openshift-ingress/e2e-ipp-cleanup-d26a0a-openshift-default-65d567cc84-47lv9"
Sep 20 02:26:32.511104 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:26:32.511018    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-d26a0a-openshift-default-65d567cc84-47lv9"
Sep 20 02:26:32.769343 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:26:32.769270    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:26:32.794900 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:26:32.794873    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-d26a0a-openshift-default-65d567cc84-47lv9"]
Sep 20 02:26:32.796708 ip-10-0-130-159 kubenswrapper[2584]: W0920 02:26:32.796682    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0bec830_952f_41b2_9077_8fd4b8c0cd89.slice/crio-5ab741228ade6ea49cd9dd67b2b08da9cb1192a46cc15dae5af49491c2702dab WatchSource:0}: Error finding container 5ab741228ade6ea49cd9dd67b2b08da9cb1192a46cc15dae5af49491c2702dab: Status 404 returned error can't find the container with id 5ab741228ade6ea49cd9dd67b2b08da9cb1192a46cc15dae5af49491c2702dab
Sep 20 02:26:32.799287 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:26:32.799255    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 20 02:26:32.799366 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:26:32.799327    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 20 02:26:32.799366 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:26:32.799355    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 20 02:26:33.178405 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:26:33.178313    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-d26a0a-openshift-default-65d567cc84-47lv9" event={"ID":"f0bec830-952f-41b2-9077-8fd4b8c0cd89","Type":"ContainerStarted","Data":"a5bd04482688be1129e37468d9526c596b8dfe3639e86a8036b8e0efe7b82b37"}
Sep 20 02:26:33.178405 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:26:33.178380    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-d26a0a-openshift-default-65d567cc84-47lv9" event={"ID":"f0bec830-952f-41b2-9077-8fd4b8c0cd89","Type":"ContainerStarted","Data":"5ab741228ade6ea49cd9dd67b2b08da9cb1192a46cc15dae5af49491c2702dab"}
Sep 20 02:26:33.202149 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:26:33.202088    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-cleanup-d26a0a-openshift-default-65d567cc84-47lv9" podStartSLOduration=1.202074364 podStartE2EDuration="1.202074364s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-20 02:26:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-20 02:26:33.200050727 +0000 UTC m=+2049.942729932" watchObservedRunningTime="2026-09-20 02:26:33.202074364 +0000 UTC m=+2049.944753568"
Sep 20 02:26:33.511703 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:26:33.511665    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-d26a0a-openshift-default-65d567cc84-47lv9"
Sep 20 02:26:33.517187 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:26:33.517160    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-d26a0a-openshift-default-65d567cc84-47lv9"
Sep 20 02:26:34.183926 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:26:34.183888    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-d26a0a-openshift-default-65d567cc84-47lv9"
Sep 20 02:26:34.184907 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:26:34.184886    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-d26a0a-openshift-default-65d567cc84-47lv9"
Sep 20 02:26:34.457966 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:26:34.457936    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:26:35.125495 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:26:35.125460    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:26:35.217876 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:26:35.217839    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:26:36.268886 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:26:36.268848    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:26:36.970435 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:26:36.970394    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:26:37.674777 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:26:37.674741    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:26:38.505960 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:26:38.505867    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d26a0a-7d8b574f66-tkx9n"]
Sep 20 02:26:38.511217 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:26:38.511190    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d26a0a-7d8b574f66-tkx9n"
Sep 20 02:26:38.513332 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:26:38.513315    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-d26a0a-serving-cert\""
Sep 20 02:26:38.517809 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:26:38.517733    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d26a0a-7d8b574f66-tkx9n"]
Sep 20 02:26:38.535683 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:26:38.535659    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:26:38.584799 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:26:38.584765    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f8d7a70a-30fd-499d-b042-3ffe40191520-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-d26a0a-7d8b574f66-tkx9n\" (UID: \"f8d7a70a-30fd-499d-b042-3ffe40191520\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d26a0a-7d8b574f66-tkx9n"
Sep 20 02:26:38.584972 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:26:38.584830    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f8d7a70a-30fd-499d-b042-3ffe40191520-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-d26a0a-7d8b574f66-tkx9n\" (UID: \"f8d7a70a-30fd-499d-b042-3ffe40191520\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d26a0a-7d8b574f66-tkx9n"
Sep 20 02:26:38.584972 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:26:38.584857    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9lwrk\" (UniqueName: \"kubernetes.io/projected/f8d7a70a-30fd-499d-b042-3ffe40191520-kube-api-access-9lwrk\") pod \"maas-api-e2e-ipp-cleanup-d26a0a-7d8b574f66-tkx9n\" (UID: \"f8d7a70a-30fd-499d-b042-3ffe40191520\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d26a0a-7d8b574f66-tkx9n"
Sep 20 02:26:38.584972 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:26:38.584892    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f8d7a70a-30fd-499d-b042-3ffe40191520-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-d26a0a-7d8b574f66-tkx9n\" (UID: \"f8d7a70a-30fd-499d-b042-3ffe40191520\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d26a0a-7d8b574f66-tkx9n"
Sep 20 02:26:38.638130 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:26:38.638094    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:26:38.686294 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:26:38.686261    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f8d7a70a-30fd-499d-b042-3ffe40191520-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-d26a0a-7d8b574f66-tkx9n\" (UID: \"f8d7a70a-30fd-499d-b042-3ffe40191520\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d26a0a-7d8b574f66-tkx9n"
Sep 20 02:26:38.686783 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:26:38.686324    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f8d7a70a-30fd-499d-b042-3ffe40191520-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-d26a0a-7d8b574f66-tkx9n\" (UID: \"f8d7a70a-30fd-499d-b042-3ffe40191520\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d26a0a-7d8b574f66-tkx9n"
Sep 20 02:26:38.686783 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:26:38.686350    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9lwrk\" (UniqueName: \"kubernetes.io/projected/f8d7a70a-30fd-499d-b042-3ffe40191520-kube-api-access-9lwrk\") pod \"maas-api-e2e-ipp-cleanup-d26a0a-7d8b574f66-tkx9n\" (UID: \"f8d7a70a-30fd-499d-b042-3ffe40191520\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d26a0a-7d8b574f66-tkx9n"
Sep 20 02:26:38.686783 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:26:38.686381    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f8d7a70a-30fd-499d-b042-3ffe40191520-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-d26a0a-7d8b574f66-tkx9n\" (UID: \"f8d7a70a-30fd-499d-b042-3ffe40191520\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d26a0a-7d8b574f66-tkx9n"
Sep 20 02:26:38.686783 ip-10-0-130-159 kubenswrapper[2584]: E0920 02:26:38.686489    2584 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d26a0a-serving-cert: secret "maas-api-e2e-ipp-cleanup-d26a0a-serving-cert" not found
Sep 20 02:26:38.686783 ip-10-0-130-159 kubenswrapper[2584]: E0920 02:26:38.686611    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f8d7a70a-30fd-499d-b042-3ffe40191520-maas-api-tls podName:f8d7a70a-30fd-499d-b042-3ffe40191520 nodeName:}" failed. No retries permitted until 2026-09-20 02:26:39.186589259 +0000 UTC m=+2055.929268446 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f8d7a70a-30fd-499d-b042-3ffe40191520-maas-api-tls") pod "maas-api-e2e-ipp-cleanup-d26a0a-7d8b574f66-tkx9n" (UID: "f8d7a70a-30fd-499d-b042-3ffe40191520") : secret "maas-api-e2e-ipp-cleanup-d26a0a-serving-cert" not found
Sep 20 02:26:38.687021 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:26:38.686998    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f8d7a70a-30fd-499d-b042-3ffe40191520-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-d26a0a-7d8b574f66-tkx9n\" (UID: \"f8d7a70a-30fd-499d-b042-3ffe40191520\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d26a0a-7d8b574f66-tkx9n"
Sep 20 02:26:38.688719 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:26:38.688702    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f8d7a70a-30fd-499d-b042-3ffe40191520-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-d26a0a-7d8b574f66-tkx9n\" (UID: \"f8d7a70a-30fd-499d-b042-3ffe40191520\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d26a0a-7d8b574f66-tkx9n"
Sep 20 02:26:38.694450 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:26:38.694430    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9lwrk\" (UniqueName: \"kubernetes.io/projected/f8d7a70a-30fd-499d-b042-3ffe40191520-kube-api-access-9lwrk\") pod \"maas-api-e2e-ipp-cleanup-d26a0a-7d8b574f66-tkx9n\" (UID: \"f8d7a70a-30fd-499d-b042-3ffe40191520\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d26a0a-7d8b574f66-tkx9n"
Sep 20 02:26:38.996820 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:26:38.996789    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:26:39.096464 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:26:39.096430    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:26:39.191978 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:26:39.191940    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f8d7a70a-30fd-499d-b042-3ffe40191520-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-d26a0a-7d8b574f66-tkx9n\" (UID: \"f8d7a70a-30fd-499d-b042-3ffe40191520\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d26a0a-7d8b574f66-tkx9n"
Sep 20 02:26:39.194414 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:26:39.194390    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f8d7a70a-30fd-499d-b042-3ffe40191520-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-d26a0a-7d8b574f66-tkx9n\" (UID: \"f8d7a70a-30fd-499d-b042-3ffe40191520\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d26a0a-7d8b574f66-tkx9n"
Sep 20 02:26:39.425974 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:26:39.425894    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d26a0a-7d8b574f66-tkx9n"
Sep 20 02:26:39.705078 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:26:39.705047    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d26a0a-7d8b574f66-tkx9n"]
Sep 20 02:26:39.706602 ip-10-0-130-159 kubenswrapper[2584]: W0920 02:26:39.706579    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8d7a70a_30fd_499d_b042_3ffe40191520.slice/crio-72ae641cca1ce67d66af7b3b0b585045204d5528ee587cda5234b39b7a4930cd WatchSource:0}: Error finding container 72ae641cca1ce67d66af7b3b0b585045204d5528ee587cda5234b39b7a4930cd: Status 404 returned error can't find the container with id 72ae641cca1ce67d66af7b3b0b585045204d5528ee587cda5234b39b7a4930cd
Sep 20 02:26:40.219302 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:26:40.219266    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d26a0a-7d8b574f66-tkx9n" event={"ID":"f8d7a70a-30fd-499d-b042-3ffe40191520","Type":"ContainerStarted","Data":"72ae641cca1ce67d66af7b3b0b585045204d5528ee587cda5234b39b7a4930cd"}
Sep 20 02:26:41.226796 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:26:41.226757    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d26a0a-7d8b574f66-tkx9n" event={"ID":"f8d7a70a-30fd-499d-b042-3ffe40191520","Type":"ContainerStarted","Data":"ceb51ab6769b1a4c63ef652643d968fd655769a6cfccba125426dcc11a2e97ab"}
Sep 20 02:26:41.227280 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:26:41.226891    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d26a0a-7d8b574f66-tkx9n"
Sep 20 02:26:41.245370 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:26:41.245322    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d26a0a-7d8b574f66-tkx9n" podStartSLOduration=2.720743045 podStartE2EDuration="3.24530863s" totalImagesPullingTime="524.565585ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-20 02:26:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-20 02:26:41.244146249 +0000 UTC m=+2057.986825454" watchObservedRunningTime="2026-09-20 02:26:41.24530863 +0000 UTC m=+2057.987987835"
Sep 20 02:26:47.238651 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:26:47.238618    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d26a0a-7d8b574f66-tkx9n"
Sep 20 02:26:56.321211 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:26:56.321172    2584 generic.go:356] "Generic (PLEG): container finished" podID="f0bec830-952f-41b2-9077-8fd4b8c0cd89" containerID="a5bd04482688be1129e37468d9526c596b8dfe3639e86a8036b8e0efe7b82b37" exitCode=0
Sep 20 02:26:56.321780 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:26:56.321248    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-d26a0a-openshift-default-65d567cc84-47lv9" event={"ID":"f0bec830-952f-41b2-9077-8fd4b8c0cd89","Type":"ContainerDied","Data":"a5bd04482688be1129e37468d9526c596b8dfe3639e86a8036b8e0efe7b82b37"}
Sep 20 02:26:56.321914 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:26:56.321815    2584 scope.go:117] "RemoveContainer" containerID="a5bd04482688be1129e37468d9526c596b8dfe3639e86a8036b8e0efe7b82b37"
Sep 20 02:26:56.323053 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:26:56.323026    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 20 02:26:56.323115 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:26:56.323096    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 20 02:26:56.323152 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:26:56.323125    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 20 02:26:57.329600 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:26:57.329555    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-d26a0a-openshift-default-65d567cc84-47lv9" event={"ID":"f0bec830-952f-41b2-9077-8fd4b8c0cd89","Type":"ContainerStarted","Data":"20f9cd885a804b96f34a739880ed26008947fd15c1b0c4915d70f852329f8255"}
Sep 20 02:26:57.511374 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:26:57.511334    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-d26a0a-openshift-default-65d567cc84-47lv9"
Sep 20 02:26:57.511617 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:26:57.511591    2584 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-d26a0a-openshift-default-65d567cc84-47lv9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused" start-of-body=
Sep 20 02:26:57.511711 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:26:57.511648    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-d26a0a-openshift-default-65d567cc84-47lv9" podUID="f0bec830-952f-41b2-9077-8fd4b8c0cd89" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused"
Sep 20 02:26:58.512089 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:26:58.512038    2584 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-d26a0a-openshift-default-65d567cc84-47lv9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused" start-of-body=
Sep 20 02:26:58.512500 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:26:58.512127    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-d26a0a-openshift-default-65d567cc84-47lv9" podUID="f0bec830-952f-41b2-9077-8fd4b8c0cd89" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused"
Sep 20 02:26:59.512142 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:26:59.512103    2584 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-d26a0a-openshift-default-65d567cc84-47lv9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused" start-of-body=
Sep 20 02:26:59.512582 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:26:59.512174    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-d26a0a-openshift-default-65d567cc84-47lv9" podUID="f0bec830-952f-41b2-9077-8fd4b8c0cd89" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused"
Sep 20 02:27:00.512354 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:00.512308    2584 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-d26a0a-openshift-default-65d567cc84-47lv9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused" start-of-body=
Sep 20 02:27:00.512806 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:00.512380    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-d26a0a-openshift-default-65d567cc84-47lv9" podUID="f0bec830-952f-41b2-9077-8fd4b8c0cd89" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused"
Sep 20 02:27:00.935067 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:00.934949    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:27:01.512277 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:01.512236    2584 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-d26a0a-openshift-default-65d567cc84-47lv9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused" start-of-body=
Sep 20 02:27:01.512458 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:01.512315    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-d26a0a-openshift-default-65d567cc84-47lv9" podUID="f0bec830-952f-41b2-9077-8fd4b8c0cd89" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused"
Sep 20 02:27:01.666495 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:01.666452    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:27:02.487440 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:02.487393    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:27:02.516161 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:02.516129    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-d26a0a-openshift-default-65d567cc84-47lv9"
Sep 20 02:27:02.516696 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:02.516554    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-d26a0a-openshift-default-65d567cc84-47lv9"
Sep 20 02:27:02.517203 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:02.517185    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-d26a0a-openshift-default-65d567cc84-47lv9"
Sep 20 02:27:03.159146 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:03.159112    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:27:03.946870 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:03.946836    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:27:04.759768 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:04.759732    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:27:05.311942 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:05.311911    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:27:06.736643 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:06.736611    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:27:07.419083 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:07.419053    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:27:08.263830 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:08.263798    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:27:08.980897 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:08.980863    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:27:09.842718 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:09.842681    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:27:10.635406 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:10.635372    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:27:16.955814 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:16.955778    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d26a0a-7d8b574f66-tkx9n"]
Sep 20 02:27:16.956311 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:16.956043    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d26a0a-7d8b574f66-tkx9n" podUID="f8d7a70a-30fd-499d-b042-3ffe40191520" containerName="maas-api" containerID="cri-o://ceb51ab6769b1a4c63ef652643d968fd655769a6cfccba125426dcc11a2e97ab" gracePeriod=30
Sep 20 02:27:16.987915 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:16.987884    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:27:17.071266 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:17.071231    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:27:17.176461 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:17.176409    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:27:17.220009 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:17.219946    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d26a0a-7d8b574f66-tkx9n"
Sep 20 02:27:17.269458 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:17.269422    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:27:17.296317 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:17.296280    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f8d7a70a-30fd-499d-b042-3ffe40191520-metrics-certs\") pod \"f8d7a70a-30fd-499d-b042-3ffe40191520\" (UID: \"f8d7a70a-30fd-499d-b042-3ffe40191520\") "
Sep 20 02:27:17.296490 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:17.296352    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f8d7a70a-30fd-499d-b042-3ffe40191520-maas-api-tls\") pod \"f8d7a70a-30fd-499d-b042-3ffe40191520\" (UID: \"f8d7a70a-30fd-499d-b042-3ffe40191520\") "
Sep 20 02:27:17.296490 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:17.296412    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f8d7a70a-30fd-499d-b042-3ffe40191520-openshift-service-ca\") pod \"f8d7a70a-30fd-499d-b042-3ffe40191520\" (UID: \"f8d7a70a-30fd-499d-b042-3ffe40191520\") "
Sep 20 02:27:17.296490 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:17.296439    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9lwrk\" (UniqueName: \"kubernetes.io/projected/f8d7a70a-30fd-499d-b042-3ffe40191520-kube-api-access-9lwrk\") pod \"f8d7a70a-30fd-499d-b042-3ffe40191520\" (UID: \"f8d7a70a-30fd-499d-b042-3ffe40191520\") "
Sep 20 02:27:17.296799 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:17.296774    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f8d7a70a-30fd-499d-b042-3ffe40191520-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f8d7a70a-30fd-499d-b042-3ffe40191520" (UID: "f8d7a70a-30fd-499d-b042-3ffe40191520"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 20 02:27:17.298727 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:17.298686    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f8d7a70a-30fd-499d-b042-3ffe40191520-kube-api-access-9lwrk" (OuterVolumeSpecName: "kube-api-access-9lwrk") pod "f8d7a70a-30fd-499d-b042-3ffe40191520" (UID: "f8d7a70a-30fd-499d-b042-3ffe40191520"). InnerVolumeSpecName "kube-api-access-9lwrk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 20 02:27:17.298845 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:17.298725    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f8d7a70a-30fd-499d-b042-3ffe40191520-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "f8d7a70a-30fd-499d-b042-3ffe40191520" (UID: "f8d7a70a-30fd-499d-b042-3ffe40191520"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 20 02:27:17.298845 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:17.298704    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f8d7a70a-30fd-499d-b042-3ffe40191520-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f8d7a70a-30fd-499d-b042-3ffe40191520" (UID: "f8d7a70a-30fd-499d-b042-3ffe40191520"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 20 02:27:17.397751 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:17.397718    2584 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f8d7a70a-30fd-499d-b042-3ffe40191520-metrics-certs\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:27:17.397751 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:17.397748    2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f8d7a70a-30fd-499d-b042-3ffe40191520-maas-api-tls\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:27:17.397751 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:17.397760    2584 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f8d7a70a-30fd-499d-b042-3ffe40191520-openshift-service-ca\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:27:17.398043 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:17.397772    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9lwrk\" (UniqueName: \"kubernetes.io/projected/f8d7a70a-30fd-499d-b042-3ffe40191520-kube-api-access-9lwrk\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:27:17.438933 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:17.438900    2584 generic.go:356] "Generic (PLEG): container finished" podID="f8d7a70a-30fd-499d-b042-3ffe40191520" containerID="ceb51ab6769b1a4c63ef652643d968fd655769a6cfccba125426dcc11a2e97ab" exitCode=0
Sep 20 02:27:17.439110 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:17.438951    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d26a0a-7d8b574f66-tkx9n"
Sep 20 02:27:17.439110 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:17.438987    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d26a0a-7d8b574f66-tkx9n" event={"ID":"f8d7a70a-30fd-499d-b042-3ffe40191520","Type":"ContainerDied","Data":"ceb51ab6769b1a4c63ef652643d968fd655769a6cfccba125426dcc11a2e97ab"}
Sep 20 02:27:17.439110 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:17.439046    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d26a0a-7d8b574f66-tkx9n" event={"ID":"f8d7a70a-30fd-499d-b042-3ffe40191520","Type":"ContainerDied","Data":"72ae641cca1ce67d66af7b3b0b585045204d5528ee587cda5234b39b7a4930cd"}
Sep 20 02:27:17.439110 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:17.439077    2584 scope.go:117] "RemoveContainer" containerID="ceb51ab6769b1a4c63ef652643d968fd655769a6cfccba125426dcc11a2e97ab"
Sep 20 02:27:17.451357 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:17.451337    2584 scope.go:117] "RemoveContainer" containerID="ceb51ab6769b1a4c63ef652643d968fd655769a6cfccba125426dcc11a2e97ab"
Sep 20 02:27:17.451695 ip-10-0-130-159 kubenswrapper[2584]: E0920 02:27:17.451673    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ceb51ab6769b1a4c63ef652643d968fd655769a6cfccba125426dcc11a2e97ab\": container with ID starting with ceb51ab6769b1a4c63ef652643d968fd655769a6cfccba125426dcc11a2e97ab not found: ID does not exist" containerID="ceb51ab6769b1a4c63ef652643d968fd655769a6cfccba125426dcc11a2e97ab"
Sep 20 02:27:17.451781 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:17.451706    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ceb51ab6769b1a4c63ef652643d968fd655769a6cfccba125426dcc11a2e97ab"} err="failed to get container status \"ceb51ab6769b1a4c63ef652643d968fd655769a6cfccba125426dcc11a2e97ab\": rpc error: code = NotFound desc = could not find container \"ceb51ab6769b1a4c63ef652643d968fd655769a6cfccba125426dcc11a2e97ab\": container with ID starting with ceb51ab6769b1a4c63ef652643d968fd655769a6cfccba125426dcc11a2e97ab not found: ID does not exist"
Sep 20 02:27:17.464244 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:17.464215    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d26a0a-7d8b574f66-tkx9n"]
Sep 20 02:27:17.469619 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:17.469592    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d26a0a-7d8b574f66-tkx9n"]
Sep 20 02:27:17.634459 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:17.634375    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:27:17.734412 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:17.734377    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:27:18.404222 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:18.404188    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:27:19.099367 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:19.099332    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f8d7a70a-30fd-499d-b042-3ffe40191520" path="/var/lib/kubelet/pods/f8d7a70a-30fd-499d-b042-3ffe40191520/volumes"
Sep 20 02:27:19.109282 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:19.109253    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:27:20.048871 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:20.048825    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:27:20.753363 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:20.753327    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:27:23.339275 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:23.339239    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:27:23.539348 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:23.539318    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:27:23.597170 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:23.597068    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-d26a0a-openshift-default-65d567cc84-47lv9"]
Sep 20 02:27:23.597356 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:23.597325    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-cleanup-d26a0a-openshift-default-65d567cc84-47lv9" podUID="f0bec830-952f-41b2-9077-8fd4b8c0cd89" containerName="istio-proxy" containerID="cri-o://20f9cd885a804b96f34a739880ed26008947fd15c1b0c4915d70f852329f8255" gracePeriod=30
Sep 20 02:27:23.621376 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:23.621345    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:27:25.264838 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:25.264800    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-ztdt2_ece82f19-68b5-4810-be50-d8678cfdee02/console-operator/1.log"
Sep 20 02:27:25.272111 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:25.272087    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-ztdt2_ece82f19-68b5-4810-be50-d8678cfdee02/console-operator/1.log"
Sep 20 02:27:27.648300 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:27.648258    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:27:28.372294 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:28.372242    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:27:28.857072 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:28.857048    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-d26a0a-openshift-default-65d567cc84-47lv9"
Sep 20 02:27:29.004208 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:29.004132    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f0bec830-952f-41b2-9077-8fd4b8c0cd89-istiod-ca-cert\") pod \"f0bec830-952f-41b2-9077-8fd4b8c0cd89\" (UID: \"f0bec830-952f-41b2-9077-8fd4b8c0cd89\") "
Sep 20 02:27:29.004208 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:29.004190    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f0bec830-952f-41b2-9077-8fd4b8c0cd89-istio-envoy\") pod \"f0bec830-952f-41b2-9077-8fd4b8c0cd89\" (UID: \"f0bec830-952f-41b2-9077-8fd4b8c0cd89\") "
Sep 20 02:27:29.004411 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:29.004245    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0bec830-952f-41b2-9077-8fd4b8c0cd89-credential-socket\") pod \"f0bec830-952f-41b2-9077-8fd4b8c0cd89\" (UID: \"f0bec830-952f-41b2-9077-8fd4b8c0cd89\") "
Sep 20 02:27:29.004411 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:29.004293    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f0bec830-952f-41b2-9077-8fd4b8c0cd89-workload-certs\") pod \"f0bec830-952f-41b2-9077-8fd4b8c0cd89\" (UID: \"f0bec830-952f-41b2-9077-8fd4b8c0cd89\") "
Sep 20 02:27:29.004411 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:29.004324    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-85qwf\" (UniqueName: \"kubernetes.io/projected/f0bec830-952f-41b2-9077-8fd4b8c0cd89-kube-api-access-85qwf\") pod \"f0bec830-952f-41b2-9077-8fd4b8c0cd89\" (UID: \"f0bec830-952f-41b2-9077-8fd4b8c0cd89\") "
Sep 20 02:27:29.004411 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:29.004368    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f0bec830-952f-41b2-9077-8fd4b8c0cd89-istio-podinfo\") pod \"f0bec830-952f-41b2-9077-8fd4b8c0cd89\" (UID: \"f0bec830-952f-41b2-9077-8fd4b8c0cd89\") "
Sep 20 02:27:29.004411 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:29.004397    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f0bec830-952f-41b2-9077-8fd4b8c0cd89-istio-token\") pod \"f0bec830-952f-41b2-9077-8fd4b8c0cd89\" (UID: \"f0bec830-952f-41b2-9077-8fd4b8c0cd89\") "
Sep 20 02:27:29.004721 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:29.004431    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0bec830-952f-41b2-9077-8fd4b8c0cd89-workload-socket\") pod \"f0bec830-952f-41b2-9077-8fd4b8c0cd89\" (UID: \"f0bec830-952f-41b2-9077-8fd4b8c0cd89\") "
Sep 20 02:27:29.004721 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:29.004509    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f0bec830-952f-41b2-9077-8fd4b8c0cd89-istio-data\") pod \"f0bec830-952f-41b2-9077-8fd4b8c0cd89\" (UID: \"f0bec830-952f-41b2-9077-8fd4b8c0cd89\") "
Sep 20 02:27:29.004721 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:29.004597    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0bec830-952f-41b2-9077-8fd4b8c0cd89-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f0bec830-952f-41b2-9077-8fd4b8c0cd89" (UID: "f0bec830-952f-41b2-9077-8fd4b8c0cd89"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 20 02:27:29.004721 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:29.004620    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0bec830-952f-41b2-9077-8fd4b8c0cd89-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f0bec830-952f-41b2-9077-8fd4b8c0cd89" (UID: "f0bec830-952f-41b2-9077-8fd4b8c0cd89"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 20 02:27:29.004935 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:29.004826    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0bec830-952f-41b2-9077-8fd4b8c0cd89-istio-data" (OuterVolumeSpecName: "istio-data") pod "f0bec830-952f-41b2-9077-8fd4b8c0cd89" (UID: "f0bec830-952f-41b2-9077-8fd4b8c0cd89"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 20 02:27:29.004935 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:29.004848    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0bec830-952f-41b2-9077-8fd4b8c0cd89-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f0bec830-952f-41b2-9077-8fd4b8c0cd89" (UID: "f0bec830-952f-41b2-9077-8fd4b8c0cd89"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 20 02:27:29.004935 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:29.004912    2584 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0bec830-952f-41b2-9077-8fd4b8c0cd89-credential-socket\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:27:29.004935 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:29.004929    2584 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f0bec830-952f-41b2-9077-8fd4b8c0cd89-workload-certs\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:27:29.005157 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:29.004942    2584 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f0bec830-952f-41b2-9077-8fd4b8c0cd89-istio-data\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:27:29.005157 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:29.004964    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f0bec830-952f-41b2-9077-8fd4b8c0cd89-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f0bec830-952f-41b2-9077-8fd4b8c0cd89" (UID: "f0bec830-952f-41b2-9077-8fd4b8c0cd89"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 20 02:27:29.006621 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:29.006583    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0bec830-952f-41b2-9077-8fd4b8c0cd89-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f0bec830-952f-41b2-9077-8fd4b8c0cd89" (UID: "f0bec830-952f-41b2-9077-8fd4b8c0cd89"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 20 02:27:29.006745 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:29.006686    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f0bec830-952f-41b2-9077-8fd4b8c0cd89-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f0bec830-952f-41b2-9077-8fd4b8c0cd89" (UID: "f0bec830-952f-41b2-9077-8fd4b8c0cd89"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 20 02:27:29.006745 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:29.006715    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f0bec830-952f-41b2-9077-8fd4b8c0cd89-istio-token" (OuterVolumeSpecName: "istio-token") pod "f0bec830-952f-41b2-9077-8fd4b8c0cd89" (UID: "f0bec830-952f-41b2-9077-8fd4b8c0cd89"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 20 02:27:29.006885 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:29.006864    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f0bec830-952f-41b2-9077-8fd4b8c0cd89-kube-api-access-85qwf" (OuterVolumeSpecName: "kube-api-access-85qwf") pod "f0bec830-952f-41b2-9077-8fd4b8c0cd89" (UID: "f0bec830-952f-41b2-9077-8fd4b8c0cd89"). InnerVolumeSpecName "kube-api-access-85qwf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 20 02:27:29.105423 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:29.105396    2584 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f0bec830-952f-41b2-9077-8fd4b8c0cd89-istiod-ca-cert\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:27:29.105423 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:29.105422    2584 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f0bec830-952f-41b2-9077-8fd4b8c0cd89-istio-envoy\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:27:29.105633 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:29.105436    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-85qwf\" (UniqueName: \"kubernetes.io/projected/f0bec830-952f-41b2-9077-8fd4b8c0cd89-kube-api-access-85qwf\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:27:29.105633 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:29.105449    2584 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f0bec830-952f-41b2-9077-8fd4b8c0cd89-istio-podinfo\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:27:29.105633 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:29.105461    2584 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f0bec830-952f-41b2-9077-8fd4b8c0cd89-istio-token\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:27:29.105633 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:29.105472    2584 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0bec830-952f-41b2-9077-8fd4b8c0cd89-workload-socket\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:27:29.288725 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:29.288614    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:27:29.504487 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:29.504451    2584 generic.go:356] "Generic (PLEG): container finished" podID="f0bec830-952f-41b2-9077-8fd4b8c0cd89" containerID="20f9cd885a804b96f34a739880ed26008947fd15c1b0c4915d70f852329f8255" exitCode=0
Sep 20 02:27:29.504686 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:29.504498    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-d26a0a-openshift-default-65d567cc84-47lv9"
Sep 20 02:27:29.504686 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:29.504548    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-d26a0a-openshift-default-65d567cc84-47lv9" event={"ID":"f0bec830-952f-41b2-9077-8fd4b8c0cd89","Type":"ContainerDied","Data":"20f9cd885a804b96f34a739880ed26008947fd15c1b0c4915d70f852329f8255"}
Sep 20 02:27:29.504686 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:29.504605    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-d26a0a-openshift-default-65d567cc84-47lv9" event={"ID":"f0bec830-952f-41b2-9077-8fd4b8c0cd89","Type":"ContainerDied","Data":"5ab741228ade6ea49cd9dd67b2b08da9cb1192a46cc15dae5af49491c2702dab"}
Sep 20 02:27:29.504686 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:29.504620    2584 scope.go:117] "RemoveContainer" containerID="20f9cd885a804b96f34a739880ed26008947fd15c1b0c4915d70f852329f8255"
Sep 20 02:27:29.514373 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:29.514352    2584 scope.go:117] "RemoveContainer" containerID="a5bd04482688be1129e37468d9526c596b8dfe3639e86a8036b8e0efe7b82b37"
Sep 20 02:27:29.523062 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:29.523039    2584 scope.go:117] "RemoveContainer" containerID="20f9cd885a804b96f34a739880ed26008947fd15c1b0c4915d70f852329f8255"
Sep 20 02:27:29.523418 ip-10-0-130-159 kubenswrapper[2584]: E0920 02:27:29.523400    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"20f9cd885a804b96f34a739880ed26008947fd15c1b0c4915d70f852329f8255\": container with ID starting with 20f9cd885a804b96f34a739880ed26008947fd15c1b0c4915d70f852329f8255 not found: ID does not exist" containerID="20f9cd885a804b96f34a739880ed26008947fd15c1b0c4915d70f852329f8255"
Sep 20 02:27:29.523483 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:29.523428    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"20f9cd885a804b96f34a739880ed26008947fd15c1b0c4915d70f852329f8255"} err="failed to get container status \"20f9cd885a804b96f34a739880ed26008947fd15c1b0c4915d70f852329f8255\": rpc error: code = NotFound desc = could not find container \"20f9cd885a804b96f34a739880ed26008947fd15c1b0c4915d70f852329f8255\": container with ID starting with 20f9cd885a804b96f34a739880ed26008947fd15c1b0c4915d70f852329f8255 not found: ID does not exist"
Sep 20 02:27:29.523483 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:29.523446    2584 scope.go:117] "RemoveContainer" containerID="a5bd04482688be1129e37468d9526c596b8dfe3639e86a8036b8e0efe7b82b37"
Sep 20 02:27:29.523766 ip-10-0-130-159 kubenswrapper[2584]: E0920 02:27:29.523750    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a5bd04482688be1129e37468d9526c596b8dfe3639e86a8036b8e0efe7b82b37\": container with ID starting with a5bd04482688be1129e37468d9526c596b8dfe3639e86a8036b8e0efe7b82b37 not found: ID does not exist" containerID="a5bd04482688be1129e37468d9526c596b8dfe3639e86a8036b8e0efe7b82b37"
Sep 20 02:27:29.523823 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:29.523771    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a5bd04482688be1129e37468d9526c596b8dfe3639e86a8036b8e0efe7b82b37"} err="failed to get container status \"a5bd04482688be1129e37468d9526c596b8dfe3639e86a8036b8e0efe7b82b37\": rpc error: code = NotFound desc = could not find container \"a5bd04482688be1129e37468d9526c596b8dfe3639e86a8036b8e0efe7b82b37\": container with ID starting with a5bd04482688be1129e37468d9526c596b8dfe3639e86a8036b8e0efe7b82b37 not found: ID does not exist"
Sep 20 02:27:29.524077 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:29.524055    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-d26a0a-openshift-default-65d567cc84-47lv9"]
Sep 20 02:27:29.530627 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:29.530596    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-d26a0a-openshift-default-65d567cc84-47lv9"]
Sep 20 02:27:30.148710 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:30.148680    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:27:31.002122 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:31.002089    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:27:31.099792 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:31.099755    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f0bec830-952f-41b2-9077-8fd4b8c0cd89" path="/var/lib/kubelet/pods/f0bec830-952f-41b2-9077-8fd4b8c0cd89/volumes"
Sep 20 02:27:31.756215 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:31.756181    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:27:41.227265 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:41.227225    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:27:42.956988 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:42.956958    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-ec8b82-59584946fc-5j586"]
Sep 20 02:27:42.957772 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:42.957739    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-ec8b82-59584946fc-5j586" podUID="3020f8fa-d45a-48b9-8ce9-f9af5b0321dc" containerName="maas-api" containerID="cri-o://b08c52e3f3163fc042063e07c63c1485d948aa7a94ae0360052ab53762c1df78" gracePeriod=30
Sep 20 02:27:42.988273 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:42.988236    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:27:43.055201 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:43.055165    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:27:43.155442 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:43.155404    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:27:43.200603 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:43.200571    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-ec8b82-59584946fc-5j586"
Sep 20 02:27:43.226195 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:43.226116    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:27:43.231073 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:43.231049    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3020f8fa-d45a-48b9-8ce9-f9af5b0321dc-openshift-service-ca\") pod \"3020f8fa-d45a-48b9-8ce9-f9af5b0321dc\" (UID: \"3020f8fa-d45a-48b9-8ce9-f9af5b0321dc\") "
Sep 20 02:27:43.231204 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:43.231093    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3020f8fa-d45a-48b9-8ce9-f9af5b0321dc-maas-api-tls\") pod \"3020f8fa-d45a-48b9-8ce9-f9af5b0321dc\" (UID: \"3020f8fa-d45a-48b9-8ce9-f9af5b0321dc\") "
Sep 20 02:27:43.231254 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:43.231209    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3020f8fa-d45a-48b9-8ce9-f9af5b0321dc-metrics-certs\") pod \"3020f8fa-d45a-48b9-8ce9-f9af5b0321dc\" (UID: \"3020f8fa-d45a-48b9-8ce9-f9af5b0321dc\") "
Sep 20 02:27:43.231322 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:43.231302    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7x724\" (UniqueName: \"kubernetes.io/projected/3020f8fa-d45a-48b9-8ce9-f9af5b0321dc-kube-api-access-7x724\") pod \"3020f8fa-d45a-48b9-8ce9-f9af5b0321dc\" (UID: \"3020f8fa-d45a-48b9-8ce9-f9af5b0321dc\") "
Sep 20 02:27:43.231481 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:43.231459    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3020f8fa-d45a-48b9-8ce9-f9af5b0321dc-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "3020f8fa-d45a-48b9-8ce9-f9af5b0321dc" (UID: "3020f8fa-d45a-48b9-8ce9-f9af5b0321dc"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 20 02:27:43.231691 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:43.231662    2584 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3020f8fa-d45a-48b9-8ce9-f9af5b0321dc-openshift-service-ca\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:27:43.233532 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:43.233493    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3020f8fa-d45a-48b9-8ce9-f9af5b0321dc-kube-api-access-7x724" (OuterVolumeSpecName: "kube-api-access-7x724") pod "3020f8fa-d45a-48b9-8ce9-f9af5b0321dc" (UID: "3020f8fa-d45a-48b9-8ce9-f9af5b0321dc"). InnerVolumeSpecName "kube-api-access-7x724". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 20 02:27:43.233630 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:43.233498    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3020f8fa-d45a-48b9-8ce9-f9af5b0321dc-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "3020f8fa-d45a-48b9-8ce9-f9af5b0321dc" (UID: "3020f8fa-d45a-48b9-8ce9-f9af5b0321dc"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 20 02:27:43.233630 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:43.233555    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3020f8fa-d45a-48b9-8ce9-f9af5b0321dc-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "3020f8fa-d45a-48b9-8ce9-f9af5b0321dc" (UID: "3020f8fa-d45a-48b9-8ce9-f9af5b0321dc"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 20 02:27:43.333665 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:43.333628    2584 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3020f8fa-d45a-48b9-8ce9-f9af5b0321dc-metrics-certs\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:27:43.333846 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:43.333661    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7x724\" (UniqueName: \"kubernetes.io/projected/3020f8fa-d45a-48b9-8ce9-f9af5b0321dc-kube-api-access-7x724\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:27:43.333846 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:43.333773    2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3020f8fa-d45a-48b9-8ce9-f9af5b0321dc-maas-api-tls\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:27:43.581583 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:43.581475    2584 generic.go:356] "Generic (PLEG): container finished" podID="3020f8fa-d45a-48b9-8ce9-f9af5b0321dc" containerID="b08c52e3f3163fc042063e07c63c1485d948aa7a94ae0360052ab53762c1df78" exitCode=0
Sep 20 02:27:43.581583 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:43.581545    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-ec8b82-59584946fc-5j586" event={"ID":"3020f8fa-d45a-48b9-8ce9-f9af5b0321dc","Type":"ContainerDied","Data":"b08c52e3f3163fc042063e07c63c1485d948aa7a94ae0360052ab53762c1df78"}
Sep 20 02:27:43.581583 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:43.581568    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-ec8b82-59584946fc-5j586"
Sep 20 02:27:43.581809 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:43.581623    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-ec8b82-59584946fc-5j586" event={"ID":"3020f8fa-d45a-48b9-8ce9-f9af5b0321dc","Type":"ContainerDied","Data":"55efcc3f6050605184c8dd7ba24d4d0b1454625a4765073d555d7b66d75115b9"}
Sep 20 02:27:43.581809 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:43.581646    2584 scope.go:117] "RemoveContainer" containerID="b08c52e3f3163fc042063e07c63c1485d948aa7a94ae0360052ab53762c1df78"
Sep 20 02:27:43.585721 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:43.585697    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:27:43.591799 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:43.591783    2584 scope.go:117] "RemoveContainer" containerID="b08c52e3f3163fc042063e07c63c1485d948aa7a94ae0360052ab53762c1df78"
Sep 20 02:27:43.592040 ip-10-0-130-159 kubenswrapper[2584]: E0920 02:27:43.592026    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b08c52e3f3163fc042063e07c63c1485d948aa7a94ae0360052ab53762c1df78\": container with ID starting with b08c52e3f3163fc042063e07c63c1485d948aa7a94ae0360052ab53762c1df78 not found: ID does not exist" containerID="b08c52e3f3163fc042063e07c63c1485d948aa7a94ae0360052ab53762c1df78"
Sep 20 02:27:43.592103 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:43.592046    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b08c52e3f3163fc042063e07c63c1485d948aa7a94ae0360052ab53762c1df78"} err="failed to get container status \"b08c52e3f3163fc042063e07c63c1485d948aa7a94ae0360052ab53762c1df78\": rpc error: code = NotFound desc = could not find container \"b08c52e3f3163fc042063e07c63c1485d948aa7a94ae0360052ab53762c1df78\": container with ID starting with b08c52e3f3163fc042063e07c63c1485d948aa7a94ae0360052ab53762c1df78 not found: ID does not exist"
Sep 20 02:27:43.602778 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:43.602729    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-ec8b82-59584946fc-5j586"]
Sep 20 02:27:43.608282 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:43.608259    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-ec8b82-59584946fc-5j586"]
Sep 20 02:27:43.685735 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:43.685707    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:27:44.334764 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:44.334724    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:27:45.050887 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:45.050857    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:27:45.092800 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:45.092767    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3020f8fa-d45a-48b9-8ce9-f9af5b0321dc" path="/var/lib/kubelet/pods/3020f8fa-d45a-48b9-8ce9-f9af5b0321dc/volumes"
Sep 20 02:27:45.848681 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:45.848650    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:27:48.663808 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:27:48.663774    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:28:25.112784 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:28:25.112753    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:28:25.804718 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:28:25.804620    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:28:31.390006 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:28:31.389972    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:28:37.193422 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:28:37.193381    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:28:37.293060 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:28:37.293025    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:28:37.445962 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:28:37.445888    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-ec8b82-openshift-default-65cf789494-k4s5q"]
Sep 20 02:28:37.446133 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:28:37.446107    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-ec8b82-openshift-default-65cf789494-k4s5q" podUID="c016c6f5-99f6-400e-ac31-292ede5ae4dd" containerName="istio-proxy" containerID="cri-o://058977c9b6a552f0538b7d7b11f5c6d5de24d571b5c228521458b3cec83d0e6f" gracePeriod=30
Sep 20 02:28:37.473416 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:28:37.473390    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:28:38.123572 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:28:38.123536    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:28:38.960490 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:28:38.960459    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:28:39.677723 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:28:39.677682    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:28:40.530664 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:28:40.530635    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:28:41.244594 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:28:41.244560    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:28:42.696624 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:28:42.696597    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-ec8b82-openshift-default-65cf789494-k4s5q"
Sep 20 02:28:42.782656 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:28:42.782582    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c016c6f5-99f6-400e-ac31-292ede5ae4dd-credential-socket\") pod \"c016c6f5-99f6-400e-ac31-292ede5ae4dd\" (UID: \"c016c6f5-99f6-400e-ac31-292ede5ae4dd\") "
Sep 20 02:28:42.782656 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:28:42.782645    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c016c6f5-99f6-400e-ac31-292ede5ae4dd-istiod-ca-cert\") pod \"c016c6f5-99f6-400e-ac31-292ede5ae4dd\" (UID: \"c016c6f5-99f6-400e-ac31-292ede5ae4dd\") "
Sep 20 02:28:42.782882 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:28:42.782677    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c016c6f5-99f6-400e-ac31-292ede5ae4dd-istio-token\") pod \"c016c6f5-99f6-400e-ac31-292ede5ae4dd\" (UID: \"c016c6f5-99f6-400e-ac31-292ede5ae4dd\") "
Sep 20 02:28:42.782882 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:28:42.782694    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c016c6f5-99f6-400e-ac31-292ede5ae4dd-workload-certs\") pod \"c016c6f5-99f6-400e-ac31-292ede5ae4dd\" (UID: \"c016c6f5-99f6-400e-ac31-292ede5ae4dd\") "
Sep 20 02:28:42.782882 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:28:42.782720    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5jqfp\" (UniqueName: \"kubernetes.io/projected/c016c6f5-99f6-400e-ac31-292ede5ae4dd-kube-api-access-5jqfp\") pod \"c016c6f5-99f6-400e-ac31-292ede5ae4dd\" (UID: \"c016c6f5-99f6-400e-ac31-292ede5ae4dd\") "
Sep 20 02:28:42.782882 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:28:42.782792    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c016c6f5-99f6-400e-ac31-292ede5ae4dd-workload-socket\") pod \"c016c6f5-99f6-400e-ac31-292ede5ae4dd\" (UID: \"c016c6f5-99f6-400e-ac31-292ede5ae4dd\") "
Sep 20 02:28:42.782882 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:28:42.782840    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c016c6f5-99f6-400e-ac31-292ede5ae4dd-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c016c6f5-99f6-400e-ac31-292ede5ae4dd" (UID: "c016c6f5-99f6-400e-ac31-292ede5ae4dd"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 20 02:28:42.782882 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:28:42.782868    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c016c6f5-99f6-400e-ac31-292ede5ae4dd-istio-podinfo\") pod \"c016c6f5-99f6-400e-ac31-292ede5ae4dd\" (UID: \"c016c6f5-99f6-400e-ac31-292ede5ae4dd\") "
Sep 20 02:28:42.783213 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:28:42.782899    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c016c6f5-99f6-400e-ac31-292ede5ae4dd-istio-data\") pod \"c016c6f5-99f6-400e-ac31-292ede5ae4dd\" (UID: \"c016c6f5-99f6-400e-ac31-292ede5ae4dd\") "
Sep 20 02:28:42.783213 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:28:42.782927    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c016c6f5-99f6-400e-ac31-292ede5ae4dd-istio-envoy\") pod \"c016c6f5-99f6-400e-ac31-292ede5ae4dd\" (UID: \"c016c6f5-99f6-400e-ac31-292ede5ae4dd\") "
Sep 20 02:28:42.783213 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:28:42.782996    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c016c6f5-99f6-400e-ac31-292ede5ae4dd-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c016c6f5-99f6-400e-ac31-292ede5ae4dd" (UID: "c016c6f5-99f6-400e-ac31-292ede5ae4dd"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 20 02:28:42.783213 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:28:42.783060    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c016c6f5-99f6-400e-ac31-292ede5ae4dd-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c016c6f5-99f6-400e-ac31-292ede5ae4dd" (UID: "c016c6f5-99f6-400e-ac31-292ede5ae4dd"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 20 02:28:42.783213 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:28:42.783083    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c016c6f5-99f6-400e-ac31-292ede5ae4dd-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c016c6f5-99f6-400e-ac31-292ede5ae4dd" (UID: "c016c6f5-99f6-400e-ac31-292ede5ae4dd"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 20 02:28:42.783476 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:28:42.783269    2584 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c016c6f5-99f6-400e-ac31-292ede5ae4dd-credential-socket\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:28:42.783476 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:28:42.783316    2584 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c016c6f5-99f6-400e-ac31-292ede5ae4dd-istiod-ca-cert\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:28:42.783476 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:28:42.783332    2584 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c016c6f5-99f6-400e-ac31-292ede5ae4dd-workload-certs\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:28:42.783476 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:28:42.783344    2584 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c016c6f5-99f6-400e-ac31-292ede5ae4dd-workload-socket\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:28:42.783476 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:28:42.783424    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c016c6f5-99f6-400e-ac31-292ede5ae4dd-istio-data" (OuterVolumeSpecName: "istio-data") pod "c016c6f5-99f6-400e-ac31-292ede5ae4dd" (UID: "c016c6f5-99f6-400e-ac31-292ede5ae4dd"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 20 02:28:42.785093 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:28:42.785058    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c016c6f5-99f6-400e-ac31-292ede5ae4dd-kube-api-access-5jqfp" (OuterVolumeSpecName: "kube-api-access-5jqfp") pod "c016c6f5-99f6-400e-ac31-292ede5ae4dd" (UID: "c016c6f5-99f6-400e-ac31-292ede5ae4dd"). InnerVolumeSpecName "kube-api-access-5jqfp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 20 02:28:42.785246 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:28:42.785220    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c016c6f5-99f6-400e-ac31-292ede5ae4dd-istio-token" (OuterVolumeSpecName: "istio-token") pod "c016c6f5-99f6-400e-ac31-292ede5ae4dd" (UID: "c016c6f5-99f6-400e-ac31-292ede5ae4dd"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 20 02:28:42.785506 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:28:42.785486    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c016c6f5-99f6-400e-ac31-292ede5ae4dd-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c016c6f5-99f6-400e-ac31-292ede5ae4dd" (UID: "c016c6f5-99f6-400e-ac31-292ede5ae4dd"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 20 02:28:42.785693 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:28:42.785479    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c016c6f5-99f6-400e-ac31-292ede5ae4dd-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c016c6f5-99f6-400e-ac31-292ede5ae4dd" (UID: "c016c6f5-99f6-400e-ac31-292ede5ae4dd"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 20 02:28:42.884506 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:28:42.884470    2584 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c016c6f5-99f6-400e-ac31-292ede5ae4dd-istio-envoy\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:28:42.884506 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:28:42.884501    2584 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c016c6f5-99f6-400e-ac31-292ede5ae4dd-istio-token\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:28:42.884726 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:28:42.884515    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5jqfp\" (UniqueName: \"kubernetes.io/projected/c016c6f5-99f6-400e-ac31-292ede5ae4dd-kube-api-access-5jqfp\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:28:42.884726 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:28:42.884560    2584 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c016c6f5-99f6-400e-ac31-292ede5ae4dd-istio-podinfo\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:28:42.884726 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:28:42.884573    2584 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c016c6f5-99f6-400e-ac31-292ede5ae4dd-istio-data\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:28:42.889178 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:28:42.889148    2584 generic.go:356] "Generic (PLEG): container finished" podID="c016c6f5-99f6-400e-ac31-292ede5ae4dd" containerID="058977c9b6a552f0538b7d7b11f5c6d5de24d571b5c228521458b3cec83d0e6f" exitCode=0
Sep 20 02:28:42.889380 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:28:42.889196    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-ec8b82-openshift-default-65cf789494-k4s5q"
Sep 20 02:28:42.889380 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:28:42.889237    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-ec8b82-openshift-default-65cf789494-k4s5q" event={"ID":"c016c6f5-99f6-400e-ac31-292ede5ae4dd","Type":"ContainerDied","Data":"058977c9b6a552f0538b7d7b11f5c6d5de24d571b5c228521458b3cec83d0e6f"}
Sep 20 02:28:42.889380 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:28:42.889340    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-ec8b82-openshift-default-65cf789494-k4s5q" event={"ID":"c016c6f5-99f6-400e-ac31-292ede5ae4dd","Type":"ContainerDied","Data":"3f21ba664888756572fa9ca2b0bca04f8ca6a461dee1d01f31a2d0ec3051ce6e"}
Sep 20 02:28:42.889380 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:28:42.889373    2584 scope.go:117] "RemoveContainer" containerID="058977c9b6a552f0538b7d7b11f5c6d5de24d571b5c228521458b3cec83d0e6f"
Sep 20 02:28:42.900122 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:28:42.900048    2584 scope.go:117] "RemoveContainer" containerID="b37a9fd4f2fa18ed669ec5c7715dd8057759eab59f3bb192844893bcd5560567"
Sep 20 02:28:42.909206 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:28:42.909189    2584 scope.go:117] "RemoveContainer" containerID="058977c9b6a552f0538b7d7b11f5c6d5de24d571b5c228521458b3cec83d0e6f"
Sep 20 02:28:42.909472 ip-10-0-130-159 kubenswrapper[2584]: E0920 02:28:42.909454    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"058977c9b6a552f0538b7d7b11f5c6d5de24d571b5c228521458b3cec83d0e6f\": container with ID starting with 058977c9b6a552f0538b7d7b11f5c6d5de24d571b5c228521458b3cec83d0e6f not found: ID does not exist" containerID="058977c9b6a552f0538b7d7b11f5c6d5de24d571b5c228521458b3cec83d0e6f"
Sep 20 02:28:42.909533 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:28:42.909481    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"058977c9b6a552f0538b7d7b11f5c6d5de24d571b5c228521458b3cec83d0e6f"} err="failed to get container status \"058977c9b6a552f0538b7d7b11f5c6d5de24d571b5c228521458b3cec83d0e6f\": rpc error: code = NotFound desc = could not find container \"058977c9b6a552f0538b7d7b11f5c6d5de24d571b5c228521458b3cec83d0e6f\": container with ID starting with 058977c9b6a552f0538b7d7b11f5c6d5de24d571b5c228521458b3cec83d0e6f not found: ID does not exist"
Sep 20 02:28:42.909533 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:28:42.909496    2584 scope.go:117] "RemoveContainer" containerID="b37a9fd4f2fa18ed669ec5c7715dd8057759eab59f3bb192844893bcd5560567"
Sep 20 02:28:42.909765 ip-10-0-130-159 kubenswrapper[2584]: E0920 02:28:42.909747    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b37a9fd4f2fa18ed669ec5c7715dd8057759eab59f3bb192844893bcd5560567\": container with ID starting with b37a9fd4f2fa18ed669ec5c7715dd8057759eab59f3bb192844893bcd5560567 not found: ID does not exist" containerID="b37a9fd4f2fa18ed669ec5c7715dd8057759eab59f3bb192844893bcd5560567"
Sep 20 02:28:42.909810 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:28:42.909769    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b37a9fd4f2fa18ed669ec5c7715dd8057759eab59f3bb192844893bcd5560567"} err="failed to get container status \"b37a9fd4f2fa18ed669ec5c7715dd8057759eab59f3bb192844893bcd5560567\": rpc error: code = NotFound desc = could not find container \"b37a9fd4f2fa18ed669ec5c7715dd8057759eab59f3bb192844893bcd5560567\": container with ID starting with b37a9fd4f2fa18ed669ec5c7715dd8057759eab59f3bb192844893bcd5560567 not found: ID does not exist"
Sep 20 02:28:42.912400 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:28:42.912379    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-ec8b82-openshift-default-65cf789494-k4s5q"]
Sep 20 02:28:42.916215 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:28:42.916194    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-ec8b82-openshift-default-65cf789494-k4s5q"]
Sep 20 02:28:43.093551 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:28:43.093462    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c016c6f5-99f6-400e-ac31-292ede5ae4dd" path="/var/lib/kubelet/pods/c016c6f5-99f6-400e-ac31-292ede5ae4dd/volumes"
Sep 20 02:28:48.356322 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:28:48.356286    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-6be25a-5ddb98f8cf-5hh4g"]
Sep 20 02:28:48.356886 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:28:48.356608    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6be25a-5ddb98f8cf-5hh4g" podUID="a81869b4-741f-43d7-a18c-041836710946" containerName="maas-api" containerID="cri-o://d5100088dce49111e396d679fbb5c6beac91c57113e44470c55a6a08e94ffbac" gracePeriod=30
Sep 20 02:28:48.390500 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:28:48.390470    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:28:48.490351 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:28:48.490311    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:28:48.575422 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:28:48.575377    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:28:48.617902 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:28:48.617834    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6be25a-5ddb98f8cf-5hh4g"
Sep 20 02:28:48.642430 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:28:48.642397    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:28:48.739999 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:28:48.739966    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a81869b4-741f-43d7-a18c-041836710946-maas-api-tls\") pod \"a81869b4-741f-43d7-a18c-041836710946\" (UID: \"a81869b4-741f-43d7-a18c-041836710946\") "
Sep 20 02:28:48.740165 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:28:48.740026    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a81869b4-741f-43d7-a18c-041836710946-metrics-certs\") pod \"a81869b4-741f-43d7-a18c-041836710946\" (UID: \"a81869b4-741f-43d7-a18c-041836710946\") "
Sep 20 02:28:48.740165 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:28:48.740086    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a81869b4-741f-43d7-a18c-041836710946-openshift-service-ca\") pod \"a81869b4-741f-43d7-a18c-041836710946\" (UID: \"a81869b4-741f-43d7-a18c-041836710946\") "
Sep 20 02:28:48.740165 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:28:48.740119    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xmp4h\" (UniqueName: \"kubernetes.io/projected/a81869b4-741f-43d7-a18c-041836710946-kube-api-access-xmp4h\") pod \"a81869b4-741f-43d7-a18c-041836710946\" (UID: \"a81869b4-741f-43d7-a18c-041836710946\") "
Sep 20 02:28:48.740497 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:28:48.740460    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a81869b4-741f-43d7-a18c-041836710946-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a81869b4-741f-43d7-a18c-041836710946" (UID: "a81869b4-741f-43d7-a18c-041836710946"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 20 02:28:48.742059 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:28:48.742038    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a81869b4-741f-43d7-a18c-041836710946-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a81869b4-741f-43d7-a18c-041836710946" (UID: "a81869b4-741f-43d7-a18c-041836710946"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 20 02:28:48.742140 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:28:48.742109    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a81869b4-741f-43d7-a18c-041836710946-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "a81869b4-741f-43d7-a18c-041836710946" (UID: "a81869b4-741f-43d7-a18c-041836710946"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 20 02:28:48.742276 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:28:48.742254    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a81869b4-741f-43d7-a18c-041836710946-kube-api-access-xmp4h" (OuterVolumeSpecName: "kube-api-access-xmp4h") pod "a81869b4-741f-43d7-a18c-041836710946" (UID: "a81869b4-741f-43d7-a18c-041836710946"). InnerVolumeSpecName "kube-api-access-xmp4h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 20 02:28:48.841354 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:28:48.841321    2584 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a81869b4-741f-43d7-a18c-041836710946-metrics-certs\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:28:48.841354 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:28:48.841348    2584 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a81869b4-741f-43d7-a18c-041836710946-openshift-service-ca\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:28:48.841354 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:28:48.841358    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xmp4h\" (UniqueName: \"kubernetes.io/projected/a81869b4-741f-43d7-a18c-041836710946-kube-api-access-xmp4h\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:28:48.841618 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:28:48.841367    2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a81869b4-741f-43d7-a18c-041836710946-maas-api-tls\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:28:48.921624 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:28:48.921507    2584 generic.go:356] "Generic (PLEG): container finished" podID="a81869b4-741f-43d7-a18c-041836710946" containerID="d5100088dce49111e396d679fbb5c6beac91c57113e44470c55a6a08e94ffbac" exitCode=0
Sep 20 02:28:48.921624 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:28:48.921563    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6be25a-5ddb98f8cf-5hh4g"
Sep 20 02:28:48.921803 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:28:48.921562    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6be25a-5ddb98f8cf-5hh4g" event={"ID":"a81869b4-741f-43d7-a18c-041836710946","Type":"ContainerDied","Data":"d5100088dce49111e396d679fbb5c6beac91c57113e44470c55a6a08e94ffbac"}
Sep 20 02:28:48.921803 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:28:48.921688    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6be25a-5ddb98f8cf-5hh4g" event={"ID":"a81869b4-741f-43d7-a18c-041836710946","Type":"ContainerDied","Data":"af2fcda1f979648c67579d4581332ca9db7b78376d92cf8931af5e73eccca336"}
Sep 20 02:28:48.921803 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:28:48.921712    2584 scope.go:117] "RemoveContainer" containerID="d5100088dce49111e396d679fbb5c6beac91c57113e44470c55a6a08e94ffbac"
Sep 20 02:28:48.931664 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:28:48.931646    2584 scope.go:117] "RemoveContainer" containerID="d5100088dce49111e396d679fbb5c6beac91c57113e44470c55a6a08e94ffbac"
Sep 20 02:28:48.931937 ip-10-0-130-159 kubenswrapper[2584]: E0920 02:28:48.931916    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d5100088dce49111e396d679fbb5c6beac91c57113e44470c55a6a08e94ffbac\": container with ID starting with d5100088dce49111e396d679fbb5c6beac91c57113e44470c55a6a08e94ffbac not found: ID does not exist" containerID="d5100088dce49111e396d679fbb5c6beac91c57113e44470c55a6a08e94ffbac"
Sep 20 02:28:48.932012 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:28:48.931945    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d5100088dce49111e396d679fbb5c6beac91c57113e44470c55a6a08e94ffbac"} err="failed to get container status \"d5100088dce49111e396d679fbb5c6beac91c57113e44470c55a6a08e94ffbac\": rpc error: code = NotFound desc = could not find container \"d5100088dce49111e396d679fbb5c6beac91c57113e44470c55a6a08e94ffbac\": container with ID starting with d5100088dce49111e396d679fbb5c6beac91c57113e44470c55a6a08e94ffbac not found: ID does not exist"
Sep 20 02:28:48.945112 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:28:48.945077    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-6be25a-5ddb98f8cf-5hh4g"]
Sep 20 02:28:48.948716 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:28:48.948691    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-6be25a-5ddb98f8cf-5hh4g"]
Sep 20 02:28:48.963655 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:28:48.963627    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:28:49.025301 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:28:49.025264    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:28:49.092836 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:28:49.092802    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a81869b4-741f-43d7-a18c-041836710946" path="/var/lib/kubelet/pods/a81869b4-741f-43d7-a18c-041836710946/volumes"
Sep 20 02:28:49.726997 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:28:49.726965    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:28:50.406747 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:28:50.406715    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:28:55.090141 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:28:55.090104    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:28:55.159424 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:28:55.159389    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:28:55.351626 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:28:55.351512    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-6be25a-openshift-default-6796dcdc8-zzwz6"]
Sep 20 02:28:55.352094 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:28:55.352057    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-6be25a-openshift-default-6796dcdc8-zzwz6" podUID="cb81a34c-3882-46fc-8436-c5c94ad03045" containerName="istio-proxy" containerID="cri-o://e4fd210fee8d33651706ffb2ec01da9f33d4af1f0998729ec8bfebe874285dc1" gracePeriod=30
Sep 20 02:28:55.371055 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:28:55.371005    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:28:56.380941 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:28:56.380895    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:28:57.147330 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:28:57.147302    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:28:58.087254 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:28:58.087224    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:29:00.601571 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:29:00.601545    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-6be25a-openshift-default-6796dcdc8-zzwz6"
Sep 20 02:29:00.745170 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:29:00.745136    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cb81a34c-3882-46fc-8436-c5c94ad03045-workload-certs\") pod \"cb81a34c-3882-46fc-8436-c5c94ad03045\" (UID: \"cb81a34c-3882-46fc-8436-c5c94ad03045\") "
Sep 20 02:29:00.745387 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:29:00.745188    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6qc6r\" (UniqueName: \"kubernetes.io/projected/cb81a34c-3882-46fc-8436-c5c94ad03045-kube-api-access-6qc6r\") pod \"cb81a34c-3882-46fc-8436-c5c94ad03045\" (UID: \"cb81a34c-3882-46fc-8436-c5c94ad03045\") "
Sep 20 02:29:00.745387 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:29:00.745208    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cb81a34c-3882-46fc-8436-c5c94ad03045-istio-podinfo\") pod \"cb81a34c-3882-46fc-8436-c5c94ad03045\" (UID: \"cb81a34c-3882-46fc-8436-c5c94ad03045\") "
Sep 20 02:29:00.745387 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:29:00.745237    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb81a34c-3882-46fc-8436-c5c94ad03045-workload-socket\") pod \"cb81a34c-3882-46fc-8436-c5c94ad03045\" (UID: \"cb81a34c-3882-46fc-8436-c5c94ad03045\") "
Sep 20 02:29:00.745387 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:29:00.745258    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cb81a34c-3882-46fc-8436-c5c94ad03045-istio-data\") pod \"cb81a34c-3882-46fc-8436-c5c94ad03045\" (UID: \"cb81a34c-3882-46fc-8436-c5c94ad03045\") "
Sep 20 02:29:00.745387 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:29:00.745341    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cb81a34c-3882-46fc-8436-c5c94ad03045-istio-envoy\") pod \"cb81a34c-3882-46fc-8436-c5c94ad03045\" (UID: \"cb81a34c-3882-46fc-8436-c5c94ad03045\") "
Sep 20 02:29:00.745696 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:29:00.745397    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb81a34c-3882-46fc-8436-c5c94ad03045-credential-socket\") pod \"cb81a34c-3882-46fc-8436-c5c94ad03045\" (UID: \"cb81a34c-3882-46fc-8436-c5c94ad03045\") "
Sep 20 02:29:00.745696 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:29:00.745432    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cb81a34c-3882-46fc-8436-c5c94ad03045-istiod-ca-cert\") pod \"cb81a34c-3882-46fc-8436-c5c94ad03045\" (UID: \"cb81a34c-3882-46fc-8436-c5c94ad03045\") "
Sep 20 02:29:00.745696 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:29:00.745457    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cb81a34c-3882-46fc-8436-c5c94ad03045-istio-token\") pod \"cb81a34c-3882-46fc-8436-c5c94ad03045\" (UID: \"cb81a34c-3882-46fc-8436-c5c94ad03045\") "
Sep 20 02:29:00.745696 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:29:00.745513    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cb81a34c-3882-46fc-8436-c5c94ad03045-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "cb81a34c-3882-46fc-8436-c5c94ad03045" (UID: "cb81a34c-3882-46fc-8436-c5c94ad03045"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 20 02:29:00.745696 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:29:00.745567    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cb81a34c-3882-46fc-8436-c5c94ad03045-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "cb81a34c-3882-46fc-8436-c5c94ad03045" (UID: "cb81a34c-3882-46fc-8436-c5c94ad03045"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 20 02:29:00.745961 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:29:00.745772    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cb81a34c-3882-46fc-8436-c5c94ad03045-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "cb81a34c-3882-46fc-8436-c5c94ad03045" (UID: "cb81a34c-3882-46fc-8436-c5c94ad03045"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 20 02:29:00.745961 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:29:00.745819    2584 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cb81a34c-3882-46fc-8436-c5c94ad03045-workload-certs\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:29:00.745961 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:29:00.745838    2584 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb81a34c-3882-46fc-8436-c5c94ad03045-workload-socket\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:29:00.745961 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:29:00.745906    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cb81a34c-3882-46fc-8436-c5c94ad03045-istio-data" (OuterVolumeSpecName: "istio-data") pod "cb81a34c-3882-46fc-8436-c5c94ad03045" (UID: "cb81a34c-3882-46fc-8436-c5c94ad03045"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 20 02:29:00.746152 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:29:00.746078    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cb81a34c-3882-46fc-8436-c5c94ad03045-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "cb81a34c-3882-46fc-8436-c5c94ad03045" (UID: "cb81a34c-3882-46fc-8436-c5c94ad03045"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 20 02:29:00.747497 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:29:00.747476    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/cb81a34c-3882-46fc-8436-c5c94ad03045-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "cb81a34c-3882-46fc-8436-c5c94ad03045" (UID: "cb81a34c-3882-46fc-8436-c5c94ad03045"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 20 02:29:00.747603 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:29:00.747549    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cb81a34c-3882-46fc-8436-c5c94ad03045-kube-api-access-6qc6r" (OuterVolumeSpecName: "kube-api-access-6qc6r") pod "cb81a34c-3882-46fc-8436-c5c94ad03045" (UID: "cb81a34c-3882-46fc-8436-c5c94ad03045"). InnerVolumeSpecName "kube-api-access-6qc6r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 20 02:29:00.747989 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:29:00.747960    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cb81a34c-3882-46fc-8436-c5c94ad03045-istio-token" (OuterVolumeSpecName: "istio-token") pod "cb81a34c-3882-46fc-8436-c5c94ad03045" (UID: "cb81a34c-3882-46fc-8436-c5c94ad03045"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 20 02:29:00.748067 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:29:00.748046    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cb81a34c-3882-46fc-8436-c5c94ad03045-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "cb81a34c-3882-46fc-8436-c5c94ad03045" (UID: "cb81a34c-3882-46fc-8436-c5c94ad03045"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 20 02:29:00.846643 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:29:00.846602    2584 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb81a34c-3882-46fc-8436-c5c94ad03045-credential-socket\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:29:00.846643 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:29:00.846633    2584 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cb81a34c-3882-46fc-8436-c5c94ad03045-istiod-ca-cert\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:29:00.846643 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:29:00.846644    2584 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cb81a34c-3882-46fc-8436-c5c94ad03045-istio-token\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:29:00.846643 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:29:00.846653    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6qc6r\" (UniqueName: \"kubernetes.io/projected/cb81a34c-3882-46fc-8436-c5c94ad03045-kube-api-access-6qc6r\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:29:00.846905 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:29:00.846661    2584 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cb81a34c-3882-46fc-8436-c5c94ad03045-istio-podinfo\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:29:00.846905 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:29:00.846669    2584 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cb81a34c-3882-46fc-8436-c5c94ad03045-istio-data\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:29:00.846905 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:29:00.846677    2584 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cb81a34c-3882-46fc-8436-c5c94ad03045-istio-envoy\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:29:00.984651 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:29:00.984615    2584 generic.go:356] "Generic (PLEG): container finished" podID="cb81a34c-3882-46fc-8436-c5c94ad03045" containerID="e4fd210fee8d33651706ffb2ec01da9f33d4af1f0998729ec8bfebe874285dc1" exitCode=0
Sep 20 02:29:00.984842 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:29:00.984666    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-6be25a-openshift-default-6796dcdc8-zzwz6" event={"ID":"cb81a34c-3882-46fc-8436-c5c94ad03045","Type":"ContainerDied","Data":"e4fd210fee8d33651706ffb2ec01da9f33d4af1f0998729ec8bfebe874285dc1"}
Sep 20 02:29:00.984842 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:29:00.984677    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-6be25a-openshift-default-6796dcdc8-zzwz6"
Sep 20 02:29:00.984842 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:29:00.984711    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-6be25a-openshift-default-6796dcdc8-zzwz6" event={"ID":"cb81a34c-3882-46fc-8436-c5c94ad03045","Type":"ContainerDied","Data":"32ba8685659a0c4057b42226426209bfac086e9511dc975ac6ab4acef08e8438"}
Sep 20 02:29:00.984842 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:29:00.984733    2584 scope.go:117] "RemoveContainer" containerID="e4fd210fee8d33651706ffb2ec01da9f33d4af1f0998729ec8bfebe874285dc1"
Sep 20 02:29:00.994084 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:29:00.994063    2584 scope.go:117] "RemoveContainer" containerID="8c3f2b4a85049fe3f8f9ec916a0bacb5fe09b2f9f4308af0420a13c36356b6e6"
Sep 20 02:29:01.004272 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:29:01.004260    2584 scope.go:117] "RemoveContainer" containerID="e4fd210fee8d33651706ffb2ec01da9f33d4af1f0998729ec8bfebe874285dc1"
Sep 20 02:29:01.004599 ip-10-0-130-159 kubenswrapper[2584]: E0920 02:29:01.004576    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e4fd210fee8d33651706ffb2ec01da9f33d4af1f0998729ec8bfebe874285dc1\": container with ID starting with e4fd210fee8d33651706ffb2ec01da9f33d4af1f0998729ec8bfebe874285dc1 not found: ID does not exist" containerID="e4fd210fee8d33651706ffb2ec01da9f33d4af1f0998729ec8bfebe874285dc1"
Sep 20 02:29:01.004709 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:29:01.004611    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e4fd210fee8d33651706ffb2ec01da9f33d4af1f0998729ec8bfebe874285dc1"} err="failed to get container status \"e4fd210fee8d33651706ffb2ec01da9f33d4af1f0998729ec8bfebe874285dc1\": rpc error: code = NotFound desc = could not find container \"e4fd210fee8d33651706ffb2ec01da9f33d4af1f0998729ec8bfebe874285dc1\": container with ID starting with e4fd210fee8d33651706ffb2ec01da9f33d4af1f0998729ec8bfebe874285dc1 not found: ID does not exist"
Sep 20 02:29:01.004709 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:29:01.004636    2584 scope.go:117] "RemoveContainer" containerID="8c3f2b4a85049fe3f8f9ec916a0bacb5fe09b2f9f4308af0420a13c36356b6e6"
Sep 20 02:29:01.004931 ip-10-0-130-159 kubenswrapper[2584]: E0920 02:29:01.004913    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8c3f2b4a85049fe3f8f9ec916a0bacb5fe09b2f9f4308af0420a13c36356b6e6\": container with ID starting with 8c3f2b4a85049fe3f8f9ec916a0bacb5fe09b2f9f4308af0420a13c36356b6e6 not found: ID does not exist" containerID="8c3f2b4a85049fe3f8f9ec916a0bacb5fe09b2f9f4308af0420a13c36356b6e6"
Sep 20 02:29:01.004972 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:29:01.004936    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c3f2b4a85049fe3f8f9ec916a0bacb5fe09b2f9f4308af0420a13c36356b6e6"} err="failed to get container status \"8c3f2b4a85049fe3f8f9ec916a0bacb5fe09b2f9f4308af0420a13c36356b6e6\": rpc error: code = NotFound desc = could not find container \"8c3f2b4a85049fe3f8f9ec916a0bacb5fe09b2f9f4308af0420a13c36356b6e6\": container with ID starting with 8c3f2b4a85049fe3f8f9ec916a0bacb5fe09b2f9f4308af0420a13c36356b6e6 not found: ID does not exist"
Sep 20 02:29:01.013389 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:29:01.012025    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-6be25a-openshift-default-6796dcdc8-zzwz6"]
Sep 20 02:29:01.016977 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:29:01.016951    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-6be25a-openshift-default-6796dcdc8-zzwz6"]
Sep 20 02:29:01.093411 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:29:01.093376    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cb81a34c-3882-46fc-8436-c5c94ad03045" path="/var/lib/kubelet/pods/cb81a34c-3882-46fc-8436-c5c94ad03045/volumes"
Sep 20 02:29:06.295117 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:29:06.295076    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:29:06.374346 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:29:06.374316    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:29:06.454996 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:29:06.454966    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:29:06.560334 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:29:06.560216    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:29:06.704810 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:29:06.704761    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:29:06.967494 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:29:06.967444    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:29:07.110564 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:29:07.110498    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:29:08.090776 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:29:08.090725    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:29:13.309855 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:29:13.309818    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:29:13.380722 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:29:13.380688    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:29:13.583397 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:29:13.583280    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:29:14.848631 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:29:14.848575    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:29:15.550713 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:29:15.550684    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:29:16.365492 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:29:16.365453    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:29:24.067586 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:29:24.067548    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:29:24.731983 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:29:24.731949    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-317780-8d7f49969-l2qbf"]
Sep 20 02:29:24.732212 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:29:24.732180    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-317780-8d7f49969-l2qbf" podUID="796b838b-dd79-4953-bf6c-91e3326f3e4c" containerName="maas-api" containerID="cri-o://4636529cb369b9fdd0314c8dde1f37c50cd646cb218b6f50c41f47a87355482e" gracePeriod=30
Sep 20 02:29:24.781378 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:29:24.781352    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:29:24.918307 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:29:24.918276    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:29:24.990194 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:29:24.990126    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:29:24.993638 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:29:24.993618    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-317780-8d7f49969-l2qbf"
Sep 20 02:29:25.063083 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:29:25.063050    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:29:25.072861 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:29:25.072835    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/796b838b-dd79-4953-bf6c-91e3326f3e4c-metrics-certs\") pod \"796b838b-dd79-4953-bf6c-91e3326f3e4c\" (UID: \"796b838b-dd79-4953-bf6c-91e3326f3e4c\") "
Sep 20 02:29:25.073322 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:29:25.072876    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/796b838b-dd79-4953-bf6c-91e3326f3e4c-openshift-service-ca\") pod \"796b838b-dd79-4953-bf6c-91e3326f3e4c\" (UID: \"796b838b-dd79-4953-bf6c-91e3326f3e4c\") "
Sep 20 02:29:25.073322 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:29:25.072916    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jck2c\" (UniqueName: \"kubernetes.io/projected/796b838b-dd79-4953-bf6c-91e3326f3e4c-kube-api-access-jck2c\") pod \"796b838b-dd79-4953-bf6c-91e3326f3e4c\" (UID: \"796b838b-dd79-4953-bf6c-91e3326f3e4c\") "
Sep 20 02:29:25.073322 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:29:25.072950    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/796b838b-dd79-4953-bf6c-91e3326f3e4c-maas-api-tls\") pod \"796b838b-dd79-4953-bf6c-91e3326f3e4c\" (UID: \"796b838b-dd79-4953-bf6c-91e3326f3e4c\") "
Sep 20 02:29:25.073322 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:29:25.073300    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/796b838b-dd79-4953-bf6c-91e3326f3e4c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "796b838b-dd79-4953-bf6c-91e3326f3e4c" (UID: "796b838b-dd79-4953-bf6c-91e3326f3e4c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 20 02:29:25.075076 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:29:25.075045    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/796b838b-dd79-4953-bf6c-91e3326f3e4c-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "796b838b-dd79-4953-bf6c-91e3326f3e4c" (UID: "796b838b-dd79-4953-bf6c-91e3326f3e4c"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 20 02:29:25.075182 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:29:25.075127    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/796b838b-dd79-4953-bf6c-91e3326f3e4c-kube-api-access-jck2c" (OuterVolumeSpecName: "kube-api-access-jck2c") pod "796b838b-dd79-4953-bf6c-91e3326f3e4c" (UID: "796b838b-dd79-4953-bf6c-91e3326f3e4c"). InnerVolumeSpecName "kube-api-access-jck2c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 20 02:29:25.075182 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:29:25.075164    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/796b838b-dd79-4953-bf6c-91e3326f3e4c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "796b838b-dd79-4953-bf6c-91e3326f3e4c" (UID: "796b838b-dd79-4953-bf6c-91e3326f3e4c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 20 02:29:25.112955 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:29:25.112923    2584 generic.go:356] "Generic (PLEG): container finished" podID="796b838b-dd79-4953-bf6c-91e3326f3e4c" containerID="4636529cb369b9fdd0314c8dde1f37c50cd646cb218b6f50c41f47a87355482e" exitCode=0
Sep 20 02:29:25.113094 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:29:25.113001    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-317780-8d7f49969-l2qbf" event={"ID":"796b838b-dd79-4953-bf6c-91e3326f3e4c","Type":"ContainerDied","Data":"4636529cb369b9fdd0314c8dde1f37c50cd646cb218b6f50c41f47a87355482e"}
Sep 20 02:29:25.113094 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:29:25.113042    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-317780-8d7f49969-l2qbf" event={"ID":"796b838b-dd79-4953-bf6c-91e3326f3e4c","Type":"ContainerDied","Data":"56719b45818c62a668b8c9df82d85dcda8bf66a34a9bdf541bbec9844c52ae01"}
Sep 20 02:29:25.113094 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:29:25.113065    2584 scope.go:117] "RemoveContainer" containerID="4636529cb369b9fdd0314c8dde1f37c50cd646cb218b6f50c41f47a87355482e"
Sep 20 02:29:25.113094 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:29:25.113087    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-317780-8d7f49969-l2qbf"
Sep 20 02:29:25.121596 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:29:25.121574    2584 scope.go:117] "RemoveContainer" containerID="4636529cb369b9fdd0314c8dde1f37c50cd646cb218b6f50c41f47a87355482e"
Sep 20 02:29:25.121936 ip-10-0-130-159 kubenswrapper[2584]: E0920 02:29:25.121917    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4636529cb369b9fdd0314c8dde1f37c50cd646cb218b6f50c41f47a87355482e\": container with ID starting with 4636529cb369b9fdd0314c8dde1f37c50cd646cb218b6f50c41f47a87355482e not found: ID does not exist" containerID="4636529cb369b9fdd0314c8dde1f37c50cd646cb218b6f50c41f47a87355482e"
Sep 20 02:29:25.122014 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:29:25.121945    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4636529cb369b9fdd0314c8dde1f37c50cd646cb218b6f50c41f47a87355482e"} err="failed to get container status \"4636529cb369b9fdd0314c8dde1f37c50cd646cb218b6f50c41f47a87355482e\": rpc error: code = NotFound desc = could not find container \"4636529cb369b9fdd0314c8dde1f37c50cd646cb218b6f50c41f47a87355482e\": container with ID starting with 4636529cb369b9fdd0314c8dde1f37c50cd646cb218b6f50c41f47a87355482e not found: ID does not exist"
Sep 20 02:29:25.142361 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:29:25.142334    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-317780-8d7f49969-l2qbf"]
Sep 20 02:29:25.146151 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:29:25.146129    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-317780-8d7f49969-l2qbf"]
Sep 20 02:29:25.173776 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:29:25.173747    2584 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/796b838b-dd79-4953-bf6c-91e3326f3e4c-metrics-certs\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:29:25.173776 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:29:25.173771    2584 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/796b838b-dd79-4953-bf6c-91e3326f3e4c-openshift-service-ca\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:29:25.173776 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:29:25.173783    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jck2c\" (UniqueName: \"kubernetes.io/projected/796b838b-dd79-4953-bf6c-91e3326f3e4c-kube-api-access-jck2c\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:29:25.173776 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:29:25.173792    2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/796b838b-dd79-4953-bf6c-91e3326f3e4c-maas-api-tls\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:29:25.180238 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:29:25.180213    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:29:25.438345 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:29:25.438254    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:29:25.597277 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:29:25.597240    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:29:26.968732 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:29:26.968698    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:29:27.092538 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:29:27.092487    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="796b838b-dd79-4953-bf6c-91e3326f3e4c" path="/var/lib/kubelet/pods/796b838b-dd79-4953-bf6c-91e3326f3e4c/volumes"
Sep 20 02:29:32.629377 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:29:32.629348    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:29:32.770717 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:29:32.770682    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:29:33.024042 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:29:33.024007    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:29:35.789744 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:29:35.789706    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6f56f85f68-2bfth"]
Sep 20 02:29:35.790287 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:29:35.789991    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-6f56f85f68-2bfth" podUID="ec515e4f-40b2-41bf-8cf8-063323865e36" containerName="manager" containerID="cri-o://402a721c4867462656354f6301ac78148ca602e2f7b695993a1f7252503dde42" gracePeriod=30
Sep 20 02:29:36.038149 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:29:36.038123    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6f56f85f68-2bfth"
Sep 20 02:29:36.173080 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:29:36.172982    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t477j\" (UniqueName: \"kubernetes.io/projected/ec515e4f-40b2-41bf-8cf8-063323865e36-kube-api-access-t477j\") pod \"ec515e4f-40b2-41bf-8cf8-063323865e36\" (UID: \"ec515e4f-40b2-41bf-8cf8-063323865e36\") "
Sep 20 02:29:36.173080 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:29:36.173018    2584 generic.go:356] "Generic (PLEG): container finished" podID="ec515e4f-40b2-41bf-8cf8-063323865e36" containerID="402a721c4867462656354f6301ac78148ca602e2f7b695993a1f7252503dde42" exitCode=0
Sep 20 02:29:36.173328 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:29:36.173095    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ec515e4f-40b2-41bf-8cf8-063323865e36-webhook-cert\") pod \"ec515e4f-40b2-41bf-8cf8-063323865e36\" (UID: \"ec515e4f-40b2-41bf-8cf8-063323865e36\") "
Sep 20 02:29:36.173328 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:29:36.173130    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ec515e4f-40b2-41bf-8cf8-063323865e36-metrics-certs\") pod \"ec515e4f-40b2-41bf-8cf8-063323865e36\" (UID: \"ec515e4f-40b2-41bf-8cf8-063323865e36\") "
Sep 20 02:29:36.173328 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:29:36.173144    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6f56f85f68-2bfth" event={"ID":"ec515e4f-40b2-41bf-8cf8-063323865e36","Type":"ContainerDied","Data":"402a721c4867462656354f6301ac78148ca602e2f7b695993a1f7252503dde42"}
Sep 20 02:29:36.173328 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:29:36.173213    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6f56f85f68-2bfth" event={"ID":"ec515e4f-40b2-41bf-8cf8-063323865e36","Type":"ContainerDied","Data":"8c7ace170704b25e94d251e85260eb7ab5e8303e4af4f6d92d6d497ee080242e"}
Sep 20 02:29:36.173328 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:29:36.173243    2584 scope.go:117] "RemoveContainer" containerID="402a721c4867462656354f6301ac78148ca602e2f7b695993a1f7252503dde42"
Sep 20 02:29:36.173645 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:29:36.173130    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6f56f85f68-2bfth"
Sep 20 02:29:36.175657 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:29:36.175628    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ec515e4f-40b2-41bf-8cf8-063323865e36-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "ec515e4f-40b2-41bf-8cf8-063323865e36" (UID: "ec515e4f-40b2-41bf-8cf8-063323865e36"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 20 02:29:36.175657 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:29:36.175638    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec515e4f-40b2-41bf-8cf8-063323865e36-kube-api-access-t477j" (OuterVolumeSpecName: "kube-api-access-t477j") pod "ec515e4f-40b2-41bf-8cf8-063323865e36" (UID: "ec515e4f-40b2-41bf-8cf8-063323865e36"). InnerVolumeSpecName "kube-api-access-t477j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 20 02:29:36.175874 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:29:36.175688    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ec515e4f-40b2-41bf-8cf8-063323865e36-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "ec515e4f-40b2-41bf-8cf8-063323865e36" (UID: "ec515e4f-40b2-41bf-8cf8-063323865e36"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 20 02:29:36.193784 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:29:36.193756    2584 scope.go:117] "RemoveContainer" containerID="402a721c4867462656354f6301ac78148ca602e2f7b695993a1f7252503dde42"
Sep 20 02:29:36.194123 ip-10-0-130-159 kubenswrapper[2584]: E0920 02:29:36.194102    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"402a721c4867462656354f6301ac78148ca602e2f7b695993a1f7252503dde42\": container with ID starting with 402a721c4867462656354f6301ac78148ca602e2f7b695993a1f7252503dde42 not found: ID does not exist" containerID="402a721c4867462656354f6301ac78148ca602e2f7b695993a1f7252503dde42"
Sep 20 02:29:36.194184 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:29:36.194133    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"402a721c4867462656354f6301ac78148ca602e2f7b695993a1f7252503dde42"} err="failed to get container status \"402a721c4867462656354f6301ac78148ca602e2f7b695993a1f7252503dde42\": rpc error: code = NotFound desc = could not find container \"402a721c4867462656354f6301ac78148ca602e2f7b695993a1f7252503dde42\": container with ID starting with 402a721c4867462656354f6301ac78148ca602e2f7b695993a1f7252503dde42 not found: ID does not exist"
Sep 20 02:29:36.274403 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:29:36.274367    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t477j\" (UniqueName: \"kubernetes.io/projected/ec515e4f-40b2-41bf-8cf8-063323865e36-kube-api-access-t477j\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:29:36.274403 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:29:36.274395    2584 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ec515e4f-40b2-41bf-8cf8-063323865e36-webhook-cert\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:29:36.274403 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:29:36.274404    2584 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ec515e4f-40b2-41bf-8cf8-063323865e36-metrics-certs\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:29:36.499044 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:29:36.498994    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6f56f85f68-2bfth"]
Sep 20 02:29:36.500838 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:29:36.500812    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6f56f85f68-2bfth"]
Sep 20 02:29:37.094603 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:29:37.094575    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ec515e4f-40b2-41bf-8cf8-063323865e36" path="/var/lib/kubelet/pods/ec515e4f-40b2-41bf-8cf8-063323865e36/volumes"
Sep 20 02:30:37.353977 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:30:37.353940    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6f56f85f68-wlvjq"]
Sep 20 02:30:37.354557 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:30:37.354459    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c016c6f5-99f6-400e-ac31-292ede5ae4dd" containerName="istio-proxy"
Sep 20 02:30:37.354557 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:30:37.354478    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="c016c6f5-99f6-400e-ac31-292ede5ae4dd" containerName="istio-proxy"
Sep 20 02:30:37.354557 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:30:37.354489    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f0bec830-952f-41b2-9077-8fd4b8c0cd89" containerName="istio-proxy"
Sep 20 02:30:37.354557 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:30:37.354498    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="f0bec830-952f-41b2-9077-8fd4b8c0cd89" containerName="istio-proxy"
Sep 20 02:30:37.354557 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:30:37.354510    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c016c6f5-99f6-400e-ac31-292ede5ae4dd" containerName="istio-proxy"
Sep 20 02:30:37.354557 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:30:37.354537    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="c016c6f5-99f6-400e-ac31-292ede5ae4dd" containerName="istio-proxy"
Sep 20 02:30:37.354557 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:30:37.354555    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ec515e4f-40b2-41bf-8cf8-063323865e36" containerName="manager"
Sep 20 02:30:37.354557 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:30:37.354562    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec515e4f-40b2-41bf-8cf8-063323865e36" containerName="manager"
Sep 20 02:30:37.354995 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:30:37.354574    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cb81a34c-3882-46fc-8436-c5c94ad03045" containerName="istio-proxy"
Sep 20 02:30:37.354995 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:30:37.354582    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="cb81a34c-3882-46fc-8436-c5c94ad03045" containerName="istio-proxy"
Sep 20 02:30:37.354995 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:30:37.354593    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a81869b4-741f-43d7-a18c-041836710946" containerName="maas-api"
Sep 20 02:30:37.354995 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:30:37.354601    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="a81869b4-741f-43d7-a18c-041836710946" containerName="maas-api"
Sep 20 02:30:37.354995 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:30:37.354612    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f8d7a70a-30fd-499d-b042-3ffe40191520" containerName="maas-api"
Sep 20 02:30:37.354995 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:30:37.354620    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="f8d7a70a-30fd-499d-b042-3ffe40191520" containerName="maas-api"
Sep 20 02:30:37.354995 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:30:37.354632    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3020f8fa-d45a-48b9-8ce9-f9af5b0321dc" containerName="maas-api"
Sep 20 02:30:37.354995 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:30:37.354640    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="3020f8fa-d45a-48b9-8ce9-f9af5b0321dc" containerName="maas-api"
Sep 20 02:30:37.354995 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:30:37.354657    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cb81a34c-3882-46fc-8436-c5c94ad03045" containerName="istio-proxy"
Sep 20 02:30:37.354995 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:30:37.354665    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="cb81a34c-3882-46fc-8436-c5c94ad03045" containerName="istio-proxy"
Sep 20 02:30:37.354995 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:30:37.354676    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="796b838b-dd79-4953-bf6c-91e3326f3e4c" containerName="maas-api"
Sep 20 02:30:37.354995 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:30:37.354684    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="796b838b-dd79-4953-bf6c-91e3326f3e4c" containerName="maas-api"
Sep 20 02:30:37.354995 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:30:37.354783    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="cb81a34c-3882-46fc-8436-c5c94ad03045" containerName="istio-proxy"
Sep 20 02:30:37.354995 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:30:37.354795    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="c016c6f5-99f6-400e-ac31-292ede5ae4dd" containerName="istio-proxy"
Sep 20 02:30:37.354995 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:30:37.354803    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="f0bec830-952f-41b2-9077-8fd4b8c0cd89" containerName="istio-proxy"
Sep 20 02:30:37.354995 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:30:37.354812    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="a81869b4-741f-43d7-a18c-041836710946" containerName="maas-api"
Sep 20 02:30:37.354995 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:30:37.354820    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="f8d7a70a-30fd-499d-b042-3ffe40191520" containerName="maas-api"
Sep 20 02:30:37.354995 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:30:37.354830    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="ec515e4f-40b2-41bf-8cf8-063323865e36" containerName="manager"
Sep 20 02:30:37.354995 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:30:37.354840    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="c016c6f5-99f6-400e-ac31-292ede5ae4dd" containerName="istio-proxy"
Sep 20 02:30:37.354995 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:30:37.354850    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="796b838b-dd79-4953-bf6c-91e3326f3e4c" containerName="maas-api"
Sep 20 02:30:37.354995 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:30:37.354859    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="3020f8fa-d45a-48b9-8ce9-f9af5b0321dc" containerName="maas-api"
Sep 20 02:30:37.359326 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:30:37.359306    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6f56f85f68-wlvjq"
Sep 20 02:30:37.362715 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:30:37.362689    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 20 02:30:37.362840 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:30:37.362695    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 20 02:30:37.362840 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:30:37.362701    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-mv62k\""
Sep 20 02:30:37.362840 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:30:37.362738    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 20 02:30:37.368041 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:30:37.368021    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6f56f85f68-wlvjq"]
Sep 20 02:30:37.496405 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:30:37.496374    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cd7550e3-b8c8-48b1-ab1c-21725e626fd8-webhook-cert\") pod \"maas-controller-6f56f85f68-wlvjq\" (UID: \"cd7550e3-b8c8-48b1-ab1c-21725e626fd8\") " pod="opendatahub/maas-controller-6f56f85f68-wlvjq"
Sep 20 02:30:37.496592 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:30:37.496421    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cd7550e3-b8c8-48b1-ab1c-21725e626fd8-metrics-certs\") pod \"maas-controller-6f56f85f68-wlvjq\" (UID: \"cd7550e3-b8c8-48b1-ab1c-21725e626fd8\") " pod="opendatahub/maas-controller-6f56f85f68-wlvjq"
Sep 20 02:30:37.496592 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:30:37.496471    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mfw2g\" (UniqueName: \"kubernetes.io/projected/cd7550e3-b8c8-48b1-ab1c-21725e626fd8-kube-api-access-mfw2g\") pod \"maas-controller-6f56f85f68-wlvjq\" (UID: \"cd7550e3-b8c8-48b1-ab1c-21725e626fd8\") " pod="opendatahub/maas-controller-6f56f85f68-wlvjq"
Sep 20 02:30:37.597682 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:30:37.597649    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cd7550e3-b8c8-48b1-ab1c-21725e626fd8-webhook-cert\") pod \"maas-controller-6f56f85f68-wlvjq\" (UID: \"cd7550e3-b8c8-48b1-ab1c-21725e626fd8\") " pod="opendatahub/maas-controller-6f56f85f68-wlvjq"
Sep 20 02:30:37.597863 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:30:37.597698    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cd7550e3-b8c8-48b1-ab1c-21725e626fd8-metrics-certs\") pod \"maas-controller-6f56f85f68-wlvjq\" (UID: \"cd7550e3-b8c8-48b1-ab1c-21725e626fd8\") " pod="opendatahub/maas-controller-6f56f85f68-wlvjq"
Sep 20 02:30:37.597863 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:30:37.597722    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mfw2g\" (UniqueName: \"kubernetes.io/projected/cd7550e3-b8c8-48b1-ab1c-21725e626fd8-kube-api-access-mfw2g\") pod \"maas-controller-6f56f85f68-wlvjq\" (UID: \"cd7550e3-b8c8-48b1-ab1c-21725e626fd8\") " pod="opendatahub/maas-controller-6f56f85f68-wlvjq"
Sep 20 02:30:37.600097 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:30:37.600073    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cd7550e3-b8c8-48b1-ab1c-21725e626fd8-webhook-cert\") pod \"maas-controller-6f56f85f68-wlvjq\" (UID: \"cd7550e3-b8c8-48b1-ab1c-21725e626fd8\") " pod="opendatahub/maas-controller-6f56f85f68-wlvjq"
Sep 20 02:30:37.600229 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:30:37.600187    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cd7550e3-b8c8-48b1-ab1c-21725e626fd8-metrics-certs\") pod \"maas-controller-6f56f85f68-wlvjq\" (UID: \"cd7550e3-b8c8-48b1-ab1c-21725e626fd8\") " pod="opendatahub/maas-controller-6f56f85f68-wlvjq"
Sep 20 02:30:37.605726 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:30:37.605668    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mfw2g\" (UniqueName: \"kubernetes.io/projected/cd7550e3-b8c8-48b1-ab1c-21725e626fd8-kube-api-access-mfw2g\") pod \"maas-controller-6f56f85f68-wlvjq\" (UID: \"cd7550e3-b8c8-48b1-ab1c-21725e626fd8\") " pod="opendatahub/maas-controller-6f56f85f68-wlvjq"
Sep 20 02:30:37.670767 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:30:37.670729    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6f56f85f68-wlvjq"
Sep 20 02:30:37.942546 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:30:37.942484    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6f56f85f68-wlvjq"]
Sep 20 02:30:37.946200 ip-10-0-130-159 kubenswrapper[2584]: W0920 02:30:37.946175    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd7550e3_b8c8_48b1_ab1c_21725e626fd8.slice/crio-d176ad4d1ed4e378463c5bbc961fa671f23936a8dad4da070b9afebdec30aa37 WatchSource:0}: Error finding container d176ad4d1ed4e378463c5bbc961fa671f23936a8dad4da070b9afebdec30aa37: Status 404 returned error can't find the container with id d176ad4d1ed4e378463c5bbc961fa671f23936a8dad4da070b9afebdec30aa37
Sep 20 02:30:37.947603 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:30:37.947583    2584 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 20 02:30:38.472876 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:30:38.472847    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6f56f85f68-wlvjq" event={"ID":"cd7550e3-b8c8-48b1-ab1c-21725e626fd8","Type":"ContainerStarted","Data":"2c113d22e9f473f84e313896ab9ae4d017ae27d5225eb3fe76ce1947c904c33b"}
Sep 20 02:30:38.473193 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:30:38.472909    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6f56f85f68-wlvjq" event={"ID":"cd7550e3-b8c8-48b1-ab1c-21725e626fd8","Type":"ContainerStarted","Data":"d176ad4d1ed4e378463c5bbc961fa671f23936a8dad4da070b9afebdec30aa37"}
Sep 20 02:30:38.473193 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:30:38.473005    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6f56f85f68-wlvjq"
Sep 20 02:30:38.491034 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:30:38.490969    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6f56f85f68-wlvjq" podStartSLOduration=1.027000855 podStartE2EDuration="1.490945863s" totalImagesPullingTime="463.945008ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-20 02:30:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-20 02:30:38.48924128 +0000 UTC m=+2295.231920487" watchObservedRunningTime="2026-09-20 02:30:38.490945863 +0000 UTC m=+2295.233625071"
Sep 20 02:30:49.480964 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:30:49.480931    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6f56f85f68-wlvjq"
Sep 20 02:30:58.793675 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:30:58.793643    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:30:58.859635 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:30:58.859600    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6f56f85f68-wlvjq"]
Sep 20 02:30:58.859869 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:30:58.859840    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-6f56f85f68-wlvjq" podUID="cd7550e3-b8c8-48b1-ab1c-21725e626fd8" containerName="manager" containerID="cri-o://2c113d22e9f473f84e313896ab9ae4d017ae27d5225eb3fe76ce1947c904c33b" gracePeriod=30
Sep 20 02:30:59.107151 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:30:59.107123    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6f56f85f68-wlvjq"
Sep 20 02:30:59.205219 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:30:59.205190    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cd7550e3-b8c8-48b1-ab1c-21725e626fd8-metrics-certs\") pod \"cd7550e3-b8c8-48b1-ab1c-21725e626fd8\" (UID: \"cd7550e3-b8c8-48b1-ab1c-21725e626fd8\") "
Sep 20 02:30:59.205423 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:30:59.205241    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mfw2g\" (UniqueName: \"kubernetes.io/projected/cd7550e3-b8c8-48b1-ab1c-21725e626fd8-kube-api-access-mfw2g\") pod \"cd7550e3-b8c8-48b1-ab1c-21725e626fd8\" (UID: \"cd7550e3-b8c8-48b1-ab1c-21725e626fd8\") "
Sep 20 02:30:59.205423 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:30:59.205337    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cd7550e3-b8c8-48b1-ab1c-21725e626fd8-webhook-cert\") pod \"cd7550e3-b8c8-48b1-ab1c-21725e626fd8\" (UID: \"cd7550e3-b8c8-48b1-ab1c-21725e626fd8\") "
Sep 20 02:30:59.207301 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:30:59.207272    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cd7550e3-b8c8-48b1-ab1c-21725e626fd8-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "cd7550e3-b8c8-48b1-ab1c-21725e626fd8" (UID: "cd7550e3-b8c8-48b1-ab1c-21725e626fd8"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 20 02:30:59.207429 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:30:59.207332    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd7550e3-b8c8-48b1-ab1c-21725e626fd8-kube-api-access-mfw2g" (OuterVolumeSpecName: "kube-api-access-mfw2g") pod "cd7550e3-b8c8-48b1-ab1c-21725e626fd8" (UID: "cd7550e3-b8c8-48b1-ab1c-21725e626fd8"). InnerVolumeSpecName "kube-api-access-mfw2g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 20 02:30:59.207429 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:30:59.207389    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cd7550e3-b8c8-48b1-ab1c-21725e626fd8-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "cd7550e3-b8c8-48b1-ab1c-21725e626fd8" (UID: "cd7550e3-b8c8-48b1-ab1c-21725e626fd8"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 20 02:30:59.307067 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:30:59.307035    2584 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cd7550e3-b8c8-48b1-ab1c-21725e626fd8-webhook-cert\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:30:59.307067 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:30:59.307060    2584 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cd7550e3-b8c8-48b1-ab1c-21725e626fd8-metrics-certs\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:30:59.307067 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:30:59.307070    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mfw2g\" (UniqueName: \"kubernetes.io/projected/cd7550e3-b8c8-48b1-ab1c-21725e626fd8-kube-api-access-mfw2g\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:30:59.579839 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:30:59.579806    2584 generic.go:356] "Generic (PLEG): container finished" podID="cd7550e3-b8c8-48b1-ab1c-21725e626fd8" containerID="2c113d22e9f473f84e313896ab9ae4d017ae27d5225eb3fe76ce1947c904c33b" exitCode=0
Sep 20 02:30:59.580051 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:30:59.579848    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6f56f85f68-wlvjq"
Sep 20 02:30:59.580051 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:30:59.579886    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6f56f85f68-wlvjq" event={"ID":"cd7550e3-b8c8-48b1-ab1c-21725e626fd8","Type":"ContainerDied","Data":"2c113d22e9f473f84e313896ab9ae4d017ae27d5225eb3fe76ce1947c904c33b"}
Sep 20 02:30:59.580051 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:30:59.579943    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6f56f85f68-wlvjq" event={"ID":"cd7550e3-b8c8-48b1-ab1c-21725e626fd8","Type":"ContainerDied","Data":"d176ad4d1ed4e378463c5bbc961fa671f23936a8dad4da070b9afebdec30aa37"}
Sep 20 02:30:59.580051 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:30:59.579973    2584 scope.go:117] "RemoveContainer" containerID="2c113d22e9f473f84e313896ab9ae4d017ae27d5225eb3fe76ce1947c904c33b"
Sep 20 02:30:59.588910 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:30:59.588893    2584 scope.go:117] "RemoveContainer" containerID="2c113d22e9f473f84e313896ab9ae4d017ae27d5225eb3fe76ce1947c904c33b"
Sep 20 02:30:59.589179 ip-10-0-130-159 kubenswrapper[2584]: E0920 02:30:59.589160    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2c113d22e9f473f84e313896ab9ae4d017ae27d5225eb3fe76ce1947c904c33b\": container with ID starting with 2c113d22e9f473f84e313896ab9ae4d017ae27d5225eb3fe76ce1947c904c33b not found: ID does not exist" containerID="2c113d22e9f473f84e313896ab9ae4d017ae27d5225eb3fe76ce1947c904c33b"
Sep 20 02:30:59.589249 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:30:59.589186    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c113d22e9f473f84e313896ab9ae4d017ae27d5225eb3fe76ce1947c904c33b"} err="failed to get container status \"2c113d22e9f473f84e313896ab9ae4d017ae27d5225eb3fe76ce1947c904c33b\": rpc error: code = NotFound desc = could not find container \"2c113d22e9f473f84e313896ab9ae4d017ae27d5225eb3fe76ce1947c904c33b\": container with ID starting with 2c113d22e9f473f84e313896ab9ae4d017ae27d5225eb3fe76ce1947c904c33b not found: ID does not exist"
Sep 20 02:30:59.601816 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:30:59.601794    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6f56f85f68-wlvjq"]
Sep 20 02:30:59.605613 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:30:59.605575    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6f56f85f68-wlvjq"]
Sep 20 02:31:00.288638 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:31:00.288600    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:31:00.398712 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:31:00.398681    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6f56f85f68-zwvbx"]
Sep 20 02:31:00.399057 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:31:00.399045    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f0bec830-952f-41b2-9077-8fd4b8c0cd89" containerName="istio-proxy"
Sep 20 02:31:00.399100 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:31:00.399059    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="f0bec830-952f-41b2-9077-8fd4b8c0cd89" containerName="istio-proxy"
Sep 20 02:31:00.399100 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:31:00.399081    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cd7550e3-b8c8-48b1-ab1c-21725e626fd8" containerName="manager"
Sep 20 02:31:00.399100 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:31:00.399088    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd7550e3-b8c8-48b1-ab1c-21725e626fd8" containerName="manager"
Sep 20 02:31:00.399200 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:31:00.399157    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="cb81a34c-3882-46fc-8436-c5c94ad03045" containerName="istio-proxy"
Sep 20 02:31:00.399200 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:31:00.399166    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="f0bec830-952f-41b2-9077-8fd4b8c0cd89" containerName="istio-proxy"
Sep 20 02:31:00.399200 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:31:00.399172    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="cd7550e3-b8c8-48b1-ab1c-21725e626fd8" containerName="manager"
Sep 20 02:31:00.403397 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:31:00.403379    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6f56f85f68-zwvbx"
Sep 20 02:31:00.407188 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:31:00.407164    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-mv62k\""
Sep 20 02:31:00.407325 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:31:00.407308    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 20 02:31:00.407394 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:31:00.407371    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 20 02:31:00.408280 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:31:00.408191    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 20 02:31:00.415020 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:31:00.414995    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6f56f85f68-zwvbx"]
Sep 20 02:31:00.517546 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:31:00.517480    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/45ae3219-4dc3-4614-82c0-2be885dee4e3-metrics-certs\") pod \"maas-controller-6f56f85f68-zwvbx\" (UID: \"45ae3219-4dc3-4614-82c0-2be885dee4e3\") " pod="opendatahub/maas-controller-6f56f85f68-zwvbx"
Sep 20 02:31:00.517721 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:31:00.517581    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/45ae3219-4dc3-4614-82c0-2be885dee4e3-webhook-cert\") pod \"maas-controller-6f56f85f68-zwvbx\" (UID: \"45ae3219-4dc3-4614-82c0-2be885dee4e3\") " pod="opendatahub/maas-controller-6f56f85f68-zwvbx"
Sep 20 02:31:00.517721 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:31:00.517619    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c4v69\" (UniqueName: \"kubernetes.io/projected/45ae3219-4dc3-4614-82c0-2be885dee4e3-kube-api-access-c4v69\") pod \"maas-controller-6f56f85f68-zwvbx\" (UID: \"45ae3219-4dc3-4614-82c0-2be885dee4e3\") " pod="opendatahub/maas-controller-6f56f85f68-zwvbx"
Sep 20 02:31:00.618348 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:31:00.618257    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/45ae3219-4dc3-4614-82c0-2be885dee4e3-webhook-cert\") pod \"maas-controller-6f56f85f68-zwvbx\" (UID: \"45ae3219-4dc3-4614-82c0-2be885dee4e3\") " pod="opendatahub/maas-controller-6f56f85f68-zwvbx"
Sep 20 02:31:00.618348 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:31:00.618309    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c4v69\" (UniqueName: \"kubernetes.io/projected/45ae3219-4dc3-4614-82c0-2be885dee4e3-kube-api-access-c4v69\") pod \"maas-controller-6f56f85f68-zwvbx\" (UID: \"45ae3219-4dc3-4614-82c0-2be885dee4e3\") " pod="opendatahub/maas-controller-6f56f85f68-zwvbx"
Sep 20 02:31:00.618630 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:31:00.618353    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/45ae3219-4dc3-4614-82c0-2be885dee4e3-metrics-certs\") pod \"maas-controller-6f56f85f68-zwvbx\" (UID: \"45ae3219-4dc3-4614-82c0-2be885dee4e3\") " pod="opendatahub/maas-controller-6f56f85f68-zwvbx"
Sep 20 02:31:00.620851 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:31:00.620823    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/45ae3219-4dc3-4614-82c0-2be885dee4e3-webhook-cert\") pod \"maas-controller-6f56f85f68-zwvbx\" (UID: \"45ae3219-4dc3-4614-82c0-2be885dee4e3\") " pod="opendatahub/maas-controller-6f56f85f68-zwvbx"
Sep 20 02:31:00.620960 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:31:00.620907    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/45ae3219-4dc3-4614-82c0-2be885dee4e3-metrics-certs\") pod \"maas-controller-6f56f85f68-zwvbx\" (UID: \"45ae3219-4dc3-4614-82c0-2be885dee4e3\") " pod="opendatahub/maas-controller-6f56f85f68-zwvbx"
Sep 20 02:31:00.626488 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:31:00.626466    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c4v69\" (UniqueName: \"kubernetes.io/projected/45ae3219-4dc3-4614-82c0-2be885dee4e3-kube-api-access-c4v69\") pod \"maas-controller-6f56f85f68-zwvbx\" (UID: \"45ae3219-4dc3-4614-82c0-2be885dee4e3\") " pod="opendatahub/maas-controller-6f56f85f68-zwvbx"
Sep 20 02:31:00.717365 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:31:00.717332    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6f56f85f68-zwvbx"
Sep 20 02:31:00.989624 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:31:00.989567    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6f56f85f68-zwvbx"]
Sep 20 02:31:00.991749 ip-10-0-130-159 kubenswrapper[2584]: W0920 02:31:00.991729    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45ae3219_4dc3_4614_82c0_2be885dee4e3.slice/crio-cfaeee32aa68c6d5156592809d8b376ded4137ec234e9e778662120e4672bc9c WatchSource:0}: Error finding container cfaeee32aa68c6d5156592809d8b376ded4137ec234e9e778662120e4672bc9c: Status 404 returned error can't find the container with id cfaeee32aa68c6d5156592809d8b376ded4137ec234e9e778662120e4672bc9c
Sep 20 02:31:01.093033 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:31:01.093001    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cd7550e3-b8c8-48b1-ab1c-21725e626fd8" path="/var/lib/kubelet/pods/cd7550e3-b8c8-48b1-ab1c-21725e626fd8/volumes"
Sep 20 02:31:01.591579 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:31:01.591543    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6f56f85f68-zwvbx" event={"ID":"45ae3219-4dc3-4614-82c0-2be885dee4e3","Type":"ContainerStarted","Data":"9a796854d99635be05abe55831ff06c9a3880a1bc142796bf61e0053ed19790c"}
Sep 20 02:31:01.592123 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:31:01.591604    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6f56f85f68-zwvbx" event={"ID":"45ae3219-4dc3-4614-82c0-2be885dee4e3","Type":"ContainerStarted","Data":"cfaeee32aa68c6d5156592809d8b376ded4137ec234e9e778662120e4672bc9c"}
Sep 20 02:31:01.592123 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:31:01.591631    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6f56f85f68-zwvbx"
Sep 20 02:31:01.614912 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:31:01.614819    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6f56f85f68-zwvbx" podStartSLOduration=1.253762742 podStartE2EDuration="1.614803085s" totalImagesPullingTime="361.040343ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-20 02:31:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-20 02:31:01.613029827 +0000 UTC m=+2318.355709034" watchObservedRunningTime="2026-09-20 02:31:01.614803085 +0000 UTC m=+2318.357482289"
Sep 20 02:31:12.600470 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:31:12.600383    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6f56f85f68-zwvbx"
Sep 20 02:31:30.529028 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:31:30.528977    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:31:30.584034 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:31:30.584004    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:31:44.174906 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:31:44.174872    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:31:44.275942 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:31:44.275904    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:31:59.875327 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:31:59.875293    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:32:05.616606 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:32:05.616567    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:32:25.297543 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:32:25.297493    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-ztdt2_ece82f19-68b5-4810-be50-d8678cfdee02/console-operator/1.log"
Sep 20 02:32:25.304649 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:32:25.304626    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-ztdt2_ece82f19-68b5-4810-be50-d8678cfdee02/console-operator/1.log"
Sep 20 02:32:40.948034 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:32:40.947949    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-gxh8t"]
Sep 20 02:32:40.948588 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:32:40.948190    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-gxh8t" podUID="af876e7a-2edc-43c3-a936-0bc828f859b9" containerName="manager" containerID="cri-o://d9510916438794c59f789dc7a50c3602e1dd4068787b53c41650a12bd34aece6" gracePeriod=10
Sep 20 02:32:41.088847 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:32:41.088808    2584 generic.go:356] "Generic (PLEG): container finished" podID="af876e7a-2edc-43c3-a936-0bc828f859b9" containerID="d9510916438794c59f789dc7a50c3602e1dd4068787b53c41650a12bd34aece6" exitCode=0
Sep 20 02:32:41.094169 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:32:41.094133    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-gxh8t" event={"ID":"af876e7a-2edc-43c3-a936-0bc828f859b9","Type":"ContainerDied","Data":"d9510916438794c59f789dc7a50c3602e1dd4068787b53c41650a12bd34aece6"}
Sep 20 02:32:41.094427 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:32:41.094411    2584 scope.go:117] "RemoveContainer" containerID="ade5cebfb9803ec0ea1070d615995ba2f73e2cc6333818c99ff51daa60905785"
Sep 20 02:32:41.211360 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:32:41.211297    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-gxh8t"
Sep 20 02:32:41.300640 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:32:41.300601    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/af876e7a-2edc-43c3-a936-0bc828f859b9-extensions-socket-volume\") pod \"af876e7a-2edc-43c3-a936-0bc828f859b9\" (UID: \"af876e7a-2edc-43c3-a936-0bc828f859b9\") "
Sep 20 02:32:41.300815 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:32:41.300678    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qs6nj\" (UniqueName: \"kubernetes.io/projected/af876e7a-2edc-43c3-a936-0bc828f859b9-kube-api-access-qs6nj\") pod \"af876e7a-2edc-43c3-a936-0bc828f859b9\" (UID: \"af876e7a-2edc-43c3-a936-0bc828f859b9\") "
Sep 20 02:32:41.300971 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:32:41.300949    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af876e7a-2edc-43c3-a936-0bc828f859b9-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "af876e7a-2edc-43c3-a936-0bc828f859b9" (UID: "af876e7a-2edc-43c3-a936-0bc828f859b9"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 20 02:32:41.302736 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:32:41.302707    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/af876e7a-2edc-43c3-a936-0bc828f859b9-kube-api-access-qs6nj" (OuterVolumeSpecName: "kube-api-access-qs6nj") pod "af876e7a-2edc-43c3-a936-0bc828f859b9" (UID: "af876e7a-2edc-43c3-a936-0bc828f859b9"). InnerVolumeSpecName "kube-api-access-qs6nj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 20 02:32:41.401936 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:32:41.401898    2584 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/af876e7a-2edc-43c3-a936-0bc828f859b9-extensions-socket-volume\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:32:41.401936 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:32:41.401931    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qs6nj\" (UniqueName: \"kubernetes.io/projected/af876e7a-2edc-43c3-a936-0bc828f859b9-kube-api-access-qs6nj\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:32:42.094941 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:32:42.094911    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-gxh8t"
Sep 20 02:32:42.095402 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:32:42.094950    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-gxh8t" event={"ID":"af876e7a-2edc-43c3-a936-0bc828f859b9","Type":"ContainerDied","Data":"ca4859fe507227054416a74e2ecaf5045381db53b0c57a57c7488e9c5e3041e9"}
Sep 20 02:32:42.095402 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:32:42.095005    2584 scope.go:117] "RemoveContainer" containerID="d9510916438794c59f789dc7a50c3602e1dd4068787b53c41650a12bd34aece6"
Sep 20 02:32:42.117742 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:32:42.117720    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-gxh8t"]
Sep 20 02:32:42.120869 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:32:42.120846    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-gxh8t"]
Sep 20 02:32:43.093253 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:32:43.093221    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="af876e7a-2edc-43c3-a936-0bc828f859b9" path="/var/lib/kubelet/pods/af876e7a-2edc-43c3-a936-0bc828f859b9/volumes"
Sep 20 02:33:41.633197 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:33:41.633166    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-xmnfb"]
Sep 20 02:33:41.633692 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:33:41.633584    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="af876e7a-2edc-43c3-a936-0bc828f859b9" containerName="manager"
Sep 20 02:33:41.633692 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:33:41.633598    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="af876e7a-2edc-43c3-a936-0bc828f859b9" containerName="manager"
Sep 20 02:33:41.633692 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:33:41.633606    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="af876e7a-2edc-43c3-a936-0bc828f859b9" containerName="manager"
Sep 20 02:33:41.633692 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:33:41.633612    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="af876e7a-2edc-43c3-a936-0bc828f859b9" containerName="manager"
Sep 20 02:33:41.633692 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:33:41.633673    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="af876e7a-2edc-43c3-a936-0bc828f859b9" containerName="manager"
Sep 20 02:33:41.633692 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:33:41.633682    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="af876e7a-2edc-43c3-a936-0bc828f859b9" containerName="manager"
Sep 20 02:33:41.637827 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:33:41.637807    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-xmnfb"
Sep 20 02:33:41.640343 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:33:41.640323    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-j9sqj\""
Sep 20 02:33:41.650367 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:33:41.650346    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-xmnfb"]
Sep 20 02:33:41.728425 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:33:41.728392    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s9b8r\" (UniqueName: \"kubernetes.io/projected/f93ff31e-f5fd-49b9-86bc-d05de963b33c-kube-api-access-s9b8r\") pod \"kuadrant-operator-controller-manager-5769f99d9f-xmnfb\" (UID: \"f93ff31e-f5fd-49b9-86bc-d05de963b33c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-xmnfb"
Sep 20 02:33:41.728599 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:33:41.728450    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f93ff31e-f5fd-49b9-86bc-d05de963b33c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-xmnfb\" (UID: \"f93ff31e-f5fd-49b9-86bc-d05de963b33c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-xmnfb"
Sep 20 02:33:41.829838 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:33:41.829797    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s9b8r\" (UniqueName: \"kubernetes.io/projected/f93ff31e-f5fd-49b9-86bc-d05de963b33c-kube-api-access-s9b8r\") pod \"kuadrant-operator-controller-manager-5769f99d9f-xmnfb\" (UID: \"f93ff31e-f5fd-49b9-86bc-d05de963b33c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-xmnfb"
Sep 20 02:33:41.830015 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:33:41.829861    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f93ff31e-f5fd-49b9-86bc-d05de963b33c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-xmnfb\" (UID: \"f93ff31e-f5fd-49b9-86bc-d05de963b33c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-xmnfb"
Sep 20 02:33:41.830205 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:33:41.830188    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f93ff31e-f5fd-49b9-86bc-d05de963b33c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-xmnfb\" (UID: \"f93ff31e-f5fd-49b9-86bc-d05de963b33c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-xmnfb"
Sep 20 02:33:41.843586 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:33:41.843555    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s9b8r\" (UniqueName: \"kubernetes.io/projected/f93ff31e-f5fd-49b9-86bc-d05de963b33c-kube-api-access-s9b8r\") pod \"kuadrant-operator-controller-manager-5769f99d9f-xmnfb\" (UID: \"f93ff31e-f5fd-49b9-86bc-d05de963b33c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-xmnfb"
Sep 20 02:33:41.947910 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:33:41.947884    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-xmnfb"
Sep 20 02:33:42.232830 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:33:42.232747    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-xmnfb"]
Sep 20 02:33:42.236458 ip-10-0-130-159 kubenswrapper[2584]: W0920 02:33:42.236428    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf93ff31e_f5fd_49b9_86bc_d05de963b33c.slice/crio-ddbee0771715fe5145fcad1ebc07e0ff169311db37997bc7dde63d52bb281238 WatchSource:0}: Error finding container ddbee0771715fe5145fcad1ebc07e0ff169311db37997bc7dde63d52bb281238: Status 404 returned error can't find the container with id ddbee0771715fe5145fcad1ebc07e0ff169311db37997bc7dde63d52bb281238
Sep 20 02:33:42.378961 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:33:42.378929    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-xmnfb" event={"ID":"f93ff31e-f5fd-49b9-86bc-d05de963b33c","Type":"ContainerStarted","Data":"b640dacbcafee51891c7fa9e2ea7e44456af15e216453afe21e5e43847e53276"}
Sep 20 02:33:42.379141 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:33:42.378990    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-xmnfb" event={"ID":"f93ff31e-f5fd-49b9-86bc-d05de963b33c","Type":"ContainerStarted","Data":"ddbee0771715fe5145fcad1ebc07e0ff169311db37997bc7dde63d52bb281238"}
Sep 20 02:33:42.379141 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:33:42.379021    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-xmnfb"
Sep 20 02:33:42.399060 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:33:42.399010    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-xmnfb" podStartSLOduration=1.398995856 podStartE2EDuration="1.398995856s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-20 02:33:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-20 02:33:42.397436358 +0000 UTC m=+2479.140115563" watchObservedRunningTime="2026-09-20 02:33:42.398995856 +0000 UTC m=+2479.141675060"
Sep 20 02:33:53.386500 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:33:53.386466    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-xmnfb"
Sep 20 02:34:18.914053 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:34:18.913966    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:34:25.923604 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:34:25.923570    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-5ce30569-kserve-74b8f647cf-8kqsm"]
Sep 20 02:34:25.927196 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:34:25.927170    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-5ce30569-kserve-74b8f647cf-8kqsm"
Sep 20 02:34:25.929887 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:34:25.929865    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-5ce30569-kserve-self-signed-certs\""
Sep 20 02:34:25.937325 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:34:25.937295    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-5ce30569-kserve-74b8f647cf-8kqsm"]
Sep 20 02:34:25.956240 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:34:25.956195    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:34:26.010258 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:34:26.010222    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/034be62d-f0ba-469b-802e-5c98745dfa95-tls-certs\") pod \"e2e-same-id-b-5ce30569-kserve-74b8f647cf-8kqsm\" (UID: \"034be62d-f0ba-469b-802e-5c98745dfa95\") " pod="llm/e2e-same-id-b-5ce30569-kserve-74b8f647cf-8kqsm"
Sep 20 02:34:26.010258 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:34:26.010266    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/034be62d-f0ba-469b-802e-5c98745dfa95-dshm\") pod \"e2e-same-id-b-5ce30569-kserve-74b8f647cf-8kqsm\" (UID: \"034be62d-f0ba-469b-802e-5c98745dfa95\") " pod="llm/e2e-same-id-b-5ce30569-kserve-74b8f647cf-8kqsm"
Sep 20 02:34:26.010580 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:34:26.010290    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lkq6b\" (UniqueName: \"kubernetes.io/projected/034be62d-f0ba-469b-802e-5c98745dfa95-kube-api-access-lkq6b\") pod \"e2e-same-id-b-5ce30569-kserve-74b8f647cf-8kqsm\" (UID: \"034be62d-f0ba-469b-802e-5c98745dfa95\") " pod="llm/e2e-same-id-b-5ce30569-kserve-74b8f647cf-8kqsm"
Sep 20 02:34:26.010580 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:34:26.010386    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/034be62d-f0ba-469b-802e-5c98745dfa95-home\") pod \"e2e-same-id-b-5ce30569-kserve-74b8f647cf-8kqsm\" (UID: \"034be62d-f0ba-469b-802e-5c98745dfa95\") " pod="llm/e2e-same-id-b-5ce30569-kserve-74b8f647cf-8kqsm"
Sep 20 02:34:26.010580 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:34:26.010504    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/034be62d-f0ba-469b-802e-5c98745dfa95-model-cache\") pod \"e2e-same-id-b-5ce30569-kserve-74b8f647cf-8kqsm\" (UID: \"034be62d-f0ba-469b-802e-5c98745dfa95\") " pod="llm/e2e-same-id-b-5ce30569-kserve-74b8f647cf-8kqsm"
Sep 20 02:34:26.010716 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:34:26.010579    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/034be62d-f0ba-469b-802e-5c98745dfa95-tmp-dir\") pod \"e2e-same-id-b-5ce30569-kserve-74b8f647cf-8kqsm\" (UID: \"034be62d-f0ba-469b-802e-5c98745dfa95\") " pod="llm/e2e-same-id-b-5ce30569-kserve-74b8f647cf-8kqsm"
Sep 20 02:34:26.111358 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:34:26.111321    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lkq6b\" (UniqueName: \"kubernetes.io/projected/034be62d-f0ba-469b-802e-5c98745dfa95-kube-api-access-lkq6b\") pod \"e2e-same-id-b-5ce30569-kserve-74b8f647cf-8kqsm\" (UID: \"034be62d-f0ba-469b-802e-5c98745dfa95\") " pod="llm/e2e-same-id-b-5ce30569-kserve-74b8f647cf-8kqsm"
Sep 20 02:34:26.111575 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:34:26.111379    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/034be62d-f0ba-469b-802e-5c98745dfa95-home\") pod \"e2e-same-id-b-5ce30569-kserve-74b8f647cf-8kqsm\" (UID: \"034be62d-f0ba-469b-802e-5c98745dfa95\") " pod="llm/e2e-same-id-b-5ce30569-kserve-74b8f647cf-8kqsm"
Sep 20 02:34:26.111656 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:34:26.111582    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/034be62d-f0ba-469b-802e-5c98745dfa95-model-cache\") pod \"e2e-same-id-b-5ce30569-kserve-74b8f647cf-8kqsm\" (UID: \"034be62d-f0ba-469b-802e-5c98745dfa95\") " pod="llm/e2e-same-id-b-5ce30569-kserve-74b8f647cf-8kqsm"
Sep 20 02:34:26.111718 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:34:26.111634    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/034be62d-f0ba-469b-802e-5c98745dfa95-tmp-dir\") pod \"e2e-same-id-b-5ce30569-kserve-74b8f647cf-8kqsm\" (UID: \"034be62d-f0ba-469b-802e-5c98745dfa95\") " pod="llm/e2e-same-id-b-5ce30569-kserve-74b8f647cf-8kqsm"
Sep 20 02:34:26.111777 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:34:26.111713    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/034be62d-f0ba-469b-802e-5c98745dfa95-tls-certs\") pod \"e2e-same-id-b-5ce30569-kserve-74b8f647cf-8kqsm\" (UID: \"034be62d-f0ba-469b-802e-5c98745dfa95\") " pod="llm/e2e-same-id-b-5ce30569-kserve-74b8f647cf-8kqsm"
Sep 20 02:34:26.111777 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:34:26.111769    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/034be62d-f0ba-469b-802e-5c98745dfa95-dshm\") pod \"e2e-same-id-b-5ce30569-kserve-74b8f647cf-8kqsm\" (UID: \"034be62d-f0ba-469b-802e-5c98745dfa95\") " pod="llm/e2e-same-id-b-5ce30569-kserve-74b8f647cf-8kqsm"
Sep 20 02:34:26.112019 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:34:26.111994    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/034be62d-f0ba-469b-802e-5c98745dfa95-home\") pod \"e2e-same-id-b-5ce30569-kserve-74b8f647cf-8kqsm\" (UID: \"034be62d-f0ba-469b-802e-5c98745dfa95\") " pod="llm/e2e-same-id-b-5ce30569-kserve-74b8f647cf-8kqsm"
Sep 20 02:34:26.112210 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:34:26.112034    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/034be62d-f0ba-469b-802e-5c98745dfa95-tmp-dir\") pod \"e2e-same-id-b-5ce30569-kserve-74b8f647cf-8kqsm\" (UID: \"034be62d-f0ba-469b-802e-5c98745dfa95\") " pod="llm/e2e-same-id-b-5ce30569-kserve-74b8f647cf-8kqsm"
Sep 20 02:34:26.112285 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:34:26.112075    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/034be62d-f0ba-469b-802e-5c98745dfa95-model-cache\") pod \"e2e-same-id-b-5ce30569-kserve-74b8f647cf-8kqsm\" (UID: \"034be62d-f0ba-469b-802e-5c98745dfa95\") " pod="llm/e2e-same-id-b-5ce30569-kserve-74b8f647cf-8kqsm"
Sep 20 02:34:26.114172 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:34:26.114147    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/034be62d-f0ba-469b-802e-5c98745dfa95-dshm\") pod \"e2e-same-id-b-5ce30569-kserve-74b8f647cf-8kqsm\" (UID: \"034be62d-f0ba-469b-802e-5c98745dfa95\") " pod="llm/e2e-same-id-b-5ce30569-kserve-74b8f647cf-8kqsm"
Sep 20 02:34:26.114501 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:34:26.114476    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/034be62d-f0ba-469b-802e-5c98745dfa95-tls-certs\") pod \"e2e-same-id-b-5ce30569-kserve-74b8f647cf-8kqsm\" (UID: \"034be62d-f0ba-469b-802e-5c98745dfa95\") " pod="llm/e2e-same-id-b-5ce30569-kserve-74b8f647cf-8kqsm"
Sep 20 02:34:26.120229 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:34:26.120205    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lkq6b\" (UniqueName: \"kubernetes.io/projected/034be62d-f0ba-469b-802e-5c98745dfa95-kube-api-access-lkq6b\") pod \"e2e-same-id-b-5ce30569-kserve-74b8f647cf-8kqsm\" (UID: \"034be62d-f0ba-469b-802e-5c98745dfa95\") " pod="llm/e2e-same-id-b-5ce30569-kserve-74b8f647cf-8kqsm"
Sep 20 02:34:26.237237 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:34:26.237196    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-5ce30569-kserve-74b8f647cf-8kqsm"
Sep 20 02:34:26.516489 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:34:26.516464    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-5ce30569-kserve-74b8f647cf-8kqsm"]
Sep 20 02:34:26.519151 ip-10-0-130-159 kubenswrapper[2584]: W0920 02:34:26.519123    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod034be62d_f0ba_469b_802e_5c98745dfa95.slice/crio-7680d5113e0688edc9d441c3623e5c3bf68db918f3b65582629e36f8e7bf779a WatchSource:0}: Error finding container 7680d5113e0688edc9d441c3623e5c3bf68db918f3b65582629e36f8e7bf779a: Status 404 returned error can't find the container with id 7680d5113e0688edc9d441c3623e5c3bf68db918f3b65582629e36f8e7bf779a
Sep 20 02:34:26.604704 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:34:26.604670    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-5ce30569-kserve-74b8f647cf-8kqsm" event={"ID":"034be62d-f0ba-469b-802e-5c98745dfa95","Type":"ContainerStarted","Data":"b368a66bce2000c3e232276f758e7642b96096325c140491ab28a84d0abaedeb"}
Sep 20 02:34:26.604862 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:34:26.604715    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-5ce30569-kserve-74b8f647cf-8kqsm" event={"ID":"034be62d-f0ba-469b-802e-5c98745dfa95","Type":"ContainerStarted","Data":"7680d5113e0688edc9d441c3623e5c3bf68db918f3b65582629e36f8e7bf779a"}
Sep 20 02:34:26.623686 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:34:26.623632    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-5ce30569-kserve-74b8f647cf-8kqsm" podStartSLOduration=1.623614849 podStartE2EDuration="1.623614849s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-20 02:34:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-20 02:34:26.622855372 +0000 UTC m=+2523.365534578" watchObservedRunningTime="2026-09-20 02:34:26.623614849 +0000 UTC m=+2523.366294056"
Sep 20 02:34:36.238099 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:34:36.238062    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-5ce30569-kserve-74b8f647cf-8kqsm"
Sep 20 02:34:36.238099 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:34:36.238108    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-5ce30569-kserve-74b8f647cf-8kqsm"
Sep 20 02:34:36.250922 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:34:36.250894    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-5ce30569-kserve-74b8f647cf-8kqsm"
Sep 20 02:34:36.666135 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:34:36.666052    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-5ce30569-kserve-74b8f647cf-8kqsm"
Sep 20 02:34:40.422541 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:34:40.422475    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:34:45.322009 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:34:45.321965    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:34:46.130571 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:34:46.130506    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-5ce30569-kserve-74b8f647cf-8kqsm"]
Sep 20 02:34:46.154245 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:34:46.154210    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:35:01.152059 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:35:01.152006    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-5ce30569-kserve-74b8f647cf-8kqsm" podUID="034be62d-f0ba-469b-802e-5c98745dfa95" containerName="main" containerID="cri-o://b368a66bce2000c3e232276f758e7642b96096325c140491ab28a84d0abaedeb" gracePeriod=45
Sep 20 02:35:01.414822 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:35:01.414746    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-5ce30569-kserve-74b8f647cf-8kqsm"
Sep 20 02:35:01.542033 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:35:01.541995    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lkq6b\" (UniqueName: \"kubernetes.io/projected/034be62d-f0ba-469b-802e-5c98745dfa95-kube-api-access-lkq6b\") pod \"034be62d-f0ba-469b-802e-5c98745dfa95\" (UID: \"034be62d-f0ba-469b-802e-5c98745dfa95\") "
Sep 20 02:35:01.542261 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:35:01.542077    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/034be62d-f0ba-469b-802e-5c98745dfa95-tmp-dir\") pod \"034be62d-f0ba-469b-802e-5c98745dfa95\" (UID: \"034be62d-f0ba-469b-802e-5c98745dfa95\") "
Sep 20 02:35:01.542261 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:35:01.542123    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/034be62d-f0ba-469b-802e-5c98745dfa95-tls-certs\") pod \"034be62d-f0ba-469b-802e-5c98745dfa95\" (UID: \"034be62d-f0ba-469b-802e-5c98745dfa95\") "
Sep 20 02:35:01.542261 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:35:01.542149    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/034be62d-f0ba-469b-802e-5c98745dfa95-model-cache\") pod \"034be62d-f0ba-469b-802e-5c98745dfa95\" (UID: \"034be62d-f0ba-469b-802e-5c98745dfa95\") "
Sep 20 02:35:01.542261 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:35:01.542178    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/034be62d-f0ba-469b-802e-5c98745dfa95-home\") pod \"034be62d-f0ba-469b-802e-5c98745dfa95\" (UID: \"034be62d-f0ba-469b-802e-5c98745dfa95\") "
Sep 20 02:35:01.542261 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:35:01.542203    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/034be62d-f0ba-469b-802e-5c98745dfa95-dshm\") pod \"034be62d-f0ba-469b-802e-5c98745dfa95\" (UID: \"034be62d-f0ba-469b-802e-5c98745dfa95\") "
Sep 20 02:35:01.542574 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:35:01.542412    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/034be62d-f0ba-469b-802e-5c98745dfa95-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "034be62d-f0ba-469b-802e-5c98745dfa95" (UID: "034be62d-f0ba-469b-802e-5c98745dfa95"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 20 02:35:01.542574 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:35:01.542477    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/034be62d-f0ba-469b-802e-5c98745dfa95-model-cache" (OuterVolumeSpecName: "model-cache") pod "034be62d-f0ba-469b-802e-5c98745dfa95" (UID: "034be62d-f0ba-469b-802e-5c98745dfa95"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 20 02:35:01.542574 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:35:01.542488    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/034be62d-f0ba-469b-802e-5c98745dfa95-home" (OuterVolumeSpecName: "home") pod "034be62d-f0ba-469b-802e-5c98745dfa95" (UID: "034be62d-f0ba-469b-802e-5c98745dfa95"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 20 02:35:01.544491 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:35:01.544465    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/034be62d-f0ba-469b-802e-5c98745dfa95-dshm" (OuterVolumeSpecName: "dshm") pod "034be62d-f0ba-469b-802e-5c98745dfa95" (UID: "034be62d-f0ba-469b-802e-5c98745dfa95"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 20 02:35:01.544610 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:35:01.544500    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/034be62d-f0ba-469b-802e-5c98745dfa95-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "034be62d-f0ba-469b-802e-5c98745dfa95" (UID: "034be62d-f0ba-469b-802e-5c98745dfa95"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 20 02:35:01.544610 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:35:01.544597    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/034be62d-f0ba-469b-802e-5c98745dfa95-kube-api-access-lkq6b" (OuterVolumeSpecName: "kube-api-access-lkq6b") pod "034be62d-f0ba-469b-802e-5c98745dfa95" (UID: "034be62d-f0ba-469b-802e-5c98745dfa95"). InnerVolumeSpecName "kube-api-access-lkq6b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 20 02:35:01.643799 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:35:01.643751    2584 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/034be62d-f0ba-469b-802e-5c98745dfa95-tls-certs\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:35:01.643799 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:35:01.643790    2584 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/034be62d-f0ba-469b-802e-5c98745dfa95-model-cache\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:35:01.643799 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:35:01.643802    2584 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/034be62d-f0ba-469b-802e-5c98745dfa95-home\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:35:01.643799 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:35:01.643811    2584 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/034be62d-f0ba-469b-802e-5c98745dfa95-dshm\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:35:01.644095 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:35:01.643823    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lkq6b\" (UniqueName: \"kubernetes.io/projected/034be62d-f0ba-469b-802e-5c98745dfa95-kube-api-access-lkq6b\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:35:01.644095 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:35:01.643836    2584 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/034be62d-f0ba-469b-802e-5c98745dfa95-tmp-dir\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:35:01.784284 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:35:01.784240    2584 generic.go:356] "Generic (PLEG): container finished" podID="034be62d-f0ba-469b-802e-5c98745dfa95" containerID="b368a66bce2000c3e232276f758e7642b96096325c140491ab28a84d0abaedeb" exitCode=0
Sep 20 02:35:01.784480 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:35:01.784306    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-5ce30569-kserve-74b8f647cf-8kqsm"
Sep 20 02:35:01.784480 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:35:01.784332    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-5ce30569-kserve-74b8f647cf-8kqsm" event={"ID":"034be62d-f0ba-469b-802e-5c98745dfa95","Type":"ContainerDied","Data":"b368a66bce2000c3e232276f758e7642b96096325c140491ab28a84d0abaedeb"}
Sep 20 02:35:01.784480 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:35:01.784389    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-5ce30569-kserve-74b8f647cf-8kqsm" event={"ID":"034be62d-f0ba-469b-802e-5c98745dfa95","Type":"ContainerDied","Data":"7680d5113e0688edc9d441c3623e5c3bf68db918f3b65582629e36f8e7bf779a"}
Sep 20 02:35:01.784480 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:35:01.784412    2584 scope.go:117] "RemoveContainer" containerID="b368a66bce2000c3e232276f758e7642b96096325c140491ab28a84d0abaedeb"
Sep 20 02:35:01.794292 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:35:01.794271    2584 scope.go:117] "RemoveContainer" containerID="b368a66bce2000c3e232276f758e7642b96096325c140491ab28a84d0abaedeb"
Sep 20 02:35:01.794610 ip-10-0-130-159 kubenswrapper[2584]: E0920 02:35:01.794588    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b368a66bce2000c3e232276f758e7642b96096325c140491ab28a84d0abaedeb\": container with ID starting with b368a66bce2000c3e232276f758e7642b96096325c140491ab28a84d0abaedeb not found: ID does not exist" containerID="b368a66bce2000c3e232276f758e7642b96096325c140491ab28a84d0abaedeb"
Sep 20 02:35:01.794670 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:35:01.794621    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b368a66bce2000c3e232276f758e7642b96096325c140491ab28a84d0abaedeb"} err="failed to get container status \"b368a66bce2000c3e232276f758e7642b96096325c140491ab28a84d0abaedeb\": rpc error: code = NotFound desc = could not find container \"b368a66bce2000c3e232276f758e7642b96096325c140491ab28a84d0abaedeb\": container with ID starting with b368a66bce2000c3e232276f758e7642b96096325c140491ab28a84d0abaedeb not found: ID does not exist"
Sep 20 02:35:01.805822 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:35:01.805792    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-5ce30569-kserve-74b8f647cf-8kqsm"]
Sep 20 02:35:01.812545 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:35:01.812505    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-5ce30569-kserve-74b8f647cf-8kqsm"]
Sep 20 02:35:03.093855 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:35:03.093822    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="034be62d-f0ba-469b-802e-5c98745dfa95" path="/var/lib/kubelet/pods/034be62d-f0ba-469b-802e-5c98745dfa95/volumes"
Sep 20 02:35:08.534682 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:35:08.534640    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-58647f984c-5pmkk"]
Sep 20 02:35:08.535246 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:35:08.535226    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="034be62d-f0ba-469b-802e-5c98745dfa95" containerName="main"
Sep 20 02:35:08.535308 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:35:08.535250    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="034be62d-f0ba-469b-802e-5c98745dfa95" containerName="main"
Sep 20 02:35:08.535356 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:35:08.535321    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="034be62d-f0ba-469b-802e-5c98745dfa95" containerName="main"
Sep 20 02:35:08.540437 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:35:08.540416    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-58647f984c-5pmkk"
Sep 20 02:35:08.548369 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:35:08.548340    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-58647f984c-5pmkk"]
Sep 20 02:35:08.602106 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:35:08.602058    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f950bc30-dbfb-437c-8ddc-05206f0d3faf-metrics-certs\") pod \"maas-api-58647f984c-5pmkk\" (UID: \"f950bc30-dbfb-437c-8ddc-05206f0d3faf\") " pod="odh-ai-gateway-infra/maas-api-58647f984c-5pmkk"
Sep 20 02:35:08.602304 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:35:08.602118    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kvqzg\" (UniqueName: \"kubernetes.io/projected/f950bc30-dbfb-437c-8ddc-05206f0d3faf-kube-api-access-kvqzg\") pod \"maas-api-58647f984c-5pmkk\" (UID: \"f950bc30-dbfb-437c-8ddc-05206f0d3faf\") " pod="odh-ai-gateway-infra/maas-api-58647f984c-5pmkk"
Sep 20 02:35:08.602304 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:35:08.602182    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f950bc30-dbfb-437c-8ddc-05206f0d3faf-maas-api-tls\") pod \"maas-api-58647f984c-5pmkk\" (UID: \"f950bc30-dbfb-437c-8ddc-05206f0d3faf\") " pod="odh-ai-gateway-infra/maas-api-58647f984c-5pmkk"
Sep 20 02:35:08.602304 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:35:08.602209    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f950bc30-dbfb-437c-8ddc-05206f0d3faf-openshift-service-ca\") pod \"maas-api-58647f984c-5pmkk\" (UID: \"f950bc30-dbfb-437c-8ddc-05206f0d3faf\") " pod="odh-ai-gateway-infra/maas-api-58647f984c-5pmkk"
Sep 20 02:35:08.703644 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:35:08.703607    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f950bc30-dbfb-437c-8ddc-05206f0d3faf-metrics-certs\") pod \"maas-api-58647f984c-5pmkk\" (UID: \"f950bc30-dbfb-437c-8ddc-05206f0d3faf\") " pod="odh-ai-gateway-infra/maas-api-58647f984c-5pmkk"
Sep 20 02:35:08.703838 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:35:08.703656    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kvqzg\" (UniqueName: \"kubernetes.io/projected/f950bc30-dbfb-437c-8ddc-05206f0d3faf-kube-api-access-kvqzg\") pod \"maas-api-58647f984c-5pmkk\" (UID: \"f950bc30-dbfb-437c-8ddc-05206f0d3faf\") " pod="odh-ai-gateway-infra/maas-api-58647f984c-5pmkk"
Sep 20 02:35:08.703838 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:35:08.703682    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f950bc30-dbfb-437c-8ddc-05206f0d3faf-maas-api-tls\") pod \"maas-api-58647f984c-5pmkk\" (UID: \"f950bc30-dbfb-437c-8ddc-05206f0d3faf\") " pod="odh-ai-gateway-infra/maas-api-58647f984c-5pmkk"
Sep 20 02:35:08.703838 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:35:08.703699    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f950bc30-dbfb-437c-8ddc-05206f0d3faf-openshift-service-ca\") pod \"maas-api-58647f984c-5pmkk\" (UID: \"f950bc30-dbfb-437c-8ddc-05206f0d3faf\") " pod="odh-ai-gateway-infra/maas-api-58647f984c-5pmkk"
Sep 20 02:35:08.704501 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:35:08.704477    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f950bc30-dbfb-437c-8ddc-05206f0d3faf-openshift-service-ca\") pod \"maas-api-58647f984c-5pmkk\" (UID: \"f950bc30-dbfb-437c-8ddc-05206f0d3faf\") " pod="odh-ai-gateway-infra/maas-api-58647f984c-5pmkk"
Sep 20 02:35:08.706309 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:35:08.706284    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f950bc30-dbfb-437c-8ddc-05206f0d3faf-metrics-certs\") pod \"maas-api-58647f984c-5pmkk\" (UID: \"f950bc30-dbfb-437c-8ddc-05206f0d3faf\") " pod="odh-ai-gateway-infra/maas-api-58647f984c-5pmkk"
Sep 20 02:35:08.706402 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:35:08.706337    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f950bc30-dbfb-437c-8ddc-05206f0d3faf-maas-api-tls\") pod \"maas-api-58647f984c-5pmkk\" (UID: \"f950bc30-dbfb-437c-8ddc-05206f0d3faf\") " pod="odh-ai-gateway-infra/maas-api-58647f984c-5pmkk"
Sep 20 02:35:08.711449 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:35:08.711423    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kvqzg\" (UniqueName: \"kubernetes.io/projected/f950bc30-dbfb-437c-8ddc-05206f0d3faf-kube-api-access-kvqzg\") pod \"maas-api-58647f984c-5pmkk\" (UID: \"f950bc30-dbfb-437c-8ddc-05206f0d3faf\") " pod="odh-ai-gateway-infra/maas-api-58647f984c-5pmkk"
Sep 20 02:35:08.854323 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:35:08.854227    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-58647f984c-5pmkk"
Sep 20 02:35:09.132757 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:35:09.132730    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-58647f984c-5pmkk"]
Sep 20 02:35:09.134833 ip-10-0-130-159 kubenswrapper[2584]: W0920 02:35:09.134808    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf950bc30_dbfb_437c_8ddc_05206f0d3faf.slice/crio-e753e3054b44ca96c785db8b2ed9c75b3dcf78c1e68fcfba9910aafad6c21391 WatchSource:0}: Error finding container e753e3054b44ca96c785db8b2ed9c75b3dcf78c1e68fcfba9910aafad6c21391: Status 404 returned error can't find the container with id e753e3054b44ca96c785db8b2ed9c75b3dcf78c1e68fcfba9910aafad6c21391
Sep 20 02:35:09.830230 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:35:09.830196    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-58647f984c-5pmkk" event={"ID":"f950bc30-dbfb-437c-8ddc-05206f0d3faf","Type":"ContainerStarted","Data":"cc0910cbb4e9138b4f50b7bb5af3e4586c64b9da1454ea92a4d158812c983de6"}
Sep 20 02:35:09.830634 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:35:09.830253    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-58647f984c-5pmkk" event={"ID":"f950bc30-dbfb-437c-8ddc-05206f0d3faf","Type":"ContainerStarted","Data":"e753e3054b44ca96c785db8b2ed9c75b3dcf78c1e68fcfba9910aafad6c21391"}
Sep 20 02:35:09.830634 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:35:09.830279    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-58647f984c-5pmkk"
Sep 20 02:35:09.851541 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:35:09.851417    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-58647f984c-5pmkk" podStartSLOduration=1.396836244 podStartE2EDuration="1.851400088s" totalImagesPullingTime="454.563844ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-20 02:35:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-20 02:35:09.849840426 +0000 UTC m=+2566.592519633" watchObservedRunningTime="2026-09-20 02:35:09.851400088 +0000 UTC m=+2566.594079292"
Sep 20 02:35:15.841968 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:35:15.841939    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-58647f984c-5pmkk"
Sep 20 02:35:24.040371 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:35:24.040333    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-5747fd5c8c-zx8np"]
Sep 20 02:35:24.040937 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:35:24.040623    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-5747fd5c8c-zx8np" podUID="b05ebed5-1002-4b61-8fac-d1227f949a39" containerName="maas-api" containerID="cri-o://cac7921b024a1727da1a9194f2158a14cc4a1fb89a61592f9486cb7879ad5493" gracePeriod=30
Sep 20 02:35:25.290680 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:35:25.290649    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5747fd5c8c-zx8np"
Sep 20 02:35:25.366078 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:35:25.365979    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b05ebed5-1002-4b61-8fac-d1227f949a39-openshift-service-ca\") pod \"b05ebed5-1002-4b61-8fac-d1227f949a39\" (UID: \"b05ebed5-1002-4b61-8fac-d1227f949a39\") "
Sep 20 02:35:25.366078 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:35:25.366064    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b05ebed5-1002-4b61-8fac-d1227f949a39-metrics-certs\") pod \"b05ebed5-1002-4b61-8fac-d1227f949a39\" (UID: \"b05ebed5-1002-4b61-8fac-d1227f949a39\") "
Sep 20 02:35:25.366338 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:35:25.366112    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v67kq\" (UniqueName: \"kubernetes.io/projected/b05ebed5-1002-4b61-8fac-d1227f949a39-kube-api-access-v67kq\") pod \"b05ebed5-1002-4b61-8fac-d1227f949a39\" (UID: \"b05ebed5-1002-4b61-8fac-d1227f949a39\") "
Sep 20 02:35:25.366338 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:35:25.366156    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b05ebed5-1002-4b61-8fac-d1227f949a39-maas-api-tls\") pod \"b05ebed5-1002-4b61-8fac-d1227f949a39\" (UID: \"b05ebed5-1002-4b61-8fac-d1227f949a39\") "
Sep 20 02:35:25.366477 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:35:25.366424    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b05ebed5-1002-4b61-8fac-d1227f949a39-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b05ebed5-1002-4b61-8fac-d1227f949a39" (UID: "b05ebed5-1002-4b61-8fac-d1227f949a39"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 20 02:35:25.368631 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:35:25.368600    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b05ebed5-1002-4b61-8fac-d1227f949a39-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "b05ebed5-1002-4b61-8fac-d1227f949a39" (UID: "b05ebed5-1002-4b61-8fac-d1227f949a39"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 20 02:35:25.368906 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:35:25.368888    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b05ebed5-1002-4b61-8fac-d1227f949a39-kube-api-access-v67kq" (OuterVolumeSpecName: "kube-api-access-v67kq") pod "b05ebed5-1002-4b61-8fac-d1227f949a39" (UID: "b05ebed5-1002-4b61-8fac-d1227f949a39"). InnerVolumeSpecName "kube-api-access-v67kq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 20 02:35:25.368985 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:35:25.368891    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b05ebed5-1002-4b61-8fac-d1227f949a39-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b05ebed5-1002-4b61-8fac-d1227f949a39" (UID: "b05ebed5-1002-4b61-8fac-d1227f949a39"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 20 02:35:25.467601 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:35:25.467556    2584 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b05ebed5-1002-4b61-8fac-d1227f949a39-metrics-certs\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:35:25.467601 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:35:25.467591    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v67kq\" (UniqueName: \"kubernetes.io/projected/b05ebed5-1002-4b61-8fac-d1227f949a39-kube-api-access-v67kq\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:35:25.467601 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:35:25.467602    2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b05ebed5-1002-4b61-8fac-d1227f949a39-maas-api-tls\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:35:25.467601 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:35:25.467611    2584 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b05ebed5-1002-4b61-8fac-d1227f949a39-openshift-service-ca\") on node \"ip-10-0-130-159.ec2.internal\" DevicePath \"\""
Sep 20 02:35:25.578818 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:35:25.578785    2584 scope.go:117] "RemoveContainer" containerID="cac7921b024a1727da1a9194f2158a14cc4a1fb89a61592f9486cb7879ad5493"
Sep 20 02:35:25.908971 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:35:25.908938    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5747fd5c8c-zx8np"
Sep 20 02:35:25.909185 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:35:25.908938    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5747fd5c8c-zx8np" event={"ID":"b05ebed5-1002-4b61-8fac-d1227f949a39","Type":"ContainerDied","Data":"cac7921b024a1727da1a9194f2158a14cc4a1fb89a61592f9486cb7879ad5493"}
Sep 20 02:35:25.909185 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:35:25.909088    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5747fd5c8c-zx8np" event={"ID":"b05ebed5-1002-4b61-8fac-d1227f949a39","Type":"ContainerDied","Data":"172733bb90a6af8bf625ca467780d1c2869964ec24ab2c2426af2c3321845319"}
Sep 20 02:35:25.938979 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:35:25.938943    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-5747fd5c8c-zx8np"]
Sep 20 02:35:25.942036 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:35:25.942003    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-5747fd5c8c-zx8np"]
Sep 20 02:35:27.092838 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:35:27.092796    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b05ebed5-1002-4b61-8fac-d1227f949a39" path="/var/lib/kubelet/pods/b05ebed5-1002-4b61-8fac-d1227f949a39/volumes"
Sep 20 02:35:27.416840 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:35:27.416760    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:35:41.436090 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:35:41.435993    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-84755fcbb8-rxcv5_807e839b-d2f0-4e52-b9b4-e98e4d2f05b5/authorino/0.log"
Sep 20 02:35:46.988349 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:35:46.988318    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-v8pxt_fe1d0a3b-8799-4617-8763-fa987349cf78/manager/0.log"
Sep 20 02:35:47.110979 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:35:47.110944    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-6vgqp_4f29b72e-019b-4f87-b9c2-9154fd7bb6ec/manager/0.log"
Sep 20 02:35:47.234788 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:35:47.234754    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6f56f85f68-zwvbx_45ae3219-4dc3-4614-82c0-2be885dee4e3/manager/0.log"
Sep 20 02:35:47.863378 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:35:47.863352    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-59d4cf4f4c-tqwb6_c7f8158a-9f79-4227-8623-1b767c7e29a3/copy-manifests/0.log"
Sep 20 02:35:47.870169 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:35:47.870144    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-59d4cf4f4c-tqwb6_c7f8158a-9f79-4227-8623-1b767c7e29a3/manager/0.log"
Sep 20 02:35:48.207510 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:35:48.207480    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-58647f984c-5pmkk_f950bc30-dbfb-437c-8ddc-05206f0d3faf/maas-api/0.log"
Sep 20 02:35:48.640511 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:35:48.640408    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-sph88_c9a02530-1010-4497-a848-8d1373a51d01/postgres/0.log"
Sep 20 02:35:49.421015 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:35:49.420977    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-84755fcbb8-rxcv5_807e839b-d2f0-4e52-b9b4-e98e4d2f05b5/authorino/0.log"
Sep 20 02:35:49.540400 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:35:49.540370    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-76b478bd78-m6b4s_464f7342-77e6-4c63-808c-cc6c73f8df11/manager/0.log"
Sep 20 02:35:49.655277 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:35:49.655240    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-7b84cd7c9c-mc75w_9002335e-a543-4c6c-ba94-cd0812a7978d/manager/0.log"
Sep 20 02:35:49.894375 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:35:49.894338    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-xmnfb_f93ff31e-f5fd-49b9-86bc-d05de963b33c/manager/0.log"
Sep 20 02:35:50.128756 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:35:50.128723    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-74495c69f8-krtwd_db69bff3-503c-495e-99a8-0d90f8f0a154/manager/0.log"
Sep 20 02:35:50.839493 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:35:50.839461    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-86bbc588c6-zlhv2_004ecde9-2ce2-41cf-b558-ae17143cd051/kube-auth-proxy/0.log"
Sep 20 02:35:51.082513 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:35:51.082451    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-z42zz_6bb258c3-e7d4-4656-9653-db0c9f68186e/payload-pre-processing/0.log"
Sep 20 02:35:51.319464 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:35:51.319423    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-95688946b-7rdfv_2dbc9737-865d-4dbf-85bf-8d2b9781a8d5/router/0.log"
Sep 20 02:35:51.888326 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:35:51.888295    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-embedding-simulated-kserve-bcc9ff8d8-sdq5l_7b10b1cd-1ae2-4ff2-be1d-b9c158059274/main/0.log"
Sep 20 02:35:57.984001 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:35:57.983962    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-gqx6h/must-gather-hdx8l"]
Sep 20 02:35:57.984812 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:35:57.984366    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b05ebed5-1002-4b61-8fac-d1227f949a39" containerName="maas-api"
Sep 20 02:35:57.984812 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:35:57.984378    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="b05ebed5-1002-4b61-8fac-d1227f949a39" containerName="maas-api"
Sep 20 02:35:57.984812 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:35:57.984459    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="b05ebed5-1002-4b61-8fac-d1227f949a39" containerName="maas-api"
Sep 20 02:35:57.987499 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:35:57.987481    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-gqx6h/must-gather-hdx8l"
Sep 20 02:35:57.989985 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:35:57.989961    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-gqx6h\"/\"openshift-service-ca.crt\""
Sep 20 02:35:57.990166 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:35:57.990150    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-gqx6h\"/\"default-dockercfg-bbk9m\""
Sep 20 02:35:57.990852 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:35:57.990835    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-gqx6h\"/\"kube-root-ca.crt\""
Sep 20 02:35:57.999435 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:35:57.999411    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-gqx6h/must-gather-hdx8l"]
Sep 20 02:35:58.010492 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:35:58.010462    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:35:58.050806 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:35:58.050766    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bpmsm\" (UniqueName: \"kubernetes.io/projected/20d5be36-ae37-4997-9bca-b538d9bbf899-kube-api-access-bpmsm\") pod \"must-gather-hdx8l\" (UID: \"20d5be36-ae37-4997-9bca-b538d9bbf899\") " pod="openshift-must-gather-gqx6h/must-gather-hdx8l"
Sep 20 02:35:58.050806 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:35:58.050804    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/20d5be36-ae37-4997-9bca-b538d9bbf899-must-gather-output\") pod \"must-gather-hdx8l\" (UID: \"20d5be36-ae37-4997-9bca-b538d9bbf899\") " pod="openshift-must-gather-gqx6h/must-gather-hdx8l"
Sep 20 02:35:58.098489 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:35:58.098452    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:35:58.151834 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:35:58.151800    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bpmsm\" (UniqueName: \"kubernetes.io/projected/20d5be36-ae37-4997-9bca-b538d9bbf899-kube-api-access-bpmsm\") pod \"must-gather-hdx8l\" (UID: \"20d5be36-ae37-4997-9bca-b538d9bbf899\") " pod="openshift-must-gather-gqx6h/must-gather-hdx8l"
Sep 20 02:35:58.151834 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:35:58.151838    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/20d5be36-ae37-4997-9bca-b538d9bbf899-must-gather-output\") pod \"must-gather-hdx8l\" (UID: \"20d5be36-ae37-4997-9bca-b538d9bbf899\") " pod="openshift-must-gather-gqx6h/must-gather-hdx8l"
Sep 20 02:35:58.152232 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:35:58.152215    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/20d5be36-ae37-4997-9bca-b538d9bbf899-must-gather-output\") pod \"must-gather-hdx8l\" (UID: \"20d5be36-ae37-4997-9bca-b538d9bbf899\") " pod="openshift-must-gather-gqx6h/must-gather-hdx8l"
Sep 20 02:35:58.166235 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:35:58.166202    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bpmsm\" (UniqueName: \"kubernetes.io/projected/20d5be36-ae37-4997-9bca-b538d9bbf899-kube-api-access-bpmsm\") pod \"must-gather-hdx8l\" (UID: \"20d5be36-ae37-4997-9bca-b538d9bbf899\") " pod="openshift-must-gather-gqx6h/must-gather-hdx8l"
Sep 20 02:35:58.296658 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:35:58.296564    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-gqx6h/must-gather-hdx8l"
Sep 20 02:35:58.569946 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:35:58.569919    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-gqx6h/must-gather-hdx8l"]
Sep 20 02:35:58.572255 ip-10-0-130-159 kubenswrapper[2584]: W0920 02:35:58.572228    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod20d5be36_ae37_4997_9bca_b538d9bbf899.slice/crio-e1c5eb9d133ce9a84e60ff8bf1ba329f0569cb2428afc7489ca8c539ca8fb84e WatchSource:0}: Error finding container e1c5eb9d133ce9a84e60ff8bf1ba329f0569cb2428afc7489ca8c539ca8fb84e: Status 404 returned error can't find the container with id e1c5eb9d133ce9a84e60ff8bf1ba329f0569cb2428afc7489ca8c539ca8fb84e
Sep 20 02:35:58.574292 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:35:58.574277    2584 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 20 02:35:59.074307 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:35:59.074248    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gqx6h/must-gather-hdx8l" event={"ID":"20d5be36-ae37-4997-9bca-b538d9bbf899","Type":"ContainerStarted","Data":"e1c5eb9d133ce9a84e60ff8bf1ba329f0569cb2428afc7489ca8c539ca8fb84e"}
Sep 20 02:36:00.085429 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:36:00.085392    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gqx6h/must-gather-hdx8l" event={"ID":"20d5be36-ae37-4997-9bca-b538d9bbf899","Type":"ContainerStarted","Data":"64b91b41a4a39677d009dc35b55c5a773b6b7719e0f117e886d8180c5d139c45"}
Sep 20 02:36:00.085961 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:36:00.085439    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gqx6h/must-gather-hdx8l" event={"ID":"20d5be36-ae37-4997-9bca-b538d9bbf899","Type":"ContainerStarted","Data":"2da8b5b4f405a21fa68d77b49ee0aeceef56a0e550b149ef42bc4d9e06380bae"}
Sep 20 02:36:00.102482 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:36:00.102413    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-gqx6h/must-gather-hdx8l" podStartSLOduration=2.35866293 podStartE2EDuration="3.102387685s" totalImagesPullingTime="743.724755ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-20 02:35:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-20 02:36:00.100918674 +0000 UTC m=+2616.843597879" watchObservedRunningTime="2026-09-20 02:36:00.102387685 +0000 UTC m=+2616.845066895"
Sep 20 02:36:00.990205 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:36:00.990120    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-llv6h_05a98348-758f-4a4a-b150-35bbdeb5b359/global-pull-secret-syncer/0.log"
Sep 20 02:36:01.056716 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:36:01.056678    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log"
Sep 20 02:36:01.173266 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:36:01.173227    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-130-159.ec2.internal_4f739e7563e31c6ca101623c66d03faf/haproxy/0.log"
Sep 20 02:36:06.171540 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:36:06.169703    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-84755fcbb8-rxcv5_807e839b-d2f0-4e52-b9b4-e98e4d2f05b5/authorino/0.log"
Sep 20 02:36:06.204077 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:36:06.203980    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-76b478bd78-m6b4s_464f7342-77e6-4c63-808c-cc6c73f8df11/manager/0.log"
Sep 20 02:36:06.236243 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:36:06.236214    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-7b84cd7c9c-mc75w_9002335e-a543-4c6c-ba94-cd0812a7978d/manager/0.log"
Sep 20 02:36:06.318872 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:36:06.318837    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-xmnfb_f93ff31e-f5fd-49b9-86bc-d05de963b33c/manager/0.log"
Sep 20 02:36:06.431107 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:36:06.431021    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-74495c69f8-krtwd_db69bff3-503c-495e-99a8-0d90f8f0a154/manager/0.log"
Sep 20 02:36:07.144574 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:36:07.144540    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:36:07.210829 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:36:07.210781    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ftlkq_d08af64d-ba39-4d61-9b95-c6febd32cbbb/konnectivity-agent/0.log" limit=1024
Sep 20 02:36:07.867465 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:36:07.867416    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-847b8bd8d7-bp5fk_092d6fa5-0734-4777-b6e0-2e1ef430076d/cluster-monitoring-operator/0.log"
Sep 20 02:36:08.084497 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:36:08.082446    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-sdbvz_d088ddb2-d975-4caa-807e-5dad7943101a/node-exporter/0.log"
Sep 20 02:36:08.123027 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:36:08.122994    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-sdbvz_d088ddb2-d975-4caa-807e-5dad7943101a/kube-rbac-proxy/0.log"
Sep 20 02:36:08.158224 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:36:08.158202    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-sdbvz_d088ddb2-d975-4caa-807e-5dad7943101a/init-textfile/0.log"
Sep 20 02:36:10.242741 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:36:10.242699    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6446c8d75d-fktjk_54d123d3-fc8b-49ee-9e26-d4a3ee494a1d/networking-console-plugin/0.log"
Sep 20 02:36:10.890986 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:36:10.890926    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-ztdt2_ece82f19-68b5-4810-be50-d8678cfdee02/console-operator/1.log"
Sep 20 02:36:10.901634 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:36:10.901599    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-ztdt2_ece82f19-68b5-4810-be50-d8678cfdee02/console-operator/2.log"
Sep 20 02:36:11.423814 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:36:11.423779    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-gqx6h/perf-node-gather-daemonset-66dzn"]
Sep 20 02:36:11.432864 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:36:11.432829    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-gqx6h/perf-node-gather-daemonset-66dzn"
Sep 20 02:36:11.438645 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:36:11.438618    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-gqx6h/perf-node-gather-daemonset-66dzn"]
Sep 20 02:36:11.509250 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:36:11.509215    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/afee4e0c-a6ba-4c96-873f-a1eb925e656f-podres\") pod \"perf-node-gather-daemonset-66dzn\" (UID: \"afee4e0c-a6ba-4c96-873f-a1eb925e656f\") " pod="openshift-must-gather-gqx6h/perf-node-gather-daemonset-66dzn"
Sep 20 02:36:11.509250 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:36:11.509252    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/afee4e0c-a6ba-4c96-873f-a1eb925e656f-proc\") pod \"perf-node-gather-daemonset-66dzn\" (UID: \"afee4e0c-a6ba-4c96-873f-a1eb925e656f\") " pod="openshift-must-gather-gqx6h/perf-node-gather-daemonset-66dzn"
Sep 20 02:36:11.509549 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:36:11.509328    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/afee4e0c-a6ba-4c96-873f-a1eb925e656f-lib-modules\") pod \"perf-node-gather-daemonset-66dzn\" (UID: \"afee4e0c-a6ba-4c96-873f-a1eb925e656f\") " pod="openshift-must-gather-gqx6h/perf-node-gather-daemonset-66dzn"
Sep 20 02:36:11.509549 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:36:11.509384    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/afee4e0c-a6ba-4c96-873f-a1eb925e656f-sys\") pod \"perf-node-gather-daemonset-66dzn\" (UID: \"afee4e0c-a6ba-4c96-873f-a1eb925e656f\") " pod="openshift-must-gather-gqx6h/perf-node-gather-daemonset-66dzn"
Sep 20 02:36:11.509549 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:36:11.509493    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-64j8g\" (UniqueName: \"kubernetes.io/projected/afee4e0c-a6ba-4c96-873f-a1eb925e656f-kube-api-access-64j8g\") pod \"perf-node-gather-daemonset-66dzn\" (UID: \"afee4e0c-a6ba-4c96-873f-a1eb925e656f\") " pod="openshift-must-gather-gqx6h/perf-node-gather-daemonset-66dzn"
Sep 20 02:36:11.610755 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:36:11.610710    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/afee4e0c-a6ba-4c96-873f-a1eb925e656f-podres\") pod \"perf-node-gather-daemonset-66dzn\" (UID: \"afee4e0c-a6ba-4c96-873f-a1eb925e656f\") " pod="openshift-must-gather-gqx6h/perf-node-gather-daemonset-66dzn"
Sep 20 02:36:11.611049 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:36:11.611023    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/afee4e0c-a6ba-4c96-873f-a1eb925e656f-proc\") pod \"perf-node-gather-daemonset-66dzn\" (UID: \"afee4e0c-a6ba-4c96-873f-a1eb925e656f\") " pod="openshift-must-gather-gqx6h/perf-node-gather-daemonset-66dzn"
Sep 20 02:36:11.611200 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:36:11.611175    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/afee4e0c-a6ba-4c96-873f-a1eb925e656f-proc\") pod \"perf-node-gather-daemonset-66dzn\" (UID: \"afee4e0c-a6ba-4c96-873f-a1eb925e656f\") " pod="openshift-must-gather-gqx6h/perf-node-gather-daemonset-66dzn"
Sep 20 02:36:11.611396 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:36:11.611195    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/afee4e0c-a6ba-4c96-873f-a1eb925e656f-lib-modules\") pod \"perf-node-gather-daemonset-66dzn\" (UID: \"afee4e0c-a6ba-4c96-873f-a1eb925e656f\") " pod="openshift-must-gather-gqx6h/perf-node-gather-daemonset-66dzn"
Sep 20 02:36:11.611570 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:36:11.611330    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/afee4e0c-a6ba-4c96-873f-a1eb925e656f-lib-modules\") pod \"perf-node-gather-daemonset-66dzn\" (UID: \"afee4e0c-a6ba-4c96-873f-a1eb925e656f\") " pod="openshift-must-gather-gqx6h/perf-node-gather-daemonset-66dzn"
Sep 20 02:36:11.611668 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:36:11.611060    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/afee4e0c-a6ba-4c96-873f-a1eb925e656f-podres\") pod \"perf-node-gather-daemonset-66dzn\" (UID: \"afee4e0c-a6ba-4c96-873f-a1eb925e656f\") " pod="openshift-must-gather-gqx6h/perf-node-gather-daemonset-66dzn"
Sep 20 02:36:11.611734 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:36:11.611685    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/afee4e0c-a6ba-4c96-873f-a1eb925e656f-sys\") pod \"perf-node-gather-daemonset-66dzn\" (UID: \"afee4e0c-a6ba-4c96-873f-a1eb925e656f\") " pod="openshift-must-gather-gqx6h/perf-node-gather-daemonset-66dzn"
Sep 20 02:36:11.611833 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:36:11.611808    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/afee4e0c-a6ba-4c96-873f-a1eb925e656f-sys\") pod \"perf-node-gather-daemonset-66dzn\" (UID: \"afee4e0c-a6ba-4c96-873f-a1eb925e656f\") " pod="openshift-must-gather-gqx6h/perf-node-gather-daemonset-66dzn"
Sep 20 02:36:11.611968 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:36:11.611949    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-64j8g\" (UniqueName: \"kubernetes.io/projected/afee4e0c-a6ba-4c96-873f-a1eb925e656f-kube-api-access-64j8g\") pod \"perf-node-gather-daemonset-66dzn\" (UID: \"afee4e0c-a6ba-4c96-873f-a1eb925e656f\") " pod="openshift-must-gather-gqx6h/perf-node-gather-daemonset-66dzn"
Sep 20 02:36:11.621451 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:36:11.621419    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-64j8g\" (UniqueName: \"kubernetes.io/projected/afee4e0c-a6ba-4c96-873f-a1eb925e656f-kube-api-access-64j8g\") pod \"perf-node-gather-daemonset-66dzn\" (UID: \"afee4e0c-a6ba-4c96-873f-a1eb925e656f\") " pod="openshift-must-gather-gqx6h/perf-node-gather-daemonset-66dzn"
Sep 20 02:36:11.747283 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:36:11.747242    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-gqx6h/perf-node-gather-daemonset-66dzn"
Sep 20 02:36:12.035219 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:36:12.035165    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-gqx6h/perf-node-gather-daemonset-66dzn"]
Sep 20 02:36:12.037371 ip-10-0-130-159 kubenswrapper[2584]: W0920 02:36:12.037335    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podafee4e0c_a6ba_4c96_873f_a1eb925e656f.slice/crio-94e56770d71b15f8ec253845b2b2f825fcc38a4c96e9e99154b0c8cb417fafbb WatchSource:0}: Error finding container 94e56770d71b15f8ec253845b2b2f825fcc38a4c96e9e99154b0c8cb417fafbb: Status 404 returned error can't find the container with id 94e56770d71b15f8ec253845b2b2f825fcc38a4c96e9e99154b0c8cb417fafbb
Sep 20 02:36:12.180893 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:36:12.180844    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gqx6h/perf-node-gather-daemonset-66dzn" event={"ID":"afee4e0c-a6ba-4c96-873f-a1eb925e656f","Type":"ContainerStarted","Data":"94e56770d71b15f8ec253845b2b2f825fcc38a4c96e9e99154b0c8cb417fafbb"}
Sep 20 02:36:12.286698 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:36:12.286615    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-5c4db8bdf9-hlwl5_a2305e14-b41d-45f8-86ce-c99c92938ba3/volume-data-source-validator/0.log"
Sep 20 02:36:13.188395 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:36:13.188351    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gqx6h/perf-node-gather-daemonset-66dzn" event={"ID":"afee4e0c-a6ba-4c96-873f-a1eb925e656f","Type":"ContainerStarted","Data":"a3581937a784c2c5ae70f8b89dac60d2cac490fff32a9f5e9b12182452aab1ad"}
Sep 20 02:36:13.189218 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:36:13.189191    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-gqx6h/perf-node-gather-daemonset-66dzn"
Sep 20 02:36:13.205756 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:36:13.205697    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-gqx6h/perf-node-gather-daemonset-66dzn" podStartSLOduration=2.205675293 podStartE2EDuration="2.205675293s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-20 02:36:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-20 02:36:13.204474706 +0000 UTC m=+2629.947153944" watchObservedRunningTime="2026-09-20 02:36:13.205675293 +0000 UTC m=+2629.948354500"
Sep 20 02:36:13.466862 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:36:13.466831    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-c9zbv_79e39187-4356-422e-b5d3-9c9f8d3a9a58/dns/0.log"
Sep 20 02:36:13.489756 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:36:13.489727    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-c9zbv_79e39187-4356-422e-b5d3-9c9f8d3a9a58/kube-rbac-proxy/0.log"
Sep 20 02:36:13.621167 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:36:13.621138    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-l9ssk_43dd43c0-ff48-4f9a-a1c5-9020836702aa/dns-node-resolver/0.log"
Sep 20 02:36:14.318510 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:36:14.318476    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-v6d2k_4fa3e7e2-7940-4d7d-af23-63ff07edba43/node-ca/0.log"
Sep 20 02:36:16.137755 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:36:16.137727    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-86bbc588c6-zlhv2_004ecde9-2ce2-41cf-b558-ae17143cd051/kube-auth-proxy/0.log"
Sep 20 02:36:16.216566 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:36:16.216515    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-z42zz_6bb258c3-e7d4-4656-9653-db0c9f68186e/payload-pre-processing/0.log"
Sep 20 02:36:16.286809 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:36:16.286780    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-95688946b-7rdfv_2dbc9737-865d-4dbf-85bf-8d2b9781a8d5/router/0.log"
Sep 20 02:36:17.222179 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:36:17.222139    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-4dsc5_2075981b-d064-43eb-ab90-373b95ad49f4/serve-healthcheck-canary/0.log"
Sep 20 02:36:17.737715 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:36:17.737663    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-6f9c644dd9-gm67p_a6fdf198-e4cd-4267-92fd-0ebee2f97522/insights-operator/0.log"
Sep 20 02:36:17.739114 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:36:17.739066    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-6f9c644dd9-gm67p_a6fdf198-e4cd-4267-92fd-0ebee2f97522/insights-operator/1.log"
Sep 20 02:36:17.825002 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:36:17.824973    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-bjhc4_62740de5-15dc-4e29-aa81-731ab68b2b3a/kube-rbac-proxy/0.log"
Sep 20 02:36:17.849496 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:36:17.849458    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-bjhc4_62740de5-15dc-4e29-aa81-731ab68b2b3a/exporter/0.log"
Sep 20 02:36:17.874251 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:36:17.874217    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-bjhc4_62740de5-15dc-4e29-aa81-731ab68b2b3a/extractor/0.log"
Sep 20 02:36:20.048255 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:36:20.048223    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-v8pxt_fe1d0a3b-8799-4617-8763-fa987349cf78/manager/0.log"
Sep 20 02:36:20.219923 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:36:20.219721    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-gqx6h/perf-node-gather-daemonset-66dzn"
Sep 20 02:36:20.651332 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:36:20.651295    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-6vgqp_4f29b72e-019b-4f87-b9c2-9154fd7bb6ec/manager/0.log"
Sep 20 02:36:20.729964 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:36:20.729928    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6f56f85f68-zwvbx_45ae3219-4dc3-4614-82c0-2be885dee4e3/manager/0.log"
Sep 20 02:36:21.102128 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:36:21.102096    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-59d4cf4f4c-tqwb6_c7f8158a-9f79-4227-8623-1b767c7e29a3/manager/0.log"
Sep 20 02:36:21.126182 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:36:21.126154    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-59d4cf4f4c-tqwb6_c7f8158a-9f79-4227-8623-1b767c7e29a3/copy-manifests/0.log"
Sep 20 02:36:23.042233 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:36:23.042197    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-86bbc588c6-zlhv2_004ecde9-2ce2-41cf-b558-ae17143cd051/kube-auth-proxy/0.log"
Sep 20 02:36:23.101117 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:36:23.101029    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-z42zz_6bb258c3-e7d4-4656-9653-db0c9f68186e/payload-pre-processing/0.log"
Sep 20 02:36:23.150818 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:36:23.150764    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-95688946b-7rdfv_2dbc9737-865d-4dbf-85bf-8d2b9781a8d5/router/0.log"
Sep 20 02:36:25.619463 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:36:25.619431    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-86bbc588c6-zlhv2_004ecde9-2ce2-41cf-b558-ae17143cd051/kube-auth-proxy/0.log"
Sep 20 02:36:25.696019 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:36:25.695986    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-z42zz_6bb258c3-e7d4-4656-9653-db0c9f68186e/payload-pre-processing/0.log"
Sep 20 02:36:25.771131 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:36:25.771087    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-95688946b-7rdfv_2dbc9737-865d-4dbf-85bf-8d2b9781a8d5/router/0.log"
Sep 20 02:36:28.329964 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:36:28.329925    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-55889f7b47-2xts8_006f93ba-2bd5-48cb-b52d-2e288401cd0f/kube-storage-version-migrator-operator/1.log"
Sep 20 02:36:28.331584 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:36:28.331557    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-55889f7b47-2xts8_006f93ba-2bd5-48cb-b52d-2e288401cd0f/kube-storage-version-migrator-operator/0.log"
Sep 20 02:36:29.487333 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:36:29.487303    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-f94fv_96a97560-011d-4806-ae10-4d53880eb9ff/kube-multus-additional-cni-plugins/0.log"
Sep 20 02:36:29.509238 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:36:29.509215    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-f94fv_96a97560-011d-4806-ae10-4d53880eb9ff/egress-router-binary-copy/0.log"
Sep 20 02:36:29.530466 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:36:29.530431    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-f94fv_96a97560-011d-4806-ae10-4d53880eb9ff/cni-plugins/0.log"
Sep 20 02:36:29.552335 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:36:29.552305    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-f94fv_96a97560-011d-4806-ae10-4d53880eb9ff/bond-cni-plugin/0.log"
Sep 20 02:36:29.574151 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:36:29.574124    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-f94fv_96a97560-011d-4806-ae10-4d53880eb9ff/routeoverride-cni/0.log"
Sep 20 02:36:29.595113 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:36:29.595088    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-f94fv_96a97560-011d-4806-ae10-4d53880eb9ff/whereabouts-cni-bincopy/0.log"
Sep 20 02:36:29.615702 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:36:29.615672    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-f94fv_96a97560-011d-4806-ae10-4d53880eb9ff/whereabouts-cni/0.log"
Sep 20 02:36:29.845937 ip-10-0-130-159 kubenswrapper[2584]: I0920 02:36:29.845841    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-k8rn8_f9800646-e98f-45d8-aa4e-5eb53056f5e8/kube-multus/0.log"