Sep 19 05:38:49.985618 ip-10-0-139-144 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Sep 19 05:38:49.985633 ip-10-0-139-144 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Sep 19 05:38:49.985643 ip-10-0-139-144 systemd[1]: kubelet.service: Failed with result 'resources'.
Sep 19 05:38:49.985968 ip-10-0-139-144 systemd[1]: Failed to start Kubernetes Kubelet.
Sep 19 05:38:59.087152 ip-10-0-139-144 systemd[1]: kubelet.service: Failed to schedule restart job: Unit openvswitch.service not found.
Sep 19 05:38:59.087164 ip-10-0-139-144 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 98e0e57f68a940bcb85138397b7cac94 --
Sep 19 05:40:39.935455 ip-10-0-139-144 systemd[1]: Starting Kubernetes Kubelet...
Sep 19 05:40:43.279723 ip-10-0-139-144 kubenswrapper[2583]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 19 05:40:43.279723 ip-10-0-139-144 kubenswrapper[2583]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 19 05:40:43.279723 ip-10-0-139-144 kubenswrapper[2583]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 19 05:40:43.279723 ip-10-0-139-144 kubenswrapper[2583]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 19 05:40:43.279723 ip-10-0-139-144 kubenswrapper[2583]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 19 05:40:43.280584 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.279798    2583 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 19 05:40:43.284924 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.284897    2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 19 05:40:43.284924 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.284917    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 19 05:40:43.284924 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.284923    2583 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 19 05:40:43.284924 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.284927    2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 19 05:40:43.284924 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.284931    2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 19 05:40:43.285260 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.284936    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 19 05:40:43.285260 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.284941    2583 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 19 05:40:43.285260 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.284945    2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 19 05:40:43.285260 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.284949    2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 19 05:40:43.285260 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.284953    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 19 05:40:43.285260 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.284956    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 19 05:40:43.285260 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.284960    2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 19 05:40:43.285260 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.284963    2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 19 05:40:43.285260 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.284969    2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 19 05:40:43.285260 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.284975    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 19 05:40:43.285260 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.284979    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 19 05:40:43.285260 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.284983    2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 19 05:40:43.285260 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.284986    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 19 05:40:43.285260 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.284990    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 19 05:40:43.285260 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.284994    2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 19 05:40:43.285260 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.284997    2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 19 05:40:43.285260 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.285001    2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 19 05:40:43.285260 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.285005    2583 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 19 05:40:43.285260 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.285009    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 19 05:40:43.286031 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.285012    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 19 05:40:43.286031 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.285016    2583 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 19 05:40:43.286031 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.285020    2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 19 05:40:43.286031 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.285024    2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 19 05:40:43.286031 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.285028    2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 19 05:40:43.286031 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.285032    2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 19 05:40:43.286031 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.285035    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 19 05:40:43.286031 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.285040    2583 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 19 05:40:43.286031 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.285044    2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 19 05:40:43.286031 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.285048    2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 19 05:40:43.286031 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.285051    2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 19 05:40:43.286031 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.285055    2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 19 05:40:43.286031 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.285059    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 19 05:40:43.286031 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.285063    2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 19 05:40:43.286031 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.285068    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 19 05:40:43.286031 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.285073    2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 19 05:40:43.286031 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.285077    2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 19 05:40:43.286031 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.285082    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 19 05:40:43.286031 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.285086    2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 19 05:40:43.286031 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.285090    2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 19 05:40:43.286902 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.285095    2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 19 05:40:43.286902 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.285103    2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 19 05:40:43.286902 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.285109    2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 19 05:40:43.286902 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.285114    2583 feature_gate.go:328] unrecognized feature gate: Example
Sep 19 05:40:43.286902 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.285119    2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 19 05:40:43.286902 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.285123    2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 19 05:40:43.286902 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.285143    2583 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 19 05:40:43.286902 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.285148    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 19 05:40:43.286902 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.285153    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 19 05:40:43.286902 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.285158    2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 19 05:40:43.286902 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.285162    2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 19 05:40:43.286902 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.285166    2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 19 05:40:43.286902 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.285171    2583 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 19 05:40:43.286902 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.285175    2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 19 05:40:43.286902 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.285179    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 19 05:40:43.286902 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.285183    2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 19 05:40:43.286902 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.285188    2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 19 05:40:43.286902 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.285192    2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 19 05:40:43.286902 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.285203    2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 19 05:40:43.286902 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.285208    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 19 05:40:43.287794 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.285212    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 19 05:40:43.287794 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.285216    2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 19 05:40:43.287794 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.285220    2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 19 05:40:43.287794 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.285224    2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 19 05:40:43.287794 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.285229    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 19 05:40:43.287794 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.285233    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 19 05:40:43.287794 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.285239    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 19 05:40:43.287794 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.285245    2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 19 05:40:43.287794 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.285249    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 19 05:40:43.287794 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.285254    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 19 05:40:43.287794 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.285258    2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 19 05:40:43.287794 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.285264    2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 19 05:40:43.287794 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.285270    2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 19 05:40:43.287794 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.285275    2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 19 05:40:43.287794 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.285280    2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 19 05:40:43.287794 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.285284    2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 19 05:40:43.287794 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.285289    2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 19 05:40:43.287794 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.285294    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 19 05:40:43.287794 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.285299    2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 19 05:40:43.288608 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.285305    2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 19 05:40:43.288608 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.285309    2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 19 05:40:43.288608 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.285314    2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 19 05:40:43.288608 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.285318    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 19 05:40:43.288608 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.285322    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 19 05:40:43.288608 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.285326    2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 19 05:40:43.288608 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.285330    2583 feature_gate.go:328] unrecognized feature gate: Example2
Sep 19 05:40:43.288608 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.285334    2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 19 05:40:43.288608 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.285338    2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 19 05:40:43.288608 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.285342    2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 19 05:40:43.288608 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.285347    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 19 05:40:43.288608 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.285351    2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 19 05:40:43.288608 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.285356    2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 19 05:40:43.288608 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.285360    2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 19 05:40:43.288608 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.285364    2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 19 05:40:43.288608 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.285369    2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 19 05:40:43.288608 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.286511    2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 19 05:40:43.288608 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.286521    2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 19 05:40:43.288608 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.286526    2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 19 05:40:43.288608 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.286531    2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 19 05:40:43.289493 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.286536    2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 19 05:40:43.289493 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.286540    2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 19 05:40:43.289493 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.286545    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 19 05:40:43.289493 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.286549    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 19 05:40:43.289493 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.286554    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 19 05:40:43.289493 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.286558    2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 19 05:40:43.289493 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.286562    2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 19 05:40:43.289493 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.286567    2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 19 05:40:43.289493 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.286571    2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 19 05:40:43.289493 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.286576    2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 19 05:40:43.289493 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.286580    2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 19 05:40:43.289493 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.286584    2583 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 19 05:40:43.289493 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.286589    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 19 05:40:43.289493 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.286594    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 19 05:40:43.289493 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.286598    2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 19 05:40:43.289493 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.286603    2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 19 05:40:43.289493 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.286607    2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 19 05:40:43.289493 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.286611    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 19 05:40:43.289493 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.286615    2583 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 19 05:40:43.289493 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.286619    2583 feature_gate.go:328] unrecognized feature gate: Example2
Sep 19 05:40:43.290082 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.286624    2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 19 05:40:43.290082 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.286628    2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 19 05:40:43.290082 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.286632    2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 19 05:40:43.290082 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.286636    2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 19 05:40:43.290082 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.286640    2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 19 05:40:43.290082 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.286644    2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 19 05:40:43.290082 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.286648    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 19 05:40:43.290082 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.286652    2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 19 05:40:43.290082 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.286656    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 19 05:40:43.290082 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.286660    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 19 05:40:43.290082 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.286664    2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 19 05:40:43.290082 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.286669    2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 19 05:40:43.290082 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.286674    2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 19 05:40:43.290082 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.286679    2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 19 05:40:43.290082 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.286685    2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 19 05:40:43.290082 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.286690    2583 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 19 05:40:43.290082 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.286694    2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 19 05:40:43.290082 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.286699    2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 19 05:40:43.290082 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.286704    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 19 05:40:43.290651 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.286708    2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 19 05:40:43.290651 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.286712    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 19 05:40:43.290651 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.286717    2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 19 05:40:43.290651 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.286721    2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 19 05:40:43.290651 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.286725    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 19 05:40:43.290651 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.286729    2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 19 05:40:43.290651 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.286733    2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 19 05:40:43.290651 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.286738    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 19 05:40:43.290651 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.286742    2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 19 05:40:43.290651 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.286747    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 19 05:40:43.290651 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.286751    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 19 05:40:43.290651 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.286755    2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 19 05:40:43.290651 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.286759    2583 feature_gate.go:328] unrecognized feature gate: Example
Sep 19 05:40:43.290651 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.286763    2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 19 05:40:43.290651 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.286768    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 19 05:40:43.290651 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.286772    2583 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 19 05:40:43.290651 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.286776    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 19 05:40:43.290651 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.286781    2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 19 05:40:43.290651 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.286786    2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 19 05:40:43.290651 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.286790    2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 19 05:40:43.291194 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.286794    2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 19 05:40:43.291194 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.286798    2583 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 19 05:40:43.291194 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.286802    2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 19 05:40:43.291194 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.286806    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 19 05:40:43.291194 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.286811    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 19 05:40:43.291194 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.286815    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 19 05:40:43.291194 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.286819    2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 19 05:40:43.291194 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.286826    2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 19 05:40:43.291194 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.286832    2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 19 05:40:43.291194 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.286837    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 19 05:40:43.291194 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.286841    2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 19 05:40:43.291194 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.286846    2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 19 05:40:43.291194 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.286850    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 19 05:40:43.291194 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.286854    2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 19 05:40:43.291194 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.286859    2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 19 05:40:43.291194 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.286863    2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 19 05:40:43.291194 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.286867    2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 19 05:40:43.291194 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.286872    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 19 05:40:43.291194 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.286876    2583 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 19 05:40:43.291194 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.286881    2583 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 19 05:40:43.291819 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.286886    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 19 05:40:43.291819 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.286892    2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 19 05:40:43.291819 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.286897    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 19 05:40:43.291819 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.286901    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 19 05:40:43.291819 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.286906    2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 19 05:40:43.291819 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.286911    2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 19 05:40:43.291819 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.286916    2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 19 05:40:43.291819 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.286920    2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 19 05:40:43.291819 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.286924    2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 19 05:40:43.291819 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.286928    2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 19 05:40:43.291819 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.286932    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 19 05:40:43.291819 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.286937    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 19 05:40:43.291819 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.286941    2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 19 05:40:43.291819 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.286945    2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 19 05:40:43.291819 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.286950    2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 19 05:40:43.291819 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.286954    2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 19 05:40:43.291819 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.287605    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 19 05:40:43.291819 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.287616    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 19 05:40:43.291819 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.287621    2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 19 05:40:43.292392 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.287626    2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 19 05:40:43.292392 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.287631    2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 19 05:40:43.292392 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.287637    2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 19 05:40:43.292392 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.287642    2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 19 05:40:43.292392 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.287646    2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 19 05:40:43.292392 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.287650    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 19 05:40:43.292392 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.287654    2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 19 05:40:43.292392 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.287659    2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 19 05:40:43.292392 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.287663    2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 19 05:40:43.292392 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.287667    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 19 05:40:43.292392 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.287674    2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 19 05:40:43.292392 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.287680    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 19 05:40:43.292392 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.287685    2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 19 05:40:43.292392 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.287690    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 19 05:40:43.292392 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.287695    2583 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 19 05:40:43.292392 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.287700    2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 19 05:40:43.292392 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.287705    2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 19 05:40:43.292392 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.287709    2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 19 05:40:43.292392 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.287713    2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 19 05:40:43.292856 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.287717    2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 19 05:40:43.292856 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.287721    2583 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 19 05:40:43.292856 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.287725    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 19 05:40:43.292856 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.287729    2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 19 05:40:43.292856 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.287733    2583 feature_gate.go:328] unrecognized feature gate: Example
Sep 19 05:40:43.292856 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.287738    2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 19 05:40:43.292856 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.287742    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 19 05:40:43.292856 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.287746    2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 19 05:40:43.292856 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.287751    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 19 05:40:43.292856 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.287755    2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 19 05:40:43.292856 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.287760    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 19 05:40:43.292856 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.287765    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 19 05:40:43.292856 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.287770    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 19 05:40:43.292856 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.287774    2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 19 05:40:43.292856 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.287781    2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 19 05:40:43.292856 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.287786    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 19 05:40:43.292856 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.287791    2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 19 05:40:43.292856 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.287795    2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 19 05:40:43.292856 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.287799    2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 19 05:40:43.293432 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.287803    2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 19 05:40:43.293432 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.287807    2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 19 05:40:43.293432 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.287811    2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 19 05:40:43.293432 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.287816    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 19 05:40:43.293432 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.287820    2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 19 05:40:43.293432 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.287824    2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 19 05:40:43.293432 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.287829    2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 19 05:40:43.293432 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.287833    2583 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 19 05:40:43.293432 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.287838    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 19 05:40:43.293432 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.287843    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 19 05:40:43.293432 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.287847    2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 19 05:40:43.293432 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.287851    2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 19 05:40:43.293432 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.287858    2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 19 05:40:43.293432 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.287862    2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 19 05:40:43.293432 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.287866    2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 19 05:40:43.293432 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.287871    2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 19 05:40:43.293432 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.287879    2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 19 05:40:43.293432 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.287885    2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 19 05:40:43.293432 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.287890    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 19 05:40:43.293923 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.287894    2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 19 05:40:43.293923 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.287898    2583 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 19 05:40:43.293923 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.287903    2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 19 05:40:43.293923 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.287907    2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 19 05:40:43.293923 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.287912    2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 19 05:40:43.293923 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.287916    2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 19 05:40:43.293923 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.287921    2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 19 05:40:43.293923 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.287925    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 19 05:40:43.293923 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.287929    2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 19 05:40:43.293923 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.287934    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 19 05:40:43.293923 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.287938    2583 feature_gate.go:328] unrecognized feature gate: Example2
Sep 19 05:40:43.293923 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.287942    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 19 05:40:43.293923 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.287946    2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 19 05:40:43.293923 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.287950    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 19 05:40:43.293923 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.287954    2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 19 05:40:43.293923 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.287958    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 19 05:40:43.293923 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.287962    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 19 05:40:43.293923 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.287967    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 19 05:40:43.293923 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.287971    2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 19 05:40:43.293923 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.287975    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 19 05:40:43.294442 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.287980    2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 19 05:40:43.294442 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.287983    2583 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 19 05:40:43.294442 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.287988    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 19 05:40:43.294442 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.287992    2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 19 05:40:43.294442 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.287997    2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 19 05:40:43.294442 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.288000    2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 19 05:40:43.294442 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.288005    2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 19 05:40:43.294442 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.288008    2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 19 05:40:43.294442 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.288013    2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 19 05:40:43.294442 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.288017    2583 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 19 05:40:43.294442 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.288021    2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 19 05:40:43.294442 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.288025    2583 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 19 05:40:43.294442 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.288029    2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 19 05:40:43.294442 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.288034    2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 19 05:40:43.294442 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.288039    2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 19 05:40:43.294442 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.288043    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 19 05:40:43.294442 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.288047    2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 19 05:40:43.294442 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.288051    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 19 05:40:43.294442 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.288055    2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 19 05:40:43.294442 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288177    2583 flags.go:64] FLAG: --address="0.0.0.0"
Sep 19 05:40:43.294442 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288228    2583 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 19 05:40:43.294958 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288237    2583 flags.go:64] FLAG: --anonymous-auth="true"
Sep 19 05:40:43.294958 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288244    2583 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 19 05:40:43.294958 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288250    2583 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 19 05:40:43.294958 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288255    2583 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 19 05:40:43.294958 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288262    2583 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 19 05:40:43.294958 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288268    2583 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 19 05:40:43.294958 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288273    2583 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 19 05:40:43.294958 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288278    2583 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 19 05:40:43.294958 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288284    2583 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 19 05:40:43.294958 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288289    2583 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 19 05:40:43.294958 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288294    2583 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 19 05:40:43.294958 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288300    2583 flags.go:64] FLAG: --cgroup-root=""
Sep 19 05:40:43.294958 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288304    2583 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 19 05:40:43.294958 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288309    2583 flags.go:64] FLAG: --client-ca-file=""
Sep 19 05:40:43.294958 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288321    2583 flags.go:64] FLAG: --cloud-provider="external"
Sep 19 05:40:43.294958 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288326    2583 flags.go:64] FLAG: --cluster-dns="[]"
Sep 19 05:40:43.294958 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288334    2583 flags.go:64] FLAG: --cluster-domain=""
Sep 19 05:40:43.294958 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288339    2583 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 19 05:40:43.294958 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288344    2583 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 19 05:40:43.294958 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288349    2583 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 19 05:40:43.294958 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288354    2583 flags.go:64] FLAG: --container-log-max-files="5"
Sep 19 05:40:43.294958 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288361    2583 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 19 05:40:43.294958 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288365    2583 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 19 05:40:43.295519 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288370    2583 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 19 05:40:43.295519 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288376    2583 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 19 05:40:43.295519 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288381    2583 flags.go:64] FLAG: --contention-profiling="false"
Sep 19 05:40:43.295519 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288386    2583 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 19 05:40:43.295519 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288391    2583 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 19 05:40:43.295519 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288396    2583 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 19 05:40:43.295519 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288401    2583 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 19 05:40:43.295519 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288407    2583 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 19 05:40:43.295519 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288412    2583 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 19 05:40:43.295519 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288417    2583 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 19 05:40:43.295519 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288421    2583 flags.go:64] FLAG: --enable-load-reader="false"
Sep 19 05:40:43.295519 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288426    2583 flags.go:64] FLAG: --enable-server="true"
Sep 19 05:40:43.295519 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288431    2583 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 19 05:40:43.295519 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288437    2583 flags.go:64] FLAG: --event-burst="100"
Sep 19 05:40:43.295519 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288442    2583 flags.go:64] FLAG: --event-qps="50"
Sep 19 05:40:43.295519 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288447    2583 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 19 05:40:43.295519 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288452    2583 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 19 05:40:43.295519 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288457    2583 flags.go:64] FLAG: --eviction-hard=""
Sep 19 05:40:43.295519 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288463    2583 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 19 05:40:43.295519 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288468    2583 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 19 05:40:43.295519 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288473    2583 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 19 05:40:43.295519 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288478    2583 flags.go:64] FLAG: --eviction-soft=""
Sep 19 05:40:43.295519 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288484    2583 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 19 05:40:43.295519 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288489    2583 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 19 05:40:43.295519 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288495    2583 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 19 05:40:43.296223 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288500    2583 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 19 05:40:43.296223 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288504    2583 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 19 05:40:43.296223 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288509    2583 flags.go:64] FLAG: --fail-swap-on="true"
Sep 19 05:40:43.296223 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288514    2583 flags.go:64] FLAG: --feature-gates=""
Sep 19 05:40:43.296223 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288520    2583 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 19 05:40:43.296223 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288525    2583 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 19 05:40:43.296223 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288530    2583 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 19 05:40:43.296223 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288535    2583 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 19 05:40:43.296223 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288544    2583 flags.go:64] FLAG: --healthz-port="10248"
Sep 19 05:40:43.296223 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288549    2583 flags.go:64] FLAG: --help="false"
Sep 19 05:40:43.296223 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288554    2583 flags.go:64] FLAG: --hostname-override="ip-10-0-139-144.ec2.internal"
Sep 19 05:40:43.296223 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288559    2583 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 19 05:40:43.296223 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288563    2583 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 19 05:40:43.296223 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288568    2583 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 19 05:40:43.296223 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288574    2583 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 19 05:40:43.296223 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288579    2583 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 19 05:40:43.296223 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288584    2583 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 19 05:40:43.296223 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288589    2583 flags.go:64] FLAG: --image-service-endpoint=""
Sep 19 05:40:43.296223 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288593    2583 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 19 05:40:43.296223 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288598    2583 flags.go:64] FLAG: --kube-api-burst="100"
Sep 19 05:40:43.296223 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288603    2583 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 19 05:40:43.296223 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288607    2583 flags.go:64] FLAG: --kube-api-qps="50"
Sep 19 05:40:43.296223 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288612    2583 flags.go:64] FLAG: --kube-reserved=""
Sep 19 05:40:43.296223 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288616    2583 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 19 05:40:43.296816 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288621    2583 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 19 05:40:43.296816 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288626    2583 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 19 05:40:43.296816 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288630    2583 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 19 05:40:43.296816 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288635    2583 flags.go:64] FLAG: --lock-file=""
Sep 19 05:40:43.296816 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288640    2583 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 19 05:40:43.296816 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288645    2583 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 19 05:40:43.296816 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288650    2583 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 19 05:40:43.296816 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288659    2583 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 19 05:40:43.296816 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288665    2583 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 19 05:40:43.296816 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288669    2583 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 19 05:40:43.296816 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288674    2583 flags.go:64] FLAG: --logging-format="text"
Sep 19 05:40:43.296816 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288679    2583 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 19 05:40:43.296816 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288684    2583 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 19 05:40:43.296816 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288689    2583 flags.go:64] FLAG: --manifest-url=""
Sep 19 05:40:43.296816 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288694    2583 flags.go:64] FLAG: --manifest-url-header=""
Sep 19 05:40:43.296816 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288700    2583 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 19 05:40:43.296816 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288707    2583 flags.go:64] FLAG: --max-open-files="1000000"
Sep 19 05:40:43.296816 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288713    2583 flags.go:64] FLAG: --max-pods="110"
Sep 19 05:40:43.296816 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288718    2583 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 19 05:40:43.296816 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288723    2583 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 19 05:40:43.296816 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288728    2583 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 19 05:40:43.296816 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288733    2583 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 19 05:40:43.296816 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288738    2583 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 19 05:40:43.296816 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288742    2583 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 19 05:40:43.296816 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288747    2583 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 19 05:40:43.297473 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288758    2583 flags.go:64] FLAG: --node-status-max-images="50"
Sep 19 05:40:43.297473 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288763    2583 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 19 05:40:43.297473 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288768    2583 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 19 05:40:43.297473 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288773    2583 flags.go:64] FLAG: --pod-cidr=""
Sep 19 05:40:43.297473 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288777    2583 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a6abe77af61c5f2e3985a83e7d43538aefc38b1b0444127da14ee0e8cf8f76fc"
Sep 19 05:40:43.297473 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288785    2583 flags.go:64] FLAG: --pod-manifest-path=""
Sep 19 05:40:43.297473 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288790    2583 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 19 05:40:43.297473 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288795    2583 flags.go:64] FLAG: --pods-per-core="0"
Sep 19 05:40:43.297473 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288799    2583 flags.go:64] FLAG: --port="10250"
Sep 19 05:40:43.297473 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288805    2583 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 19 05:40:43.297473 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288811    2583 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0f71e3f40a4a1bb72"
Sep 19 05:40:43.297473 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288817    2583 flags.go:64] FLAG: --qos-reserved=""
Sep 19 05:40:43.297473 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288822    2583 flags.go:64] FLAG: --read-only-port="10255"
Sep 19 05:40:43.297473 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288827    2583 flags.go:64] FLAG: --register-node="true"
Sep 19 05:40:43.297473 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288831    2583 flags.go:64] FLAG: --register-with-taints=""
Sep 19 05:40:43.297473 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288837    2583 flags.go:64] FLAG: --registry-burst="10"
Sep 19 05:40:43.297473 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288843    2583 flags.go:64] FLAG: --registry-qps="5"
Sep 19 05:40:43.297473 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288847    2583 flags.go:64] FLAG: --reserved-cpus=""
Sep 19 05:40:43.297473 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288852    2583 flags.go:64] FLAG: --reserved-memory=""
Sep 19 05:40:43.297473 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288858    2583 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 19 05:40:43.297473 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288864    2583 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 19 05:40:43.297473 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288869    2583 flags.go:64] FLAG: --rotate-certificates="false"
Sep 19 05:40:43.297473 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288874    2583 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 19 05:40:43.297473 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288879    2583 flags.go:64] FLAG: --runonce="false"
Sep 19 05:40:43.297473 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288885    2583 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 19 05:40:43.298076 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288891    2583 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 19 05:40:43.298076 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288896    2583 flags.go:64] FLAG: --seccomp-default="false"
Sep 19 05:40:43.298076 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288900    2583 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 19 05:40:43.298076 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288905    2583 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 19 05:40:43.298076 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288910    2583 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 19 05:40:43.298076 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288915    2583 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 19 05:40:43.298076 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288919    2583 flags.go:64] FLAG: --storage-driver-password="root"
Sep 19 05:40:43.298076 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288924    2583 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 19 05:40:43.298076 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288929    2583 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 19 05:40:43.298076 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288934    2583 flags.go:64] FLAG: --storage-driver-user="root"
Sep 19 05:40:43.298076 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288939    2583 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 19 05:40:43.298076 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288943    2583 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 19 05:40:43.298076 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288948    2583 flags.go:64] FLAG: --system-cgroups=""
Sep 19 05:40:43.298076 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288953    2583 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 19 05:40:43.298076 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288961    2583 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 19 05:40:43.298076 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288966    2583 flags.go:64] FLAG: --tls-cert-file=""
Sep 19 05:40:43.298076 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288970    2583 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 19 05:40:43.298076 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288980    2583 flags.go:64] FLAG: --tls-min-version=""
Sep 19 05:40:43.298076 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288984    2583 flags.go:64] FLAG: --tls-private-key-file=""
Sep 19 05:40:43.298076 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288989    2583 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 19 05:40:43.298076 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288993    2583 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 19 05:40:43.298076 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.288998    2583 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 19 05:40:43.298076 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.289003    2583 flags.go:64] FLAG: --v="2"
Sep 19 05:40:43.298076 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.289009    2583 flags.go:64] FLAG: --version="false"
Sep 19 05:40:43.298076 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.289017    2583 flags.go:64] FLAG: --vmodule=""
Sep 19 05:40:43.298706 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.289023    2583 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 19 05:40:43.298706 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.289029    2583 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 19 05:40:43.298706 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.289229    2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 19 05:40:43.298706 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.289236    2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 19 05:40:43.298706 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.289243    2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 19 05:40:43.298706 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.289250    2583 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 19 05:40:43.298706 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.289258    2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 19 05:40:43.298706 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.289262    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 19 05:40:43.298706 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.289269    2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 19 05:40:43.298706 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.289275    2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 19 05:40:43.298706 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.289279    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 19 05:40:43.298706 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.289284    2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 19 05:40:43.298706 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.289288    2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 19 05:40:43.298706 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.289292    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 19 05:40:43.298706 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.289297    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 19 05:40:43.298706 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.289301    2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 19 05:40:43.298706 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.289305    2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 19 05:40:43.298706 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.289310    2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 19 05:40:43.298706 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.289314    2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 19 05:40:43.299185 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.289319    2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 19 05:40:43.299185 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.289323    2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 19 05:40:43.299185 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.289327    2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 19 05:40:43.299185 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.289331    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 19 05:40:43.299185 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.289335    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 19 05:40:43.299185 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.289342    2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 19 05:40:43.299185 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.289346    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 19 05:40:43.299185 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.289350    2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 19 05:40:43.299185 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.289354    2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 19 05:40:43.299185 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.289359    2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 19 05:40:43.299185 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.289363    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 19 05:40:43.299185 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.289367    2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 19 05:40:43.299185 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.289371    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 19 05:40:43.299185 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.289376    2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 19 05:40:43.299185 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.289381    2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 19 05:40:43.299185 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.289385    2583 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 19 05:40:43.299185 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.289390    2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 19 05:40:43.299185 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.289394    2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 19 05:40:43.299185 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.289398    2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 19 05:40:43.299185 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.289403    2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 19 05:40:43.299666 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.289407    2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 19 05:40:43.299666 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.289412    2583 feature_gate.go:328] unrecognized feature gate: Example
Sep 19 05:40:43.299666 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.289416    2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 19 05:40:43.299666 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.289420    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 19 05:40:43.299666 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.289424    2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 19 05:40:43.299666 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.289429    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 19 05:40:43.299666 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.289433    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 19 05:40:43.299666 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.289438    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 19 05:40:43.299666 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.289442    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 19 05:40:43.299666 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.289446    2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 19 05:40:43.299666 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.289450    2583 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 19 05:40:43.299666 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.289455    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 19 05:40:43.299666 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.289459    2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 19 05:40:43.299666 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.289463    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 19 05:40:43.299666 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.289467    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 19 05:40:43.299666 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.289471    2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 19 05:40:43.299666 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.289475    2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 19 05:40:43.299666 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.289481    2583 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 19 05:40:43.299666 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.289486    2583 feature_gate.go:328] unrecognized feature gate: Example2
Sep 19 05:40:43.299666 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.289490    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 19 05:40:43.299666 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.289494    2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 19 05:40:43.300194 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.289498    2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 19 05:40:43.300194 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.289502    2583 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 19 05:40:43.300194 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.289506    2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 19 05:40:43.300194 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.289510    2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 19 05:40:43.300194 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.289514    2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 19 05:40:43.300194 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.289519    2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 19 05:40:43.300194 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.289523    2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 19 05:40:43.300194 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.289528    2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 19 05:40:43.300194 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.289532    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 19 05:40:43.300194 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.289536    2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 19 05:40:43.300194 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.289544    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 19 05:40:43.300194 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.289548    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 19 05:40:43.300194 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.289552    2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 19 05:40:43.300194 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.289557    2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 19 05:40:43.300194 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.289561    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 19 05:40:43.300194 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.289565    2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 19 05:40:43.300194 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.289569    2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 19 05:40:43.300194 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.289574    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 19 05:40:43.300194 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.289578    2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 19 05:40:43.300194 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.289582    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 19 05:40:43.300717 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.289586    2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 19 05:40:43.300717 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.289593    2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 19 05:40:43.300717 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.289598    2583 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 19 05:40:43.300717 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.289603    2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 19 05:40:43.300717 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.289608    2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 19 05:40:43.300717 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.289612    2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 19 05:40:43.300717 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.289617    2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 19 05:40:43.300717 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.289622    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 19 05:40:43.300717 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.289630    2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 19 05:40:43.300717 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.289634    2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 19 05:40:43.300717 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.289638    2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 19 05:40:43.300717 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.289642    2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 19 05:40:43.300717 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.289646    2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 19 05:40:43.300717 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.289650    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 19 05:40:43.300717 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.289654    2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 19 05:40:43.300717 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.289658    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 19 05:40:43.300717 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.289662    2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 19 05:40:43.300717 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.289667    2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 19 05:40:43.300717 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.289672    2583 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 19 05:40:43.301205 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.289676    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 19 05:40:43.301205 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.289680    2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 19 05:40:43.301205 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.290221    2583 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 19 05:40:43.301205 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.297358    2583 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 19 05:40:43.301205 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.297376    2583 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 19 05:40:43.301205 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.297397    2583 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 19 05:40:43.301205 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.297403    2583 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 19 05:40:43.301205 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297457    2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 19 05:40:43.301205 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297462    2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 19 05:40:43.301205 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297466    2583 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 19 05:40:43.301205 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297468    2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 19 05:40:43.301205 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297471    2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 19 05:40:43.301205 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297474    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 19 05:40:43.301205 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297477    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 19 05:40:43.301205 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297480    2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 19 05:40:43.301577 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297483    2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 19 05:40:43.301577 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297486    2583 feature_gate.go:328] unrecognized feature gate: Example2
Sep 19 05:40:43.301577 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297489    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 19 05:40:43.301577 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297492    2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 19 05:40:43.301577 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297495    2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 19 05:40:43.301577 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297499    2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 19 05:40:43.301577 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297503    2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 19 05:40:43.301577 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297507    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 19 05:40:43.301577 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297509    2583 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 19 05:40:43.301577 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297512    2583 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 19 05:40:43.301577 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297515    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 19 05:40:43.301577 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297517    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 19 05:40:43.301577 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297520    2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 19 05:40:43.301577 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297522    2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 19 05:40:43.301577 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297525    2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 19 05:40:43.301577 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297528    2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 19 05:40:43.301577 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297530    2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 19 05:40:43.301577 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297533    2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 19 05:40:43.301577 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297535    2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 19 05:40:43.301577 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297538    2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 19 05:40:43.302050 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297542    2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 19 05:40:43.302050 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297547    2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 19 05:40:43.302050 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297550    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 19 05:40:43.302050 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297553    2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 19 05:40:43.302050 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297556    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 19 05:40:43.302050 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297559    2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 19 05:40:43.302050 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297562    2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 19 05:40:43.302050 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297564    2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 19 05:40:43.302050 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297567    2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 19 05:40:43.302050 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297570    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 19 05:40:43.302050 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297573    2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 19 05:40:43.302050 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297575    2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 19 05:40:43.302050 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297578    2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 19 05:40:43.302050 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297580    2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 19 05:40:43.302050 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297583    2583 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 19 05:40:43.302050 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297585    2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 19 05:40:43.302050 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297588    2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 19 05:40:43.302050 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297591    2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 19 05:40:43.302050 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297593    2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 19 05:40:43.302582 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297596    2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 19 05:40:43.302582 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297598    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 19 05:40:43.302582 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297601    2583 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 19 05:40:43.302582 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297604    2583 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 19 05:40:43.302582 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297606    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 19 05:40:43.302582 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297609    2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 19 05:40:43.302582 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297611    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 19 05:40:43.302582 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297614    2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 19 05:40:43.302582 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297617    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 19 05:40:43.302582 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297620    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 19 05:40:43.302582 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297622    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 19 05:40:43.302582 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297625    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 19 05:40:43.302582 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297628    2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 19 05:40:43.302582 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297630    2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 19 05:40:43.302582 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297633    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 19 05:40:43.302582 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297637    2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 19 05:40:43.302582 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297639    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 19 05:40:43.302582 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297642    2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 19 05:40:43.302582 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297645    2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 19 05:40:43.302582 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297647    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 19 05:40:43.303079 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297650    2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 19 05:40:43.303079 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297652    2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 19 05:40:43.303079 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297655    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 19 05:40:43.303079 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297658    2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 19 05:40:43.303079 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297660    2583 feature_gate.go:328] unrecognized feature gate: Example
Sep 19 05:40:43.303079 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297663    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 19 05:40:43.303079 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297665    2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 19 05:40:43.303079 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297668    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 19 05:40:43.303079 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297670    2583 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 19 05:40:43.303079 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297673    2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 19 05:40:43.303079 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297675    2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 19 05:40:43.303079 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297678    2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 19 05:40:43.303079 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297680    2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 19 05:40:43.303079 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297683    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 19 05:40:43.303079 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297686    2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 19 05:40:43.303079 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297688    2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 19 05:40:43.303079 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297691    2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 19 05:40:43.303079 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297693    2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 19 05:40:43.303079 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297696    2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 19 05:40:43.303079 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297698    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 19 05:40:43.303605 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297701    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 19 05:40:43.303605 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297703    2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 19 05:40:43.303605 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297706    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 19 05:40:43.303605 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297708    2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 19 05:40:43.303605 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297713    2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 19 05:40:43.303605 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297716    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 19 05:40:43.303605 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297720    2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 19 05:40:43.303605 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297723    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 19 05:40:43.303605 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297726    2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 19 05:40:43.303605 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297729    2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 19 05:40:43.303605 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297732    2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 19 05:40:43.303605 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297735    2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 19 05:40:43.303605 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.297740    2583 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 19 05:40:43.303605 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297851    2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 19 05:40:43.303961 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297856    2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 19 05:40:43.303961 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297859    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 19 05:40:43.303961 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297862    2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 19 05:40:43.303961 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297865    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 19 05:40:43.303961 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297867    2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 19 05:40:43.303961 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297870    2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 19 05:40:43.303961 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297872    2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 19 05:40:43.303961 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297875    2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 19 05:40:43.303961 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297877    2583 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 19 05:40:43.303961 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297880    2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 19 05:40:43.303961 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297883    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 19 05:40:43.303961 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297885    2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 19 05:40:43.303961 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297888    2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 19 05:40:43.303961 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297891    2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 19 05:40:43.303961 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297893    2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 19 05:40:43.303961 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297896    2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 19 05:40:43.303961 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297898    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 19 05:40:43.303961 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297901    2583 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 19 05:40:43.303961 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297903    2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 19 05:40:43.303961 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297906    2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 19 05:40:43.304455 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297908    2583 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 19 05:40:43.304455 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297911    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 19 05:40:43.304455 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297913    2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 19 05:40:43.304455 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297916    2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 19 05:40:43.304455 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297919    2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 19 05:40:43.304455 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297922    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 19 05:40:43.304455 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297924    2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 19 05:40:43.304455 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297927    2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 19 05:40:43.304455 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297930    2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 19 05:40:43.304455 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297932    2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 19 05:40:43.304455 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297935    2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 19 05:40:43.304455 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297937    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 19 05:40:43.304455 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297940    2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 19 05:40:43.304455 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297943    2583 feature_gate.go:328] unrecognized feature gate: Example
Sep 19 05:40:43.304455 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297945    2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 19 05:40:43.304455 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297949    2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 19 05:40:43.304455 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297952    2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 19 05:40:43.304455 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297955    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 19 05:40:43.304455 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297958    2583 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 19 05:40:43.304455 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297960    2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 19 05:40:43.304939 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297963    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 19 05:40:43.304939 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297966    2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 19 05:40:43.304939 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297968    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 19 05:40:43.304939 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297971    2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 19 05:40:43.304939 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297973    2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 19 05:40:43.304939 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297976    2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 19 05:40:43.304939 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297978    2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 19 05:40:43.304939 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297981    2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 19 05:40:43.304939 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297983    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 19 05:40:43.304939 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297986    2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 19 05:40:43.304939 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297988    2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 19 05:40:43.304939 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297991    2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 19 05:40:43.304939 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297993    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 19 05:40:43.304939 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297996    2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 19 05:40:43.304939 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.297998    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 19 05:40:43.304939 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.298002    2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 19 05:40:43.304939 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.298006    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 19 05:40:43.304939 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.298009    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 19 05:40:43.304939 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.298012    2583 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 19 05:40:43.305445 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.298014    2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 19 05:40:43.305445 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.298018    2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 19 05:40:43.305445 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.298020    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 19 05:40:43.305445 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.298023    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 19 05:40:43.305445 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.298025    2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 19 05:40:43.305445 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.298028    2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 19 05:40:43.305445 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.298030    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 19 05:40:43.305445 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.298033    2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 19 05:40:43.305445 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.298035    2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 19 05:40:43.305445 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.298038    2583 feature_gate.go:328] unrecognized feature gate: Example2
Sep 19 05:40:43.305445 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.298041    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 19 05:40:43.305445 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.298043    2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 19 05:40:43.305445 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.298046    2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 19 05:40:43.305445 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.298048    2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 19 05:40:43.305445 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.298051    2583 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 19 05:40:43.305445 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.298053    2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 19 05:40:43.305445 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.298056    2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 19 05:40:43.305445 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.298059    2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 19 05:40:43.305445 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.298061    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 19 05:40:43.305445 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.298064    2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 19 05:40:43.306228 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.298066    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 19 05:40:43.306228 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.298070    2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 19 05:40:43.306228 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.298074    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 19 05:40:43.306228 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.298077    2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 19 05:40:43.306228 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.298080    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 19 05:40:43.306228 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.298083    2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 19 05:40:43.306228 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.298086    2583 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 19 05:40:43.306228 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.298089    2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 19 05:40:43.306228 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.298091    2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 19 05:40:43.306228 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.298095    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 19 05:40:43.306228 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.298098    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 19 05:40:43.306228 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.298100    2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 19 05:40:43.306228 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.298103    2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 19 05:40:43.306228 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.298106    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 19 05:40:43.306228 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.298109    2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 19 05:40:43.306228 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.298112    2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 19 05:40:43.306228 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.298114    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 19 05:40:43.306228 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.298117    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 19 05:40:43.306228 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:43.298119    2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 19 05:40:43.306789 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.298125    2583 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 19 05:40:43.306789 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.298240    2583 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 19 05:40:43.306789 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.300144    2583 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 19 05:40:43.306789 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.301283    2583 server.go:1019] "Starting client certificate rotation"
Sep 19 05:40:43.306789 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.301386    2583 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 19 05:40:43.306789 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.301421    2583 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 19 05:40:43.306789 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.304074    2583 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 19 05:40:43.306789 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.305254    2583 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 19 05:40:43.310628 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.310613    2583 log.go:25] "Validated CRI v1 runtime API"
Sep 19 05:40:43.313513 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.313500    2583 log.go:25] "Validated CRI v1 image API"
Sep 19 05:40:43.314682 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.314663    2583 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 19 05:40:43.320413 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.320393    2583 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 9126a8be-cda2-4f2c-871a-e0c883b98084:/dev/nvme0n1p3 b8b527b2-8c05-4412-8bc1-b4a74d0e079f:/dev/nvme0n1p4]
Sep 19 05:40:43.320507 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.320411    2583 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 19 05:40:43.324570 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.324456    2583 manager.go:216] Machine: {Timestamp:2026-09-19 05:40:43.323963833 +0000 UTC m=+1.982418381 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099660 MemoryCapacity:33164472320 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec215ef9e6b996b166ef9776df5b3fb4 SystemUUID:ec215ef9-e6b9-96b1-66ef-9776df5b3fb4 BootID:98e0e57f-68a9-40bc-b851-38397b7cac94 Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582234112 Type:vfs Inodes:4048397 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632894464 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582238208 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:ef:70:51:39 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:ef:70:51:39 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:8a:11:7b:e1:70:4e Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164472320 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 19 05:40:43.324570 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.324558    2583 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 19 05:40:43.324732 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.324652    2583 manager.go:232] Version: {KernelVersion:5.14.0-570.141.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260914-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 19 05:40:43.326333 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.326304    2583 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 19 05:40:43.326496 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.326335    2583 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-139-144.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 19 05:40:43.327572 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.327560    2583 topology_manager.go:138] "Creating topology manager with none policy"
Sep 19 05:40:43.327637 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.327576    2583 container_manager_linux.go:309] "Creating device plugin manager"
Sep 19 05:40:43.327637 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.327588    2583 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 19 05:40:43.327637 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.327602    2583 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 19 05:40:43.327769 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.327691    2583 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 19 05:40:43.330409 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.330392    2583 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 19 05:40:43.330571 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.330558    2583 state_mem.go:36] "Initialized new in-memory state store"
Sep 19 05:40:43.330695 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.330685    2583 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 19 05:40:43.330752 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.330720    2583 kubelet.go:486] "Attempting to sync node with API server"
Sep 19 05:40:43.330752 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.330742    2583 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 19 05:40:43.330843 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.330758    2583 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 19 05:40:43.330843 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.330771    2583 kubelet.go:398] "Adding apiserver pod source"
Sep 19 05:40:43.330843 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.330784    2583 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 19 05:40:43.331140 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.331111    2583 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 19 05:40:43.331212 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.331149    2583 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 19 05:40:43.333456 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.333440    2583 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.13-6.rhaos4.21.git6253162.el9" apiVersion="v1"
Sep 19 05:40:43.335632 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.335619    2583 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 19 05:40:43.335675 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.335650    2583 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 19 05:40:43.336796 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.336785    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 19 05:40:43.336854 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.336801    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 19 05:40:43.336854 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.336808    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 19 05:40:43.336854 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.336813    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 19 05:40:43.336854 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.336824    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 19 05:40:43.336854 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.336834    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 19 05:40:43.336854 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.336852    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 19 05:40:43.336854 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.336857    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 19 05:40:43.337035 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.336864    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 19 05:40:43.337035 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.336870    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 19 05:40:43.337035 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.336884    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 19 05:40:43.337035 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.336890    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 19 05:40:43.337035 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.336913    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 19 05:40:43.337035 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.336918    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 19 05:40:43.340548 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.340526    2583 server.go:1268] "Started kubelet"
Sep 19 05:40:43.340739 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.340699    2583 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 19 05:40:43.340826 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.340748    2583 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 19 05:40:43.341264 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.341241    2583 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 19 05:40:43.341442 ip-10-0-139-144 systemd[1]: Started Kubernetes Kubelet.
Sep 19 05:40:43.342269 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.342250    2583 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 19 05:40:43.379805 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.379762    2583 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-139-144.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 19 05:40:43.380142 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.380078    2583 server.go:310] "Adding debug handlers to kubelet server"
Sep 19 05:40:43.380709 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:40:43.380657    2583 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-139-144.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 19 05:40:43.380709 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:40:43.380671    2583 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 19 05:40:43.380878 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:40:43.380854    2583 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 19 05:40:43.384415 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.384395    2583 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 19 05:40:43.384415 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.384405    2583 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 19 05:40:43.384599 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:40:43.384543    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-144.ec2.internal\" not found"
Sep 19 05:40:43.384974 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.384956    2583 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 19 05:40:43.384974 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.384972    2583 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 19 05:40:43.385105 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.384955    2583 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 19 05:40:43.385176 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.385165    2583 reconstruct.go:97] "Volume reconstruction finished"
Sep 19 05:40:43.385176 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.385176    2583 reconciler.go:29] "Reconciler: start to sync state"
Sep 19 05:40:43.385278 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.385258    2583 factory.go:55] Registering systemd factory
Sep 19 05:40:43.385278 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.385271    2583 factory.go:223] Registration of the systemd container factory successfully
Sep 19 05:40:43.385677 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.385658    2583 factory.go:153] Registering CRI-O factory
Sep 19 05:40:43.385761 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.385680    2583 factory.go:223] Registration of the crio container factory successfully
Sep 19 05:40:43.385761 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.385728    2583 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 19 05:40:43.385761 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.385747    2583 factory.go:103] Registering Raw factory
Sep 19 05:40:43.385761 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.385762    2583 manager.go:1196] Started watching for new ooms in manager
Sep 19 05:40:43.386653 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.386319    2583 manager.go:349] Starting recovery of all containers
Sep 19 05:40:43.386748 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:40:43.385423    2583 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-144.ec2.internal.18d6a261e99173d7  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-144.ec2.internal,UID:ip-10-0-139-144.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-139-144.ec2.internal,},FirstTimestamp:2026-09-19 05:40:43.340501975 +0000 UTC m=+1.998956523,LastTimestamp:2026-09-19 05:40:43.340501975 +0000 UTC m=+1.998956523,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-144.ec2.internal,}"
Sep 19 05:40:43.389042 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.389023    2583 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-g6rrb"
Sep 19 05:40:43.394038 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:40:43.393881    2583 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 19 05:40:43.394038 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:40:43.394012    2583 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-139-144.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 19 05:40:43.395297 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.395273    2583 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-g6rrb"
Sep 19 05:40:43.396778 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.396758    2583 manager.go:354] Recovery completed
Sep 19 05:40:43.400731 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.400718    2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 19 05:40:43.402493 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.402474    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-144.ec2.internal" event="NodeHasSufficientMemory"
Sep 19 05:40:43.402588 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.402507    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-144.ec2.internal" event="NodeHasNoDiskPressure"
Sep 19 05:40:43.402588 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.402522    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-144.ec2.internal" event="NodeHasSufficientPID"
Sep 19 05:40:43.403026 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.403012    2583 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 19 05:40:43.403026 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.403026    2583 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 19 05:40:43.403117 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.403044    2583 state_mem.go:36] "Initialized new in-memory state store"
Sep 19 05:40:43.403117 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.403018    2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 19 05:40:43.403730 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.403717    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-144.ec2.internal" event="NodeHasSufficientMemory"
Sep 19 05:40:43.403796 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.403740    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-144.ec2.internal" event="NodeHasNoDiskPressure"
Sep 19 05:40:43.403796 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.403749    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-144.ec2.internal" event="NodeHasSufficientPID"
Sep 19 05:40:43.404024 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.404012    2583 policy_none.go:49] "None policy: Start"
Sep 19 05:40:43.404055 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.404028    2583 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 19 05:40:43.404055 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.404037    2583 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 19 05:40:43.404640 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:40:43.404549    2583 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-144.ec2.internal.18d6a261ed43557a  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-144.ec2.internal,UID:ip-10-0-139-144.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-139-144.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-139-144.ec2.internal,},FirstTimestamp:2026-09-19 05:40:43.402491258 +0000 UTC m=+2.060945812,LastTimestamp:2026-09-19 05:40:43.402491258 +0000 UTC m=+2.060945812,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-144.ec2.internal,}"
Sep 19 05:40:43.404933 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.404923    2583 policy_none.go:47] "Start"
Sep 19 05:40:43.443730 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.443717    2583 manager.go:337] "Starting Device Plugin manager"
Sep 19 05:40:43.455508 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:40:43.443750    2583 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 19 05:40:43.455508 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.443759    2583 server.go:87] "Starting device plugin registration server"
Sep 19 05:40:43.455508 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.443973    2583 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 19 05:40:43.455508 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.443992    2583 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 19 05:40:43.455508 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.444106    2583 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 19 05:40:43.455508 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.444195    2583 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 19 05:40:43.455508 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.444204    2583 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 19 05:40:43.455508 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:40:43.444704    2583 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 19 05:40:43.455508 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:40:43.444743    2583 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-139-144.ec2.internal\" not found"
Sep 19 05:40:43.490601 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.490569    2583 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 19 05:40:43.491782 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.491766    2583 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 19 05:40:43.491849 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.491814    2583 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 19 05:40:43.491849 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.491834    2583 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 19 05:40:43.491925 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:40:43.491870    2583 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 19 05:40:43.493952 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.493933    2583 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 19 05:40:43.544541 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.544490    2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 19 05:40:43.545331 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.545317    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-144.ec2.internal" event="NodeHasSufficientMemory"
Sep 19 05:40:43.545405 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.545344    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-144.ec2.internal" event="NodeHasNoDiskPressure"
Sep 19 05:40:43.545405 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.545354    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-144.ec2.internal" event="NodeHasSufficientPID"
Sep 19 05:40:43.545405 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.545375    2583 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-139-144.ec2.internal"
Sep 19 05:40:43.558342 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.558324    2583 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-139-144.ec2.internal"
Sep 19 05:40:43.558407 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:40:43.558345    2583 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-139-144.ec2.internal\": node \"ip-10-0-139-144.ec2.internal\" not found"
Sep 19 05:40:43.592281 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.592261    2583 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-144.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-139-144.ec2.internal"]
Sep 19 05:40:43.592353 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.592316    2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 19 05:40:43.593569 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.593553    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-144.ec2.internal" event="NodeHasSufficientMemory"
Sep 19 05:40:43.593614 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.593581    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-144.ec2.internal" event="NodeHasNoDiskPressure"
Sep 19 05:40:43.593614 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.593592    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-144.ec2.internal" event="NodeHasSufficientPID"
Sep 19 05:40:43.594759 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.594747    2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 19 05:40:43.594881 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.594866    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-144.ec2.internal"
Sep 19 05:40:43.594921 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.594895    2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 19 05:40:43.595448 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.595425    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-144.ec2.internal" event="NodeHasSufficientMemory"
Sep 19 05:40:43.595536 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.595459    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-144.ec2.internal" event="NodeHasNoDiskPressure"
Sep 19 05:40:43.595536 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.595473    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-144.ec2.internal" event="NodeHasSufficientPID"
Sep 19 05:40:43.595536 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.595432    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-144.ec2.internal" event="NodeHasSufficientMemory"
Sep 19 05:40:43.595536 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.595533    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-144.ec2.internal" event="NodeHasNoDiskPressure"
Sep 19 05:40:43.595708 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.595544    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-144.ec2.internal" event="NodeHasSufficientPID"
Sep 19 05:40:43.596588 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.596572    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-144.ec2.internal"
Sep 19 05:40:43.596678 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.596595    2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 19 05:40:43.597162 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.597150    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-144.ec2.internal" event="NodeHasSufficientMemory"
Sep 19 05:40:43.597239 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.597173    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-144.ec2.internal" event="NodeHasNoDiskPressure"
Sep 19 05:40:43.597239 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.597184    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-144.ec2.internal" event="NodeHasSufficientPID"
Sep 19 05:40:43.613660 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:40:43.613638    2583 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-144.ec2.internal\" not found" node="ip-10-0-139-144.ec2.internal"
Sep 19 05:40:43.617384 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:40:43.617370    2583 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-144.ec2.internal\" not found" node="ip-10-0-139-144.ec2.internal"
Sep 19 05:40:43.639013 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:40:43.638992    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-144.ec2.internal\" not found"
Sep 19 05:40:43.686408 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.686383    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9a68be2b93f8cd1218394798eccfafaf-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-144.ec2.internal\" (UID: \"9a68be2b93f8cd1218394798eccfafaf\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-144.ec2.internal"
Sep 19 05:40:43.686505 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.686414    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/96c6a7a003ab3151e3ef151f14cc508f-config\") pod \"kube-apiserver-proxy-ip-10-0-139-144.ec2.internal\" (UID: \"96c6a7a003ab3151e3ef151f14cc508f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-144.ec2.internal"
Sep 19 05:40:43.686505 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.686437    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/9a68be2b93f8cd1218394798eccfafaf-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-144.ec2.internal\" (UID: \"9a68be2b93f8cd1218394798eccfafaf\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-144.ec2.internal"
Sep 19 05:40:43.739704 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:40:43.739683    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-144.ec2.internal\" not found"
Sep 19 05:40:43.786533 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.786513    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/9a68be2b93f8cd1218394798eccfafaf-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-144.ec2.internal\" (UID: \"9a68be2b93f8cd1218394798eccfafaf\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-144.ec2.internal"
Sep 19 05:40:43.786599 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.786538    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9a68be2b93f8cd1218394798eccfafaf-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-144.ec2.internal\" (UID: \"9a68be2b93f8cd1218394798eccfafaf\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-144.ec2.internal"
Sep 19 05:40:43.786599 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.786556    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/96c6a7a003ab3151e3ef151f14cc508f-config\") pod \"kube-apiserver-proxy-ip-10-0-139-144.ec2.internal\" (UID: \"96c6a7a003ab3151e3ef151f14cc508f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-144.ec2.internal"
Sep 19 05:40:43.786599 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.786580    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/96c6a7a003ab3151e3ef151f14cc508f-config\") pod \"kube-apiserver-proxy-ip-10-0-139-144.ec2.internal\" (UID: \"96c6a7a003ab3151e3ef151f14cc508f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-144.ec2.internal"
Sep 19 05:40:43.786695 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.786606    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/9a68be2b93f8cd1218394798eccfafaf-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-144.ec2.internal\" (UID: \"9a68be2b93f8cd1218394798eccfafaf\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-144.ec2.internal"
Sep 19 05:40:43.786695 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.786610    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9a68be2b93f8cd1218394798eccfafaf-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-144.ec2.internal\" (UID: \"9a68be2b93f8cd1218394798eccfafaf\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-144.ec2.internal"
Sep 19 05:40:43.839774 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:40:43.839720    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-144.ec2.internal\" not found"
Sep 19 05:40:43.917775 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.917753    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-144.ec2.internal"
Sep 19 05:40:43.920657 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:43.920633    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-144.ec2.internal"
Sep 19 05:40:43.940406 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:40:43.940384    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-144.ec2.internal\" not found"
Sep 19 05:40:44.040854 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:40:44.040823    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-144.ec2.internal\" not found"
Sep 19 05:40:44.141395 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:40:44.141337    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-144.ec2.internal\" not found"
Sep 19 05:40:44.226325 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.226299    2583 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 19 05:40:44.227555 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.227539    2583 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 19 05:40:44.285428 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.285398    2583 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-144.ec2.internal"
Sep 19 05:40:44.300718 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.300694    2583 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 19 05:40:44.300864 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.300841    2583 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 19 05:40:44.300921 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.300872    2583 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 19 05:40:44.300921 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:40:44.300880    2583 kubelet.go:3250] "Failed creating a mirror pod" err="Post \"https://a649b4ae7478e45d885e4645c81a90cb-d7e4a185d46e1b3e.elb.us-east-1.amazonaws.com:6443/api/v1/namespaces/kube-system/pods\": read tcp 10.0.139.144:45598->13.216.58.162:6443: use of closed network connection" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-144.ec2.internal"
Sep 19 05:40:44.300921 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.300895    2583 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 19 05:40:44.301021 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.300906    2583 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-144.ec2.internal"
Sep 19 05:40:44.331221 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.331190    2583 apiserver.go:52] "Watching apiserver"
Sep 19 05:40:44.341120 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.341099    2583 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 19 05:40:44.341584 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.341567    2583 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 19 05:40:44.341911 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.341891    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-node-tuning-operator/tuned-bghpv","openshift-dns/node-resolver-9pctd","openshift-image-registry/node-ca-xw29h","openshift-multus/multus-additional-cni-plugins-9rczd","openshift-multus/network-metrics-daemon-nrtjk","openshift-network-diagnostics/network-check-target-x5dk8","openshift-ovn-kubernetes/ovnkube-node-gj7h8","kube-system/global-pull-secret-syncer-fpnw2","kube-system/konnectivity-agent-7qkz7","kube-system/kube-apiserver-proxy-ip-10-0-139-144.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s8j7q","openshift-multus/multus-s2vw4","openshift-network-operator/iptables-alerter-bzgxk"]
Sep 19 05:40:44.343346 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.343319    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-gj7h8"
Sep 19 05:40:44.344404 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.344387    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-9pctd"
Sep 19 05:40:44.345511 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.345494    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-xw29h"
Sep 19 05:40:44.346602 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.346590    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-9rczd"
Sep 19 05:40:44.347765 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.347744    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nrtjk"
Sep 19 05:40:44.347837 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:40:44.347819    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nrtjk" podUID="b291ff5e-1667-4066-98d5-21495cd76311"
Sep 19 05:40:44.348707 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.348690    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 19 05:40:44.348800 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.348754    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 19 05:40:44.348878 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.348849    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-28qr6\""
Sep 19 05:40:44.348975 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.348959    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x5dk8"
Sep 19 05:40:44.349068 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:40:44.349017    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x5dk8" podUID="379278e6-0c96-4ff2-ba60-71ab1d88ed6d"
Sep 19 05:40:44.350227 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.350206    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-bghpv"
Sep 19 05:40:44.353425 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.353404    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 19 05:40:44.354216 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.354194    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fpnw2"
Sep 19 05:40:44.354664 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:40:44.354289    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fpnw2" podUID="4bd3176a-beea-424d-b4c2-dfc6f27298d8"
Sep 19 05:40:44.354664 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.354332    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 19 05:40:44.354664 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.354335    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 19 05:40:44.354664 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.354428    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-c95xg\""
Sep 19 05:40:44.354664 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.354429    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 19 05:40:44.355387 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.355371    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-7qkz7"
Sep 19 05:40:44.356455 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.356436    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 19 05:40:44.356920 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.356904    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s8j7q"
Sep 19 05:40:44.357242 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.357033    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 19 05:40:44.357242 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.357236    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-rc6pn\""
Sep 19 05:40:44.357461 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.357433    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 19 05:40:44.357943 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.357925    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 19 05:40:44.358038 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.357945    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 19 05:40:44.358369 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.358352    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-s2vw4"
Sep 19 05:40:44.358803 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.358786    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 19 05:40:44.358898 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.358852    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 19 05:40:44.358898 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.358854    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 19 05:40:44.359562 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.359540    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-144.ec2.internal"]
Sep 19 05:40:44.359644 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.359607    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-bzgxk"
Sep 19 05:40:44.365408 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.365392    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 19 05:40:44.365475 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.365417    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-db2d6\""
Sep 19 05:40:44.365734 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.365721    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 19 05:40:44.365786 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.365753    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 19 05:40:44.366490 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.366463    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 19 05:40:44.369800 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.369543    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 19 05:40:44.369800 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.369637    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-sbt5m\""
Sep 19 05:40:44.369800 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.369663    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-qpkfm\""
Sep 19 05:40:44.369800 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.369641    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 19 05:40:44.370740 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.370720    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 19 05:40:44.372495 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.372391    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 19 05:40:44.372495 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.372474    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 19 05:40:44.373306 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.373287    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 19 05:40:44.373395 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.373347    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 19 05:40:44.373453 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.373424    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-7wpl7\""
Sep 19 05:40:44.373453 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.373428    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 19 05:40:44.373652 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.373634    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-lmx88\""
Sep 19 05:40:44.373729 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.373651    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 19 05:40:44.377348 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.377330    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-wdx4s\""
Sep 19 05:40:44.384514 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.384500    2583 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 19 05:40:44.386114 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.386099    2583 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 19 05:40:44.390991 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.390972    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4081f4f1-e0f7-4cd4-afda-36669fc8d48a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-s8j7q\" (UID: \"4081f4f1-e0f7-4cd4-afda-36669fc8d48a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s8j7q"
Sep 19 05:40:44.391077 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.391002    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/32511ff0-ad60-4238-af22-53ccabc2c35b-cnibin\") pod \"multus-s2vw4\" (UID: \"32511ff0-ad60-4238-af22-53ccabc2c35b\") " pod="openshift-multus/multus-s2vw4"
Sep 19 05:40:44.391077 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.391024    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8d5d7036-c95e-4cc8-ad8b-c369bfb980b3-systemd-units\") pod \"ovnkube-node-gj7h8\" (UID: \"8d5d7036-c95e-4cc8-ad8b-c369bfb980b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-gj7h8"
Sep 19 05:40:44.391181 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.391078    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8d5d7036-c95e-4cc8-ad8b-c369bfb980b3-var-lib-openvswitch\") pod \"ovnkube-node-gj7h8\" (UID: \"8d5d7036-c95e-4cc8-ad8b-c369bfb980b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-gj7h8"
Sep 19 05:40:44.391181 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.391102    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e1f08f48-c39a-458a-9f90-87fb34abc301-etc-tuned\") pod \"tuned-bghpv\" (UID: \"e1f08f48-c39a-458a-9f90-87fb34abc301\") " pod="openshift-cluster-node-tuning-operator/tuned-bghpv"
Sep 19 05:40:44.391181 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.391124    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/4bd3176a-beea-424d-b4c2-dfc6f27298d8-dbus\") pod \"global-pull-secret-syncer-fpnw2\" (UID: \"4bd3176a-beea-424d-b4c2-dfc6f27298d8\") " pod="kube-system/global-pull-secret-syncer-fpnw2"
Sep 19 05:40:44.391181 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.391166    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4bd3176a-beea-424d-b4c2-dfc6f27298d8-original-pull-secret\") pod \"global-pull-secret-syncer-fpnw2\" (UID: \"4bd3176a-beea-424d-b4c2-dfc6f27298d8\") " pod="kube-system/global-pull-secret-syncer-fpnw2"
Sep 19 05:40:44.391331 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.391189    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e1f08f48-c39a-458a-9f90-87fb34abc301-var-lib-kubelet\") pod \"tuned-bghpv\" (UID: \"e1f08f48-c39a-458a-9f90-87fb34abc301\") " pod="openshift-cluster-node-tuning-operator/tuned-bghpv"
Sep 19 05:40:44.391331 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.391206    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ef2336d0-3175-4773-92fd-bafecb4f21f8-agent-certs\") pod \"konnectivity-agent-7qkz7\" (UID: \"ef2336d0-3175-4773-92fd-bafecb4f21f8\") " pod="kube-system/konnectivity-agent-7qkz7"
Sep 19 05:40:44.391331 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.391220    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8d5d7036-c95e-4cc8-ad8b-c369bfb980b3-run-ovn\") pod \"ovnkube-node-gj7h8\" (UID: \"8d5d7036-c95e-4cc8-ad8b-c369bfb980b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-gj7h8"
Sep 19 05:40:44.391331 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.391236    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8d5d7036-c95e-4cc8-ad8b-c369bfb980b3-env-overrides\") pod \"ovnkube-node-gj7h8\" (UID: \"8d5d7036-c95e-4cc8-ad8b-c369bfb980b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-gj7h8"
Sep 19 05:40:44.391331 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.391255    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/95fdae8d-ad6b-451b-bc3d-555fd930f300-cnibin\") pod \"multus-additional-cni-plugins-9rczd\" (UID: \"95fdae8d-ad6b-451b-bc3d-555fd930f300\") " pod="openshift-multus/multus-additional-cni-plugins-9rczd"
Sep 19 05:40:44.391331 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.391269    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/95fdae8d-ad6b-451b-bc3d-555fd930f300-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-9rczd\" (UID: \"95fdae8d-ad6b-451b-bc3d-555fd930f300\") " pod="openshift-multus/multus-additional-cni-plugins-9rczd"
Sep 19 05:40:44.391331 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.391283    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e1f08f48-c39a-458a-9f90-87fb34abc301-etc-modprobe-d\") pod \"tuned-bghpv\" (UID: \"e1f08f48-c39a-458a-9f90-87fb34abc301\") " pod="openshift-cluster-node-tuning-operator/tuned-bghpv"
Sep 19 05:40:44.391331 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.391297    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e1f08f48-c39a-458a-9f90-87fb34abc301-sys\") pod \"tuned-bghpv\" (UID: \"e1f08f48-c39a-458a-9f90-87fb34abc301\") " pod="openshift-cluster-node-tuning-operator/tuned-bghpv"
Sep 19 05:40:44.391331 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.391332    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/733a5136-d60a-4843-a361-463ac9830695-tmp-dir\") pod \"node-resolver-9pctd\" (UID: \"733a5136-d60a-4843-a361-463ac9830695\") " pod="openshift-dns/node-resolver-9pctd"
Sep 19 05:40:44.391650 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.391365    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4081f4f1-e0f7-4cd4-afda-36669fc8d48a-socket-dir\") pod \"aws-ebs-csi-driver-node-s8j7q\" (UID: \"4081f4f1-e0f7-4cd4-afda-36669fc8d48a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s8j7q"
Sep 19 05:40:44.391650 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.391382    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8d5d7036-c95e-4cc8-ad8b-c369bfb980b3-log-socket\") pod \"ovnkube-node-gj7h8\" (UID: \"8d5d7036-c95e-4cc8-ad8b-c369bfb980b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-gj7h8"
Sep 19 05:40:44.391650 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.391405    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8d5d7036-c95e-4cc8-ad8b-c369bfb980b3-host-cni-netd\") pod \"ovnkube-node-gj7h8\" (UID: \"8d5d7036-c95e-4cc8-ad8b-c369bfb980b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-gj7h8"
Sep 19 05:40:44.391650 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.391424    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/220708f7-de57-4213-ae89-7193308011cf-host\") pod \"node-ca-xw29h\" (UID: \"220708f7-de57-4213-ae89-7193308011cf\") " pod="openshift-image-registry/node-ca-xw29h"
Sep 19 05:40:44.391650 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.391487    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e1f08f48-c39a-458a-9f90-87fb34abc301-etc-kubernetes\") pod \"tuned-bghpv\" (UID: \"e1f08f48-c39a-458a-9f90-87fb34abc301\") " pod="openshift-cluster-node-tuning-operator/tuned-bghpv"
Sep 19 05:40:44.391650 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.391512    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4081f4f1-e0f7-4cd4-afda-36669fc8d48a-registration-dir\") pod \"aws-ebs-csi-driver-node-s8j7q\" (UID: \"4081f4f1-e0f7-4cd4-afda-36669fc8d48a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s8j7q"
Sep 19 05:40:44.391650 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.391530    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/32511ff0-ad60-4238-af22-53ccabc2c35b-host-var-lib-cni-multus\") pod \"multus-s2vw4\" (UID: \"32511ff0-ad60-4238-af22-53ccabc2c35b\") " pod="openshift-multus/multus-s2vw4"
Sep 19 05:40:44.391650 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.391547    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/32511ff0-ad60-4238-af22-53ccabc2c35b-etc-kubernetes\") pod \"multus-s2vw4\" (UID: \"32511ff0-ad60-4238-af22-53ccabc2c35b\") " pod="openshift-multus/multus-s2vw4"
Sep 19 05:40:44.391650 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.391561    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8d5d7036-c95e-4cc8-ad8b-c369bfb980b3-ovnkube-script-lib\") pod \"ovnkube-node-gj7h8\" (UID: \"8d5d7036-c95e-4cc8-ad8b-c369bfb980b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-gj7h8"
Sep 19 05:40:44.391650 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.391578    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e1f08f48-c39a-458a-9f90-87fb34abc301-etc-systemd\") pod \"tuned-bghpv\" (UID: \"e1f08f48-c39a-458a-9f90-87fb34abc301\") " pod="openshift-cluster-node-tuning-operator/tuned-bghpv"
Sep 19 05:40:44.391650 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.391592    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l79p8\" (UniqueName: \"kubernetes.io/projected/e1f08f48-c39a-458a-9f90-87fb34abc301-kube-api-access-l79p8\") pod \"tuned-bghpv\" (UID: \"e1f08f48-c39a-458a-9f90-87fb34abc301\") " pod="openshift-cluster-node-tuning-operator/tuned-bghpv"
Sep 19 05:40:44.391650 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.391607    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r46gv\" (UniqueName: \"kubernetes.io/projected/f39e3781-cfce-4a68-8f06-5c5903423038-kube-api-access-r46gv\") pod \"iptables-alerter-bzgxk\" (UID: \"f39e3781-cfce-4a68-8f06-5c5903423038\") " pod="openshift-network-operator/iptables-alerter-bzgxk"
Sep 19 05:40:44.391650 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.391642    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8d5d7036-c95e-4cc8-ad8b-c369bfb980b3-ovnkube-config\") pod \"ovnkube-node-gj7h8\" (UID: \"8d5d7036-c95e-4cc8-ad8b-c369bfb980b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-gj7h8"
Sep 19 05:40:44.392073 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.391678    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/733a5136-d60a-4843-a361-463ac9830695-hosts-file\") pod \"node-resolver-9pctd\" (UID: \"733a5136-d60a-4843-a361-463ac9830695\") " pod="openshift-dns/node-resolver-9pctd"
Sep 19 05:40:44.392073 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.391714    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t584t\" (UniqueName: \"kubernetes.io/projected/b291ff5e-1667-4066-98d5-21495cd76311-kube-api-access-t584t\") pod \"network-metrics-daemon-nrtjk\" (UID: \"b291ff5e-1667-4066-98d5-21495cd76311\") " pod="openshift-multus/network-metrics-daemon-nrtjk"
Sep 19 05:40:44.392073 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.391745    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/32511ff0-ad60-4238-af22-53ccabc2c35b-host-var-lib-kubelet\") pod \"multus-s2vw4\" (UID: \"32511ff0-ad60-4238-af22-53ccabc2c35b\") " pod="openshift-multus/multus-s2vw4"
Sep 19 05:40:44.392073 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.391768    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-krk9m\" (UniqueName: \"kubernetes.io/projected/32511ff0-ad60-4238-af22-53ccabc2c35b-kube-api-access-krk9m\") pod \"multus-s2vw4\" (UID: \"32511ff0-ad60-4238-af22-53ccabc2c35b\") " pod="openshift-multus/multus-s2vw4"
Sep 19 05:40:44.392073 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.391795    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8d5d7036-c95e-4cc8-ad8b-c369bfb980b3-etc-openvswitch\") pod \"ovnkube-node-gj7h8\" (UID: \"8d5d7036-c95e-4cc8-ad8b-c369bfb980b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-gj7h8"
Sep 19 05:40:44.392073 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.391811    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8d5d7036-c95e-4cc8-ad8b-c369bfb980b3-node-log\") pod \"ovnkube-node-gj7h8\" (UID: \"8d5d7036-c95e-4cc8-ad8b-c369bfb980b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-gj7h8"
Sep 19 05:40:44.392073 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.391833    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/220708f7-de57-4213-ae89-7193308011cf-serviceca\") pod \"node-ca-xw29h\" (UID: \"220708f7-de57-4213-ae89-7193308011cf\") " pod="openshift-image-registry/node-ca-xw29h"
Sep 19 05:40:44.392073 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.391852    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f39e3781-cfce-4a68-8f06-5c5903423038-iptables-alerter-script\") pod \"iptables-alerter-bzgxk\" (UID: \"f39e3781-cfce-4a68-8f06-5c5903423038\") " pod="openshift-network-operator/iptables-alerter-bzgxk"
Sep 19 05:40:44.392073 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.391871    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/32511ff0-ad60-4238-af22-53ccabc2c35b-host-var-lib-cni-bin\") pod \"multus-s2vw4\" (UID: \"32511ff0-ad60-4238-af22-53ccabc2c35b\") " pod="openshift-multus/multus-s2vw4"
Sep 19 05:40:44.392073 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.391889    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8d5d7036-c95e-4cc8-ad8b-c369bfb980b3-host-kubelet\") pod \"ovnkube-node-gj7h8\" (UID: \"8d5d7036-c95e-4cc8-ad8b-c369bfb980b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-gj7h8"
Sep 19 05:40:44.392073 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.391906    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8d5d7036-c95e-4cc8-ad8b-c369bfb980b3-host-slash\") pod \"ovnkube-node-gj7h8\" (UID: \"8d5d7036-c95e-4cc8-ad8b-c369bfb980b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-gj7h8"
Sep 19 05:40:44.392073 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.391920    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8d5d7036-c95e-4cc8-ad8b-c369bfb980b3-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-gj7h8\" (UID: \"8d5d7036-c95e-4cc8-ad8b-c369bfb980b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-gj7h8"
Sep 19 05:40:44.392073 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.391939    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fjcg7\" (UniqueName: \"kubernetes.io/projected/8d5d7036-c95e-4cc8-ad8b-c369bfb980b3-kube-api-access-fjcg7\") pod \"ovnkube-node-gj7h8\" (UID: \"8d5d7036-c95e-4cc8-ad8b-c369bfb980b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-gj7h8"
Sep 19 05:40:44.392073 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.391953    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/32511ff0-ad60-4238-af22-53ccabc2c35b-cni-binary-copy\") pod \"multus-s2vw4\" (UID: \"32511ff0-ad60-4238-af22-53ccabc2c35b\") " pod="openshift-multus/multus-s2vw4"
Sep 19 05:40:44.392073 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.391967    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/32511ff0-ad60-4238-af22-53ccabc2c35b-multus-daemon-config\") pod \"multus-s2vw4\" (UID: \"32511ff0-ad60-4238-af22-53ccabc2c35b\") " pod="openshift-multus/multus-s2vw4"
Sep 19 05:40:44.392073 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.391982    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/32511ff0-ad60-4238-af22-53ccabc2c35b-host-run-multus-certs\") pod \"multus-s2vw4\" (UID: \"32511ff0-ad60-4238-af22-53ccabc2c35b\") " pod="openshift-multus/multus-s2vw4"
Sep 19 05:40:44.392729 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.391996    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8d5d7036-c95e-4cc8-ad8b-c369bfb980b3-host-run-netns\") pod \"ovnkube-node-gj7h8\" (UID: \"8d5d7036-c95e-4cc8-ad8b-c369bfb980b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-gj7h8"
Sep 19 05:40:44.392729 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.392009    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8d5d7036-c95e-4cc8-ad8b-c369bfb980b3-run-openvswitch\") pod \"ovnkube-node-gj7h8\" (UID: \"8d5d7036-c95e-4cc8-ad8b-c369bfb980b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-gj7h8"
Sep 19 05:40:44.392729 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.392023    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ef2336d0-3175-4773-92fd-bafecb4f21f8-konnectivity-ca\") pod \"konnectivity-agent-7qkz7\" (UID: \"ef2336d0-3175-4773-92fd-bafecb4f21f8\") " pod="kube-system/konnectivity-agent-7qkz7"
Sep 19 05:40:44.392729 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.392048    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/32511ff0-ad60-4238-af22-53ccabc2c35b-host-run-k8s-cni-cncf-io\") pod \"multus-s2vw4\" (UID: \"32511ff0-ad60-4238-af22-53ccabc2c35b\") " pod="openshift-multus/multus-s2vw4"
Sep 19 05:40:44.392729 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.392075    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e1f08f48-c39a-458a-9f90-87fb34abc301-etc-sysconfig\") pod \"tuned-bghpv\" (UID: \"e1f08f48-c39a-458a-9f90-87fb34abc301\") " pod="openshift-cluster-node-tuning-operator/tuned-bghpv"
Sep 19 05:40:44.392729 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.392104    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7tlns\" (UniqueName: \"kubernetes.io/projected/733a5136-d60a-4843-a361-463ac9830695-kube-api-access-7tlns\") pod \"node-resolver-9pctd\" (UID: \"733a5136-d60a-4843-a361-463ac9830695\") " pod="openshift-dns/node-resolver-9pctd"
Sep 19 05:40:44.392729 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.392143    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8d5d7036-c95e-4cc8-ad8b-c369bfb980b3-host-run-ovn-kubernetes\") pod \"ovnkube-node-gj7h8\" (UID: \"8d5d7036-c95e-4cc8-ad8b-c369bfb980b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-gj7h8"
Sep 19 05:40:44.392729 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.392168    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8d5d7036-c95e-4cc8-ad8b-c369bfb980b3-host-cni-bin\") pod \"ovnkube-node-gj7h8\" (UID: \"8d5d7036-c95e-4cc8-ad8b-c369bfb980b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-gj7h8"
Sep 19 05:40:44.392729 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.392202    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/95fdae8d-ad6b-451b-bc3d-555fd930f300-tuning-conf-dir\") pod \"multus-additional-cni-plugins-9rczd\" (UID: \"95fdae8d-ad6b-451b-bc3d-555fd930f300\") " pod="openshift-multus/multus-additional-cni-plugins-9rczd"
Sep 19 05:40:44.392729 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.392235    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5ffc4\" (UniqueName: \"kubernetes.io/projected/95fdae8d-ad6b-451b-bc3d-555fd930f300-kube-api-access-5ffc4\") pod \"multus-additional-cni-plugins-9rczd\" (UID: \"95fdae8d-ad6b-451b-bc3d-555fd930f300\") " pod="openshift-multus/multus-additional-cni-plugins-9rczd"
Sep 19 05:40:44.392729 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.392261    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4081f4f1-e0f7-4cd4-afda-36669fc8d48a-etc-selinux\") pod \"aws-ebs-csi-driver-node-s8j7q\" (UID: \"4081f4f1-e0f7-4cd4-afda-36669fc8d48a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s8j7q"
Sep 19 05:40:44.392729 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.392289    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4081f4f1-e0f7-4cd4-afda-36669fc8d48a-sys-fs\") pod \"aws-ebs-csi-driver-node-s8j7q\" (UID: \"4081f4f1-e0f7-4cd4-afda-36669fc8d48a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s8j7q"
Sep 19 05:40:44.392729 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.392326    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/95fdae8d-ad6b-451b-bc3d-555fd930f300-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-9rczd\" (UID: \"95fdae8d-ad6b-451b-bc3d-555fd930f300\") " pod="openshift-multus/multus-additional-cni-plugins-9rczd"
Sep 19 05:40:44.392729 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.392356    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/4bd3176a-beea-424d-b4c2-dfc6f27298d8-kubelet-config\") pod \"global-pull-secret-syncer-fpnw2\" (UID: \"4bd3176a-beea-424d-b4c2-dfc6f27298d8\") " pod="kube-system/global-pull-secret-syncer-fpnw2"
Sep 19 05:40:44.392729 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.392380    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f39e3781-cfce-4a68-8f06-5c5903423038-host-slash\") pod \"iptables-alerter-bzgxk\" (UID: \"f39e3781-cfce-4a68-8f06-5c5903423038\") " pod="openshift-network-operator/iptables-alerter-bzgxk"
Sep 19 05:40:44.392729 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.392407    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-khkc6\" (UniqueName: \"kubernetes.io/projected/379278e6-0c96-4ff2-ba60-71ab1d88ed6d-kube-api-access-khkc6\") pod \"network-check-target-x5dk8\" (UID: \"379278e6-0c96-4ff2-ba60-71ab1d88ed6d\") " pod="openshift-network-diagnostics/network-check-target-x5dk8"
Sep 19 05:40:44.393257 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.392424    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4081f4f1-e0f7-4cd4-afda-36669fc8d48a-device-dir\") pod \"aws-ebs-csi-driver-node-s8j7q\" (UID: \"4081f4f1-e0f7-4cd4-afda-36669fc8d48a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s8j7q"
Sep 19 05:40:44.393257 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.392437    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e1f08f48-c39a-458a-9f90-87fb34abc301-tmp\") pod \"tuned-bghpv\" (UID: \"e1f08f48-c39a-458a-9f90-87fb34abc301\") " pod="openshift-cluster-node-tuning-operator/tuned-bghpv"
Sep 19 05:40:44.393257 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.392457    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b291ff5e-1667-4066-98d5-21495cd76311-metrics-certs\") pod \"network-metrics-daemon-nrtjk\" (UID: \"b291ff5e-1667-4066-98d5-21495cd76311\") " pod="openshift-multus/network-metrics-daemon-nrtjk"
Sep 19 05:40:44.393257 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.392485    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5qf77\" (UniqueName: \"kubernetes.io/projected/4081f4f1-e0f7-4cd4-afda-36669fc8d48a-kube-api-access-5qf77\") pod \"aws-ebs-csi-driver-node-s8j7q\" (UID: \"4081f4f1-e0f7-4cd4-afda-36669fc8d48a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s8j7q"
Sep 19 05:40:44.393257 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.392508    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/32511ff0-ad60-4238-af22-53ccabc2c35b-host-run-netns\") pod \"multus-s2vw4\" (UID: \"32511ff0-ad60-4238-af22-53ccabc2c35b\") " pod="openshift-multus/multus-s2vw4"
Sep 19 05:40:44.393257 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.392530    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/32511ff0-ad60-4238-af22-53ccabc2c35b-hostroot\") pod \"multus-s2vw4\" (UID: \"32511ff0-ad60-4238-af22-53ccabc2c35b\") " pod="openshift-multus/multus-s2vw4"
Sep 19 05:40:44.393257 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.392550    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/32511ff0-ad60-4238-af22-53ccabc2c35b-multus-conf-dir\") pod \"multus-s2vw4\" (UID: \"32511ff0-ad60-4238-af22-53ccabc2c35b\") " pod="openshift-multus/multus-s2vw4"
Sep 19 05:40:44.393257 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.392571    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e1f08f48-c39a-458a-9f90-87fb34abc301-run\") pod \"tuned-bghpv\" (UID: \"e1f08f48-c39a-458a-9f90-87fb34abc301\") " pod="openshift-cluster-node-tuning-operator/tuned-bghpv"
Sep 19 05:40:44.393257 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.392590    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e1f08f48-c39a-458a-9f90-87fb34abc301-host\") pod \"tuned-bghpv\" (UID: \"e1f08f48-c39a-458a-9f90-87fb34abc301\") " pod="openshift-cluster-node-tuning-operator/tuned-bghpv"
Sep 19 05:40:44.393257 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.392613    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8d5d7036-c95e-4cc8-ad8b-c369bfb980b3-run-systemd\") pod \"ovnkube-node-gj7h8\" (UID: \"8d5d7036-c95e-4cc8-ad8b-c369bfb980b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-gj7h8"
Sep 19 05:40:44.393257 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.392628    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e1f08f48-c39a-458a-9f90-87fb34abc301-etc-sysctl-d\") pod \"tuned-bghpv\" (UID: \"e1f08f48-c39a-458a-9f90-87fb34abc301\") " pod="openshift-cluster-node-tuning-operator/tuned-bghpv"
Sep 19 05:40:44.393257 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.392648    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e1f08f48-c39a-458a-9f90-87fb34abc301-etc-sysctl-conf\") pod \"tuned-bghpv\" (UID: \"e1f08f48-c39a-458a-9f90-87fb34abc301\") " pod="openshift-cluster-node-tuning-operator/tuned-bghpv"
Sep 19 05:40:44.393257 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.392661    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/32511ff0-ad60-4238-af22-53ccabc2c35b-system-cni-dir\") pod \"multus-s2vw4\" (UID: \"32511ff0-ad60-4238-af22-53ccabc2c35b\") " pod="openshift-multus/multus-s2vw4"
Sep 19 05:40:44.393257 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.392686    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/32511ff0-ad60-4238-af22-53ccabc2c35b-multus-cni-dir\") pod \"multus-s2vw4\" (UID: \"32511ff0-ad60-4238-af22-53ccabc2c35b\") " pod="openshift-multus/multus-s2vw4"
Sep 19 05:40:44.393257 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.392715    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/32511ff0-ad60-4238-af22-53ccabc2c35b-multus-socket-dir-parent\") pod \"multus-s2vw4\" (UID: \"32511ff0-ad60-4238-af22-53ccabc2c35b\") " pod="openshift-multus/multus-s2vw4"
Sep 19 05:40:44.393257 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.392736    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8d5d7036-c95e-4cc8-ad8b-c369bfb980b3-ovn-node-metrics-cert\") pod \"ovnkube-node-gj7h8\" (UID: \"8d5d7036-c95e-4cc8-ad8b-c369bfb980b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-gj7h8"
Sep 19 05:40:44.393257 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.392760    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pmbdz\" (UniqueName: \"kubernetes.io/projected/220708f7-de57-4213-ae89-7193308011cf-kube-api-access-pmbdz\") pod \"node-ca-xw29h\" (UID: \"220708f7-de57-4213-ae89-7193308011cf\") " pod="openshift-image-registry/node-ca-xw29h"
Sep 19 05:40:44.393769 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.392784    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/95fdae8d-ad6b-451b-bc3d-555fd930f300-system-cni-dir\") pod \"multus-additional-cni-plugins-9rczd\" (UID: \"95fdae8d-ad6b-451b-bc3d-555fd930f300\") " pod="openshift-multus/multus-additional-cni-plugins-9rczd"
Sep 19 05:40:44.393769 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.392806    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/95fdae8d-ad6b-451b-bc3d-555fd930f300-cni-binary-copy\") pod \"multus-additional-cni-plugins-9rczd\" (UID: \"95fdae8d-ad6b-451b-bc3d-555fd930f300\") " pod="openshift-multus/multus-additional-cni-plugins-9rczd"
Sep 19 05:40:44.393769 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.392828    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e1f08f48-c39a-458a-9f90-87fb34abc301-lib-modules\") pod \"tuned-bghpv\" (UID: \"e1f08f48-c39a-458a-9f90-87fb34abc301\") " pod="openshift-cluster-node-tuning-operator/tuned-bghpv"
Sep 19 05:40:44.399539 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.399470    2583 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-18 05:35:43 +0000 UTC" deadline="2028-03-04 01:42:15.024758366 +0000 UTC"
Sep 19 05:40:44.399588 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.399541    2583 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12764h1m30.625222092s"
Sep 19 05:40:44.406618 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.406600    2583 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 19 05:40:44.493242 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.493192    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/95fdae8d-ad6b-451b-bc3d-555fd930f300-cnibin\") pod \"multus-additional-cni-plugins-9rczd\" (UID: \"95fdae8d-ad6b-451b-bc3d-555fd930f300\") " pod="openshift-multus/multus-additional-cni-plugins-9rczd"
Sep 19 05:40:44.493242 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.493235    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/95fdae8d-ad6b-451b-bc3d-555fd930f300-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-9rczd\" (UID: \"95fdae8d-ad6b-451b-bc3d-555fd930f300\") " pod="openshift-multus/multus-additional-cni-plugins-9rczd"
Sep 19 05:40:44.493402 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.493260    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e1f08f48-c39a-458a-9f90-87fb34abc301-etc-modprobe-d\") pod \"tuned-bghpv\" (UID: \"e1f08f48-c39a-458a-9f90-87fb34abc301\") " pod="openshift-cluster-node-tuning-operator/tuned-bghpv"
Sep 19 05:40:44.493402 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.493265    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/95fdae8d-ad6b-451b-bc3d-555fd930f300-cnibin\") pod \"multus-additional-cni-plugins-9rczd\" (UID: \"95fdae8d-ad6b-451b-bc3d-555fd930f300\") " pod="openshift-multus/multus-additional-cni-plugins-9rczd"
Sep 19 05:40:44.493402 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.493281    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e1f08f48-c39a-458a-9f90-87fb34abc301-sys\") pod \"tuned-bghpv\" (UID: \"e1f08f48-c39a-458a-9f90-87fb34abc301\") " pod="openshift-cluster-node-tuning-operator/tuned-bghpv"
Sep 19 05:40:44.493402 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.493340    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/733a5136-d60a-4843-a361-463ac9830695-tmp-dir\") pod \"node-resolver-9pctd\" (UID: \"733a5136-d60a-4843-a361-463ac9830695\") " pod="openshift-dns/node-resolver-9pctd"
Sep 19 05:40:44.493402 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.493378    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e1f08f48-c39a-458a-9f90-87fb34abc301-etc-modprobe-d\") pod \"tuned-bghpv\" (UID: \"e1f08f48-c39a-458a-9f90-87fb34abc301\") " pod="openshift-cluster-node-tuning-operator/tuned-bghpv"
Sep 19 05:40:44.493402 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.493399    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4081f4f1-e0f7-4cd4-afda-36669fc8d48a-socket-dir\") pod \"aws-ebs-csi-driver-node-s8j7q\" (UID: \"4081f4f1-e0f7-4cd4-afda-36669fc8d48a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s8j7q"
Sep 19 05:40:44.493680 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.493427    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8d5d7036-c95e-4cc8-ad8b-c369bfb980b3-log-socket\") pod \"ovnkube-node-gj7h8\" (UID: \"8d5d7036-c95e-4cc8-ad8b-c369bfb980b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-gj7h8"
Sep 19 05:40:44.493680 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.493604    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4081f4f1-e0f7-4cd4-afda-36669fc8d48a-socket-dir\") pod \"aws-ebs-csi-driver-node-s8j7q\" (UID: \"4081f4f1-e0f7-4cd4-afda-36669fc8d48a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s8j7q"
Sep 19 05:40:44.493774 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.493679    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/733a5136-d60a-4843-a361-463ac9830695-tmp-dir\") pod \"node-resolver-9pctd\" (UID: \"733a5136-d60a-4843-a361-463ac9830695\") " pod="openshift-dns/node-resolver-9pctd"
Sep 19 05:40:44.493774 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.493693    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8d5d7036-c95e-4cc8-ad8b-c369bfb980b3-host-cni-netd\") pod \"ovnkube-node-gj7h8\" (UID: \"8d5d7036-c95e-4cc8-ad8b-c369bfb980b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-gj7h8"
Sep 19 05:40:44.493774 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.493703    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8d5d7036-c95e-4cc8-ad8b-c369bfb980b3-log-socket\") pod \"ovnkube-node-gj7h8\" (UID: \"8d5d7036-c95e-4cc8-ad8b-c369bfb980b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-gj7h8"
Sep 19 05:40:44.493774 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.493725    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/220708f7-de57-4213-ae89-7193308011cf-host\") pod \"node-ca-xw29h\" (UID: \"220708f7-de57-4213-ae89-7193308011cf\") " pod="openshift-image-registry/node-ca-xw29h"
Sep 19 05:40:44.493774 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.493763    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e1f08f48-c39a-458a-9f90-87fb34abc301-sys\") pod \"tuned-bghpv\" (UID: \"e1f08f48-c39a-458a-9f90-87fb34abc301\") " pod="openshift-cluster-node-tuning-operator/tuned-bghpv"
Sep 19 05:40:44.494006 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.493807    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8d5d7036-c95e-4cc8-ad8b-c369bfb980b3-host-cni-netd\") pod \"ovnkube-node-gj7h8\" (UID: \"8d5d7036-c95e-4cc8-ad8b-c369bfb980b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-gj7h8"
Sep 19 05:40:44.494006 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.493827    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e1f08f48-c39a-458a-9f90-87fb34abc301-etc-kubernetes\") pod \"tuned-bghpv\" (UID: \"e1f08f48-c39a-458a-9f90-87fb34abc301\") " pod="openshift-cluster-node-tuning-operator/tuned-bghpv"
Sep 19 05:40:44.494006 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.493837    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/220708f7-de57-4213-ae89-7193308011cf-host\") pod \"node-ca-xw29h\" (UID: \"220708f7-de57-4213-ae89-7193308011cf\") " pod="openshift-image-registry/node-ca-xw29h"
Sep 19 05:40:44.494006 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.493880    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4081f4f1-e0f7-4cd4-afda-36669fc8d48a-registration-dir\") pod \"aws-ebs-csi-driver-node-s8j7q\" (UID: \"4081f4f1-e0f7-4cd4-afda-36669fc8d48a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s8j7q"
Sep 19 05:40:44.494006 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.493894    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/95fdae8d-ad6b-451b-bc3d-555fd930f300-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-9rczd\" (UID: \"95fdae8d-ad6b-451b-bc3d-555fd930f300\") " pod="openshift-multus/multus-additional-cni-plugins-9rczd"
Sep 19 05:40:44.494006 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.493907    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/32511ff0-ad60-4238-af22-53ccabc2c35b-host-var-lib-cni-multus\") pod \"multus-s2vw4\" (UID: \"32511ff0-ad60-4238-af22-53ccabc2c35b\") " pod="openshift-multus/multus-s2vw4"
Sep 19 05:40:44.494006 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.493911    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e1f08f48-c39a-458a-9f90-87fb34abc301-etc-kubernetes\") pod \"tuned-bghpv\" (UID: \"e1f08f48-c39a-458a-9f90-87fb34abc301\") " pod="openshift-cluster-node-tuning-operator/tuned-bghpv"
Sep 19 05:40:44.494006 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.493935    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/32511ff0-ad60-4238-af22-53ccabc2c35b-etc-kubernetes\") pod \"multus-s2vw4\" (UID: \"32511ff0-ad60-4238-af22-53ccabc2c35b\") " pod="openshift-multus/multus-s2vw4"
Sep 19 05:40:44.494006 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.493939    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/32511ff0-ad60-4238-af22-53ccabc2c35b-host-var-lib-cni-multus\") pod \"multus-s2vw4\" (UID: \"32511ff0-ad60-4238-af22-53ccabc2c35b\") " pod="openshift-multus/multus-s2vw4"
Sep 19 05:40:44.494006 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.493957    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4081f4f1-e0f7-4cd4-afda-36669fc8d48a-registration-dir\") pod \"aws-ebs-csi-driver-node-s8j7q\" (UID: \"4081f4f1-e0f7-4cd4-afda-36669fc8d48a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s8j7q"
Sep 19 05:40:44.494006 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.493961    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8d5d7036-c95e-4cc8-ad8b-c369bfb980b3-ovnkube-script-lib\") pod \"ovnkube-node-gj7h8\" (UID: \"8d5d7036-c95e-4cc8-ad8b-c369bfb980b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-gj7h8"
Sep 19 05:40:44.494006 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.493996    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e1f08f48-c39a-458a-9f90-87fb34abc301-etc-systemd\") pod \"tuned-bghpv\" (UID: \"e1f08f48-c39a-458a-9f90-87fb34abc301\") " pod="openshift-cluster-node-tuning-operator/tuned-bghpv"
Sep 19 05:40:44.494793 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.494019    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l79p8\" (UniqueName: \"kubernetes.io/projected/e1f08f48-c39a-458a-9f90-87fb34abc301-kube-api-access-l79p8\") pod \"tuned-bghpv\" (UID: \"e1f08f48-c39a-458a-9f90-87fb34abc301\") " pod="openshift-cluster-node-tuning-operator/tuned-bghpv"
Sep 19 05:40:44.494793 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.494023    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/32511ff0-ad60-4238-af22-53ccabc2c35b-etc-kubernetes\") pod \"multus-s2vw4\" (UID: \"32511ff0-ad60-4238-af22-53ccabc2c35b\") " pod="openshift-multus/multus-s2vw4"
Sep 19 05:40:44.494793 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.494042    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r46gv\" (UniqueName: \"kubernetes.io/projected/f39e3781-cfce-4a68-8f06-5c5903423038-kube-api-access-r46gv\") pod \"iptables-alerter-bzgxk\" (UID: \"f39e3781-cfce-4a68-8f06-5c5903423038\") " pod="openshift-network-operator/iptables-alerter-bzgxk"
Sep 19 05:40:44.494793 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.494066    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8d5d7036-c95e-4cc8-ad8b-c369bfb980b3-ovnkube-config\") pod \"ovnkube-node-gj7h8\" (UID: \"8d5d7036-c95e-4cc8-ad8b-c369bfb980b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-gj7h8"
Sep 19 05:40:44.494793 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.494077    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e1f08f48-c39a-458a-9f90-87fb34abc301-etc-systemd\") pod \"tuned-bghpv\" (UID: \"e1f08f48-c39a-458a-9f90-87fb34abc301\") " pod="openshift-cluster-node-tuning-operator/tuned-bghpv"
Sep 19 05:40:44.494793 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.494087    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/733a5136-d60a-4843-a361-463ac9830695-hosts-file\") pod \"node-resolver-9pctd\" (UID: \"733a5136-d60a-4843-a361-463ac9830695\") " pod="openshift-dns/node-resolver-9pctd"
Sep 19 05:40:44.494793 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.494121    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t584t\" (UniqueName: \"kubernetes.io/projected/b291ff5e-1667-4066-98d5-21495cd76311-kube-api-access-t584t\") pod \"network-metrics-daemon-nrtjk\" (UID: \"b291ff5e-1667-4066-98d5-21495cd76311\") " pod="openshift-multus/network-metrics-daemon-nrtjk"
Sep 19 05:40:44.494793 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.494154    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/733a5136-d60a-4843-a361-463ac9830695-hosts-file\") pod \"node-resolver-9pctd\" (UID: \"733a5136-d60a-4843-a361-463ac9830695\") " pod="openshift-dns/node-resolver-9pctd"
Sep 19 05:40:44.494793 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.494161    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/32511ff0-ad60-4238-af22-53ccabc2c35b-host-var-lib-kubelet\") pod \"multus-s2vw4\" (UID: \"32511ff0-ad60-4238-af22-53ccabc2c35b\") " pod="openshift-multus/multus-s2vw4"
Sep 19 05:40:44.494793 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.494185    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-krk9m\" (UniqueName: \"kubernetes.io/projected/32511ff0-ad60-4238-af22-53ccabc2c35b-kube-api-access-krk9m\") pod \"multus-s2vw4\" (UID: \"32511ff0-ad60-4238-af22-53ccabc2c35b\") " pod="openshift-multus/multus-s2vw4"
Sep 19 05:40:44.494793 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.494206    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8d5d7036-c95e-4cc8-ad8b-c369bfb980b3-etc-openvswitch\") pod \"ovnkube-node-gj7h8\" (UID: \"8d5d7036-c95e-4cc8-ad8b-c369bfb980b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-gj7h8"
Sep 19 05:40:44.494793 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.494229    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8d5d7036-c95e-4cc8-ad8b-c369bfb980b3-node-log\") pod \"ovnkube-node-gj7h8\" (UID: \"8d5d7036-c95e-4cc8-ad8b-c369bfb980b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-gj7h8"
Sep 19 05:40:44.494793 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.494249    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/220708f7-de57-4213-ae89-7193308011cf-serviceca\") pod \"node-ca-xw29h\" (UID: \"220708f7-de57-4213-ae89-7193308011cf\") " pod="openshift-image-registry/node-ca-xw29h"
Sep 19 05:40:44.494793 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.494279    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f39e3781-cfce-4a68-8f06-5c5903423038-iptables-alerter-script\") pod \"iptables-alerter-bzgxk\" (UID: \"f39e3781-cfce-4a68-8f06-5c5903423038\") " pod="openshift-network-operator/iptables-alerter-bzgxk"
Sep 19 05:40:44.494793 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.494301    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/32511ff0-ad60-4238-af22-53ccabc2c35b-host-var-lib-cni-bin\") pod \"multus-s2vw4\" (UID: \"32511ff0-ad60-4238-af22-53ccabc2c35b\") " pod="openshift-multus/multus-s2vw4"
Sep 19 05:40:44.494793 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.494324    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8d5d7036-c95e-4cc8-ad8b-c369bfb980b3-host-kubelet\") pod \"ovnkube-node-gj7h8\" (UID: \"8d5d7036-c95e-4cc8-ad8b-c369bfb980b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-gj7h8"
Sep 19 05:40:44.494793 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.494350    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8d5d7036-c95e-4cc8-ad8b-c369bfb980b3-host-slash\") pod \"ovnkube-node-gj7h8\" (UID: \"8d5d7036-c95e-4cc8-ad8b-c369bfb980b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-gj7h8"
Sep 19 05:40:44.495590 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.494373    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8d5d7036-c95e-4cc8-ad8b-c369bfb980b3-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-gj7h8\" (UID: \"8d5d7036-c95e-4cc8-ad8b-c369bfb980b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-gj7h8"
Sep 19 05:40:44.495590 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.494394    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/32511ff0-ad60-4238-af22-53ccabc2c35b-host-var-lib-kubelet\") pod \"multus-s2vw4\" (UID: \"32511ff0-ad60-4238-af22-53ccabc2c35b\") " pod="openshift-multus/multus-s2vw4"
Sep 19 05:40:44.495590 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.494407    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fjcg7\" (UniqueName: \"kubernetes.io/projected/8d5d7036-c95e-4cc8-ad8b-c369bfb980b3-kube-api-access-fjcg7\") pod \"ovnkube-node-gj7h8\" (UID: \"8d5d7036-c95e-4cc8-ad8b-c369bfb980b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-gj7h8"
Sep 19 05:40:44.495590 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.494430    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/32511ff0-ad60-4238-af22-53ccabc2c35b-cni-binary-copy\") pod \"multus-s2vw4\" (UID: \"32511ff0-ad60-4238-af22-53ccabc2c35b\") " pod="openshift-multus/multus-s2vw4"
Sep 19 05:40:44.495590 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.494458    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/32511ff0-ad60-4238-af22-53ccabc2c35b-multus-daemon-config\") pod \"multus-s2vw4\" (UID: \"32511ff0-ad60-4238-af22-53ccabc2c35b\") " pod="openshift-multus/multus-s2vw4"
Sep 19 05:40:44.495590 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.494474    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8d5d7036-c95e-4cc8-ad8b-c369bfb980b3-host-slash\") pod \"ovnkube-node-gj7h8\" (UID: \"8d5d7036-c95e-4cc8-ad8b-c369bfb980b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-gj7h8"
Sep 19 05:40:44.495590 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.494485    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/32511ff0-ad60-4238-af22-53ccabc2c35b-host-run-multus-certs\") pod \"multus-s2vw4\" (UID: \"32511ff0-ad60-4238-af22-53ccabc2c35b\") " pod="openshift-multus/multus-s2vw4"
Sep 19 05:40:44.495590 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.494508    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8d5d7036-c95e-4cc8-ad8b-c369bfb980b3-host-run-netns\") pod \"ovnkube-node-gj7h8\" (UID: \"8d5d7036-c95e-4cc8-ad8b-c369bfb980b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-gj7h8"
Sep 19 05:40:44.495590 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.494520    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/32511ff0-ad60-4238-af22-53ccabc2c35b-host-var-lib-cni-bin\") pod \"multus-s2vw4\" (UID: \"32511ff0-ad60-4238-af22-53ccabc2c35b\") " pod="openshift-multus/multus-s2vw4"
Sep 19 05:40:44.495590 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.494531    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8d5d7036-c95e-4cc8-ad8b-c369bfb980b3-run-openvswitch\") pod \"ovnkube-node-gj7h8\" (UID: \"8d5d7036-c95e-4cc8-ad8b-c369bfb980b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-gj7h8"
Sep 19 05:40:44.495590 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.494553    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ef2336d0-3175-4773-92fd-bafecb4f21f8-konnectivity-ca\") pod \"konnectivity-agent-7qkz7\" (UID: \"ef2336d0-3175-4773-92fd-bafecb4f21f8\") " pod="kube-system/konnectivity-agent-7qkz7"
Sep 19 05:40:44.495590 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.494558    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8d5d7036-c95e-4cc8-ad8b-c369bfb980b3-host-kubelet\") pod \"ovnkube-node-gj7h8\" (UID: \"8d5d7036-c95e-4cc8-ad8b-c369bfb980b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-gj7h8"
Sep 19 05:40:44.495590 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.494585    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/32511ff0-ad60-4238-af22-53ccabc2c35b-host-run-k8s-cni-cncf-io\") pod \"multus-s2vw4\" (UID: \"32511ff0-ad60-4238-af22-53ccabc2c35b\") " pod="openshift-multus/multus-s2vw4"
Sep 19 05:40:44.495590 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.494585    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8d5d7036-c95e-4cc8-ad8b-c369bfb980b3-ovnkube-script-lib\") pod \"ovnkube-node-gj7h8\" (UID: \"8d5d7036-c95e-4cc8-ad8b-c369bfb980b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-gj7h8"
Sep 19 05:40:44.495590 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.494610    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e1f08f48-c39a-458a-9f90-87fb34abc301-etc-sysconfig\") pod \"tuned-bghpv\" (UID: \"e1f08f48-c39a-458a-9f90-87fb34abc301\") " pod="openshift-cluster-node-tuning-operator/tuned-bghpv"
Sep 19 05:40:44.495590 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.494616    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8d5d7036-c95e-4cc8-ad8b-c369bfb980b3-ovnkube-config\") pod \"ovnkube-node-gj7h8\" (UID: \"8d5d7036-c95e-4cc8-ad8b-c369bfb980b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-gj7h8"
Sep 19 05:40:44.495590 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.494634    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7tlns\" (UniqueName: \"kubernetes.io/projected/733a5136-d60a-4843-a361-463ac9830695-kube-api-access-7tlns\") pod \"node-resolver-9pctd\" (UID: \"733a5136-d60a-4843-a361-463ac9830695\") " pod="openshift-dns/node-resolver-9pctd"
Sep 19 05:40:44.496237 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.494647    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8d5d7036-c95e-4cc8-ad8b-c369bfb980b3-node-log\") pod \"ovnkube-node-gj7h8\" (UID: \"8d5d7036-c95e-4cc8-ad8b-c369bfb980b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-gj7h8"
Sep 19 05:40:44.496237 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.494659    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8d5d7036-c95e-4cc8-ad8b-c369bfb980b3-host-run-ovn-kubernetes\") pod \"ovnkube-node-gj7h8\" (UID: \"8d5d7036-c95e-4cc8-ad8b-c369bfb980b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-gj7h8"
Sep 19 05:40:44.496237 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.494672    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8d5d7036-c95e-4cc8-ad8b-c369bfb980b3-etc-openvswitch\") pod \"ovnkube-node-gj7h8\" (UID: \"8d5d7036-c95e-4cc8-ad8b-c369bfb980b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-gj7h8"
Sep 19 05:40:44.496237 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.494685    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8d5d7036-c95e-4cc8-ad8b-c369bfb980b3-host-cni-bin\") pod \"ovnkube-node-gj7h8\" (UID: \"8d5d7036-c95e-4cc8-ad8b-c369bfb980b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-gj7h8"
Sep 19 05:40:44.496237 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.494701    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8d5d7036-c95e-4cc8-ad8b-c369bfb980b3-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-gj7h8\" (UID: \"8d5d7036-c95e-4cc8-ad8b-c369bfb980b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-gj7h8"
Sep 19 05:40:44.496237 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.494712    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/95fdae8d-ad6b-451b-bc3d-555fd930f300-tuning-conf-dir\") pod \"multus-additional-cni-plugins-9rczd\" (UID: \"95fdae8d-ad6b-451b-bc3d-555fd930f300\") " pod="openshift-multus/multus-additional-cni-plugins-9rczd"
Sep 19 05:40:44.496237 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.494736    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5ffc4\" (UniqueName: \"kubernetes.io/projected/95fdae8d-ad6b-451b-bc3d-555fd930f300-kube-api-access-5ffc4\") pod \"multus-additional-cni-plugins-9rczd\" (UID: \"95fdae8d-ad6b-451b-bc3d-555fd930f300\") " pod="openshift-multus/multus-additional-cni-plugins-9rczd"
Sep 19 05:40:44.496237 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.494760    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4081f4f1-e0f7-4cd4-afda-36669fc8d48a-etc-selinux\") pod \"aws-ebs-csi-driver-node-s8j7q\" (UID: \"4081f4f1-e0f7-4cd4-afda-36669fc8d48a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s8j7q"
Sep 19 05:40:44.496237 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.494783    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4081f4f1-e0f7-4cd4-afda-36669fc8d48a-sys-fs\") pod \"aws-ebs-csi-driver-node-s8j7q\" (UID: \"4081f4f1-e0f7-4cd4-afda-36669fc8d48a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s8j7q"
Sep 19 05:40:44.496237 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.494813    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/95fdae8d-ad6b-451b-bc3d-555fd930f300-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-9rczd\" (UID: \"95fdae8d-ad6b-451b-bc3d-555fd930f300\") " pod="openshift-multus/multus-additional-cni-plugins-9rczd"
Sep 19 05:40:44.496237 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.494837    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/4bd3176a-beea-424d-b4c2-dfc6f27298d8-kubelet-config\") pod \"global-pull-secret-syncer-fpnw2\" (UID: \"4bd3176a-beea-424d-b4c2-dfc6f27298d8\") " pod="kube-system/global-pull-secret-syncer-fpnw2"
Sep 19 05:40:44.496237 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.494861    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f39e3781-cfce-4a68-8f06-5c5903423038-host-slash\") pod \"iptables-alerter-bzgxk\" (UID: \"f39e3781-cfce-4a68-8f06-5c5903423038\") " pod="openshift-network-operator/iptables-alerter-bzgxk"
Sep 19 05:40:44.496237 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.494887    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-khkc6\" (UniqueName: \"kubernetes.io/projected/379278e6-0c96-4ff2-ba60-71ab1d88ed6d-kube-api-access-khkc6\") pod \"network-check-target-x5dk8\" (UID: \"379278e6-0c96-4ff2-ba60-71ab1d88ed6d\") " pod="openshift-network-diagnostics/network-check-target-x5dk8"
Sep 19 05:40:44.496237 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.494912    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4081f4f1-e0f7-4cd4-afda-36669fc8d48a-device-dir\") pod \"aws-ebs-csi-driver-node-s8j7q\" (UID: \"4081f4f1-e0f7-4cd4-afda-36669fc8d48a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s8j7q"
Sep 19 05:40:44.496237 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.494955    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/32511ff0-ad60-4238-af22-53ccabc2c35b-host-run-k8s-cni-cncf-io\") pod \"multus-s2vw4\" (UID: \"32511ff0-ad60-4238-af22-53ccabc2c35b\") " pod="openshift-multus/multus-s2vw4"
Sep 19 05:40:44.496237 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.494971    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4081f4f1-e0f7-4cd4-afda-36669fc8d48a-device-dir\") pod \"aws-ebs-csi-driver-node-s8j7q\" (UID: \"4081f4f1-e0f7-4cd4-afda-36669fc8d48a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s8j7q"
Sep 19 05:40:44.496237 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.494990    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e1f08f48-c39a-458a-9f90-87fb34abc301-tmp\") pod \"tuned-bghpv\" (UID: \"e1f08f48-c39a-458a-9f90-87fb34abc301\") " pod="openshift-cluster-node-tuning-operator/tuned-bghpv"
Sep 19 05:40:44.496858 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.495010    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8d5d7036-c95e-4cc8-ad8b-c369bfb980b3-host-run-ovn-kubernetes\") pod \"ovnkube-node-gj7h8\" (UID: \"8d5d7036-c95e-4cc8-ad8b-c369bfb980b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-gj7h8"
Sep 19 05:40:44.496858 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.495018    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b291ff5e-1667-4066-98d5-21495cd76311-metrics-certs\") pod \"network-metrics-daemon-nrtjk\" (UID: \"b291ff5e-1667-4066-98d5-21495cd76311\") " pod="openshift-multus/network-metrics-daemon-nrtjk"
Sep 19 05:40:44.496858 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.495044    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qf77\" (UniqueName: \"kubernetes.io/projected/4081f4f1-e0f7-4cd4-afda-36669fc8d48a-kube-api-access-5qf77\") pod \"aws-ebs-csi-driver-node-s8j7q\" (UID: \"4081f4f1-e0f7-4cd4-afda-36669fc8d48a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s8j7q"
Sep 19 05:40:44.496858 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.495068    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/32511ff0-ad60-4238-af22-53ccabc2c35b-host-run-netns\") pod \"multus-s2vw4\" (UID: \"32511ff0-ad60-4238-af22-53ccabc2c35b\") " pod="openshift-multus/multus-s2vw4"
Sep 19 05:40:44.496858 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.495071    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/220708f7-de57-4213-ae89-7193308011cf-serviceca\") pod \"node-ca-xw29h\" (UID: \"220708f7-de57-4213-ae89-7193308011cf\") " pod="openshift-image-registry/node-ca-xw29h"
Sep 19 05:40:44.496858 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.495083    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ef2336d0-3175-4773-92fd-bafecb4f21f8-konnectivity-ca\") pod \"konnectivity-agent-7qkz7\" (UID: \"ef2336d0-3175-4773-92fd-bafecb4f21f8\") " pod="kube-system/konnectivity-agent-7qkz7"
Sep 19 05:40:44.496858 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.495070    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f39e3781-cfce-4a68-8f06-5c5903423038-iptables-alerter-script\") pod \"iptables-alerter-bzgxk\" (UID: \"f39e3781-cfce-4a68-8f06-5c5903423038\") " pod="openshift-network-operator/iptables-alerter-bzgxk"
Sep 19 05:40:44.496858 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.495095    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/32511ff0-ad60-4238-af22-53ccabc2c35b-hostroot\") pod \"multus-s2vw4\" (UID: \"32511ff0-ad60-4238-af22-53ccabc2c35b\") " pod="openshift-multus/multus-s2vw4"
Sep 19 05:40:44.496858 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.495101    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4081f4f1-e0f7-4cd4-afda-36669fc8d48a-sys-fs\") pod \"aws-ebs-csi-driver-node-s8j7q\" (UID: \"4081f4f1-e0f7-4cd4-afda-36669fc8d48a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s8j7q"
Sep 19 05:40:44.496858 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.495148    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/32511ff0-ad60-4238-af22-53ccabc2c35b-hostroot\") pod \"multus-s2vw4\" (UID: \"32511ff0-ad60-4238-af22-53ccabc2c35b\") " pod="openshift-multus/multus-s2vw4"
Sep 19 05:40:44.496858 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.495149    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/32511ff0-ad60-4238-af22-53ccabc2c35b-multus-conf-dir\") pod \"multus-s2vw4\" (UID: \"32511ff0-ad60-4238-af22-53ccabc2c35b\") " pod="openshift-multus/multus-s2vw4"
Sep 19 05:40:44.496858 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.495171    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8d5d7036-c95e-4cc8-ad8b-c369bfb980b3-run-openvswitch\") pod \"ovnkube-node-gj7h8\" (UID: \"8d5d7036-c95e-4cc8-ad8b-c369bfb980b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-gj7h8"
Sep 19 05:40:44.496858 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.495188    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/32511ff0-ad60-4238-af22-53ccabc2c35b-host-run-multus-certs\") pod \"multus-s2vw4\" (UID: \"32511ff0-ad60-4238-af22-53ccabc2c35b\") " pod="openshift-multus/multus-s2vw4"
Sep 19 05:40:44.496858 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.495187    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e1f08f48-c39a-458a-9f90-87fb34abc301-run\") pod \"tuned-bghpv\" (UID: \"e1f08f48-c39a-458a-9f90-87fb34abc301\") " pod="openshift-cluster-node-tuning-operator/tuned-bghpv"
Sep 19 05:40:44.496858 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.495222    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e1f08f48-c39a-458a-9f90-87fb34abc301-host\") pod \"tuned-bghpv\" (UID: \"e1f08f48-c39a-458a-9f90-87fb34abc301\") " pod="openshift-cluster-node-tuning-operator/tuned-bghpv"
Sep 19 05:40:44.496858 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.495225    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8d5d7036-c95e-4cc8-ad8b-c369bfb980b3-host-run-netns\") pod \"ovnkube-node-gj7h8\" (UID: \"8d5d7036-c95e-4cc8-ad8b-c369bfb980b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-gj7h8"
Sep 19 05:40:44.496858 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.495227    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/32511ff0-ad60-4238-af22-53ccabc2c35b-multus-conf-dir\") pod \"multus-s2vw4\" (UID: \"32511ff0-ad60-4238-af22-53ccabc2c35b\") " pod="openshift-multus/multus-s2vw4"
Sep 19 05:40:44.496858 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.495272    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e1f08f48-c39a-458a-9f90-87fb34abc301-host\") pod \"tuned-bghpv\" (UID: \"e1f08f48-c39a-458a-9f90-87fb34abc301\") " pod="openshift-cluster-node-tuning-operator/tuned-bghpv"
Sep 19 05:40:44.497686 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.495275    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/95fdae8d-ad6b-451b-bc3d-555fd930f300-tuning-conf-dir\") pod \"multus-additional-cni-plugins-9rczd\" (UID: \"95fdae8d-ad6b-451b-bc3d-555fd930f300\") " pod="openshift-multus/multus-additional-cni-plugins-9rczd"
Sep 19 05:40:44.497686 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.495308    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4081f4f1-e0f7-4cd4-afda-36669fc8d48a-etc-selinux\") pod \"aws-ebs-csi-driver-node-s8j7q\" (UID: \"4081f4f1-e0f7-4cd4-afda-36669fc8d48a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s8j7q"
Sep 19 05:40:44.497686 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.495311    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f39e3781-cfce-4a68-8f06-5c5903423038-host-slash\") pod \"iptables-alerter-bzgxk\" (UID: \"f39e3781-cfce-4a68-8f06-5c5903423038\") " pod="openshift-network-operator/iptables-alerter-bzgxk"
Sep 19 05:40:44.497686 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.495325    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8d5d7036-c95e-4cc8-ad8b-c369bfb980b3-host-cni-bin\") pod \"ovnkube-node-gj7h8\" (UID: \"8d5d7036-c95e-4cc8-ad8b-c369bfb980b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-gj7h8"
Sep 19 05:40:44.497686 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.495342    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/32511ff0-ad60-4238-af22-53ccabc2c35b-host-run-netns\") pod \"multus-s2vw4\" (UID: \"32511ff0-ad60-4238-af22-53ccabc2c35b\") " pod="openshift-multus/multus-s2vw4"
Sep 19 05:40:44.497686 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.495343    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8d5d7036-c95e-4cc8-ad8b-c369bfb980b3-run-systemd\") pod \"ovnkube-node-gj7h8\" (UID: \"8d5d7036-c95e-4cc8-ad8b-c369bfb980b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-gj7h8"
Sep 19 05:40:44.497686 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.495329    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/32511ff0-ad60-4238-af22-53ccabc2c35b-multus-daemon-config\") pod \"multus-s2vw4\" (UID: \"32511ff0-ad60-4238-af22-53ccabc2c35b\") " pod="openshift-multus/multus-s2vw4"
Sep 19 05:40:44.497686 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.495381    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8d5d7036-c95e-4cc8-ad8b-c369bfb980b3-run-systemd\") pod \"ovnkube-node-gj7h8\" (UID: \"8d5d7036-c95e-4cc8-ad8b-c369bfb980b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-gj7h8"
Sep 19 05:40:44.497686 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.495396    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/4bd3176a-beea-424d-b4c2-dfc6f27298d8-kubelet-config\") pod \"global-pull-secret-syncer-fpnw2\" (UID: \"4bd3176a-beea-424d-b4c2-dfc6f27298d8\") " pod="kube-system/global-pull-secret-syncer-fpnw2"
Sep 19 05:40:44.497686 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.495418    2583 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 19 05:40:44.497686 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.495485    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e1f08f48-c39a-458a-9f90-87fb34abc301-etc-sysctl-d\") pod \"tuned-bghpv\" (UID: \"e1f08f48-c39a-458a-9f90-87fb34abc301\") " pod="openshift-cluster-node-tuning-operator/tuned-bghpv"
Sep 19 05:40:44.497686 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.495557    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e1f08f48-c39a-458a-9f90-87fb34abc301-run\") pod \"tuned-bghpv\" (UID: \"e1f08f48-c39a-458a-9f90-87fb34abc301\") " pod="openshift-cluster-node-tuning-operator/tuned-bghpv"
Sep 19 05:40:44.497686 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.495601    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e1f08f48-c39a-458a-9f90-87fb34abc301-etc-sysconfig\") pod \"tuned-bghpv\" (UID: \"e1f08f48-c39a-458a-9f90-87fb34abc301\") " pod="openshift-cluster-node-tuning-operator/tuned-bghpv"
Sep 19 05:40:44.497686 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:40:44.495646    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 05:40:44.497686 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.495709    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e1f08f48-c39a-458a-9f90-87fb34abc301-etc-sysctl-d\") pod \"tuned-bghpv\" (UID: \"e1f08f48-c39a-458a-9f90-87fb34abc301\") " pod="openshift-cluster-node-tuning-operator/tuned-bghpv"
Sep 19 05:40:44.497686 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:40:44.495717    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b291ff5e-1667-4066-98d5-21495cd76311-metrics-certs podName:b291ff5e-1667-4066-98d5-21495cd76311 nodeName:}" failed. No retries permitted until 2026-09-19 05:40:44.995697198 +0000 UTC m=+3.654151747 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b291ff5e-1667-4066-98d5-21495cd76311-metrics-certs") pod "network-metrics-daemon-nrtjk" (UID: "b291ff5e-1667-4066-98d5-21495cd76311") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 05:40:44.497686 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.495800    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/95fdae8d-ad6b-451b-bc3d-555fd930f300-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-9rczd\" (UID: \"95fdae8d-ad6b-451b-bc3d-555fd930f300\") " pod="openshift-multus/multus-additional-cni-plugins-9rczd"
Sep 19 05:40:44.497686 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.495801    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e1f08f48-c39a-458a-9f90-87fb34abc301-etc-sysctl-conf\") pod \"tuned-bghpv\" (UID: \"e1f08f48-c39a-458a-9f90-87fb34abc301\") " pod="openshift-cluster-node-tuning-operator/tuned-bghpv"
Sep 19 05:40:44.498523 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.495841    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/32511ff0-ad60-4238-af22-53ccabc2c35b-cni-binary-copy\") pod \"multus-s2vw4\" (UID: \"32511ff0-ad60-4238-af22-53ccabc2c35b\") " pod="openshift-multus/multus-s2vw4"
Sep 19 05:40:44.498523 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.496002    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e1f08f48-c39a-458a-9f90-87fb34abc301-etc-sysctl-conf\") pod \"tuned-bghpv\" (UID: \"e1f08f48-c39a-458a-9f90-87fb34abc301\") " pod="openshift-cluster-node-tuning-operator/tuned-bghpv"
Sep 19 05:40:44.498523 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.496037    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/32511ff0-ad60-4238-af22-53ccabc2c35b-system-cni-dir\") pod \"multus-s2vw4\" (UID: \"32511ff0-ad60-4238-af22-53ccabc2c35b\") " pod="openshift-multus/multus-s2vw4"
Sep 19 05:40:44.498523 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.496061    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/32511ff0-ad60-4238-af22-53ccabc2c35b-multus-cni-dir\") pod \"multus-s2vw4\" (UID: \"32511ff0-ad60-4238-af22-53ccabc2c35b\") " pod="openshift-multus/multus-s2vw4"
Sep 19 05:40:44.498523 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.496084    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/32511ff0-ad60-4238-af22-53ccabc2c35b-multus-socket-dir-parent\") pod \"multus-s2vw4\" (UID: \"32511ff0-ad60-4238-af22-53ccabc2c35b\") " pod="openshift-multus/multus-s2vw4"
Sep 19 05:40:44.498523 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.496108    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8d5d7036-c95e-4cc8-ad8b-c369bfb980b3-ovn-node-metrics-cert\") pod \"ovnkube-node-gj7h8\" (UID: \"8d5d7036-c95e-4cc8-ad8b-c369bfb980b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-gj7h8"
Sep 19 05:40:44.498523 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.496151    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pmbdz\" (UniqueName: \"kubernetes.io/projected/220708f7-de57-4213-ae89-7193308011cf-kube-api-access-pmbdz\") pod \"node-ca-xw29h\" (UID: \"220708f7-de57-4213-ae89-7193308011cf\") " pod="openshift-image-registry/node-ca-xw29h"
Sep 19 05:40:44.498523 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.496176    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/95fdae8d-ad6b-451b-bc3d-555fd930f300-system-cni-dir\") pod \"multus-additional-cni-plugins-9rczd\" (UID: \"95fdae8d-ad6b-451b-bc3d-555fd930f300\") " pod="openshift-multus/multus-additional-cni-plugins-9rczd"
Sep 19 05:40:44.498523 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.496199    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/95fdae8d-ad6b-451b-bc3d-555fd930f300-cni-binary-copy\") pod \"multus-additional-cni-plugins-9rczd\" (UID: \"95fdae8d-ad6b-451b-bc3d-555fd930f300\") " pod="openshift-multus/multus-additional-cni-plugins-9rczd"
Sep 19 05:40:44.498523 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.496222    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e1f08f48-c39a-458a-9f90-87fb34abc301-lib-modules\") pod \"tuned-bghpv\" (UID: \"e1f08f48-c39a-458a-9f90-87fb34abc301\") " pod="openshift-cluster-node-tuning-operator/tuned-bghpv"
Sep 19 05:40:44.498523 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.496247    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4081f4f1-e0f7-4cd4-afda-36669fc8d48a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-s8j7q\" (UID: \"4081f4f1-e0f7-4cd4-afda-36669fc8d48a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s8j7q"
Sep 19 05:40:44.498523 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.496269    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/32511ff0-ad60-4238-af22-53ccabc2c35b-cnibin\") pod \"multus-s2vw4\" (UID: \"32511ff0-ad60-4238-af22-53ccabc2c35b\") " pod="openshift-multus/multus-s2vw4"
Sep 19 05:40:44.498523 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.496300    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8d5d7036-c95e-4cc8-ad8b-c369bfb980b3-systemd-units\") pod \"ovnkube-node-gj7h8\" (UID: \"8d5d7036-c95e-4cc8-ad8b-c369bfb980b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-gj7h8"
Sep 19 05:40:44.498523 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.496325    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8d5d7036-c95e-4cc8-ad8b-c369bfb980b3-var-lib-openvswitch\") pod \"ovnkube-node-gj7h8\" (UID: \"8d5d7036-c95e-4cc8-ad8b-c369bfb980b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-gj7h8"
Sep 19 05:40:44.498523 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.496349    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e1f08f48-c39a-458a-9f90-87fb34abc301-etc-tuned\") pod \"tuned-bghpv\" (UID: \"e1f08f48-c39a-458a-9f90-87fb34abc301\") " pod="openshift-cluster-node-tuning-operator/tuned-bghpv"
Sep 19 05:40:44.498523 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.496372    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/4bd3176a-beea-424d-b4c2-dfc6f27298d8-dbus\") pod \"global-pull-secret-syncer-fpnw2\" (UID: \"4bd3176a-beea-424d-b4c2-dfc6f27298d8\") " pod="kube-system/global-pull-secret-syncer-fpnw2"
Sep 19 05:40:44.498523 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.496397    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4bd3176a-beea-424d-b4c2-dfc6f27298d8-original-pull-secret\") pod \"global-pull-secret-syncer-fpnw2\" (UID: \"4bd3176a-beea-424d-b4c2-dfc6f27298d8\") " pod="kube-system/global-pull-secret-syncer-fpnw2"
Sep 19 05:40:44.499235 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.496422    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e1f08f48-c39a-458a-9f90-87fb34abc301-var-lib-kubelet\") pod \"tuned-bghpv\" (UID: \"e1f08f48-c39a-458a-9f90-87fb34abc301\") " pod="openshift-cluster-node-tuning-operator/tuned-bghpv"
Sep 19 05:40:44.499235 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.496449    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ef2336d0-3175-4773-92fd-bafecb4f21f8-agent-certs\") pod \"konnectivity-agent-7qkz7\" (UID: \"ef2336d0-3175-4773-92fd-bafecb4f21f8\") " pod="kube-system/konnectivity-agent-7qkz7"
Sep 19 05:40:44.499235 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.496470    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8d5d7036-c95e-4cc8-ad8b-c369bfb980b3-run-ovn\") pod \"ovnkube-node-gj7h8\" (UID: \"8d5d7036-c95e-4cc8-ad8b-c369bfb980b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-gj7h8"
Sep 19 05:40:44.499235 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.496531    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8d5d7036-c95e-4cc8-ad8b-c369bfb980b3-var-lib-openvswitch\") pod \"ovnkube-node-gj7h8\" (UID: \"8d5d7036-c95e-4cc8-ad8b-c369bfb980b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-gj7h8"
Sep 19 05:40:44.499235 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.496583    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/32511ff0-ad60-4238-af22-53ccabc2c35b-multus-socket-dir-parent\") pod \"multus-s2vw4\" (UID: \"32511ff0-ad60-4238-af22-53ccabc2c35b\") " pod="openshift-multus/multus-s2vw4"
Sep 19 05:40:44.499235 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.496593    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/32511ff0-ad60-4238-af22-53ccabc2c35b-cnibin\") pod \"multus-s2vw4\" (UID: \"32511ff0-ad60-4238-af22-53ccabc2c35b\") " pod="openshift-multus/multus-s2vw4"
Sep 19 05:40:44.499235 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.496640    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8d5d7036-c95e-4cc8-ad8b-c369bfb980b3-systemd-units\") pod \"ovnkube-node-gj7h8\" (UID: \"8d5d7036-c95e-4cc8-ad8b-c369bfb980b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-gj7h8"
Sep 19 05:40:44.499235 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.496641    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/32511ff0-ad60-4238-af22-53ccabc2c35b-system-cni-dir\") pod \"multus-s2vw4\" (UID: \"32511ff0-ad60-4238-af22-53ccabc2c35b\") " pod="openshift-multus/multus-s2vw4"
Sep 19 05:40:44.499235 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.496685    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/32511ff0-ad60-4238-af22-53ccabc2c35b-multus-cni-dir\") pod \"multus-s2vw4\" (UID: \"32511ff0-ad60-4238-af22-53ccabc2c35b\") " pod="openshift-multus/multus-s2vw4"
Sep 19 05:40:44.499235 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.496767    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/95fdae8d-ad6b-451b-bc3d-555fd930f300-cni-binary-copy\") pod \"multus-additional-cni-plugins-9rczd\" (UID: \"95fdae8d-ad6b-451b-bc3d-555fd930f300\") " pod="openshift-multus/multus-additional-cni-plugins-9rczd"
Sep 19 05:40:44.499235 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.496792    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e1f08f48-c39a-458a-9f90-87fb34abc301-lib-modules\") pod \"tuned-bghpv\" (UID: \"e1f08f48-c39a-458a-9f90-87fb34abc301\") " pod="openshift-cluster-node-tuning-operator/tuned-bghpv"
Sep 19 05:40:44.499235 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.496820    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/95fdae8d-ad6b-451b-bc3d-555fd930f300-system-cni-dir\") pod \"multus-additional-cni-plugins-9rczd\" (UID: \"95fdae8d-ad6b-451b-bc3d-555fd930f300\") " pod="openshift-multus/multus-additional-cni-plugins-9rczd"
Sep 19 05:40:44.499235 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:40:44.496864    2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 19 05:40:44.499235 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:40:44.496912    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4bd3176a-beea-424d-b4c2-dfc6f27298d8-original-pull-secret podName:4bd3176a-beea-424d-b4c2-dfc6f27298d8 nodeName:}" failed. No retries permitted until 2026-09-19 05:40:44.996896664 +0000 UTC m=+3.655351214 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4bd3176a-beea-424d-b4c2-dfc6f27298d8-original-pull-secret") pod "global-pull-secret-syncer-fpnw2" (UID: "4bd3176a-beea-424d-b4c2-dfc6f27298d8") : object "kube-system"/"original-pull-secret" not registered
Sep 19 05:40:44.499235 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.496913    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/4bd3176a-beea-424d-b4c2-dfc6f27298d8-dbus\") pod \"global-pull-secret-syncer-fpnw2\" (UID: \"4bd3176a-beea-424d-b4c2-dfc6f27298d8\") " pod="kube-system/global-pull-secret-syncer-fpnw2"
Sep 19 05:40:44.499235 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.496964    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8d5d7036-c95e-4cc8-ad8b-c369bfb980b3-run-ovn\") pod \"ovnkube-node-gj7h8\" (UID: \"8d5d7036-c95e-4cc8-ad8b-c369bfb980b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-gj7h8"
Sep 19 05:40:44.499235 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.497010    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e1f08f48-c39a-458a-9f90-87fb34abc301-var-lib-kubelet\") pod \"tuned-bghpv\" (UID: \"e1f08f48-c39a-458a-9f90-87fb34abc301\") " pod="openshift-cluster-node-tuning-operator/tuned-bghpv"
Sep 19 05:40:44.499881 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.497203    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4081f4f1-e0f7-4cd4-afda-36669fc8d48a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-s8j7q\" (UID: \"4081f4f1-e0f7-4cd4-afda-36669fc8d48a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s8j7q"
Sep 19 05:40:44.499881 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.497479    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8d5d7036-c95e-4cc8-ad8b-c369bfb980b3-env-overrides\") pod \"ovnkube-node-gj7h8\" (UID: \"8d5d7036-c95e-4cc8-ad8b-c369bfb980b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-gj7h8"
Sep 19 05:40:44.499881 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.496492    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8d5d7036-c95e-4cc8-ad8b-c369bfb980b3-env-overrides\") pod \"ovnkube-node-gj7h8\" (UID: \"8d5d7036-c95e-4cc8-ad8b-c369bfb980b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-gj7h8"
Sep 19 05:40:44.499881 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:44.498591    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a68be2b93f8cd1218394798eccfafaf.slice/crio-923116d856feca4c79379be04e4880950243a33150c4c94faa97f16458fedbe0 WatchSource:0}: Error finding container 923116d856feca4c79379be04e4880950243a33150c4c94faa97f16458fedbe0: Status 404 returned error can't find the container with id 923116d856feca4c79379be04e4880950243a33150c4c94faa97f16458fedbe0
Sep 19 05:40:44.499881 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.499406    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e1f08f48-c39a-458a-9f90-87fb34abc301-etc-tuned\") pod \"tuned-bghpv\" (UID: \"e1f08f48-c39a-458a-9f90-87fb34abc301\") " pod="openshift-cluster-node-tuning-operator/tuned-bghpv"
Sep 19 05:40:44.499881 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:44.499462    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96c6a7a003ab3151e3ef151f14cc508f.slice/crio-7b613ce0750270338c1b4a9e794448c3a658355af4cd4ba3f422ab5b238c81ae WatchSource:0}: Error finding container 7b613ce0750270338c1b4a9e794448c3a658355af4cd4ba3f422ab5b238c81ae: Status 404 returned error can't find the container with id 7b613ce0750270338c1b4a9e794448c3a658355af4cd4ba3f422ab5b238c81ae
Sep 19 05:40:44.499881 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.499499    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8d5d7036-c95e-4cc8-ad8b-c369bfb980b3-ovn-node-metrics-cert\") pod \"ovnkube-node-gj7h8\" (UID: \"8d5d7036-c95e-4cc8-ad8b-c369bfb980b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-gj7h8"
Sep 19 05:40:44.499881 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.499541    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ef2336d0-3175-4773-92fd-bafecb4f21f8-agent-certs\") pod \"konnectivity-agent-7qkz7\" (UID: \"ef2336d0-3175-4773-92fd-bafecb4f21f8\") " pod="kube-system/konnectivity-agent-7qkz7"
Sep 19 05:40:44.499881 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.499606    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e1f08f48-c39a-458a-9f90-87fb34abc301-tmp\") pod \"tuned-bghpv\" (UID: \"e1f08f48-c39a-458a-9f90-87fb34abc301\") " pod="openshift-cluster-node-tuning-operator/tuned-bghpv"
Sep 19 05:40:44.504573 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.504557    2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 19 05:40:44.508649 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.508628    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t584t\" (UniqueName: \"kubernetes.io/projected/b291ff5e-1667-4066-98d5-21495cd76311-kube-api-access-t584t\") pod \"network-metrics-daemon-nrtjk\" (UID: \"b291ff5e-1667-4066-98d5-21495cd76311\") " pod="openshift-multus/network-metrics-daemon-nrtjk"
Sep 19 05:40:44.511165 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.511148    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fjcg7\" (UniqueName: \"kubernetes.io/projected/8d5d7036-c95e-4cc8-ad8b-c369bfb980b3-kube-api-access-fjcg7\") pod \"ovnkube-node-gj7h8\" (UID: \"8d5d7036-c95e-4cc8-ad8b-c369bfb980b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-gj7h8"
Sep 19 05:40:44.511862 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.511845    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-krk9m\" (UniqueName: \"kubernetes.io/projected/32511ff0-ad60-4238-af22-53ccabc2c35b-kube-api-access-krk9m\") pod \"multus-s2vw4\" (UID: \"32511ff0-ad60-4238-af22-53ccabc2c35b\") " pod="openshift-multus/multus-s2vw4"
Sep 19 05:40:44.513752 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.513726    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5qf77\" (UniqueName: \"kubernetes.io/projected/4081f4f1-e0f7-4cd4-afda-36669fc8d48a-kube-api-access-5qf77\") pod \"aws-ebs-csi-driver-node-s8j7q\" (UID: \"4081f4f1-e0f7-4cd4-afda-36669fc8d48a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s8j7q"
Sep 19 05:40:44.513752 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:40:44.513741    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 19 05:40:44.513861 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:40:44.513767    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 19 05:40:44.513861 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:40:44.513781    2583 projected.go:196] Error preparing data for projected volume kube-api-access-khkc6 for pod openshift-network-diagnostics/network-check-target-x5dk8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 19 05:40:44.513861 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:40:44.513843    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/379278e6-0c96-4ff2-ba60-71ab1d88ed6d-kube-api-access-khkc6 podName:379278e6-0c96-4ff2-ba60-71ab1d88ed6d nodeName:}" failed. No retries permitted until 2026-09-19 05:40:45.013826822 +0000 UTC m=+3.672281377 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-khkc6" (UniqueName: "kubernetes.io/projected/379278e6-0c96-4ff2-ba60-71ab1d88ed6d-kube-api-access-khkc6") pod "network-check-target-x5dk8" (UID: "379278e6-0c96-4ff2-ba60-71ab1d88ed6d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 19 05:40:44.513965 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.513956    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r46gv\" (UniqueName: \"kubernetes.io/projected/f39e3781-cfce-4a68-8f06-5c5903423038-kube-api-access-r46gv\") pod \"iptables-alerter-bzgxk\" (UID: \"f39e3781-cfce-4a68-8f06-5c5903423038\") " pod="openshift-network-operator/iptables-alerter-bzgxk"
Sep 19 05:40:44.517705 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.517687    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l79p8\" (UniqueName: \"kubernetes.io/projected/e1f08f48-c39a-458a-9f90-87fb34abc301-kube-api-access-l79p8\") pod \"tuned-bghpv\" (UID: \"e1f08f48-c39a-458a-9f90-87fb34abc301\") " pod="openshift-cluster-node-tuning-operator/tuned-bghpv"
Sep 19 05:40:44.518983 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.518961    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pmbdz\" (UniqueName: \"kubernetes.io/projected/220708f7-de57-4213-ae89-7193308011cf-kube-api-access-pmbdz\") pod \"node-ca-xw29h\" (UID: \"220708f7-de57-4213-ae89-7193308011cf\") " pod="openshift-image-registry/node-ca-xw29h"
Sep 19 05:40:44.520501 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.520482    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7tlns\" (UniqueName: \"kubernetes.io/projected/733a5136-d60a-4843-a361-463ac9830695-kube-api-access-7tlns\") pod \"node-resolver-9pctd\" (UID: \"733a5136-d60a-4843-a361-463ac9830695\") " pod="openshift-dns/node-resolver-9pctd"
Sep 19 05:40:44.522758 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.522739    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5ffc4\" (UniqueName: \"kubernetes.io/projected/95fdae8d-ad6b-451b-bc3d-555fd930f300-kube-api-access-5ffc4\") pod \"multus-additional-cni-plugins-9rczd\" (UID: \"95fdae8d-ad6b-451b-bc3d-555fd930f300\") " pod="openshift-multus/multus-additional-cni-plugins-9rczd"
Sep 19 05:40:44.568523 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.568505    2583 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-hfntw"
Sep 19 05:40:44.573847 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.573831    2583 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 19 05:40:44.581022 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.581003    2583 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-hfntw"
Sep 19 05:40:44.670691 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.670624    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-gj7h8"
Sep 19 05:40:44.677304 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:44.677283    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d5d7036_c95e_4cc8_ad8b_c369bfb980b3.slice/crio-f07259fb38aeb3959e078ee7df9ef160bfc1ecec0fc0edc0193ad5dd02f533a4 WatchSource:0}: Error finding container f07259fb38aeb3959e078ee7df9ef160bfc1ecec0fc0edc0193ad5dd02f533a4: Status 404 returned error can't find the container with id f07259fb38aeb3959e078ee7df9ef160bfc1ecec0fc0edc0193ad5dd02f533a4
Sep 19 05:40:44.687950 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.687934    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-9pctd"
Sep 19 05:40:44.695313 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:44.695294    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod733a5136_d60a_4843_a361_463ac9830695.slice/crio-f9a6d6d9c8815c055334ca113fd684dac60194df1b7668290cac27fbf49ae7d6 WatchSource:0}: Error finding container f9a6d6d9c8815c055334ca113fd684dac60194df1b7668290cac27fbf49ae7d6: Status 404 returned error can't find the container with id f9a6d6d9c8815c055334ca113fd684dac60194df1b7668290cac27fbf49ae7d6
Sep 19 05:40:44.701484 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.701465    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-xw29h"
Sep 19 05:40:44.707843 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:44.707819    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod220708f7_de57_4213_ae89_7193308011cf.slice/crio-e2b727c6f8178fd7a7e3f9e168e9e3cbc4ede9124bbb5cff5ce70159b6e84b75 WatchSource:0}: Error finding container e2b727c6f8178fd7a7e3f9e168e9e3cbc4ede9124bbb5cff5ce70159b6e84b75: Status 404 returned error can't find the container with id e2b727c6f8178fd7a7e3f9e168e9e3cbc4ede9124bbb5cff5ce70159b6e84b75
Sep 19 05:40:44.717585 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.717570    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-9rczd"
Sep 19 05:40:44.724399 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:44.724377    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95fdae8d_ad6b_451b_bc3d_555fd930f300.slice/crio-46740716f9a0c4c5ba280172a0f7c11b1ec6dce8f8f0605165c6f2402f66734e WatchSource:0}: Error finding container 46740716f9a0c4c5ba280172a0f7c11b1ec6dce8f8f0605165c6f2402f66734e: Status 404 returned error can't find the container with id 46740716f9a0c4c5ba280172a0f7c11b1ec6dce8f8f0605165c6f2402f66734e
Sep 19 05:40:44.733838 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.733821    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-bghpv"
Sep 19 05:40:44.740224 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:44.740197    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1f08f48_c39a_458a_9f90_87fb34abc301.slice/crio-8f082c211df8d1181744606bf5df6f11982f8cd2e1ca684784eeadb1cb319aae WatchSource:0}: Error finding container 8f082c211df8d1181744606bf5df6f11982f8cd2e1ca684784eeadb1cb319aae: Status 404 returned error can't find the container with id 8f082c211df8d1181744606bf5df6f11982f8cd2e1ca684784eeadb1cb319aae
Sep 19 05:40:44.761438 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.761415    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-7qkz7"
Sep 19 05:40:44.767720 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:44.767700    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef2336d0_3175_4773_92fd_bafecb4f21f8.slice/crio-328cbec8150307085288b8e51522890d7f0821a2b7c97e8743939bcea0325194 WatchSource:0}: Error finding container 328cbec8150307085288b8e51522890d7f0821a2b7c97e8743939bcea0325194: Status 404 returned error can't find the container with id 328cbec8150307085288b8e51522890d7f0821a2b7c97e8743939bcea0325194
Sep 19 05:40:44.778237 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.778219    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s8j7q"
Sep 19 05:40:44.784558 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:44.784539    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4081f4f1_e0f7_4cd4_afda_36669fc8d48a.slice/crio-0d47fce8403dfaa7c5cc98fc355eec5aeadef7e379b72cd996a111e979db88f6 WatchSource:0}: Error finding container 0d47fce8403dfaa7c5cc98fc355eec5aeadef7e379b72cd996a111e979db88f6: Status 404 returned error can't find the container with id 0d47fce8403dfaa7c5cc98fc355eec5aeadef7e379b72cd996a111e979db88f6
Sep 19 05:40:44.796525 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.796509    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-s2vw4"
Sep 19 05:40:44.803025 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:44.803006    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-bzgxk"
Sep 19 05:40:44.803718 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:44.803698    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32511ff0_ad60_4238_af22_53ccabc2c35b.slice/crio-1a484a93150a0c95d034b99e5cc009be0de8f2bd3413f0e434f9f6d6363d0d0a WatchSource:0}: Error finding container 1a484a93150a0c95d034b99e5cc009be0de8f2bd3413f0e434f9f6d6363d0d0a: Status 404 returned error can't find the container with id 1a484a93150a0c95d034b99e5cc009be0de8f2bd3413f0e434f9f6d6363d0d0a
Sep 19 05:40:44.809652 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:40:44.809636    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf39e3781_cfce_4a68_8f06_5c5903423038.slice/crio-c3c42d6091e4184ab658c2bc212518bce7de7a51dcf75d7a56383ddabf79bb09 WatchSource:0}: Error finding container c3c42d6091e4184ab658c2bc212518bce7de7a51dcf75d7a56383ddabf79bb09: Status 404 returned error can't find the container with id c3c42d6091e4184ab658c2bc212518bce7de7a51dcf75d7a56383ddabf79bb09
Sep 19 05:40:45.000502 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:45.000427    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b291ff5e-1667-4066-98d5-21495cd76311-metrics-certs\") pod \"network-metrics-daemon-nrtjk\" (UID: \"b291ff5e-1667-4066-98d5-21495cd76311\") " pod="openshift-multus/network-metrics-daemon-nrtjk"
Sep 19 05:40:45.000502 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:45.000475    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4bd3176a-beea-424d-b4c2-dfc6f27298d8-original-pull-secret\") pod \"global-pull-secret-syncer-fpnw2\" (UID: \"4bd3176a-beea-424d-b4c2-dfc6f27298d8\") " pod="kube-system/global-pull-secret-syncer-fpnw2"
Sep 19 05:40:45.000711 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:40:45.000583    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 05:40:45.000711 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:40:45.000659    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b291ff5e-1667-4066-98d5-21495cd76311-metrics-certs podName:b291ff5e-1667-4066-98d5-21495cd76311 nodeName:}" failed. No retries permitted until 2026-09-19 05:40:46.000643243 +0000 UTC m=+4.659097779 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b291ff5e-1667-4066-98d5-21495cd76311-metrics-certs") pod "network-metrics-daemon-nrtjk" (UID: "b291ff5e-1667-4066-98d5-21495cd76311") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 05:40:45.000711 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:40:45.000591    2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 19 05:40:45.000853 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:40:45.000722    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4bd3176a-beea-424d-b4c2-dfc6f27298d8-original-pull-secret podName:4bd3176a-beea-424d-b4c2-dfc6f27298d8 nodeName:}" failed. No retries permitted until 2026-09-19 05:40:46.000708829 +0000 UTC m=+4.659163366 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4bd3176a-beea-424d-b4c2-dfc6f27298d8-original-pull-secret") pod "global-pull-secret-syncer-fpnw2" (UID: "4bd3176a-beea-424d-b4c2-dfc6f27298d8") : object "kube-system"/"original-pull-secret" not registered
Sep 19 05:40:45.100866 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:45.100822    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-khkc6\" (UniqueName: \"kubernetes.io/projected/379278e6-0c96-4ff2-ba60-71ab1d88ed6d-kube-api-access-khkc6\") pod \"network-check-target-x5dk8\" (UID: \"379278e6-0c96-4ff2-ba60-71ab1d88ed6d\") " pod="openshift-network-diagnostics/network-check-target-x5dk8"
Sep 19 05:40:45.101062 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:40:45.101040    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 19 05:40:45.101224 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:40:45.101073    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 19 05:40:45.101224 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:40:45.101111    2583 projected.go:196] Error preparing data for projected volume kube-api-access-khkc6 for pod openshift-network-diagnostics/network-check-target-x5dk8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 19 05:40:45.101224 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:40:45.101201    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/379278e6-0c96-4ff2-ba60-71ab1d88ed6d-kube-api-access-khkc6 podName:379278e6-0c96-4ff2-ba60-71ab1d88ed6d nodeName:}" failed. No retries permitted until 2026-09-19 05:40:46.101174497 +0000 UTC m=+4.759629033 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-khkc6" (UniqueName: "kubernetes.io/projected/379278e6-0c96-4ff2-ba60-71ab1d88ed6d-kube-api-access-khkc6") pod "network-check-target-x5dk8" (UID: "379278e6-0c96-4ff2-ba60-71ab1d88ed6d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 19 05:40:45.495060 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:45.494926    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fpnw2"
Sep 19 05:40:45.495060 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:40:45.495048    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fpnw2" podUID="4bd3176a-beea-424d-b4c2-dfc6f27298d8"
Sep 19 05:40:45.523404 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:45.523303    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-bzgxk" event={"ID":"f39e3781-cfce-4a68-8f06-5c5903423038","Type":"ContainerStarted","Data":"c3c42d6091e4184ab658c2bc212518bce7de7a51dcf75d7a56383ddabf79bb09"}
Sep 19 05:40:45.532437 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:45.532401    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s8j7q" event={"ID":"4081f4f1-e0f7-4cd4-afda-36669fc8d48a","Type":"ContainerStarted","Data":"0d47fce8403dfaa7c5cc98fc355eec5aeadef7e379b72cd996a111e979db88f6"}
Sep 19 05:40:45.542602 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:45.542572    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9rczd" event={"ID":"95fdae8d-ad6b-451b-bc3d-555fd930f300","Type":"ContainerStarted","Data":"46740716f9a0c4c5ba280172a0f7c11b1ec6dce8f8f0605165c6f2402f66734e"}
Sep 19 05:40:45.561612 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:45.561516    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-xw29h" event={"ID":"220708f7-de57-4213-ae89-7193308011cf","Type":"ContainerStarted","Data":"e2b727c6f8178fd7a7e3f9e168e9e3cbc4ede9124bbb5cff5ce70159b6e84b75"}
Sep 19 05:40:45.584275 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:45.584194    2583 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-18 05:35:44 +0000 UTC" deadline="2028-03-25 13:01:57.722921466 +0000 UTC"
Sep 19 05:40:45.584275 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:45.584227    2583 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13279h21m12.138698743s"
Sep 19 05:40:45.595393 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:45.595326    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-9pctd" event={"ID":"733a5136-d60a-4843-a361-463ac9830695","Type":"ContainerStarted","Data":"f9a6d6d9c8815c055334ca113fd684dac60194df1b7668290cac27fbf49ae7d6"}
Sep 19 05:40:45.603424 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:45.603391    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-s2vw4" event={"ID":"32511ff0-ad60-4238-af22-53ccabc2c35b","Type":"ContainerStarted","Data":"1a484a93150a0c95d034b99e5cc009be0de8f2bd3413f0e434f9f6d6363d0d0a"}
Sep 19 05:40:45.621596 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:45.621547    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-7qkz7" event={"ID":"ef2336d0-3175-4773-92fd-bafecb4f21f8","Type":"ContainerStarted","Data":"328cbec8150307085288b8e51522890d7f0821a2b7c97e8743939bcea0325194"}
Sep 19 05:40:45.648635 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:45.648599    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-bghpv" event={"ID":"e1f08f48-c39a-458a-9f90-87fb34abc301","Type":"ContainerStarted","Data":"8f082c211df8d1181744606bf5df6f11982f8cd2e1ca684784eeadb1cb319aae"}
Sep 19 05:40:45.659775 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:45.659738    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gj7h8" event={"ID":"8d5d7036-c95e-4cc8-ad8b-c369bfb980b3","Type":"ContainerStarted","Data":"f07259fb38aeb3959e078ee7df9ef160bfc1ecec0fc0edc0193ad5dd02f533a4"}
Sep 19 05:40:45.681708 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:45.681617    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-144.ec2.internal" event={"ID":"96c6a7a003ab3151e3ef151f14cc508f","Type":"ContainerStarted","Data":"7b613ce0750270338c1b4a9e794448c3a658355af4cd4ba3f422ab5b238c81ae"}
Sep 19 05:40:45.696152 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:45.695857    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-144.ec2.internal" event={"ID":"9a68be2b93f8cd1218394798eccfafaf","Type":"ContainerStarted","Data":"923116d856feca4c79379be04e4880950243a33150c4c94faa97f16458fedbe0"}
Sep 19 05:40:45.844068 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:45.843990    2583 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 19 05:40:46.009225 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:46.008608    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b291ff5e-1667-4066-98d5-21495cd76311-metrics-certs\") pod \"network-metrics-daemon-nrtjk\" (UID: \"b291ff5e-1667-4066-98d5-21495cd76311\") " pod="openshift-multus/network-metrics-daemon-nrtjk"
Sep 19 05:40:46.009225 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:46.008668    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4bd3176a-beea-424d-b4c2-dfc6f27298d8-original-pull-secret\") pod \"global-pull-secret-syncer-fpnw2\" (UID: \"4bd3176a-beea-424d-b4c2-dfc6f27298d8\") " pod="kube-system/global-pull-secret-syncer-fpnw2"
Sep 19 05:40:46.009225 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:40:46.008809    2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 19 05:40:46.009225 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:40:46.008870    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4bd3176a-beea-424d-b4c2-dfc6f27298d8-original-pull-secret podName:4bd3176a-beea-424d-b4c2-dfc6f27298d8 nodeName:}" failed. No retries permitted until 2026-09-19 05:40:48.008852348 +0000 UTC m=+6.667306888 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4bd3176a-beea-424d-b4c2-dfc6f27298d8-original-pull-secret") pod "global-pull-secret-syncer-fpnw2" (UID: "4bd3176a-beea-424d-b4c2-dfc6f27298d8") : object "kube-system"/"original-pull-secret" not registered
Sep 19 05:40:46.009579 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:40:46.009280    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 05:40:46.009579 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:40:46.009333    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b291ff5e-1667-4066-98d5-21495cd76311-metrics-certs podName:b291ff5e-1667-4066-98d5-21495cd76311 nodeName:}" failed. No retries permitted until 2026-09-19 05:40:48.009318586 +0000 UTC m=+6.667773129 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b291ff5e-1667-4066-98d5-21495cd76311-metrics-certs") pod "network-metrics-daemon-nrtjk" (UID: "b291ff5e-1667-4066-98d5-21495cd76311") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 05:40:46.109414 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:46.109323    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-khkc6\" (UniqueName: \"kubernetes.io/projected/379278e6-0c96-4ff2-ba60-71ab1d88ed6d-kube-api-access-khkc6\") pod \"network-check-target-x5dk8\" (UID: \"379278e6-0c96-4ff2-ba60-71ab1d88ed6d\") " pod="openshift-network-diagnostics/network-check-target-x5dk8"
Sep 19 05:40:46.109578 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:40:46.109476    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 19 05:40:46.109578 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:40:46.109494    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 19 05:40:46.109578 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:40:46.109507    2583 projected.go:196] Error preparing data for projected volume kube-api-access-khkc6 for pod openshift-network-diagnostics/network-check-target-x5dk8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 19 05:40:46.109578 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:40:46.109573    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/379278e6-0c96-4ff2-ba60-71ab1d88ed6d-kube-api-access-khkc6 podName:379278e6-0c96-4ff2-ba60-71ab1d88ed6d nodeName:}" failed. No retries permitted until 2026-09-19 05:40:48.109555298 +0000 UTC m=+6.768009841 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-khkc6" (UniqueName: "kubernetes.io/projected/379278e6-0c96-4ff2-ba60-71ab1d88ed6d-kube-api-access-khkc6") pod "network-check-target-x5dk8" (UID: "379278e6-0c96-4ff2-ba60-71ab1d88ed6d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 19 05:40:46.492391 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:46.492313    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x5dk8"
Sep 19 05:40:46.492548 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:40:46.492437    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x5dk8" podUID="379278e6-0c96-4ff2-ba60-71ab1d88ed6d"
Sep 19 05:40:46.492826 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:46.492809    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nrtjk"
Sep 19 05:40:46.492926 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:40:46.492909    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nrtjk" podUID="b291ff5e-1667-4066-98d5-21495cd76311"
Sep 19 05:40:46.506441 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:46.506406    2583 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 19 05:40:46.584753 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:46.584712    2583 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-18 05:35:44 +0000 UTC" deadline="2028-04-28 19:59:07.17633051 +0000 UTC"
Sep 19 05:40:46.584753 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:46.584751    2583 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14102h18m20.591584638s"
Sep 19 05:40:47.107799 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:47.107768    2583 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 19 05:40:47.493647 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:47.492528    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fpnw2"
Sep 19 05:40:47.493813 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:40:47.493641    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fpnw2" podUID="4bd3176a-beea-424d-b4c2-dfc6f27298d8"
Sep 19 05:40:48.024346 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:48.024304    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b291ff5e-1667-4066-98d5-21495cd76311-metrics-certs\") pod \"network-metrics-daemon-nrtjk\" (UID: \"b291ff5e-1667-4066-98d5-21495cd76311\") " pod="openshift-multus/network-metrics-daemon-nrtjk"
Sep 19 05:40:48.024787 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:48.024373    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4bd3176a-beea-424d-b4c2-dfc6f27298d8-original-pull-secret\") pod \"global-pull-secret-syncer-fpnw2\" (UID: \"4bd3176a-beea-424d-b4c2-dfc6f27298d8\") " pod="kube-system/global-pull-secret-syncer-fpnw2"
Sep 19 05:40:48.024787 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:40:48.024535    2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 19 05:40:48.024787 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:40:48.024597    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4bd3176a-beea-424d-b4c2-dfc6f27298d8-original-pull-secret podName:4bd3176a-beea-424d-b4c2-dfc6f27298d8 nodeName:}" failed. No retries permitted until 2026-09-19 05:40:52.024579719 +0000 UTC m=+10.683034257 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4bd3176a-beea-424d-b4c2-dfc6f27298d8-original-pull-secret") pod "global-pull-secret-syncer-fpnw2" (UID: "4bd3176a-beea-424d-b4c2-dfc6f27298d8") : object "kube-system"/"original-pull-secret" not registered
Sep 19 05:40:48.025075 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:40:48.024998    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 05:40:48.025075 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:40:48.025048    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b291ff5e-1667-4066-98d5-21495cd76311-metrics-certs podName:b291ff5e-1667-4066-98d5-21495cd76311 nodeName:}" failed. No retries permitted until 2026-09-19 05:40:52.025032125 +0000 UTC m=+10.683486660 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b291ff5e-1667-4066-98d5-21495cd76311-metrics-certs") pod "network-metrics-daemon-nrtjk" (UID: "b291ff5e-1667-4066-98d5-21495cd76311") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 05:40:48.125037 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:48.124995    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-khkc6\" (UniqueName: \"kubernetes.io/projected/379278e6-0c96-4ff2-ba60-71ab1d88ed6d-kube-api-access-khkc6\") pod \"network-check-target-x5dk8\" (UID: \"379278e6-0c96-4ff2-ba60-71ab1d88ed6d\") " pod="openshift-network-diagnostics/network-check-target-x5dk8"
Sep 19 05:40:48.125263 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:40:48.125186    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 19 05:40:48.125263 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:40:48.125218    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 19 05:40:48.125263 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:40:48.125230    2583 projected.go:196] Error preparing data for projected volume kube-api-access-khkc6 for pod openshift-network-diagnostics/network-check-target-x5dk8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 19 05:40:48.125435 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:40:48.125296    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/379278e6-0c96-4ff2-ba60-71ab1d88ed6d-kube-api-access-khkc6 podName:379278e6-0c96-4ff2-ba60-71ab1d88ed6d nodeName:}" failed. No retries permitted until 2026-09-19 05:40:52.125277879 +0000 UTC m=+10.783732418 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-khkc6" (UniqueName: "kubernetes.io/projected/379278e6-0c96-4ff2-ba60-71ab1d88ed6d-kube-api-access-khkc6") pod "network-check-target-x5dk8" (UID: "379278e6-0c96-4ff2-ba60-71ab1d88ed6d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 19 05:40:48.492378 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:48.492304    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x5dk8"
Sep 19 05:40:48.492520 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:40:48.492436    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x5dk8" podUID="379278e6-0c96-4ff2-ba60-71ab1d88ed6d"
Sep 19 05:40:48.492520 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:48.492451    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nrtjk"
Sep 19 05:40:48.492634 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:40:48.492573    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nrtjk" podUID="b291ff5e-1667-4066-98d5-21495cd76311"
Sep 19 05:40:49.496450 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:49.496419    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fpnw2"
Sep 19 05:40:49.496902 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:40:49.496542    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fpnw2" podUID="4bd3176a-beea-424d-b4c2-dfc6f27298d8"
Sep 19 05:40:50.492359 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:50.492325    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nrtjk"
Sep 19 05:40:50.492556 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:40:50.492468    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nrtjk" podUID="b291ff5e-1667-4066-98d5-21495cd76311"
Sep 19 05:40:50.492637 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:50.492561    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x5dk8"
Sep 19 05:40:50.492691 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:40:50.492655    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x5dk8" podUID="379278e6-0c96-4ff2-ba60-71ab1d88ed6d"
Sep 19 05:40:51.492869 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:51.492830    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fpnw2"
Sep 19 05:40:51.493294 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:40:51.492972    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fpnw2" podUID="4bd3176a-beea-424d-b4c2-dfc6f27298d8"
Sep 19 05:40:52.057859 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:52.057558    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b291ff5e-1667-4066-98d5-21495cd76311-metrics-certs\") pod \"network-metrics-daemon-nrtjk\" (UID: \"b291ff5e-1667-4066-98d5-21495cd76311\") " pod="openshift-multus/network-metrics-daemon-nrtjk"
Sep 19 05:40:52.057859 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:52.057715    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4bd3176a-beea-424d-b4c2-dfc6f27298d8-original-pull-secret\") pod \"global-pull-secret-syncer-fpnw2\" (UID: \"4bd3176a-beea-424d-b4c2-dfc6f27298d8\") " pod="kube-system/global-pull-secret-syncer-fpnw2"
Sep 19 05:40:52.057859 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:40:52.057737    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 05:40:52.057859 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:40:52.057808    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b291ff5e-1667-4066-98d5-21495cd76311-metrics-certs podName:b291ff5e-1667-4066-98d5-21495cd76311 nodeName:}" failed. No retries permitted until 2026-09-19 05:41:00.057787811 +0000 UTC m=+18.716242350 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b291ff5e-1667-4066-98d5-21495cd76311-metrics-certs") pod "network-metrics-daemon-nrtjk" (UID: "b291ff5e-1667-4066-98d5-21495cd76311") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 05:40:52.057859 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:40:52.057842    2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 19 05:40:52.058275 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:40:52.057891    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4bd3176a-beea-424d-b4c2-dfc6f27298d8-original-pull-secret podName:4bd3176a-beea-424d-b4c2-dfc6f27298d8 nodeName:}" failed. No retries permitted until 2026-09-19 05:41:00.057876406 +0000 UTC m=+18.716330947 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4bd3176a-beea-424d-b4c2-dfc6f27298d8-original-pull-secret") pod "global-pull-secret-syncer-fpnw2" (UID: "4bd3176a-beea-424d-b4c2-dfc6f27298d8") : object "kube-system"/"original-pull-secret" not registered
Sep 19 05:40:52.158561 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:52.158521    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-khkc6\" (UniqueName: \"kubernetes.io/projected/379278e6-0c96-4ff2-ba60-71ab1d88ed6d-kube-api-access-khkc6\") pod \"network-check-target-x5dk8\" (UID: \"379278e6-0c96-4ff2-ba60-71ab1d88ed6d\") " pod="openshift-network-diagnostics/network-check-target-x5dk8"
Sep 19 05:40:52.158720 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:40:52.158681    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 19 05:40:52.158720 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:40:52.158708    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 19 05:40:52.158835 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:40:52.158723    2583 projected.go:196] Error preparing data for projected volume kube-api-access-khkc6 for pod openshift-network-diagnostics/network-check-target-x5dk8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 19 05:40:52.158835 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:40:52.158786    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/379278e6-0c96-4ff2-ba60-71ab1d88ed6d-kube-api-access-khkc6 podName:379278e6-0c96-4ff2-ba60-71ab1d88ed6d nodeName:}" failed. No retries permitted until 2026-09-19 05:41:00.15876934 +0000 UTC m=+18.817223875 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-khkc6" (UniqueName: "kubernetes.io/projected/379278e6-0c96-4ff2-ba60-71ab1d88ed6d-kube-api-access-khkc6") pod "network-check-target-x5dk8" (UID: "379278e6-0c96-4ff2-ba60-71ab1d88ed6d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 19 05:40:52.492452 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:52.492358    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x5dk8"
Sep 19 05:40:52.492616 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:40:52.492500    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x5dk8" podUID="379278e6-0c96-4ff2-ba60-71ab1d88ed6d"
Sep 19 05:40:52.492616 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:52.492591    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nrtjk"
Sep 19 05:40:52.492768 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:40:52.492726    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nrtjk" podUID="b291ff5e-1667-4066-98d5-21495cd76311"
Sep 19 05:40:53.493259 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:53.493121    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fpnw2"
Sep 19 05:40:53.493259 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:40:53.493253    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fpnw2" podUID="4bd3176a-beea-424d-b4c2-dfc6f27298d8"
Sep 19 05:40:54.492296 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:54.492268    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x5dk8"
Sep 19 05:40:54.492483 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:54.492268    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nrtjk"
Sep 19 05:40:54.492483 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:40:54.492360    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x5dk8" podUID="379278e6-0c96-4ff2-ba60-71ab1d88ed6d"
Sep 19 05:40:54.492483 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:40:54.492443    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nrtjk" podUID="b291ff5e-1667-4066-98d5-21495cd76311"
Sep 19 05:40:55.492226 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:55.492188    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fpnw2"
Sep 19 05:40:55.492672 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:40:55.492321    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fpnw2" podUID="4bd3176a-beea-424d-b4c2-dfc6f27298d8"
Sep 19 05:40:56.492403 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:56.492363    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nrtjk"
Sep 19 05:40:56.492811 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:40:56.492502    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nrtjk" podUID="b291ff5e-1667-4066-98d5-21495cd76311"
Sep 19 05:40:56.492811 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:56.492551    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x5dk8"
Sep 19 05:40:56.492811 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:40:56.492648    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x5dk8" podUID="379278e6-0c96-4ff2-ba60-71ab1d88ed6d"
Sep 19 05:40:57.493531 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:57.493046    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fpnw2"
Sep 19 05:40:57.493531 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:40:57.493433    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fpnw2" podUID="4bd3176a-beea-424d-b4c2-dfc6f27298d8"
Sep 19 05:40:58.492697 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:58.492663    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x5dk8"
Sep 19 05:40:58.492888 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:58.492674    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nrtjk"
Sep 19 05:40:58.492888 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:40:58.492773    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x5dk8" podUID="379278e6-0c96-4ff2-ba60-71ab1d88ed6d"
Sep 19 05:40:58.492888 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:40:58.492851    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nrtjk" podUID="b291ff5e-1667-4066-98d5-21495cd76311"
Sep 19 05:40:59.492998 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:40:59.492966    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fpnw2"
Sep 19 05:40:59.493512 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:40:59.493080    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fpnw2" podUID="4bd3176a-beea-424d-b4c2-dfc6f27298d8"
Sep 19 05:41:00.114218 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:00.114177    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b291ff5e-1667-4066-98d5-21495cd76311-metrics-certs\") pod \"network-metrics-daemon-nrtjk\" (UID: \"b291ff5e-1667-4066-98d5-21495cd76311\") " pod="openshift-multus/network-metrics-daemon-nrtjk"
Sep 19 05:41:00.114457 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:00.114237    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4bd3176a-beea-424d-b4c2-dfc6f27298d8-original-pull-secret\") pod \"global-pull-secret-syncer-fpnw2\" (UID: \"4bd3176a-beea-424d-b4c2-dfc6f27298d8\") " pod="kube-system/global-pull-secret-syncer-fpnw2"
Sep 19 05:41:00.114457 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:41:00.114330    2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 19 05:41:00.114457 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:41:00.114336    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 05:41:00.114457 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:41:00.114393    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4bd3176a-beea-424d-b4c2-dfc6f27298d8-original-pull-secret podName:4bd3176a-beea-424d-b4c2-dfc6f27298d8 nodeName:}" failed. No retries permitted until 2026-09-19 05:41:16.114376877 +0000 UTC m=+34.772831418 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4bd3176a-beea-424d-b4c2-dfc6f27298d8-original-pull-secret") pod "global-pull-secret-syncer-fpnw2" (UID: "4bd3176a-beea-424d-b4c2-dfc6f27298d8") : object "kube-system"/"original-pull-secret" not registered
Sep 19 05:41:00.114457 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:41:00.114407    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b291ff5e-1667-4066-98d5-21495cd76311-metrics-certs podName:b291ff5e-1667-4066-98d5-21495cd76311 nodeName:}" failed. No retries permitted until 2026-09-19 05:41:16.11440033 +0000 UTC m=+34.772854866 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b291ff5e-1667-4066-98d5-21495cd76311-metrics-certs") pod "network-metrics-daemon-nrtjk" (UID: "b291ff5e-1667-4066-98d5-21495cd76311") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 05:41:00.215454 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:00.215418    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-khkc6\" (UniqueName: \"kubernetes.io/projected/379278e6-0c96-4ff2-ba60-71ab1d88ed6d-kube-api-access-khkc6\") pod \"network-check-target-x5dk8\" (UID: \"379278e6-0c96-4ff2-ba60-71ab1d88ed6d\") " pod="openshift-network-diagnostics/network-check-target-x5dk8"
Sep 19 05:41:00.215621 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:41:00.215588    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 19 05:41:00.215621 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:41:00.215618    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 19 05:41:00.215694 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:41:00.215630    2583 projected.go:196] Error preparing data for projected volume kube-api-access-khkc6 for pod openshift-network-diagnostics/network-check-target-x5dk8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 19 05:41:00.215694 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:41:00.215686    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/379278e6-0c96-4ff2-ba60-71ab1d88ed6d-kube-api-access-khkc6 podName:379278e6-0c96-4ff2-ba60-71ab1d88ed6d nodeName:}" failed. No retries permitted until 2026-09-19 05:41:16.21566934 +0000 UTC m=+34.874123883 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-khkc6" (UniqueName: "kubernetes.io/projected/379278e6-0c96-4ff2-ba60-71ab1d88ed6d-kube-api-access-khkc6") pod "network-check-target-x5dk8" (UID: "379278e6-0c96-4ff2-ba60-71ab1d88ed6d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 19 05:41:00.492643 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:00.492576    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x5dk8"
Sep 19 05:41:00.492887 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:00.492590    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nrtjk"
Sep 19 05:41:00.492887 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:41:00.492705    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x5dk8" podUID="379278e6-0c96-4ff2-ba60-71ab1d88ed6d"
Sep 19 05:41:00.492887 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:41:00.492756    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nrtjk" podUID="b291ff5e-1667-4066-98d5-21495cd76311"
Sep 19 05:41:01.492411 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:01.492381    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fpnw2"
Sep 19 05:41:01.492838 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:41:01.492508    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fpnw2" podUID="4bd3176a-beea-424d-b4c2-dfc6f27298d8"
Sep 19 05:41:02.492064 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:02.492027    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x5dk8"
Sep 19 05:41:02.492253 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:02.492027    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nrtjk"
Sep 19 05:41:02.492253 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:41:02.492124    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x5dk8" podUID="379278e6-0c96-4ff2-ba60-71ab1d88ed6d"
Sep 19 05:41:02.492347 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:41:02.492251    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nrtjk" podUID="b291ff5e-1667-4066-98d5-21495cd76311"
Sep 19 05:41:03.478013 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:41:03.477863    2583 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95fdae8d_ad6b_451b_bc3d_555fd930f300.slice/crio-c3c3a46bbc540024f5e7e576ac0473d1993e3414a285a4d6f75cdcb1b4d53ffd.scope\": RecentStats: unable to find data in memory cache]"
Sep 19 05:41:03.493597 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:03.493568    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fpnw2"
Sep 19 05:41:03.493708 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:41:03.493655    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fpnw2" podUID="4bd3176a-beea-424d-b4c2-dfc6f27298d8"
Sep 19 05:41:03.741117 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:03.741086    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-s2vw4" event={"ID":"32511ff0-ad60-4238-af22-53ccabc2c35b","Type":"ContainerStarted","Data":"701a995d9964980f355cd814b8b8accce9802d32f0e8a3a927a1035ee2e4529f"}
Sep 19 05:41:03.742501 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:03.742474    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-7qkz7" event={"ID":"ef2336d0-3175-4773-92fd-bafecb4f21f8","Type":"ContainerStarted","Data":"f9c0afe81eae9d802223dda561e9a36f759b1fb4500d0bd59be0225a6497ff17"}
Sep 19 05:41:03.743814 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:03.743790    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-bghpv" event={"ID":"e1f08f48-c39a-458a-9f90-87fb34abc301","Type":"ContainerStarted","Data":"e9434c5c5bdb56055d225f3e0e0886ed13145c44956024ce83d8b575a71cdb52"}
Sep 19 05:41:03.746813 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:03.746794    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gj7h8" event={"ID":"8d5d7036-c95e-4cc8-ad8b-c369bfb980b3","Type":"ContainerStarted","Data":"0a62ca7e85ab4a27758dfad3ba8b81d6699aad74da6fe92fa494d97a0758f262"}
Sep 19 05:41:03.746889 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:03.746847    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gj7h8" event={"ID":"8d5d7036-c95e-4cc8-ad8b-c369bfb980b3","Type":"ContainerStarted","Data":"6f50e01ec6b0ff3a4f1eff2a31240761b200f8a7e88f512f52f8ec72b1c87f5c"}
Sep 19 05:41:03.746947 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:03.746928    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gj7h8" event={"ID":"8d5d7036-c95e-4cc8-ad8b-c369bfb980b3","Type":"ContainerStarted","Data":"9eb8a1d0727c2de88172863ca03c26b29653596c5d7700df718b1633b9fd652d"}
Sep 19 05:41:03.746991 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:03.746959    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gj7h8" event={"ID":"8d5d7036-c95e-4cc8-ad8b-c369bfb980b3","Type":"ContainerStarted","Data":"0f7bdb224bdb1dfef961175c3d3fc8ec9704cf6f3c1469534e5a5cc3eefc58fe"}
Sep 19 05:41:03.746991 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:03.746986    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gj7h8" event={"ID":"8d5d7036-c95e-4cc8-ad8b-c369bfb980b3","Type":"ContainerStarted","Data":"ce3a5d4aa30c4ebdad732aba2d95fb9fe31055b794b2c1f40ba9093ad2772318"}
Sep 19 05:41:03.747069 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:03.747019    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gj7h8" event={"ID":"8d5d7036-c95e-4cc8-ad8b-c369bfb980b3","Type":"ContainerStarted","Data":"96b84170910b3ac323bd63833adbe5b966621053a847141bc61d21cbc5560a6d"}
Sep 19 05:41:03.748108 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:03.748090    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-144.ec2.internal" event={"ID":"96c6a7a003ab3151e3ef151f14cc508f","Type":"ContainerStarted","Data":"095b4a203b13f37767fd75806ed95ddad3c342b8c172555b93d1e3b5f95de9b3"}
Sep 19 05:41:03.749454 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:03.749432    2583 generic.go:356] "Generic (PLEG): container finished" podID="9a68be2b93f8cd1218394798eccfafaf" containerID="f60a97fed0d6887a5539f884888cff2f141b8252021008a4832c5f3cda198118" exitCode=0
Sep 19 05:41:03.749550 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:03.749472    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-144.ec2.internal" event={"ID":"9a68be2b93f8cd1218394798eccfafaf","Type":"ContainerDied","Data":"f60a97fed0d6887a5539f884888cff2f141b8252021008a4832c5f3cda198118"}
Sep 19 05:41:03.750871 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:03.750853    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s8j7q" event={"ID":"4081f4f1-e0f7-4cd4-afda-36669fc8d48a","Type":"ContainerStarted","Data":"664c6df9262a2fce61f6e02f96a8670bde9b21a27ebdb180c7e0741bc52fb733"}
Sep 19 05:41:03.752250 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:03.752231    2583 generic.go:356] "Generic (PLEG): container finished" podID="95fdae8d-ad6b-451b-bc3d-555fd930f300" containerID="c3c3a46bbc540024f5e7e576ac0473d1993e3414a285a4d6f75cdcb1b4d53ffd" exitCode=0
Sep 19 05:41:03.752334 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:03.752293    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9rczd" event={"ID":"95fdae8d-ad6b-451b-bc3d-555fd930f300","Type":"ContainerDied","Data":"c3c3a46bbc540024f5e7e576ac0473d1993e3414a285a4d6f75cdcb1b4d53ffd"}
Sep 19 05:41:03.753913 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:03.753875    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-xw29h" event={"ID":"220708f7-de57-4213-ae89-7193308011cf","Type":"ContainerStarted","Data":"66e0bbad3b371f40bf14fe94a77f62409c856b116d42ec69ec8e769c1f5a760b"}
Sep 19 05:41:03.755292 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:03.755271    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-9pctd" event={"ID":"733a5136-d60a-4843-a361-463ac9830695","Type":"ContainerStarted","Data":"782555a54761fd87e70dc25f24a9990da476724991cf9ced7fd209fc1735223b"}
Sep 19 05:41:03.766411 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:03.766209    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-s2vw4" podStartSLOduration=2.524788392 podStartE2EDuration="20.766185032s" totalImagesPullingTime="18.24139664s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 05:40:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:41:03.765338307 +0000 UTC m=+22.423792868" watchObservedRunningTime="2026-09-19 05:41:03.766185032 +0000 UTC m=+22.424639592"
Sep 19 05:41:03.813407 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:03.813370    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-9pctd" podStartSLOduration=2.921874361 podStartE2EDuration="20.813356s" totalImagesPullingTime="17.891481639s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 05:40:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:41:03.796680952 +0000 UTC m=+22.455135511" watchObservedRunningTime="2026-09-19 05:41:03.813356 +0000 UTC m=+22.471810621"
Sep 19 05:41:03.813502 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:03.813431    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-xw29h" podStartSLOduration=6.691717475 podStartE2EDuration="20.813427122s" totalImagesPullingTime="14.121709647s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 05:40:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:41:03.813108013 +0000 UTC m=+22.471562573" watchObservedRunningTime="2026-09-19 05:41:03.813427122 +0000 UTC m=+22.471881679"
Sep 19 05:41:03.869539 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:03.869475    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-bghpv" podStartSLOduration=2.6029206670000002 podStartE2EDuration="20.869453956s" totalImagesPullingTime="18.266533289s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 05:40:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:41:03.868534155 +0000 UTC m=+22.526988713" watchObservedRunningTime="2026-09-19 05:41:03.869453956 +0000 UTC m=+22.527908517"
Sep 19 05:41:03.891595 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:03.891546    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-7qkz7" podStartSLOduration=10.10596865 podStartE2EDuration="20.891527451s" totalImagesPullingTime="10.785558801s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:40:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:41:03.891456309 +0000 UTC m=+22.549910868" watchObservedRunningTime="2026-09-19 05:41:03.891527451 +0000 UTC m=+22.549982008"
Sep 19 05:41:04.301360 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:04.301336    2583 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 19 05:41:04.459990 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:04.459763    2583 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-19T05:41:04.301355301Z","UUID":"418adc49-acdf-4f71-ae65-cc10f0995dcb","Handler":null,"Name":"","Endpoint":""}
Sep 19 05:41:04.461611 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:04.461590    2583 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 19 05:41:04.461754 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:04.461623    2583 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 19 05:41:04.492900 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:04.492870    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x5dk8"
Sep 19 05:41:04.493351 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:04.492910    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nrtjk"
Sep 19 05:41:04.493351 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:41:04.492991    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x5dk8" podUID="379278e6-0c96-4ff2-ba60-71ab1d88ed6d"
Sep 19 05:41:04.493351 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:41:04.493148    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nrtjk" podUID="b291ff5e-1667-4066-98d5-21495cd76311"
Sep 19 05:41:04.560222 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:04.560184    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-7qkz7"
Sep 19 05:41:04.760197 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:04.760049    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-bzgxk" event={"ID":"f39e3781-cfce-4a68-8f06-5c5903423038","Type":"ContainerStarted","Data":"3508da820ab9b54520ea613e43c788fb886448fb265fed81bf56e865f8f3fcb9"}
Sep 19 05:41:04.763075 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:04.763043    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s8j7q" event={"ID":"4081f4f1-e0f7-4cd4-afda-36669fc8d48a","Type":"ContainerStarted","Data":"a535db5af50dccc211952ee3f900357252e40ac5f0534affd4ca5e9df35c8347"}
Sep 19 05:41:04.765574 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:04.765537    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-144.ec2.internal" event={"ID":"9a68be2b93f8cd1218394798eccfafaf","Type":"ContainerStarted","Data":"75618b058dbf92f61a3b124d0880e0d94857b0deadc997d478e79c81c5391da5"}
Sep 19 05:41:04.778135 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:04.778094    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-144.ec2.internal" podStartSLOduration=20.778082803 podStartE2EDuration="20.778082803s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 05:40:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:41:03.929300642 +0000 UTC m=+22.587755201" watchObservedRunningTime="2026-09-19 05:41:04.778082803 +0000 UTC m=+23.436537394"
Sep 19 05:41:04.778341 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:04.778317    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-bzgxk" podStartSLOduration=3.611416464 podStartE2EDuration="21.778308897s" totalImagesPullingTime="18.166892433s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 05:40:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:41:04.777689568 +0000 UTC m=+23.436144126" watchObservedRunningTime="2026-09-19 05:41:04.778308897 +0000 UTC m=+23.436763456"
Sep 19 05:41:04.793825 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:04.793737    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-144.ec2.internal" podStartSLOduration=20.793722662 podStartE2EDuration="20.793722662s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 05:40:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:41:04.793721279 +0000 UTC m=+23.452175840" watchObservedRunningTime="2026-09-19 05:41:04.793722662 +0000 UTC m=+23.452177244"
Sep 19 05:41:05.289825 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:05.289743    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-7qkz7"
Sep 19 05:41:05.290356 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:05.290331    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-7qkz7"
Sep 19 05:41:05.495615 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:05.495591    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fpnw2"
Sep 19 05:41:05.495961 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:41:05.495680    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fpnw2" podUID="4bd3176a-beea-424d-b4c2-dfc6f27298d8"
Sep 19 05:41:05.770761 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:05.770732    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s8j7q" event={"ID":"4081f4f1-e0f7-4cd4-afda-36669fc8d48a","Type":"ContainerStarted","Data":"6a0f09d97de44b77e2f324a3628b1d4459efb5cd673211b05a5796c1c3ea6a23"}
Sep 19 05:41:05.774301 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:05.774278    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gj7h8" event={"ID":"8d5d7036-c95e-4cc8-ad8b-c369bfb980b3","Type":"ContainerStarted","Data":"54573fb3e2124d354ef8db0161c820c2d4ff85f88370f61809596402a7ac3242"}
Sep 19 05:41:05.775151 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:05.775118    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-7qkz7"
Sep 19 05:41:05.788243 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:05.788175    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s8j7q" podStartSLOduration=3.455248872 podStartE2EDuration="22.78815471s" totalImagesPullingTime="19.332905838s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 05:40:43 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:41:05.787557033 +0000 UTC m=+24.446011592" watchObservedRunningTime="2026-09-19 05:41:05.78815471 +0000 UTC m=+24.446609271"
Sep 19 05:41:06.493116 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:06.493084    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x5dk8"
Sep 19 05:41:06.493301 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:41:06.493219    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x5dk8" podUID="379278e6-0c96-4ff2-ba60-71ab1d88ed6d"
Sep 19 05:41:06.493301 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:06.493277    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nrtjk"
Sep 19 05:41:06.493430 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:41:06.493401    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nrtjk" podUID="b291ff5e-1667-4066-98d5-21495cd76311"
Sep 19 05:41:07.495974 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:07.495946    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fpnw2"
Sep 19 05:41:07.496388 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:41:07.496036    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fpnw2" podUID="4bd3176a-beea-424d-b4c2-dfc6f27298d8"
Sep 19 05:41:08.492338 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:08.492304    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x5dk8"
Sep 19 05:41:08.492338 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:08.492304    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nrtjk"
Sep 19 05:41:08.492584 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:41:08.492419    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x5dk8" podUID="379278e6-0c96-4ff2-ba60-71ab1d88ed6d"
Sep 19 05:41:08.492584 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:41:08.492554    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nrtjk" podUID="b291ff5e-1667-4066-98d5-21495cd76311"
Sep 19 05:41:09.495952 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:09.495703    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fpnw2"
Sep 19 05:41:09.496520 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:41:09.496071    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fpnw2" podUID="4bd3176a-beea-424d-b4c2-dfc6f27298d8"
Sep 19 05:41:09.786202 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:09.786155    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gj7h8" event={"ID":"8d5d7036-c95e-4cc8-ad8b-c369bfb980b3","Type":"ContainerStarted","Data":"7f276b87256c6f866365cac1e4e7c2c50ff20cfdbeabf5c595f5ed4afa17c70f"}
Sep 19 05:41:09.786464 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:09.786438    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-gj7h8"
Sep 19 05:41:09.788017 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:09.787988    2583 generic.go:356] "Generic (PLEG): container finished" podID="95fdae8d-ad6b-451b-bc3d-555fd930f300" containerID="6a1150b3ed1e06bf4c5e0902a8be88cd0dce842d3b2d860fe9393cabcfd4d0f4" exitCode=0
Sep 19 05:41:09.788151 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:09.788023    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9rczd" event={"ID":"95fdae8d-ad6b-451b-bc3d-555fd930f300","Type":"ContainerDied","Data":"6a1150b3ed1e06bf4c5e0902a8be88cd0dce842d3b2d860fe9393cabcfd4d0f4"}
Sep 19 05:41:09.802141 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:09.802066    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-gj7h8"
Sep 19 05:41:09.819534 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:09.819488    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-gj7h8" podStartSLOduration=8.481472573 podStartE2EDuration="26.819475984s" totalImagesPullingTime="18.338003411s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 05:40:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:41:09.817717796 +0000 UTC m=+28.476172353" watchObservedRunningTime="2026-09-19 05:41:09.819475984 +0000 UTC m=+28.477930542"
Sep 19 05:41:10.492765 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:10.492731    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x5dk8"
Sep 19 05:41:10.492907 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:41:10.492832    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x5dk8" podUID="379278e6-0c96-4ff2-ba60-71ab1d88ed6d"
Sep 19 05:41:10.492907 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:10.492890    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nrtjk"
Sep 19 05:41:10.493034 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:41:10.493014    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nrtjk" podUID="b291ff5e-1667-4066-98d5-21495cd76311"
Sep 19 05:41:10.791993 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:10.791765    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-gj7h8"
Sep 19 05:41:10.792366 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:10.792041    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-gj7h8"
Sep 19 05:41:10.811728 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:10.811703    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-gj7h8"
Sep 19 05:41:11.046548 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:11.046516    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-fpnw2"]
Sep 19 05:41:11.046742 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:11.046632    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fpnw2"
Sep 19 05:41:11.046742 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:41:11.046731    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fpnw2" podUID="4bd3176a-beea-424d-b4c2-dfc6f27298d8"
Sep 19 05:41:11.049659 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:11.049636    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-nrtjk"]
Sep 19 05:41:11.049823 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:11.049722    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nrtjk"
Sep 19 05:41:11.049823 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:41:11.049802    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nrtjk" podUID="b291ff5e-1667-4066-98d5-21495cd76311"
Sep 19 05:41:11.052765 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:11.052743    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-x5dk8"]
Sep 19 05:41:11.052866 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:11.052816    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x5dk8"
Sep 19 05:41:11.052924 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:41:11.052895    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x5dk8" podUID="379278e6-0c96-4ff2-ba60-71ab1d88ed6d"
Sep 19 05:41:11.795424 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:11.795388    2583 generic.go:356] "Generic (PLEG): container finished" podID="95fdae8d-ad6b-451b-bc3d-555fd930f300" containerID="23c45ad9f28d01968e2e7c932ae8c17b2463375f455a333bb0ed51041418224a" exitCode=0
Sep 19 05:41:11.796035 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:11.795474    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9rczd" event={"ID":"95fdae8d-ad6b-451b-bc3d-555fd930f300","Type":"ContainerDied","Data":"23c45ad9f28d01968e2e7c932ae8c17b2463375f455a333bb0ed51041418224a"}
Sep 19 05:41:12.492847 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:12.492812    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nrtjk"
Sep 19 05:41:12.492847 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:12.492843    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x5dk8"
Sep 19 05:41:12.493046 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:12.492846    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fpnw2"
Sep 19 05:41:12.493046 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:41:12.492931    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nrtjk" podUID="b291ff5e-1667-4066-98d5-21495cd76311"
Sep 19 05:41:12.493046 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:41:12.493004    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x5dk8" podUID="379278e6-0c96-4ff2-ba60-71ab1d88ed6d"
Sep 19 05:41:12.493190 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:41:12.493095    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fpnw2" podUID="4bd3176a-beea-424d-b4c2-dfc6f27298d8"
Sep 19 05:41:12.799956 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:12.799928    2583 generic.go:356] "Generic (PLEG): container finished" podID="95fdae8d-ad6b-451b-bc3d-555fd930f300" containerID="69f63570d4a6ebcb0838dd82eb2e52e53bd8a9b4534cb5790b7f798ade0897fc" exitCode=0
Sep 19 05:41:12.800321 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:12.800013    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9rczd" event={"ID":"95fdae8d-ad6b-451b-bc3d-555fd930f300","Type":"ContainerDied","Data":"69f63570d4a6ebcb0838dd82eb2e52e53bd8a9b4534cb5790b7f798ade0897fc"}
Sep 19 05:41:14.492225 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:14.492181    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fpnw2"
Sep 19 05:41:14.492682 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:41:14.492435    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fpnw2" podUID="4bd3176a-beea-424d-b4c2-dfc6f27298d8"
Sep 19 05:41:14.492682 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:14.492527    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nrtjk"
Sep 19 05:41:14.492682 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:41:14.492653    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nrtjk" podUID="b291ff5e-1667-4066-98d5-21495cd76311"
Sep 19 05:41:14.492857 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:14.492708    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x5dk8"
Sep 19 05:41:14.492857 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:41:14.492783    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x5dk8" podUID="379278e6-0c96-4ff2-ba60-71ab1d88ed6d"
Sep 19 05:41:16.132204 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:16.132166    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4bd3176a-beea-424d-b4c2-dfc6f27298d8-original-pull-secret\") pod \"global-pull-secret-syncer-fpnw2\" (UID: \"4bd3176a-beea-424d-b4c2-dfc6f27298d8\") " pod="kube-system/global-pull-secret-syncer-fpnw2"
Sep 19 05:41:16.132617 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:16.132256    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b291ff5e-1667-4066-98d5-21495cd76311-metrics-certs\") pod \"network-metrics-daemon-nrtjk\" (UID: \"b291ff5e-1667-4066-98d5-21495cd76311\") " pod="openshift-multus/network-metrics-daemon-nrtjk"
Sep 19 05:41:16.132617 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:41:16.132310    2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 19 05:41:16.132617 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:41:16.132333    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 05:41:16.132617 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:41:16.132374    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b291ff5e-1667-4066-98d5-21495cd76311-metrics-certs podName:b291ff5e-1667-4066-98d5-21495cd76311 nodeName:}" failed. No retries permitted until 2026-09-19 05:41:48.132355365 +0000 UTC m=+66.790809901 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b291ff5e-1667-4066-98d5-21495cd76311-metrics-certs") pod "network-metrics-daemon-nrtjk" (UID: "b291ff5e-1667-4066-98d5-21495cd76311") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 05:41:16.132617 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:41:16.132392    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4bd3176a-beea-424d-b4c2-dfc6f27298d8-original-pull-secret podName:4bd3176a-beea-424d-b4c2-dfc6f27298d8 nodeName:}" failed. No retries permitted until 2026-09-19 05:41:48.132382539 +0000 UTC m=+66.790837085 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4bd3176a-beea-424d-b4c2-dfc6f27298d8-original-pull-secret") pod "global-pull-secret-syncer-fpnw2" (UID: "4bd3176a-beea-424d-b4c2-dfc6f27298d8") : object "kube-system"/"original-pull-secret" not registered
Sep 19 05:41:16.233480 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:16.233388    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-khkc6\" (UniqueName: \"kubernetes.io/projected/379278e6-0c96-4ff2-ba60-71ab1d88ed6d-kube-api-access-khkc6\") pod \"network-check-target-x5dk8\" (UID: \"379278e6-0c96-4ff2-ba60-71ab1d88ed6d\") " pod="openshift-network-diagnostics/network-check-target-x5dk8"
Sep 19 05:41:16.233642 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:41:16.233549    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 19 05:41:16.233642 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:41:16.233568    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 19 05:41:16.233642 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:41:16.233577    2583 projected.go:196] Error preparing data for projected volume kube-api-access-khkc6 for pod openshift-network-diagnostics/network-check-target-x5dk8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 19 05:41:16.233642 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:41:16.233629    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/379278e6-0c96-4ff2-ba60-71ab1d88ed6d-kube-api-access-khkc6 podName:379278e6-0c96-4ff2-ba60-71ab1d88ed6d nodeName:}" failed. No retries permitted until 2026-09-19 05:41:48.233612767 +0000 UTC m=+66.892067327 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-khkc6" (UniqueName: "kubernetes.io/projected/379278e6-0c96-4ff2-ba60-71ab1d88ed6d-kube-api-access-khkc6") pod "network-check-target-x5dk8" (UID: "379278e6-0c96-4ff2-ba60-71ab1d88ed6d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 19 05:41:16.492368 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:16.492286    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fpnw2"
Sep 19 05:41:16.492368 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:16.492322    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nrtjk"
Sep 19 05:41:16.492368 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:16.492286    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x5dk8"
Sep 19 05:41:16.492634 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:41:16.492430    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fpnw2" podUID="4bd3176a-beea-424d-b4c2-dfc6f27298d8"
Sep 19 05:41:16.492634 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:41:16.492560    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nrtjk" podUID="b291ff5e-1667-4066-98d5-21495cd76311"
Sep 19 05:41:16.492731 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:41:16.492657    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x5dk8" podUID="379278e6-0c96-4ff2-ba60-71ab1d88ed6d"
Sep 19 05:41:16.837311 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:16.837282    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-144.ec2.internal" event="NodeReady"
Sep 19 05:41:16.837511 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:16.837402    2583 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 19 05:41:16.881187 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:16.881148    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-55d7655b79-9sb7m"]
Sep 19 05:41:16.911477 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:16.911451    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-656cq"]
Sep 19 05:41:16.911652 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:16.911574    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-55d7655b79-9sb7m"
Sep 19 05:41:16.918346 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:16.918158    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 19 05:41:16.918520 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:16.918434    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 19 05:41:16.919351 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:16.919163    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-sp8wq\""
Sep 19 05:41:16.919351 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:16.919270    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 19 05:41:16.925764 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:16.925739    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 19 05:41:16.933060 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:16.933034    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-55d7655b79-9sb7m"]
Sep 19 05:41:16.933192 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:16.933074    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-znm4v"]
Sep 19 05:41:16.933258 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:16.933184    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-656cq"
Sep 19 05:41:16.935864 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:16.935844    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 19 05:41:16.936015 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:16.935923    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 19 05:41:16.936015 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:16.935979    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-mbwrd\""
Sep 19 05:41:16.961162 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:16.961139    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-656cq"]
Sep 19 05:41:16.961275 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:16.961190    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-znm4v"]
Sep 19 05:41:16.961275 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:16.961255    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-znm4v"
Sep 19 05:41:16.963722 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:16.963701    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 19 05:41:16.963861 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:16.963804    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-q45g7\""
Sep 19 05:41:16.964051 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:16.964028    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 19 05:41:16.964373 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:16.964354    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 19 05:41:17.037920 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:17.037891    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7d39ddd1-8d0b-4dee-8c11-a67822704f32-image-registry-private-configuration\") pod \"image-registry-55d7655b79-9sb7m\" (UID: \"7d39ddd1-8d0b-4dee-8c11-a67822704f32\") " pod="openshift-image-registry/image-registry-55d7655b79-9sb7m"
Sep 19 05:41:17.037920 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:17.037922    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7d39ddd1-8d0b-4dee-8c11-a67822704f32-registry-certificates\") pod \"image-registry-55d7655b79-9sb7m\" (UID: \"7d39ddd1-8d0b-4dee-8c11-a67822704f32\") " pod="openshift-image-registry/image-registry-55d7655b79-9sb7m"
Sep 19 05:41:17.038115 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:17.037943    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7d39ddd1-8d0b-4dee-8c11-a67822704f32-trusted-ca\") pod \"image-registry-55d7655b79-9sb7m\" (UID: \"7d39ddd1-8d0b-4dee-8c11-a67822704f32\") " pod="openshift-image-registry/image-registry-55d7655b79-9sb7m"
Sep 19 05:41:17.038115 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:17.037970    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8f5dj\" (UniqueName: \"kubernetes.io/projected/7d39ddd1-8d0b-4dee-8c11-a67822704f32-kube-api-access-8f5dj\") pod \"image-registry-55d7655b79-9sb7m\" (UID: \"7d39ddd1-8d0b-4dee-8c11-a67822704f32\") " pod="openshift-image-registry/image-registry-55d7655b79-9sb7m"
Sep 19 05:41:17.038115 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:17.037998    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7d39ddd1-8d0b-4dee-8c11-a67822704f32-installation-pull-secrets\") pod \"image-registry-55d7655b79-9sb7m\" (UID: \"7d39ddd1-8d0b-4dee-8c11-a67822704f32\") " pod="openshift-image-registry/image-registry-55d7655b79-9sb7m"
Sep 19 05:41:17.038115 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:17.038016    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n758r\" (UniqueName: \"kubernetes.io/projected/6a315987-3770-4424-8864-36e28a3fc43b-kube-api-access-n758r\") pod \"dns-default-656cq\" (UID: \"6a315987-3770-4424-8864-36e28a3fc43b\") " pod="openshift-dns/dns-default-656cq"
Sep 19 05:41:17.038115 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:17.038037    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7d39ddd1-8d0b-4dee-8c11-a67822704f32-bound-sa-token\") pod \"image-registry-55d7655b79-9sb7m\" (UID: \"7d39ddd1-8d0b-4dee-8c11-a67822704f32\") " pod="openshift-image-registry/image-registry-55d7655b79-9sb7m"
Sep 19 05:41:17.038115 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:17.038093    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7d39ddd1-8d0b-4dee-8c11-a67822704f32-registry-tls\") pod \"image-registry-55d7655b79-9sb7m\" (UID: \"7d39ddd1-8d0b-4dee-8c11-a67822704f32\") " pod="openshift-image-registry/image-registry-55d7655b79-9sb7m"
Sep 19 05:41:17.038333 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:17.038116    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7d39ddd1-8d0b-4dee-8c11-a67822704f32-ca-trust-extracted\") pod \"image-registry-55d7655b79-9sb7m\" (UID: \"7d39ddd1-8d0b-4dee-8c11-a67822704f32\") " pod="openshift-image-registry/image-registry-55d7655b79-9sb7m"
Sep 19 05:41:17.038333 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:17.038165    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6a315987-3770-4424-8864-36e28a3fc43b-tmp-dir\") pod \"dns-default-656cq\" (UID: \"6a315987-3770-4424-8864-36e28a3fc43b\") " pod="openshift-dns/dns-default-656cq"
Sep 19 05:41:17.038333 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:17.038198    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b839569f-a0b2-4046-aee5-5e4ba7b0ef9f-cert\") pod \"ingress-canary-znm4v\" (UID: \"b839569f-a0b2-4046-aee5-5e4ba7b0ef9f\") " pod="openshift-ingress-canary/ingress-canary-znm4v"
Sep 19 05:41:17.038333 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:17.038236    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gc6ps\" (UniqueName: \"kubernetes.io/projected/b839569f-a0b2-4046-aee5-5e4ba7b0ef9f-kube-api-access-gc6ps\") pod \"ingress-canary-znm4v\" (UID: \"b839569f-a0b2-4046-aee5-5e4ba7b0ef9f\") " pod="openshift-ingress-canary/ingress-canary-znm4v"
Sep 19 05:41:17.038333 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:17.038306    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/6a315987-3770-4424-8864-36e28a3fc43b-config-volume\") pod \"dns-default-656cq\" (UID: \"6a315987-3770-4424-8864-36e28a3fc43b\") " pod="openshift-dns/dns-default-656cq"
Sep 19 05:41:17.038333 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:17.038321    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6a315987-3770-4424-8864-36e28a3fc43b-metrics-tls\") pod \"dns-default-656cq\" (UID: \"6a315987-3770-4424-8864-36e28a3fc43b\") " pod="openshift-dns/dns-default-656cq"
Sep 19 05:41:17.095417 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:17.095337    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7qkz7_ef2336d0-3175-4773-92fd-bafecb4f21f8/konnectivity-agent/0.log" limit=1024
Sep 19 05:41:17.139299 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:17.139274    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/6a315987-3770-4424-8864-36e28a3fc43b-config-volume\") pod \"dns-default-656cq\" (UID: \"6a315987-3770-4424-8864-36e28a3fc43b\") " pod="openshift-dns/dns-default-656cq"
Sep 19 05:41:17.139675 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:17.139302    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6a315987-3770-4424-8864-36e28a3fc43b-metrics-tls\") pod \"dns-default-656cq\" (UID: \"6a315987-3770-4424-8864-36e28a3fc43b\") " pod="openshift-dns/dns-default-656cq"
Sep 19 05:41:17.139675 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:17.139342    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7d39ddd1-8d0b-4dee-8c11-a67822704f32-image-registry-private-configuration\") pod \"image-registry-55d7655b79-9sb7m\" (UID: \"7d39ddd1-8d0b-4dee-8c11-a67822704f32\") " pod="openshift-image-registry/image-registry-55d7655b79-9sb7m"
Sep 19 05:41:17.139675 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:17.139357    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7d39ddd1-8d0b-4dee-8c11-a67822704f32-registry-certificates\") pod \"image-registry-55d7655b79-9sb7m\" (UID: \"7d39ddd1-8d0b-4dee-8c11-a67822704f32\") " pod="openshift-image-registry/image-registry-55d7655b79-9sb7m"
Sep 19 05:41:17.139675 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:17.139374    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7d39ddd1-8d0b-4dee-8c11-a67822704f32-trusted-ca\") pod \"image-registry-55d7655b79-9sb7m\" (UID: \"7d39ddd1-8d0b-4dee-8c11-a67822704f32\") " pod="openshift-image-registry/image-registry-55d7655b79-9sb7m"
Sep 19 05:41:17.139675 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:17.139398    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8f5dj\" (UniqueName: \"kubernetes.io/projected/7d39ddd1-8d0b-4dee-8c11-a67822704f32-kube-api-access-8f5dj\") pod \"image-registry-55d7655b79-9sb7m\" (UID: \"7d39ddd1-8d0b-4dee-8c11-a67822704f32\") " pod="openshift-image-registry/image-registry-55d7655b79-9sb7m"
Sep 19 05:41:17.139675 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:17.139418    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7d39ddd1-8d0b-4dee-8c11-a67822704f32-installation-pull-secrets\") pod \"image-registry-55d7655b79-9sb7m\" (UID: \"7d39ddd1-8d0b-4dee-8c11-a67822704f32\") " pod="openshift-image-registry/image-registry-55d7655b79-9sb7m"
Sep 19 05:41:17.139675 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:17.139437    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n758r\" (UniqueName: \"kubernetes.io/projected/6a315987-3770-4424-8864-36e28a3fc43b-kube-api-access-n758r\") pod \"dns-default-656cq\" (UID: \"6a315987-3770-4424-8864-36e28a3fc43b\") " pod="openshift-dns/dns-default-656cq"
Sep 19 05:41:17.139675 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:41:17.139444    2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 19 05:41:17.139675 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:17.139497    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7d39ddd1-8d0b-4dee-8c11-a67822704f32-bound-sa-token\") pod \"image-registry-55d7655b79-9sb7m\" (UID: \"7d39ddd1-8d0b-4dee-8c11-a67822704f32\") " pod="openshift-image-registry/image-registry-55d7655b79-9sb7m"
Sep 19 05:41:17.139675 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:41:17.139510    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6a315987-3770-4424-8864-36e28a3fc43b-metrics-tls podName:6a315987-3770-4424-8864-36e28a3fc43b nodeName:}" failed. No retries permitted until 2026-09-19 05:41:17.63949205 +0000 UTC m=+36.297946598 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6a315987-3770-4424-8864-36e28a3fc43b-metrics-tls") pod "dns-default-656cq" (UID: "6a315987-3770-4424-8864-36e28a3fc43b") : secret "dns-default-metrics-tls" not found
Sep 19 05:41:17.139675 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:17.139552    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7d39ddd1-8d0b-4dee-8c11-a67822704f32-registry-tls\") pod \"image-registry-55d7655b79-9sb7m\" (UID: \"7d39ddd1-8d0b-4dee-8c11-a67822704f32\") " pod="openshift-image-registry/image-registry-55d7655b79-9sb7m"
Sep 19 05:41:17.139675 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:17.139579    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7d39ddd1-8d0b-4dee-8c11-a67822704f32-ca-trust-extracted\") pod \"image-registry-55d7655b79-9sb7m\" (UID: \"7d39ddd1-8d0b-4dee-8c11-a67822704f32\") " pod="openshift-image-registry/image-registry-55d7655b79-9sb7m"
Sep 19 05:41:17.139675 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:17.139610    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6a315987-3770-4424-8864-36e28a3fc43b-tmp-dir\") pod \"dns-default-656cq\" (UID: \"6a315987-3770-4424-8864-36e28a3fc43b\") " pod="openshift-dns/dns-default-656cq"
Sep 19 05:41:17.139675 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:17.139633    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b839569f-a0b2-4046-aee5-5e4ba7b0ef9f-cert\") pod \"ingress-canary-znm4v\" (UID: \"b839569f-a0b2-4046-aee5-5e4ba7b0ef9f\") " pod="openshift-ingress-canary/ingress-canary-znm4v"
Sep 19 05:41:17.139675 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:17.139654    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gc6ps\" (UniqueName: \"kubernetes.io/projected/b839569f-a0b2-4046-aee5-5e4ba7b0ef9f-kube-api-access-gc6ps\") pod \"ingress-canary-znm4v\" (UID: \"b839569f-a0b2-4046-aee5-5e4ba7b0ef9f\") " pod="openshift-ingress-canary/ingress-canary-znm4v"
Sep 19 05:41:17.140390 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:17.139923    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/6a315987-3770-4424-8864-36e28a3fc43b-config-volume\") pod \"dns-default-656cq\" (UID: \"6a315987-3770-4424-8864-36e28a3fc43b\") " pod="openshift-dns/dns-default-656cq"
Sep 19 05:41:17.140390 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:17.139938    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7d39ddd1-8d0b-4dee-8c11-a67822704f32-registry-certificates\") pod \"image-registry-55d7655b79-9sb7m\" (UID: \"7d39ddd1-8d0b-4dee-8c11-a67822704f32\") " pod="openshift-image-registry/image-registry-55d7655b79-9sb7m"
Sep 19 05:41:17.140390 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:41:17.140269    2583 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 19 05:41:17.140390 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:41:17.140284    2583 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-55d7655b79-9sb7m: secret "image-registry-tls" not found
Sep 19 05:41:17.140390 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:41:17.140333    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7d39ddd1-8d0b-4dee-8c11-a67822704f32-registry-tls podName:7d39ddd1-8d0b-4dee-8c11-a67822704f32 nodeName:}" failed. No retries permitted until 2026-09-19 05:41:17.640315436 +0000 UTC m=+36.298769983 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/7d39ddd1-8d0b-4dee-8c11-a67822704f32-registry-tls") pod "image-registry-55d7655b79-9sb7m" (UID: "7d39ddd1-8d0b-4dee-8c11-a67822704f32") : secret "image-registry-tls" not found
Sep 19 05:41:17.140390 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:17.140381    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6a315987-3770-4424-8864-36e28a3fc43b-tmp-dir\") pod \"dns-default-656cq\" (UID: \"6a315987-3770-4424-8864-36e28a3fc43b\") " pod="openshift-dns/dns-default-656cq"
Sep 19 05:41:17.140581 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:17.140401    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7d39ddd1-8d0b-4dee-8c11-a67822704f32-ca-trust-extracted\") pod \"image-registry-55d7655b79-9sb7m\" (UID: \"7d39ddd1-8d0b-4dee-8c11-a67822704f32\") " pod="openshift-image-registry/image-registry-55d7655b79-9sb7m"
Sep 19 05:41:17.140581 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:17.140464    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7d39ddd1-8d0b-4dee-8c11-a67822704f32-trusted-ca\") pod \"image-registry-55d7655b79-9sb7m\" (UID: \"7d39ddd1-8d0b-4dee-8c11-a67822704f32\") " pod="openshift-image-registry/image-registry-55d7655b79-9sb7m"
Sep 19 05:41:17.143662 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:17.143506    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7d39ddd1-8d0b-4dee-8c11-a67822704f32-installation-pull-secrets\") pod \"image-registry-55d7655b79-9sb7m\" (UID: \"7d39ddd1-8d0b-4dee-8c11-a67822704f32\") " pod="openshift-image-registry/image-registry-55d7655b79-9sb7m"
Sep 19 05:41:17.143754 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:17.143522    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b839569f-a0b2-4046-aee5-5e4ba7b0ef9f-cert\") pod \"ingress-canary-znm4v\" (UID: \"b839569f-a0b2-4046-aee5-5e4ba7b0ef9f\") " pod="openshift-ingress-canary/ingress-canary-znm4v"
Sep 19 05:41:17.143754 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:17.143636    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7d39ddd1-8d0b-4dee-8c11-a67822704f32-image-registry-private-configuration\") pod \"image-registry-55d7655b79-9sb7m\" (UID: \"7d39ddd1-8d0b-4dee-8c11-a67822704f32\") " pod="openshift-image-registry/image-registry-55d7655b79-9sb7m"
Sep 19 05:41:17.148355 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:17.148333    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n758r\" (UniqueName: \"kubernetes.io/projected/6a315987-3770-4424-8864-36e28a3fc43b-kube-api-access-n758r\") pod \"dns-default-656cq\" (UID: \"6a315987-3770-4424-8864-36e28a3fc43b\") " pod="openshift-dns/dns-default-656cq"
Sep 19 05:41:17.150715 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:17.150689    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7d39ddd1-8d0b-4dee-8c11-a67822704f32-bound-sa-token\") pod \"image-registry-55d7655b79-9sb7m\" (UID: \"7d39ddd1-8d0b-4dee-8c11-a67822704f32\") " pod="openshift-image-registry/image-registry-55d7655b79-9sb7m"
Sep 19 05:41:17.151355 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:17.151339    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8f5dj\" (UniqueName: \"kubernetes.io/projected/7d39ddd1-8d0b-4dee-8c11-a67822704f32-kube-api-access-8f5dj\") pod \"image-registry-55d7655b79-9sb7m\" (UID: \"7d39ddd1-8d0b-4dee-8c11-a67822704f32\") " pod="openshift-image-registry/image-registry-55d7655b79-9sb7m"
Sep 19 05:41:17.151476 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:17.151454    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gc6ps\" (UniqueName: \"kubernetes.io/projected/b839569f-a0b2-4046-aee5-5e4ba7b0ef9f-kube-api-access-gc6ps\") pod \"ingress-canary-znm4v\" (UID: \"b839569f-a0b2-4046-aee5-5e4ba7b0ef9f\") " pod="openshift-ingress-canary/ingress-canary-znm4v"
Sep 19 05:41:17.197170 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:17.197122    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7qkz7_ef2336d0-3175-4773-92fd-bafecb4f21f8/konnectivity-agent/0.log" limit=1024
Sep 19 05:41:17.271417 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:17.271390    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-znm4v"
Sep 19 05:41:17.642826 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:17.642802    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6a315987-3770-4424-8864-36e28a3fc43b-metrics-tls\") pod \"dns-default-656cq\" (UID: \"6a315987-3770-4424-8864-36e28a3fc43b\") " pod="openshift-dns/dns-default-656cq"
Sep 19 05:41:17.642931 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:17.642876    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7d39ddd1-8d0b-4dee-8c11-a67822704f32-registry-tls\") pod \"image-registry-55d7655b79-9sb7m\" (UID: \"7d39ddd1-8d0b-4dee-8c11-a67822704f32\") " pod="openshift-image-registry/image-registry-55d7655b79-9sb7m"
Sep 19 05:41:17.642968 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:41:17.642935    2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 19 05:41:17.642968 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:41:17.642958    2583 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 19 05:41:17.642968 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:41:17.642967    2583 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-55d7655b79-9sb7m: secret "image-registry-tls" not found
Sep 19 05:41:17.643070 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:41:17.642999    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6a315987-3770-4424-8864-36e28a3fc43b-metrics-tls podName:6a315987-3770-4424-8864-36e28a3fc43b nodeName:}" failed. No retries permitted until 2026-09-19 05:41:18.642984 +0000 UTC m=+37.301438536 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6a315987-3770-4424-8864-36e28a3fc43b-metrics-tls") pod "dns-default-656cq" (UID: "6a315987-3770-4424-8864-36e28a3fc43b") : secret "dns-default-metrics-tls" not found
Sep 19 05:41:17.643070 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:41:17.643011    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7d39ddd1-8d0b-4dee-8c11-a67822704f32-registry-tls podName:7d39ddd1-8d0b-4dee-8c11-a67822704f32 nodeName:}" failed. No retries permitted until 2026-09-19 05:41:18.643005959 +0000 UTC m=+37.301460495 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/7d39ddd1-8d0b-4dee-8c11-a67822704f32-registry-tls") pod "image-registry-55d7655b79-9sb7m" (UID: "7d39ddd1-8d0b-4dee-8c11-a67822704f32") : secret "image-registry-tls" not found
Sep 19 05:41:17.771435 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:17.771366    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-znm4v"]
Sep 19 05:41:17.774827 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:41:17.774792    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb839569f_a0b2_4046_aee5_5e4ba7b0ef9f.slice/crio-0d86f7757839df5539418698c69fdbad94c73f83cdcdf08bd3bfcf60d283d5dd WatchSource:0}: Error finding container 0d86f7757839df5539418698c69fdbad94c73f83cdcdf08bd3bfcf60d283d5dd: Status 404 returned error can't find the container with id 0d86f7757839df5539418698c69fdbad94c73f83cdcdf08bd3bfcf60d283d5dd
Sep 19 05:41:17.814512 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:17.814487    2583 generic.go:356] "Generic (PLEG): container finished" podID="95fdae8d-ad6b-451b-bc3d-555fd930f300" containerID="bf2bb580cf1c7ec6f9c968d1c27cb6fad8b08d1924c3129a906ef929b10c2786" exitCode=0
Sep 19 05:41:17.814592 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:17.814540    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9rczd" event={"ID":"95fdae8d-ad6b-451b-bc3d-555fd930f300","Type":"ContainerDied","Data":"bf2bb580cf1c7ec6f9c968d1c27cb6fad8b08d1924c3129a906ef929b10c2786"}
Sep 19 05:41:17.815570 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:17.815550    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-znm4v" event={"ID":"b839569f-a0b2-4046-aee5-5e4ba7b0ef9f","Type":"ContainerStarted","Data":"0d86f7757839df5539418698c69fdbad94c73f83cdcdf08bd3bfcf60d283d5dd"}
Sep 19 05:41:17.871975 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:17.871946    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7qkz7_ef2336d0-3175-4773-92fd-bafecb4f21f8/konnectivity-agent/0.log" limit=1024
Sep 19 05:41:17.970681 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:17.970655    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7qkz7_ef2336d0-3175-4773-92fd-bafecb4f21f8/konnectivity-agent/0.log" limit=1024
Sep 19 05:41:18.492597 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:18.492369    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x5dk8"
Sep 19 05:41:18.493536 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:18.492421    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nrtjk"
Sep 19 05:41:18.493536 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:18.492428    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fpnw2"
Sep 19 05:41:18.495328 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:18.495300    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 19 05:41:18.495459 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:18.495295    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 19 05:41:18.495459 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:18.495411    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 19 05:41:18.497144 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:18.496452    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-b2b8g\""
Sep 19 05:41:18.497144 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:18.496572    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 19 05:41:18.497144 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:18.496720    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-tjt75\""
Sep 19 05:41:18.650694 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:18.650660    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7d39ddd1-8d0b-4dee-8c11-a67822704f32-registry-tls\") pod \"image-registry-55d7655b79-9sb7m\" (UID: \"7d39ddd1-8d0b-4dee-8c11-a67822704f32\") " pod="openshift-image-registry/image-registry-55d7655b79-9sb7m"
Sep 19 05:41:18.650861 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:41:18.650792    2583 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 19 05:41:18.650861 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:41:18.650809    2583 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-55d7655b79-9sb7m: secret "image-registry-tls" not found
Sep 19 05:41:18.650861 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:18.650825    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6a315987-3770-4424-8864-36e28a3fc43b-metrics-tls\") pod \"dns-default-656cq\" (UID: \"6a315987-3770-4424-8864-36e28a3fc43b\") " pod="openshift-dns/dns-default-656cq"
Sep 19 05:41:18.650861 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:41:18.650862    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7d39ddd1-8d0b-4dee-8c11-a67822704f32-registry-tls podName:7d39ddd1-8d0b-4dee-8c11-a67822704f32 nodeName:}" failed. No retries permitted until 2026-09-19 05:41:20.650844752 +0000 UTC m=+39.309299289 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/7d39ddd1-8d0b-4dee-8c11-a67822704f32-registry-tls") pod "image-registry-55d7655b79-9sb7m" (UID: "7d39ddd1-8d0b-4dee-8c11-a67822704f32") : secret "image-registry-tls" not found
Sep 19 05:41:18.651052 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:41:18.650915    2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 19 05:41:18.651052 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:41:18.650979    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6a315987-3770-4424-8864-36e28a3fc43b-metrics-tls podName:6a315987-3770-4424-8864-36e28a3fc43b nodeName:}" failed. No retries permitted until 2026-09-19 05:41:20.650963575 +0000 UTC m=+39.309418118 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6a315987-3770-4424-8864-36e28a3fc43b-metrics-tls") pod "dns-default-656cq" (UID: "6a315987-3770-4424-8864-36e28a3fc43b") : secret "dns-default-metrics-tls" not found
Sep 19 05:41:18.796646 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:18.796607    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7qkz7_ef2336d0-3175-4773-92fd-bafecb4f21f8/konnectivity-agent/0.log" limit=1024
Sep 19 05:41:18.821141 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:18.821101    2583 generic.go:356] "Generic (PLEG): container finished" podID="95fdae8d-ad6b-451b-bc3d-555fd930f300" containerID="8a0c0aad279c76b3902a2e0f0ad25f6a05615f493a429331543b9c7e76fa2113" exitCode=0
Sep 19 05:41:18.821314 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:18.821197    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9rczd" event={"ID":"95fdae8d-ad6b-451b-bc3d-555fd930f300","Type":"ContainerDied","Data":"8a0c0aad279c76b3902a2e0f0ad25f6a05615f493a429331543b9c7e76fa2113"}
Sep 19 05:41:18.863584 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:18.863379    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7qkz7_ef2336d0-3175-4773-92fd-bafecb4f21f8/konnectivity-agent/0.log" limit=1024
Sep 19 05:41:19.828345 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:19.828312    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9rczd" event={"ID":"95fdae8d-ad6b-451b-bc3d-555fd930f300","Type":"ContainerStarted","Data":"fdb82d1185e6c6f7afdd85eea30fe7e39161add11a63c9188ded183502b7fd6b"}
Sep 19 05:41:19.851221 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:19.851176    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-9rczd" podStartSLOduration=6.729290552 podStartE2EDuration="36.851159021s" totalImagesPullingTime="30.121868469s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 05:40:43 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:41:19.849271604 +0000 UTC m=+38.507726161" watchObservedRunningTime="2026-09-19 05:41:19.851159021 +0000 UTC m=+38.509613579"
Sep 19 05:41:19.973473 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:19.973432    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7qkz7_ef2336d0-3175-4773-92fd-bafecb4f21f8/konnectivity-agent/0.log" limit=1024
Sep 19 05:41:20.076029 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:20.075981    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7qkz7_ef2336d0-3175-4773-92fd-bafecb4f21f8/konnectivity-agent/0.log" limit=1024
Sep 19 05:41:20.668664 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:20.668627    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7d39ddd1-8d0b-4dee-8c11-a67822704f32-registry-tls\") pod \"image-registry-55d7655b79-9sb7m\" (UID: \"7d39ddd1-8d0b-4dee-8c11-a67822704f32\") " pod="openshift-image-registry/image-registry-55d7655b79-9sb7m"
Sep 19 05:41:20.668825 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:20.668675    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6a315987-3770-4424-8864-36e28a3fc43b-metrics-tls\") pod \"dns-default-656cq\" (UID: \"6a315987-3770-4424-8864-36e28a3fc43b\") " pod="openshift-dns/dns-default-656cq"
Sep 19 05:41:20.668825 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:41:20.668782    2583 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 19 05:41:20.668825 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:41:20.668797    2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 19 05:41:20.668825 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:41:20.668803    2583 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-55d7655b79-9sb7m: secret "image-registry-tls" not found
Sep 19 05:41:20.668952 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:41:20.668854    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6a315987-3770-4424-8864-36e28a3fc43b-metrics-tls podName:6a315987-3770-4424-8864-36e28a3fc43b nodeName:}" failed. No retries permitted until 2026-09-19 05:41:24.668836995 +0000 UTC m=+43.327291532 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6a315987-3770-4424-8864-36e28a3fc43b-metrics-tls") pod "dns-default-656cq" (UID: "6a315987-3770-4424-8864-36e28a3fc43b") : secret "dns-default-metrics-tls" not found
Sep 19 05:41:20.668952 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:41:20.668873    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7d39ddd1-8d0b-4dee-8c11-a67822704f32-registry-tls podName:7d39ddd1-8d0b-4dee-8c11-a67822704f32 nodeName:}" failed. No retries permitted until 2026-09-19 05:41:24.668864407 +0000 UTC m=+43.327318943 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/7d39ddd1-8d0b-4dee-8c11-a67822704f32-registry-tls") pod "image-registry-55d7655b79-9sb7m" (UID: "7d39ddd1-8d0b-4dee-8c11-a67822704f32") : secret "image-registry-tls" not found
Sep 19 05:41:20.776494 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:20.776466    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7qkz7_ef2336d0-3175-4773-92fd-bafecb4f21f8/konnectivity-agent/0.log" limit=1024
Sep 19 05:41:20.832205 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:20.832178    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-znm4v" event={"ID":"b839569f-a0b2-4046-aee5-5e4ba7b0ef9f","Type":"ContainerStarted","Data":"fa747b9fb1b39e203ecb14d211346b9100f8c01d1439208f68af7d06f3e2a387"}
Sep 19 05:41:20.849065 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:20.848997    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-znm4v" podStartSLOduration=2.444369863 podStartE2EDuration="4.848982135s" totalImagesPullingTime="2.404612272s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:41:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:41:20.848155432 +0000 UTC m=+39.506609987" watchObservedRunningTime="2026-09-19 05:41:20.848982135 +0000 UTC m=+39.507436672"
Sep 19 05:41:21.402572 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:21.402538    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7qkz7_ef2336d0-3175-4773-92fd-bafecb4f21f8/konnectivity-agent/0.log" limit=1024
Sep 19 05:41:21.465618 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:21.465578    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7qkz7_ef2336d0-3175-4773-92fd-bafecb4f21f8/konnectivity-agent/0.log" limit=1024
Sep 19 05:41:24.698953 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:24.698915    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7d39ddd1-8d0b-4dee-8c11-a67822704f32-registry-tls\") pod \"image-registry-55d7655b79-9sb7m\" (UID: \"7d39ddd1-8d0b-4dee-8c11-a67822704f32\") " pod="openshift-image-registry/image-registry-55d7655b79-9sb7m"
Sep 19 05:41:24.698953 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:24.698962    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6a315987-3770-4424-8864-36e28a3fc43b-metrics-tls\") pod \"dns-default-656cq\" (UID: \"6a315987-3770-4424-8864-36e28a3fc43b\") " pod="openshift-dns/dns-default-656cq"
Sep 19 05:41:24.699389 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:41:24.699058    2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 19 05:41:24.699389 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:41:24.699064    2583 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 19 05:41:24.699389 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:41:24.699084    2583 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-55d7655b79-9sb7m: secret "image-registry-tls" not found
Sep 19 05:41:24.699389 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:41:24.699107    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6a315987-3770-4424-8864-36e28a3fc43b-metrics-tls podName:6a315987-3770-4424-8864-36e28a3fc43b nodeName:}" failed. No retries permitted until 2026-09-19 05:41:32.699094551 +0000 UTC m=+51.357549087 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6a315987-3770-4424-8864-36e28a3fc43b-metrics-tls") pod "dns-default-656cq" (UID: "6a315987-3770-4424-8864-36e28a3fc43b") : secret "dns-default-metrics-tls" not found
Sep 19 05:41:24.699389 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:41:24.699146    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7d39ddd1-8d0b-4dee-8c11-a67822704f32-registry-tls podName:7d39ddd1-8d0b-4dee-8c11-a67822704f32 nodeName:}" failed. No retries permitted until 2026-09-19 05:41:32.69911866 +0000 UTC m=+51.357573196 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/7d39ddd1-8d0b-4dee-8c11-a67822704f32-registry-tls") pod "image-registry-55d7655b79-9sb7m" (UID: "7d39ddd1-8d0b-4dee-8c11-a67822704f32") : secret "image-registry-tls" not found
Sep 19 05:41:26.635944 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:26.635907    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7qkz7_ef2336d0-3175-4773-92fd-bafecb4f21f8/konnectivity-agent/0.log" limit=1024
Sep 19 05:41:26.710145 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:26.710104    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7qkz7_ef2336d0-3175-4773-92fd-bafecb4f21f8/konnectivity-agent/0.log" limit=1024
Sep 19 05:41:31.392204 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:31.392172    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7cbc45fddc-gvxmv"]
Sep 19 05:41:31.394113 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:31.394097    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-gvxmv"
Sep 19 05:41:31.396834 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:31.396810    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-mvr7b\""
Sep 19 05:41:31.398443 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:31.398420    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-9nkln"]
Sep 19 05:41:31.400041 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:31.400013    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-9nkln"
Sep 19 05:41:31.403908 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:31.403886    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 19 05:41:31.405056 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:31.405035    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Sep 19 05:41:31.405152 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:31.405085    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-gc5t7\""
Sep 19 05:41:31.405444 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:31.405426    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 19 05:41:31.405502 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:31.405490    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Sep 19 05:41:31.405998 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:31.405981    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-57cdc4b768-t949s"]
Sep 19 05:41:31.408786 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:31.408758    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-6f9c644dd9-k9mzm"]
Sep 19 05:41:31.410593 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:31.410543    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-t949s"
Sep 19 05:41:31.411240 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:31.411218    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-6f9c644dd9-k9mzm"
Sep 19 05:41:31.413014 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:31.412968    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7cbc45fddc-gvxmv"]
Sep 19 05:41:31.413436 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:31.413205    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Sep 19 05:41:31.413436 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:31.413223    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-5hw4t\""
Sep 19 05:41:31.413436 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:31.413226    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Sep 19 05:41:31.413436 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:31.413326    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Sep 19 05:41:31.413436 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:31.413404    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Sep 19 05:41:31.418192 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:31.418117    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Sep 19 05:41:31.418302 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:31.418264    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 19 05:41:31.418302 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:31.418296    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Sep 19 05:41:31.418737 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:31.418714    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-9nkln"]
Sep 19 05:41:31.419226 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:31.419211    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 19 05:41:31.419530 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:31.419508    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-plkwm\""
Sep 19 05:41:31.425283 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:31.425263    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Sep 19 05:41:31.437228 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:31.437203    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-57cdc4b768-t949s"]
Sep 19 05:41:31.437923 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:31.437904    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-6f9c644dd9-k9mzm"]
Sep 19 05:41:31.514350 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:31.514318    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-npzd4"]
Sep 19 05:41:31.516475 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:31.516454    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-npzd4"
Sep 19 05:41:31.519034 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:31.519014    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Sep 19 05:41:31.520368 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:31.520349    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Sep 19 05:41:31.520596 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:31.520575    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-sc65q\""
Sep 19 05:41:31.520675 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:31.520656    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Sep 19 05:41:31.543466 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:31.543430    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/18abc165-1181-4294-965e-b444f51b576c-serving-cert\") pod \"service-ca-operator-57cdc4b768-t949s\" (UID: \"18abc165-1181-4294-965e-b444f51b576c\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-t949s"
Sep 19 05:41:31.543587 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:31.543532    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e3671a49-9883-42c5-b0cf-726774c62cd1-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-847b8bd8d7-9nkln\" (UID: \"e3671a49-9883-42c5-b0cf-726774c62cd1\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-9nkln"
Sep 19 05:41:31.543651 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:31.543579    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rfgfc\" (UniqueName: \"kubernetes.io/projected/6ed851cc-ce44-4d1f-bbd1-1b740629c78d-kube-api-access-rfgfc\") pod \"network-check-source-7cbc45fddc-gvxmv\" (UID: \"6ed851cc-ce44-4d1f-bbd1-1b740629c78d\") " pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-gvxmv"
Sep 19 05:41:31.543651 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:31.543611    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/18abc165-1181-4294-965e-b444f51b576c-config\") pod \"service-ca-operator-57cdc4b768-t949s\" (UID: \"18abc165-1181-4294-965e-b444f51b576c\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-t949s"
Sep 19 05:41:31.543651 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:31.543636    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/965fac7e-ec07-48bb-8d5b-202ddd313625-trusted-ca-bundle\") pod \"insights-operator-6f9c644dd9-k9mzm\" (UID: \"965fac7e-ec07-48bb-8d5b-202ddd313625\") " pod="openshift-insights/insights-operator-6f9c644dd9-k9mzm"
Sep 19 05:41:31.543862 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:31.543733    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/e3671a49-9883-42c5-b0cf-726774c62cd1-telemetry-config\") pod \"cluster-monitoring-operator-847b8bd8d7-9nkln\" (UID: \"e3671a49-9883-42c5-b0cf-726774c62cd1\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-9nkln"
Sep 19 05:41:31.543862 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:31.543758    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jpl8k\" (UniqueName: \"kubernetes.io/projected/18abc165-1181-4294-965e-b444f51b576c-kube-api-access-jpl8k\") pod \"service-ca-operator-57cdc4b768-t949s\" (UID: \"18abc165-1181-4294-965e-b444f51b576c\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-t949s"
Sep 19 05:41:31.543862 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:31.543782    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/965fac7e-ec07-48bb-8d5b-202ddd313625-service-ca-bundle\") pod \"insights-operator-6f9c644dd9-k9mzm\" (UID: \"965fac7e-ec07-48bb-8d5b-202ddd313625\") " pod="openshift-insights/insights-operator-6f9c644dd9-k9mzm"
Sep 19 05:41:31.543862 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:31.543808    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/965fac7e-ec07-48bb-8d5b-202ddd313625-serving-cert\") pod \"insights-operator-6f9c644dd9-k9mzm\" (UID: \"965fac7e-ec07-48bb-8d5b-202ddd313625\") " pod="openshift-insights/insights-operator-6f9c644dd9-k9mzm"
Sep 19 05:41:31.543862 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:31.543849    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w4qqd\" (UniqueName: \"kubernetes.io/projected/e3671a49-9883-42c5-b0cf-726774c62cd1-kube-api-access-w4qqd\") pod \"cluster-monitoring-operator-847b8bd8d7-9nkln\" (UID: \"e3671a49-9883-42c5-b0cf-726774c62cd1\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-9nkln"
Sep 19 05:41:31.544090 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:31.543879    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/965fac7e-ec07-48bb-8d5b-202ddd313625-tmp\") pod \"insights-operator-6f9c644dd9-k9mzm\" (UID: \"965fac7e-ec07-48bb-8d5b-202ddd313625\") " pod="openshift-insights/insights-operator-6f9c644dd9-k9mzm"
Sep 19 05:41:31.544090 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:31.543923    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/965fac7e-ec07-48bb-8d5b-202ddd313625-snapshots\") pod \"insights-operator-6f9c644dd9-k9mzm\" (UID: \"965fac7e-ec07-48bb-8d5b-202ddd313625\") " pod="openshift-insights/insights-operator-6f9c644dd9-k9mzm"
Sep 19 05:41:31.544090 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:31.543951    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ffgvd\" (UniqueName: \"kubernetes.io/projected/965fac7e-ec07-48bb-8d5b-202ddd313625-kube-api-access-ffgvd\") pod \"insights-operator-6f9c644dd9-k9mzm\" (UID: \"965fac7e-ec07-48bb-8d5b-202ddd313625\") " pod="openshift-insights/insights-operator-6f9c644dd9-k9mzm"
Sep 19 05:41:31.544899 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:31.544877    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-npzd4"]
Sep 19 05:41:31.645159 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:31.645058    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rfgfc\" (UniqueName: \"kubernetes.io/projected/6ed851cc-ce44-4d1f-bbd1-1b740629c78d-kube-api-access-rfgfc\") pod \"network-check-source-7cbc45fddc-gvxmv\" (UID: \"6ed851cc-ce44-4d1f-bbd1-1b740629c78d\") " pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-gvxmv"
Sep 19 05:41:31.645159 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:31.645093    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/965fac7e-ec07-48bb-8d5b-202ddd313625-trusted-ca-bundle\") pod \"insights-operator-6f9c644dd9-k9mzm\" (UID: \"965fac7e-ec07-48bb-8d5b-202ddd313625\") " pod="openshift-insights/insights-operator-6f9c644dd9-k9mzm"
Sep 19 05:41:31.645159 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:31.645148    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/965fac7e-ec07-48bb-8d5b-202ddd313625-service-ca-bundle\") pod \"insights-operator-6f9c644dd9-k9mzm\" (UID: \"965fac7e-ec07-48bb-8d5b-202ddd313625\") " pod="openshift-insights/insights-operator-6f9c644dd9-k9mzm"
Sep 19 05:41:31.645424 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:31.645179    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/8b4a7ecf-a718-4241-a172-7334070ef48e-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-npzd4\" (UID: \"8b4a7ecf-a718-4241-a172-7334070ef48e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-npzd4"
Sep 19 05:41:31.645424 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:31.645199    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jpl8k\" (UniqueName: \"kubernetes.io/projected/18abc165-1181-4294-965e-b444f51b576c-kube-api-access-jpl8k\") pod \"service-ca-operator-57cdc4b768-t949s\" (UID: \"18abc165-1181-4294-965e-b444f51b576c\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-t949s"
Sep 19 05:41:31.645424 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:31.645219    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q5g5k\" (UniqueName: \"kubernetes.io/projected/8b4a7ecf-a718-4241-a172-7334070ef48e-kube-api-access-q5g5k\") pod \"cluster-samples-operator-685d7d68b5-npzd4\" (UID: \"8b4a7ecf-a718-4241-a172-7334070ef48e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-npzd4"
Sep 19 05:41:31.645424 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:31.645245    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/965fac7e-ec07-48bb-8d5b-202ddd313625-tmp\") pod \"insights-operator-6f9c644dd9-k9mzm\" (UID: \"965fac7e-ec07-48bb-8d5b-202ddd313625\") " pod="openshift-insights/insights-operator-6f9c644dd9-k9mzm"
Sep 19 05:41:31.645424 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:31.645389    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/18abc165-1181-4294-965e-b444f51b576c-serving-cert\") pod \"service-ca-operator-57cdc4b768-t949s\" (UID: \"18abc165-1181-4294-965e-b444f51b576c\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-t949s"
Sep 19 05:41:31.645655 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:31.645424    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e3671a49-9883-42c5-b0cf-726774c62cd1-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-847b8bd8d7-9nkln\" (UID: \"e3671a49-9883-42c5-b0cf-726774c62cd1\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-9nkln"
Sep 19 05:41:31.645655 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:31.645461    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/18abc165-1181-4294-965e-b444f51b576c-config\") pod \"service-ca-operator-57cdc4b768-t949s\" (UID: \"18abc165-1181-4294-965e-b444f51b576c\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-t949s"
Sep 19 05:41:31.645655 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:31.645524    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/e3671a49-9883-42c5-b0cf-726774c62cd1-telemetry-config\") pod \"cluster-monitoring-operator-847b8bd8d7-9nkln\" (UID: \"e3671a49-9883-42c5-b0cf-726774c62cd1\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-9nkln"
Sep 19 05:41:31.645655 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:41:31.645528    2583 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 19 05:41:31.645655 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:41:31.645632    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3671a49-9883-42c5-b0cf-726774c62cd1-cluster-monitoring-operator-tls podName:e3671a49-9883-42c5-b0cf-726774c62cd1 nodeName:}" failed. No retries permitted until 2026-09-19 05:41:32.145609478 +0000 UTC m=+50.804064021 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e3671a49-9883-42c5-b0cf-726774c62cd1-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-847b8bd8d7-9nkln" (UID: "e3671a49-9883-42c5-b0cf-726774c62cd1") : secret "cluster-monitoring-operator-tls" not found
Sep 19 05:41:31.645834 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:31.645550    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/965fac7e-ec07-48bb-8d5b-202ddd313625-serving-cert\") pod \"insights-operator-6f9c644dd9-k9mzm\" (UID: \"965fac7e-ec07-48bb-8d5b-202ddd313625\") " pod="openshift-insights/insights-operator-6f9c644dd9-k9mzm"
Sep 19 05:41:31.645892 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:31.645874    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w4qqd\" (UniqueName: \"kubernetes.io/projected/e3671a49-9883-42c5-b0cf-726774c62cd1-kube-api-access-w4qqd\") pod \"cluster-monitoring-operator-847b8bd8d7-9nkln\" (UID: \"e3671a49-9883-42c5-b0cf-726774c62cd1\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-9nkln"
Sep 19 05:41:31.645937 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:31.645920    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/965fac7e-ec07-48bb-8d5b-202ddd313625-snapshots\") pod \"insights-operator-6f9c644dd9-k9mzm\" (UID: \"965fac7e-ec07-48bb-8d5b-202ddd313625\") " pod="openshift-insights/insights-operator-6f9c644dd9-k9mzm"
Sep 19 05:41:31.645977 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:31.645950    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ffgvd\" (UniqueName: \"kubernetes.io/projected/965fac7e-ec07-48bb-8d5b-202ddd313625-kube-api-access-ffgvd\") pod \"insights-operator-6f9c644dd9-k9mzm\" (UID: \"965fac7e-ec07-48bb-8d5b-202ddd313625\") " pod="openshift-insights/insights-operator-6f9c644dd9-k9mzm"
Sep 19 05:41:31.646141 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:31.646108    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/18abc165-1181-4294-965e-b444f51b576c-config\") pod \"service-ca-operator-57cdc4b768-t949s\" (UID: \"18abc165-1181-4294-965e-b444f51b576c\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-t949s"
Sep 19 05:41:31.646258 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:31.646193    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/e3671a49-9883-42c5-b0cf-726774c62cd1-telemetry-config\") pod \"cluster-monitoring-operator-847b8bd8d7-9nkln\" (UID: \"e3671a49-9883-42c5-b0cf-726774c62cd1\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-9nkln"
Sep 19 05:41:31.646391 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:31.646371    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/965fac7e-ec07-48bb-8d5b-202ddd313625-tmp\") pod \"insights-operator-6f9c644dd9-k9mzm\" (UID: \"965fac7e-ec07-48bb-8d5b-202ddd313625\") " pod="openshift-insights/insights-operator-6f9c644dd9-k9mzm"
Sep 19 05:41:31.646463 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:31.646406    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/965fac7e-ec07-48bb-8d5b-202ddd313625-service-ca-bundle\") pod \"insights-operator-6f9c644dd9-k9mzm\" (UID: \"965fac7e-ec07-48bb-8d5b-202ddd313625\") " pod="openshift-insights/insights-operator-6f9c644dd9-k9mzm"
Sep 19 05:41:31.646693 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:31.646666    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/965fac7e-ec07-48bb-8d5b-202ddd313625-snapshots\") pod \"insights-operator-6f9c644dd9-k9mzm\" (UID: \"965fac7e-ec07-48bb-8d5b-202ddd313625\") " pod="openshift-insights/insights-operator-6f9c644dd9-k9mzm"
Sep 19 05:41:31.646819 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:31.646800    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/965fac7e-ec07-48bb-8d5b-202ddd313625-trusted-ca-bundle\") pod \"insights-operator-6f9c644dd9-k9mzm\" (UID: \"965fac7e-ec07-48bb-8d5b-202ddd313625\") " pod="openshift-insights/insights-operator-6f9c644dd9-k9mzm"
Sep 19 05:41:31.648955 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:31.648934    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/18abc165-1181-4294-965e-b444f51b576c-serving-cert\") pod \"service-ca-operator-57cdc4b768-t949s\" (UID: \"18abc165-1181-4294-965e-b444f51b576c\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-t949s"
Sep 19 05:41:31.649037 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:31.648937    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/965fac7e-ec07-48bb-8d5b-202ddd313625-serving-cert\") pod \"insights-operator-6f9c644dd9-k9mzm\" (UID: \"965fac7e-ec07-48bb-8d5b-202ddd313625\") " pod="openshift-insights/insights-operator-6f9c644dd9-k9mzm"
Sep 19 05:41:31.659399 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:31.659378    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rfgfc\" (UniqueName: \"kubernetes.io/projected/6ed851cc-ce44-4d1f-bbd1-1b740629c78d-kube-api-access-rfgfc\") pod \"network-check-source-7cbc45fddc-gvxmv\" (UID: \"6ed851cc-ce44-4d1f-bbd1-1b740629c78d\") " pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-gvxmv"
Sep 19 05:41:31.662228 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:31.662205    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ffgvd\" (UniqueName: \"kubernetes.io/projected/965fac7e-ec07-48bb-8d5b-202ddd313625-kube-api-access-ffgvd\") pod \"insights-operator-6f9c644dd9-k9mzm\" (UID: \"965fac7e-ec07-48bb-8d5b-202ddd313625\") " pod="openshift-insights/insights-operator-6f9c644dd9-k9mzm"
Sep 19 05:41:31.662832 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:31.662810    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jpl8k\" (UniqueName: \"kubernetes.io/projected/18abc165-1181-4294-965e-b444f51b576c-kube-api-access-jpl8k\") pod \"service-ca-operator-57cdc4b768-t949s\" (UID: \"18abc165-1181-4294-965e-b444f51b576c\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-t949s"
Sep 19 05:41:31.663113 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:31.663097    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w4qqd\" (UniqueName: \"kubernetes.io/projected/e3671a49-9883-42c5-b0cf-726774c62cd1-kube-api-access-w4qqd\") pod \"cluster-monitoring-operator-847b8bd8d7-9nkln\" (UID: \"e3671a49-9883-42c5-b0cf-726774c62cd1\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-9nkln"
Sep 19 05:41:31.702893 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:31.702852    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-gvxmv"
Sep 19 05:41:31.704167 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:31.704123    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7qkz7_ef2336d0-3175-4773-92fd-bafecb4f21f8/konnectivity-agent/0.log" limit=1024
Sep 19 05:41:31.721935 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:31.721915    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-t949s"
Sep 19 05:41:31.727806 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:31.727786    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-6f9c644dd9-k9mzm"
Sep 19 05:41:31.746804 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:31.746765    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/8b4a7ecf-a718-4241-a172-7334070ef48e-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-npzd4\" (UID: \"8b4a7ecf-a718-4241-a172-7334070ef48e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-npzd4"
Sep 19 05:41:31.747455 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:41:31.747062    2583 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 19 05:41:31.747455 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:41:31.747219    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8b4a7ecf-a718-4241-a172-7334070ef48e-samples-operator-tls podName:8b4a7ecf-a718-4241-a172-7334070ef48e nodeName:}" failed. No retries permitted until 2026-09-19 05:41:32.247197629 +0000 UTC m=+50.905652165 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/8b4a7ecf-a718-4241-a172-7334070ef48e-samples-operator-tls") pod "cluster-samples-operator-685d7d68b5-npzd4" (UID: "8b4a7ecf-a718-4241-a172-7334070ef48e") : secret "samples-operator-tls" not found
Sep 19 05:41:31.747646 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:31.747121    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q5g5k\" (UniqueName: \"kubernetes.io/projected/8b4a7ecf-a718-4241-a172-7334070ef48e-kube-api-access-q5g5k\") pod \"cluster-samples-operator-685d7d68b5-npzd4\" (UID: \"8b4a7ecf-a718-4241-a172-7334070ef48e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-npzd4"
Sep 19 05:41:31.764448 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:31.764405    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q5g5k\" (UniqueName: \"kubernetes.io/projected/8b4a7ecf-a718-4241-a172-7334070ef48e-kube-api-access-q5g5k\") pod \"cluster-samples-operator-685d7d68b5-npzd4\" (UID: \"8b4a7ecf-a718-4241-a172-7334070ef48e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-npzd4"
Sep 19 05:41:31.807858 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:31.807832    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7qkz7_ef2336d0-3175-4773-92fd-bafecb4f21f8/konnectivity-agent/0.log" limit=1024
Sep 19 05:41:31.956853 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:31.956789    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7qkz7_ef2336d0-3175-4773-92fd-bafecb4f21f8/konnectivity-agent/0.log" limit=1024
Sep 19 05:41:32.001305 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:32.001275    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7cbc45fddc-gvxmv"]
Sep 19 05:41:32.008701 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:41:32.008671    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ed851cc_ce44_4d1f_bbd1_1b740629c78d.slice/crio-f1315bd8eb75aa2852b1eedd1ff8500ad57d9ba70e919feee39dbdab0ac2e2ea WatchSource:0}: Error finding container f1315bd8eb75aa2852b1eedd1ff8500ad57d9ba70e919feee39dbdab0ac2e2ea: Status 404 returned error can't find the container with id f1315bd8eb75aa2852b1eedd1ff8500ad57d9ba70e919feee39dbdab0ac2e2ea
Sep 19 05:41:32.018890 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:32.018869    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-57cdc4b768-t949s"]
Sep 19 05:41:32.021563 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:32.021541    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-6f9c644dd9-k9mzm"]
Sep 19 05:41:32.021737 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:41:32.021711    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18abc165_1181_4294_965e_b444f51b576c.slice/crio-fb693acf86ff81141cd074345e7e7229b31a9bd0dca8c4ec04fa1f26f0751fe9 WatchSource:0}: Error finding container fb693acf86ff81141cd074345e7e7229b31a9bd0dca8c4ec04fa1f26f0751fe9: Status 404 returned error can't find the container with id fb693acf86ff81141cd074345e7e7229b31a9bd0dca8c4ec04fa1f26f0751fe9
Sep 19 05:41:32.024124 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:41:32.024106    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod965fac7e_ec07_48bb_8d5b_202ddd313625.slice/crio-f19359ddf58ff4a792fa031d023a4953f461b60f91cbe1af3df177c2f714741d WatchSource:0}: Error finding container f19359ddf58ff4a792fa031d023a4953f461b60f91cbe1af3df177c2f714741d: Status 404 returned error can't find the container with id f19359ddf58ff4a792fa031d023a4953f461b60f91cbe1af3df177c2f714741d
Sep 19 05:41:32.152108 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:32.152080    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e3671a49-9883-42c5-b0cf-726774c62cd1-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-847b8bd8d7-9nkln\" (UID: \"e3671a49-9883-42c5-b0cf-726774c62cd1\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-9nkln"
Sep 19 05:41:32.152257 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:41:32.152236    2583 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 19 05:41:32.152307 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:41:32.152297    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3671a49-9883-42c5-b0cf-726774c62cd1-cluster-monitoring-operator-tls podName:e3671a49-9883-42c5-b0cf-726774c62cd1 nodeName:}" failed. No retries permitted until 2026-09-19 05:41:33.152282778 +0000 UTC m=+51.810737313 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e3671a49-9883-42c5-b0cf-726774c62cd1-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-847b8bd8d7-9nkln" (UID: "e3671a49-9883-42c5-b0cf-726774c62cd1") : secret "cluster-monitoring-operator-tls" not found
Sep 19 05:41:32.253427 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:32.253358    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/8b4a7ecf-a718-4241-a172-7334070ef48e-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-npzd4\" (UID: \"8b4a7ecf-a718-4241-a172-7334070ef48e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-npzd4"
Sep 19 05:41:32.253526 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:41:32.253492    2583 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 19 05:41:32.253565 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:41:32.253549    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8b4a7ecf-a718-4241-a172-7334070ef48e-samples-operator-tls podName:8b4a7ecf-a718-4241-a172-7334070ef48e nodeName:}" failed. No retries permitted until 2026-09-19 05:41:33.253535099 +0000 UTC m=+51.911989635 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/8b4a7ecf-a718-4241-a172-7334070ef48e-samples-operator-tls") pod "cluster-samples-operator-685d7d68b5-npzd4" (UID: "8b4a7ecf-a718-4241-a172-7334070ef48e") : secret "samples-operator-tls" not found
Sep 19 05:41:32.759012 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:32.758293    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7d39ddd1-8d0b-4dee-8c11-a67822704f32-registry-tls\") pod \"image-registry-55d7655b79-9sb7m\" (UID: \"7d39ddd1-8d0b-4dee-8c11-a67822704f32\") " pod="openshift-image-registry/image-registry-55d7655b79-9sb7m"
Sep 19 05:41:32.759012 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:32.758355    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6a315987-3770-4424-8864-36e28a3fc43b-metrics-tls\") pod \"dns-default-656cq\" (UID: \"6a315987-3770-4424-8864-36e28a3fc43b\") " pod="openshift-dns/dns-default-656cq"
Sep 19 05:41:32.759012 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:41:32.758491    2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 19 05:41:32.759012 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:41:32.758552    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6a315987-3770-4424-8864-36e28a3fc43b-metrics-tls podName:6a315987-3770-4424-8864-36e28a3fc43b nodeName:}" failed. No retries permitted until 2026-09-19 05:41:48.758533615 +0000 UTC m=+67.416988172 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6a315987-3770-4424-8864-36e28a3fc43b-metrics-tls") pod "dns-default-656cq" (UID: "6a315987-3770-4424-8864-36e28a3fc43b") : secret "dns-default-metrics-tls" not found
Sep 19 05:41:32.759012 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:41:32.758921    2583 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 19 05:41:32.759012 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:41:32.758937    2583 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-55d7655b79-9sb7m: secret "image-registry-tls" not found
Sep 19 05:41:32.759012 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:41:32.758980    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7d39ddd1-8d0b-4dee-8c11-a67822704f32-registry-tls podName:7d39ddd1-8d0b-4dee-8c11-a67822704f32 nodeName:}" failed. No retries permitted until 2026-09-19 05:41:48.758965773 +0000 UTC m=+67.417420312 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/7d39ddd1-8d0b-4dee-8c11-a67822704f32-registry-tls") pod "image-registry-55d7655b79-9sb7m" (UID: "7d39ddd1-8d0b-4dee-8c11-a67822704f32") : secret "image-registry-tls" not found
Sep 19 05:41:32.860533 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:32.860498    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-6f9c644dd9-k9mzm" event={"ID":"965fac7e-ec07-48bb-8d5b-202ddd313625","Type":"ContainerStarted","Data":"f19359ddf58ff4a792fa031d023a4953f461b60f91cbe1af3df177c2f714741d"}
Sep 19 05:41:32.863093 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:32.863062    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-t949s" event={"ID":"18abc165-1181-4294-965e-b444f51b576c","Type":"ContainerStarted","Data":"fb693acf86ff81141cd074345e7e7229b31a9bd0dca8c4ec04fa1f26f0751fe9"}
Sep 19 05:41:32.864652 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:32.864605    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-gvxmv" event={"ID":"6ed851cc-ce44-4d1f-bbd1-1b740629c78d","Type":"ContainerStarted","Data":"f1315bd8eb75aa2852b1eedd1ff8500ad57d9ba70e919feee39dbdab0ac2e2ea"}
Sep 19 05:41:33.162533 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:33.162101    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e3671a49-9883-42c5-b0cf-726774c62cd1-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-847b8bd8d7-9nkln\" (UID: \"e3671a49-9883-42c5-b0cf-726774c62cd1\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-9nkln"
Sep 19 05:41:33.162533 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:41:33.162398    2583 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 19 05:41:33.162533 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:41:33.162469    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3671a49-9883-42c5-b0cf-726774c62cd1-cluster-monitoring-operator-tls podName:e3671a49-9883-42c5-b0cf-726774c62cd1 nodeName:}" failed. No retries permitted until 2026-09-19 05:41:35.162451206 +0000 UTC m=+53.820905744 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e3671a49-9883-42c5-b0cf-726774c62cd1-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-847b8bd8d7-9nkln" (UID: "e3671a49-9883-42c5-b0cf-726774c62cd1") : secret "cluster-monitoring-operator-tls" not found
Sep 19 05:41:33.263623 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:33.263585    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/8b4a7ecf-a718-4241-a172-7334070ef48e-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-npzd4\" (UID: \"8b4a7ecf-a718-4241-a172-7334070ef48e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-npzd4"
Sep 19 05:41:33.263807 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:41:33.263729    2583 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 19 05:41:33.263807 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:41:33.263791    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8b4a7ecf-a718-4241-a172-7334070ef48e-samples-operator-tls podName:8b4a7ecf-a718-4241-a172-7334070ef48e nodeName:}" failed. No retries permitted until 2026-09-19 05:41:35.263773278 +0000 UTC m=+53.922227827 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/8b4a7ecf-a718-4241-a172-7334070ef48e-samples-operator-tls") pod "cluster-samples-operator-685d7d68b5-npzd4" (UID: "8b4a7ecf-a718-4241-a172-7334070ef48e") : secret "samples-operator-tls" not found
Sep 19 05:41:34.659043 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:34.659009    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7qkz7_ef2336d0-3175-4773-92fd-bafecb4f21f8/konnectivity-agent/0.log" limit=1024
Sep 19 05:41:35.180496 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:35.180456    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e3671a49-9883-42c5-b0cf-726774c62cd1-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-847b8bd8d7-9nkln\" (UID: \"e3671a49-9883-42c5-b0cf-726774c62cd1\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-9nkln"
Sep 19 05:41:35.180688 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:41:35.180630    2583 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 19 05:41:35.180752 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:41:35.180707    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3671a49-9883-42c5-b0cf-726774c62cd1-cluster-monitoring-operator-tls podName:e3671a49-9883-42c5-b0cf-726774c62cd1 nodeName:}" failed. No retries permitted until 2026-09-19 05:41:39.180687452 +0000 UTC m=+57.839142002 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e3671a49-9883-42c5-b0cf-726774c62cd1-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-847b8bd8d7-9nkln" (UID: "e3671a49-9883-42c5-b0cf-726774c62cd1") : secret "cluster-monitoring-operator-tls" not found
Sep 19 05:41:35.281522 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:35.281488    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/8b4a7ecf-a718-4241-a172-7334070ef48e-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-npzd4\" (UID: \"8b4a7ecf-a718-4241-a172-7334070ef48e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-npzd4"
Sep 19 05:41:35.281718 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:41:35.281673    2583 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 19 05:41:35.281772 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:41:35.281753    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8b4a7ecf-a718-4241-a172-7334070ef48e-samples-operator-tls podName:8b4a7ecf-a718-4241-a172-7334070ef48e nodeName:}" failed. No retries permitted until 2026-09-19 05:41:39.281731442 +0000 UTC m=+57.940185984 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/8b4a7ecf-a718-4241-a172-7334070ef48e-samples-operator-tls") pod "cluster-samples-operator-685d7d68b5-npzd4" (UID: "8b4a7ecf-a718-4241-a172-7334070ef48e") : secret "samples-operator-tls" not found
Sep 19 05:41:35.875461 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:35.875419    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-6f9c644dd9-k9mzm" event={"ID":"965fac7e-ec07-48bb-8d5b-202ddd313625","Type":"ContainerStarted","Data":"fb9d7f3d50cbe98f3890594288d3bcded9e737c1bf92ab014b72ffcec2ba2827"}
Sep 19 05:41:35.877350 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:35.877317    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-t949s" event={"ID":"18abc165-1181-4294-965e-b444f51b576c","Type":"ContainerStarted","Data":"53bf208e82fdc97f904d1337b5435d7eec08d85204b7fdfc038c055b188853f6"}
Sep 19 05:41:35.878994 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:35.878968    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-gvxmv" event={"ID":"6ed851cc-ce44-4d1f-bbd1-1b740629c78d","Type":"ContainerStarted","Data":"b140c83335592f5acd532ef3425d021188d547a961def21f6a5ad8dedfc663d8"}
Sep 19 05:41:35.898900 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:35.898846    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-6f9c644dd9-k9mzm" podStartSLOduration=1.469286575 podStartE2EDuration="4.898830498s" totalImagesPullingTime="3.429543923s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:41:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:41:35.897520116 +0000 UTC m=+54.555974675" watchObservedRunningTime="2026-09-19 05:41:35.898830498 +0000 UTC m=+54.557285103"
Sep 19 05:41:35.926174 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:35.926096    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-t949s" podStartSLOduration=1.495121779 podStartE2EDuration="4.926078568s" totalImagesPullingTime="3.430956789s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:41:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:41:35.925262301 +0000 UTC m=+54.583716861" watchObservedRunningTime="2026-09-19 05:41:35.926078568 +0000 UTC m=+54.584533127"
Sep 19 05:41:35.943405 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:35.943347    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-gvxmv" podStartSLOduration=1.485846944 podStartE2EDuration="4.94332775s" totalImagesPullingTime="3.457480806s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:41:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:41:35.943140928 +0000 UTC m=+54.601595480" watchObservedRunningTime="2026-09-19 05:41:35.94332775 +0000 UTC m=+54.601782311"
Sep 19 05:41:36.611361 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:36.611326    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-jgkc7"]
Sep 19 05:41:36.615614 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:36.615590    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-jgkc7"
Sep 19 05:41:36.619394 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:36.619366    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 19 05:41:36.619529 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:36.619455    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-s598g\""
Sep 19 05:41:36.620044 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:36.620026    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 19 05:41:36.632433 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:36.632412    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-jgkc7"]
Sep 19 05:41:36.696532 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:36.696493    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/098cb613-a7e2-46de-a6c6-898a7ccb1a14-crio-socket\") pod \"insights-runtime-extractor-jgkc7\" (UID: \"098cb613-a7e2-46de-a6c6-898a7ccb1a14\") " pod="openshift-insights/insights-runtime-extractor-jgkc7"
Sep 19 05:41:36.696704 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:36.696564    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/098cb613-a7e2-46de-a6c6-898a7ccb1a14-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jgkc7\" (UID: \"098cb613-a7e2-46de-a6c6-898a7ccb1a14\") " pod="openshift-insights/insights-runtime-extractor-jgkc7"
Sep 19 05:41:36.696704 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:36.696594    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/098cb613-a7e2-46de-a6c6-898a7ccb1a14-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jgkc7\" (UID: \"098cb613-a7e2-46de-a6c6-898a7ccb1a14\") " pod="openshift-insights/insights-runtime-extractor-jgkc7"
Sep 19 05:41:36.696704 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:36.696687    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4fsrd\" (UniqueName: \"kubernetes.io/projected/098cb613-a7e2-46de-a6c6-898a7ccb1a14-kube-api-access-4fsrd\") pod \"insights-runtime-extractor-jgkc7\" (UID: \"098cb613-a7e2-46de-a6c6-898a7ccb1a14\") " pod="openshift-insights/insights-runtime-extractor-jgkc7"
Sep 19 05:41:36.696815 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:36.696760    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/098cb613-a7e2-46de-a6c6-898a7ccb1a14-data-volume\") pod \"insights-runtime-extractor-jgkc7\" (UID: \"098cb613-a7e2-46de-a6c6-898a7ccb1a14\") " pod="openshift-insights/insights-runtime-extractor-jgkc7"
Sep 19 05:41:36.797854 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:36.797816    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/098cb613-a7e2-46de-a6c6-898a7ccb1a14-data-volume\") pod \"insights-runtime-extractor-jgkc7\" (UID: \"098cb613-a7e2-46de-a6c6-898a7ccb1a14\") " pod="openshift-insights/insights-runtime-extractor-jgkc7"
Sep 19 05:41:36.797854 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:36.797855    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/098cb613-a7e2-46de-a6c6-898a7ccb1a14-crio-socket\") pod \"insights-runtime-extractor-jgkc7\" (UID: \"098cb613-a7e2-46de-a6c6-898a7ccb1a14\") " pod="openshift-insights/insights-runtime-extractor-jgkc7"
Sep 19 05:41:36.798061 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:36.797887    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/098cb613-a7e2-46de-a6c6-898a7ccb1a14-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jgkc7\" (UID: \"098cb613-a7e2-46de-a6c6-898a7ccb1a14\") " pod="openshift-insights/insights-runtime-extractor-jgkc7"
Sep 19 05:41:36.798061 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:36.797904    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/098cb613-a7e2-46de-a6c6-898a7ccb1a14-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jgkc7\" (UID: \"098cb613-a7e2-46de-a6c6-898a7ccb1a14\") " pod="openshift-insights/insights-runtime-extractor-jgkc7"
Sep 19 05:41:36.798061 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:41:36.797972    2583 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 19 05:41:36.798061 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:36.798041    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/098cb613-a7e2-46de-a6c6-898a7ccb1a14-crio-socket\") pod \"insights-runtime-extractor-jgkc7\" (UID: \"098cb613-a7e2-46de-a6c6-898a7ccb1a14\") " pod="openshift-insights/insights-runtime-extractor-jgkc7"
Sep 19 05:41:36.798268 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:41:36.798053    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/098cb613-a7e2-46de-a6c6-898a7ccb1a14-kube-rbac-proxy-cm podName:098cb613-a7e2-46de-a6c6-898a7ccb1a14 nodeName:}" failed. No retries permitted until 2026-09-19 05:41:37.298032045 +0000 UTC m=+55.956486601 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/098cb613-a7e2-46de-a6c6-898a7ccb1a14-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jgkc7" (UID: "098cb613-a7e2-46de-a6c6-898a7ccb1a14") : configmap "kube-rbac-proxy" not found
Sep 19 05:41:36.798268 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:36.797974    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4fsrd\" (UniqueName: \"kubernetes.io/projected/098cb613-a7e2-46de-a6c6-898a7ccb1a14-kube-api-access-4fsrd\") pod \"insights-runtime-extractor-jgkc7\" (UID: \"098cb613-a7e2-46de-a6c6-898a7ccb1a14\") " pod="openshift-insights/insights-runtime-extractor-jgkc7"
Sep 19 05:41:36.798268 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:41:36.798081    2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 19 05:41:36.798268 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:41:36.798227    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/098cb613-a7e2-46de-a6c6-898a7ccb1a14-insights-runtime-extractor-tls podName:098cb613-a7e2-46de-a6c6-898a7ccb1a14 nodeName:}" failed. No retries permitted until 2026-09-19 05:41:37.298209575 +0000 UTC m=+55.956664116 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/098cb613-a7e2-46de-a6c6-898a7ccb1a14-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jgkc7" (UID: "098cb613-a7e2-46de-a6c6-898a7ccb1a14") : secret "insights-runtime-extractor-tls" not found
Sep 19 05:41:36.798268 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:36.798226    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/098cb613-a7e2-46de-a6c6-898a7ccb1a14-data-volume\") pod \"insights-runtime-extractor-jgkc7\" (UID: \"098cb613-a7e2-46de-a6c6-898a7ccb1a14\") " pod="openshift-insights/insights-runtime-extractor-jgkc7"
Sep 19 05:41:36.811695 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:36.811660    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4fsrd\" (UniqueName: \"kubernetes.io/projected/098cb613-a7e2-46de-a6c6-898a7ccb1a14-kube-api-access-4fsrd\") pod \"insights-runtime-extractor-jgkc7\" (UID: \"098cb613-a7e2-46de-a6c6-898a7ccb1a14\") " pod="openshift-insights/insights-runtime-extractor-jgkc7"
Sep 19 05:41:37.301904 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:37.301855    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/098cb613-a7e2-46de-a6c6-898a7ccb1a14-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jgkc7\" (UID: \"098cb613-a7e2-46de-a6c6-898a7ccb1a14\") " pod="openshift-insights/insights-runtime-extractor-jgkc7"
Sep 19 05:41:37.301904 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:37.301909    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/098cb613-a7e2-46de-a6c6-898a7ccb1a14-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jgkc7\" (UID: \"098cb613-a7e2-46de-a6c6-898a7ccb1a14\") " pod="openshift-insights/insights-runtime-extractor-jgkc7"
Sep 19 05:41:37.302370 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:41:37.301980    2583 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 19 05:41:37.302370 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:41:37.302050    2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 19 05:41:37.302370 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:41:37.302057    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/098cb613-a7e2-46de-a6c6-898a7ccb1a14-kube-rbac-proxy-cm podName:098cb613-a7e2-46de-a6c6-898a7ccb1a14 nodeName:}" failed. No retries permitted until 2026-09-19 05:41:38.302040643 +0000 UTC m=+56.960495179 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/098cb613-a7e2-46de-a6c6-898a7ccb1a14-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jgkc7" (UID: "098cb613-a7e2-46de-a6c6-898a7ccb1a14") : configmap "kube-rbac-proxy" not found
Sep 19 05:41:37.302370 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:41:37.302092    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/098cb613-a7e2-46de-a6c6-898a7ccb1a14-insights-runtime-extractor-tls podName:098cb613-a7e2-46de-a6c6-898a7ccb1a14 nodeName:}" failed. No retries permitted until 2026-09-19 05:41:38.302081209 +0000 UTC m=+56.960535745 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/098cb613-a7e2-46de-a6c6-898a7ccb1a14-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jgkc7" (UID: "098cb613-a7e2-46de-a6c6-898a7ccb1a14") : secret "insights-runtime-extractor-tls" not found
Sep 19 05:41:37.925648 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:37.925621    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-9pctd_733a5136-d60a-4843-a361-463ac9830695/dns-node-resolver/0.log"
Sep 19 05:41:38.309927 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:38.309893    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/098cb613-a7e2-46de-a6c6-898a7ccb1a14-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jgkc7\" (UID: \"098cb613-a7e2-46de-a6c6-898a7ccb1a14\") " pod="openshift-insights/insights-runtime-extractor-jgkc7"
Sep 19 05:41:38.309927 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:38.309931    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/098cb613-a7e2-46de-a6c6-898a7ccb1a14-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jgkc7\" (UID: \"098cb613-a7e2-46de-a6c6-898a7ccb1a14\") " pod="openshift-insights/insights-runtime-extractor-jgkc7"
Sep 19 05:41:38.310338 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:41:38.310017    2583 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 19 05:41:38.310338 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:41:38.310039    2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 19 05:41:38.310338 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:41:38.310086    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/098cb613-a7e2-46de-a6c6-898a7ccb1a14-kube-rbac-proxy-cm podName:098cb613-a7e2-46de-a6c6-898a7ccb1a14 nodeName:}" failed. No retries permitted until 2026-09-19 05:41:40.310071652 +0000 UTC m=+58.968526193 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/098cb613-a7e2-46de-a6c6-898a7ccb1a14-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jgkc7" (UID: "098cb613-a7e2-46de-a6c6-898a7ccb1a14") : configmap "kube-rbac-proxy" not found
Sep 19 05:41:38.310338 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:41:38.310101    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/098cb613-a7e2-46de-a6c6-898a7ccb1a14-insights-runtime-extractor-tls podName:098cb613-a7e2-46de-a6c6-898a7ccb1a14 nodeName:}" failed. No retries permitted until 2026-09-19 05:41:40.310094787 +0000 UTC m=+58.968549323 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/098cb613-a7e2-46de-a6c6-898a7ccb1a14-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jgkc7" (UID: "098cb613-a7e2-46de-a6c6-898a7ccb1a14") : secret "insights-runtime-extractor-tls" not found
Sep 19 05:41:39.216320 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:39.216279    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e3671a49-9883-42c5-b0cf-726774c62cd1-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-847b8bd8d7-9nkln\" (UID: \"e3671a49-9883-42c5-b0cf-726774c62cd1\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-9nkln"
Sep 19 05:41:39.216502 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:41:39.216425    2583 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 19 05:41:39.216502 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:41:39.216488    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3671a49-9883-42c5-b0cf-726774c62cd1-cluster-monitoring-operator-tls podName:e3671a49-9883-42c5-b0cf-726774c62cd1 nodeName:}" failed. No retries permitted until 2026-09-19 05:41:47.216474257 +0000 UTC m=+65.874928793 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e3671a49-9883-42c5-b0cf-726774c62cd1-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-847b8bd8d7-9nkln" (UID: "e3671a49-9883-42c5-b0cf-726774c62cd1") : secret "cluster-monitoring-operator-tls" not found
Sep 19 05:41:39.249279 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:39.249253    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-55f6fc7b86-6bllk"]
Sep 19 05:41:39.281530 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:39.281504    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-55f6fc7b86-6bllk"]
Sep 19 05:41:39.281643 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:39.281589    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-55f6fc7b86-6bllk"
Sep 19 05:41:39.284729 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:39.284711    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Sep 19 05:41:39.284852 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:39.284840    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-hsrkn\""
Sep 19 05:41:39.285318 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:39.285300    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Sep 19 05:41:39.286307 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:39.286290    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Sep 19 05:41:39.286566 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:39.286554    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Sep 19 05:41:39.317560 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:39.317540    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/a8813434-d7a3-4064-a634-9afb552c54c5-signing-cabundle\") pod \"service-ca-55f6fc7b86-6bllk\" (UID: \"a8813434-d7a3-4064-a634-9afb552c54c5\") " pod="openshift-service-ca/service-ca-55f6fc7b86-6bllk"
Sep 19 05:41:39.317856 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:39.317580    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/a8813434-d7a3-4064-a634-9afb552c54c5-signing-key\") pod \"service-ca-55f6fc7b86-6bllk\" (UID: \"a8813434-d7a3-4064-a634-9afb552c54c5\") " pod="openshift-service-ca/service-ca-55f6fc7b86-6bllk"
Sep 19 05:41:39.317856 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:39.317611    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zs8zb\" (UniqueName: \"kubernetes.io/projected/a8813434-d7a3-4064-a634-9afb552c54c5-kube-api-access-zs8zb\") pod \"service-ca-55f6fc7b86-6bllk\" (UID: \"a8813434-d7a3-4064-a634-9afb552c54c5\") " pod="openshift-service-ca/service-ca-55f6fc7b86-6bllk"
Sep 19 05:41:39.317856 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:39.317699    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/8b4a7ecf-a718-4241-a172-7334070ef48e-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-npzd4\" (UID: \"8b4a7ecf-a718-4241-a172-7334070ef48e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-npzd4"
Sep 19 05:41:39.317856 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:41:39.317783    2583 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 19 05:41:39.317856 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:41:39.317835    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8b4a7ecf-a718-4241-a172-7334070ef48e-samples-operator-tls podName:8b4a7ecf-a718-4241-a172-7334070ef48e nodeName:}" failed. No retries permitted until 2026-09-19 05:41:47.317821491 +0000 UTC m=+65.976276027 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/8b4a7ecf-a718-4241-a172-7334070ef48e-samples-operator-tls") pod "cluster-samples-operator-685d7d68b5-npzd4" (UID: "8b4a7ecf-a718-4241-a172-7334070ef48e") : secret "samples-operator-tls" not found
Sep 19 05:41:39.329535 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:39.329513    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-xw29h_220708f7-de57-4213-ae89-7193308011cf/node-ca/0.log"
Sep 19 05:41:39.418852 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:39.418812    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/a8813434-d7a3-4064-a634-9afb552c54c5-signing-cabundle\") pod \"service-ca-55f6fc7b86-6bllk\" (UID: \"a8813434-d7a3-4064-a634-9afb552c54c5\") " pod="openshift-service-ca/service-ca-55f6fc7b86-6bllk"
Sep 19 05:41:39.419032 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:39.418866    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/a8813434-d7a3-4064-a634-9afb552c54c5-signing-key\") pod \"service-ca-55f6fc7b86-6bllk\" (UID: \"a8813434-d7a3-4064-a634-9afb552c54c5\") " pod="openshift-service-ca/service-ca-55f6fc7b86-6bllk"
Sep 19 05:41:39.419032 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:39.418902    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zs8zb\" (UniqueName: \"kubernetes.io/projected/a8813434-d7a3-4064-a634-9afb552c54c5-kube-api-access-zs8zb\") pod \"service-ca-55f6fc7b86-6bllk\" (UID: \"a8813434-d7a3-4064-a634-9afb552c54c5\") " pod="openshift-service-ca/service-ca-55f6fc7b86-6bllk"
Sep 19 05:41:39.419493 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:39.419467    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/a8813434-d7a3-4064-a634-9afb552c54c5-signing-cabundle\") pod \"service-ca-55f6fc7b86-6bllk\" (UID: \"a8813434-d7a3-4064-a634-9afb552c54c5\") " pod="openshift-service-ca/service-ca-55f6fc7b86-6bllk"
Sep 19 05:41:39.421227 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:39.421204    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/a8813434-d7a3-4064-a634-9afb552c54c5-signing-key\") pod \"service-ca-55f6fc7b86-6bllk\" (UID: \"a8813434-d7a3-4064-a634-9afb552c54c5\") " pod="openshift-service-ca/service-ca-55f6fc7b86-6bllk"
Sep 19 05:41:39.428476 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:39.428452    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zs8zb\" (UniqueName: \"kubernetes.io/projected/a8813434-d7a3-4064-a634-9afb552c54c5-kube-api-access-zs8zb\") pod \"service-ca-55f6fc7b86-6bllk\" (UID: \"a8813434-d7a3-4064-a634-9afb552c54c5\") " pod="openshift-service-ca/service-ca-55f6fc7b86-6bllk"
Sep 19 05:41:39.468864 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:39.468802    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-76794cbd6c-gsnhm"]
Sep 19 05:41:39.490913 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:39.490890    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-76794cbd6c-gsnhm"]
Sep 19 05:41:39.491037 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:39.490982    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-76794cbd6c-gsnhm"
Sep 19 05:41:39.493642 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:39.493619    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Sep 19 05:41:39.493769 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:39.493621    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-wh2bj\""
Sep 19 05:41:39.494354 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:39.494335    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Sep 19 05:41:39.519979 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:39.519952    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-glqsh\" (UniqueName: \"kubernetes.io/projected/63c5a37c-f796-4baf-8c1f-e090bcae95d3-kube-api-access-glqsh\") pod \"migrator-76794cbd6c-gsnhm\" (UID: \"63c5a37c-f796-4baf-8c1f-e090bcae95d3\") " pod="openshift-kube-storage-version-migrator/migrator-76794cbd6c-gsnhm"
Sep 19 05:41:39.590953 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:39.590925    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-55f6fc7b86-6bllk"
Sep 19 05:41:39.620539 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:39.620511    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-glqsh\" (UniqueName: \"kubernetes.io/projected/63c5a37c-f796-4baf-8c1f-e090bcae95d3-kube-api-access-glqsh\") pod \"migrator-76794cbd6c-gsnhm\" (UID: \"63c5a37c-f796-4baf-8c1f-e090bcae95d3\") " pod="openshift-kube-storage-version-migrator/migrator-76794cbd6c-gsnhm"
Sep 19 05:41:39.632521 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:39.632494    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-glqsh\" (UniqueName: \"kubernetes.io/projected/63c5a37c-f796-4baf-8c1f-e090bcae95d3-kube-api-access-glqsh\") pod \"migrator-76794cbd6c-gsnhm\" (UID: \"63c5a37c-f796-4baf-8c1f-e090bcae95d3\") " pod="openshift-kube-storage-version-migrator/migrator-76794cbd6c-gsnhm"
Sep 19 05:41:39.800235 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:39.800201    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-76794cbd6c-gsnhm"
Sep 19 05:41:39.881704 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:39.881675    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-55f6fc7b86-6bllk"]
Sep 19 05:41:39.883763 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:41:39.883734    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8813434_d7a3_4064_a634_9afb552c54c5.slice/crio-0772a4be79f791f47cae7048a8279f71cf2c2068b30c7483d82d28b4e164880e WatchSource:0}: Error finding container 0772a4be79f791f47cae7048a8279f71cf2c2068b30c7483d82d28b4e164880e: Status 404 returned error can't find the container with id 0772a4be79f791f47cae7048a8279f71cf2c2068b30c7483d82d28b4e164880e
Sep 19 05:41:39.890758 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:39.890728    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-55f6fc7b86-6bllk" event={"ID":"a8813434-d7a3-4064-a634-9afb552c54c5","Type":"ContainerStarted","Data":"0772a4be79f791f47cae7048a8279f71cf2c2068b30c7483d82d28b4e164880e"}
Sep 19 05:41:39.936675 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:39.936656    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-znm4v_b839569f-a0b2-4046-aee5-5e4ba7b0ef9f/serve-healthcheck-canary/0.log"
Sep 19 05:41:40.098042 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:40.097973    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-76794cbd6c-gsnhm"]
Sep 19 05:41:40.101798 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:41:40.101772    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63c5a37c_f796_4baf_8c1f_e090bcae95d3.slice/crio-cba7751d61bfec58f434170a6244c72263b2325338466acd26eed18d477b8905 WatchSource:0}: Error finding container cba7751d61bfec58f434170a6244c72263b2325338466acd26eed18d477b8905: Status 404 returned error can't find the container with id cba7751d61bfec58f434170a6244c72263b2325338466acd26eed18d477b8905
Sep 19 05:41:40.326489 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:40.326456    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/098cb613-a7e2-46de-a6c6-898a7ccb1a14-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jgkc7\" (UID: \"098cb613-a7e2-46de-a6c6-898a7ccb1a14\") " pod="openshift-insights/insights-runtime-extractor-jgkc7"
Sep 19 05:41:40.326489 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:40.326493    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/098cb613-a7e2-46de-a6c6-898a7ccb1a14-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jgkc7\" (UID: \"098cb613-a7e2-46de-a6c6-898a7ccb1a14\") " pod="openshift-insights/insights-runtime-extractor-jgkc7"
Sep 19 05:41:40.326886 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:41:40.326589    2583 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 19 05:41:40.326886 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:41:40.326612    2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 19 05:41:40.326886 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:41:40.326651    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/098cb613-a7e2-46de-a6c6-898a7ccb1a14-kube-rbac-proxy-cm podName:098cb613-a7e2-46de-a6c6-898a7ccb1a14 nodeName:}" failed. No retries permitted until 2026-09-19 05:41:44.326635571 +0000 UTC m=+62.985090107 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/098cb613-a7e2-46de-a6c6-898a7ccb1a14-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jgkc7" (UID: "098cb613-a7e2-46de-a6c6-898a7ccb1a14") : configmap "kube-rbac-proxy" not found
Sep 19 05:41:40.326886 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:41:40.326665    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/098cb613-a7e2-46de-a6c6-898a7ccb1a14-insights-runtime-extractor-tls podName:098cb613-a7e2-46de-a6c6-898a7ccb1a14 nodeName:}" failed. No retries permitted until 2026-09-19 05:41:44.326658914 +0000 UTC m=+62.985113450 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/098cb613-a7e2-46de-a6c6-898a7ccb1a14-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jgkc7" (UID: "098cb613-a7e2-46de-a6c6-898a7ccb1a14") : secret "insights-runtime-extractor-tls" not found
Sep 19 05:41:40.894844 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:40.894802    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-55f6fc7b86-6bllk" event={"ID":"a8813434-d7a3-4064-a634-9afb552c54c5","Type":"ContainerStarted","Data":"41a41eb00f13cef3b0714d30b5972bbee7393e9a2bb23902f47a2c3073b2ca98"}
Sep 19 05:41:40.895978 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:40.895952    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-76794cbd6c-gsnhm" event={"ID":"63c5a37c-f796-4baf-8c1f-e090bcae95d3","Type":"ContainerStarted","Data":"cba7751d61bfec58f434170a6244c72263b2325338466acd26eed18d477b8905"}
Sep 19 05:41:40.926068 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:40.926020    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-55f6fc7b86-6bllk" podStartSLOduration=1.926004381 podStartE2EDuration="1.926004381s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:41:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:41:40.92525708 +0000 UTC m=+59.583711639" watchObservedRunningTime="2026-09-19 05:41:40.926004381 +0000 UTC m=+59.584458939"
Sep 19 05:41:41.902318 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:41.902269    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-76794cbd6c-gsnhm" event={"ID":"63c5a37c-f796-4baf-8c1f-e090bcae95d3","Type":"ContainerStarted","Data":"e640388419afe176b3986705e5ff3df94e9f3e6a02e64f284cb728058df5adc1"}
Sep 19 05:41:41.902762 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:41.902375    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-76794cbd6c-gsnhm" event={"ID":"63c5a37c-f796-4baf-8c1f-e090bcae95d3","Type":"ContainerStarted","Data":"0bebb198af1146bdf64c4bd22ac2352887b978e2ca8e256bc176f95ad5794319"}
Sep 19 05:41:41.945629 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:41.945577    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-76794cbd6c-gsnhm" podStartSLOduration=1.706971712 podStartE2EDuration="2.945561048s" totalImagesPullingTime="1.238589336s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:41:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:41:41.94327884 +0000 UTC m=+60.601733399" watchObservedRunningTime="2026-09-19 05:41:41.945561048 +0000 UTC m=+60.604015606"
Sep 19 05:41:42.810348 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:42.810322    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-gj7h8"
Sep 19 05:41:44.359889 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:44.359850    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/098cb613-a7e2-46de-a6c6-898a7ccb1a14-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jgkc7\" (UID: \"098cb613-a7e2-46de-a6c6-898a7ccb1a14\") " pod="openshift-insights/insights-runtime-extractor-jgkc7"
Sep 19 05:41:44.359889 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:44.359888    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/098cb613-a7e2-46de-a6c6-898a7ccb1a14-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jgkc7\" (UID: \"098cb613-a7e2-46de-a6c6-898a7ccb1a14\") " pod="openshift-insights/insights-runtime-extractor-jgkc7"
Sep 19 05:41:44.360502 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:41:44.359967    2583 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 19 05:41:44.360502 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:41:44.360038    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/098cb613-a7e2-46de-a6c6-898a7ccb1a14-kube-rbac-proxy-cm podName:098cb613-a7e2-46de-a6c6-898a7ccb1a14 nodeName:}" failed. No retries permitted until 2026-09-19 05:41:52.360023077 +0000 UTC m=+71.018477612 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/098cb613-a7e2-46de-a6c6-898a7ccb1a14-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jgkc7" (UID: "098cb613-a7e2-46de-a6c6-898a7ccb1a14") : configmap "kube-rbac-proxy" not found
Sep 19 05:41:44.360502 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:41:44.360038    2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 19 05:41:44.360502 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:41:44.360080    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/098cb613-a7e2-46de-a6c6-898a7ccb1a14-insights-runtime-extractor-tls podName:098cb613-a7e2-46de-a6c6-898a7ccb1a14 nodeName:}" failed. No retries permitted until 2026-09-19 05:41:52.360068289 +0000 UTC m=+71.018522825 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/098cb613-a7e2-46de-a6c6-898a7ccb1a14-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jgkc7" (UID: "098cb613-a7e2-46de-a6c6-898a7ccb1a14") : secret "insights-runtime-extractor-tls" not found
Sep 19 05:41:47.284554 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:47.284497    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e3671a49-9883-42c5-b0cf-726774c62cd1-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-847b8bd8d7-9nkln\" (UID: \"e3671a49-9883-42c5-b0cf-726774c62cd1\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-9nkln"
Sep 19 05:41:47.284925 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:41:47.284656    2583 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 19 05:41:47.284925 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:41:47.284730    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3671a49-9883-42c5-b0cf-726774c62cd1-cluster-monitoring-operator-tls podName:e3671a49-9883-42c5-b0cf-726774c62cd1 nodeName:}" failed. No retries permitted until 2026-09-19 05:42:03.284710333 +0000 UTC m=+81.943164871 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e3671a49-9883-42c5-b0cf-726774c62cd1-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-847b8bd8d7-9nkln" (UID: "e3671a49-9883-42c5-b0cf-726774c62cd1") : secret "cluster-monitoring-operator-tls" not found
Sep 19 05:41:47.385213 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:47.385167    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/8b4a7ecf-a718-4241-a172-7334070ef48e-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-npzd4\" (UID: \"8b4a7ecf-a718-4241-a172-7334070ef48e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-npzd4"
Sep 19 05:41:47.387698 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:47.387663    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/8b4a7ecf-a718-4241-a172-7334070ef48e-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-npzd4\" (UID: \"8b4a7ecf-a718-4241-a172-7334070ef48e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-npzd4"
Sep 19 05:41:47.427136 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:47.427110    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-sc65q\""
Sep 19 05:41:47.435422 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:47.435407    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-npzd4"
Sep 19 05:41:47.732055 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:47.731855    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-npzd4"]
Sep 19 05:41:47.922378 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:47.922340    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-npzd4" event={"ID":"8b4a7ecf-a718-4241-a172-7334070ef48e","Type":"ContainerStarted","Data":"51299943097bda6b3e3ba05d319d1f932100b31abf7a803f8524eee826aed56f"}
Sep 19 05:41:48.191439 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:48.191405    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4bd3176a-beea-424d-b4c2-dfc6f27298d8-original-pull-secret\") pod \"global-pull-secret-syncer-fpnw2\" (UID: \"4bd3176a-beea-424d-b4c2-dfc6f27298d8\") " pod="kube-system/global-pull-secret-syncer-fpnw2"
Sep 19 05:41:48.191709 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:48.191477    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b291ff5e-1667-4066-98d5-21495cd76311-metrics-certs\") pod \"network-metrics-daemon-nrtjk\" (UID: \"b291ff5e-1667-4066-98d5-21495cd76311\") " pod="openshift-multus/network-metrics-daemon-nrtjk"
Sep 19 05:41:48.194033 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:48.194013    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 19 05:41:48.194090 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:48.194043    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 19 05:41:48.202316 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:41:48.202294    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 19 05:41:48.202494 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:41:48.202351    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b291ff5e-1667-4066-98d5-21495cd76311-metrics-certs podName:b291ff5e-1667-4066-98d5-21495cd76311 nodeName:}" failed. No retries permitted until 2026-09-19 05:42:52.202335522 +0000 UTC m=+130.860790057 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b291ff5e-1667-4066-98d5-21495cd76311-metrics-certs") pod "network-metrics-daemon-nrtjk" (UID: "b291ff5e-1667-4066-98d5-21495cd76311") : secret "metrics-daemon-secret" not found
Sep 19 05:41:48.204400 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:48.204380    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4bd3176a-beea-424d-b4c2-dfc6f27298d8-original-pull-secret\") pod \"global-pull-secret-syncer-fpnw2\" (UID: \"4bd3176a-beea-424d-b4c2-dfc6f27298d8\") " pod="kube-system/global-pull-secret-syncer-fpnw2"
Sep 19 05:41:48.219670 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:48.219647    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fpnw2"
Sep 19 05:41:48.293279 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:48.292828    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-khkc6\" (UniqueName: \"kubernetes.io/projected/379278e6-0c96-4ff2-ba60-71ab1d88ed6d-kube-api-access-khkc6\") pod \"network-check-target-x5dk8\" (UID: \"379278e6-0c96-4ff2-ba60-71ab1d88ed6d\") " pod="openshift-network-diagnostics/network-check-target-x5dk8"
Sep 19 05:41:48.295873 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:48.295820    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-khkc6\" (UniqueName: \"kubernetes.io/projected/379278e6-0c96-4ff2-ba60-71ab1d88ed6d-kube-api-access-khkc6\") pod \"network-check-target-x5dk8\" (UID: \"379278e6-0c96-4ff2-ba60-71ab1d88ed6d\") " pod="openshift-network-diagnostics/network-check-target-x5dk8"
Sep 19 05:41:48.495884 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:48.495858    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-fpnw2"]
Sep 19 05:41:48.498431 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:41:48.498402    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4bd3176a_beea_424d_b4c2_dfc6f27298d8.slice/crio-d94d8a4657e89f85b2b0ce4c1f0b1282e2fb80b4b3855c32095475c679e0c7e3 WatchSource:0}: Error finding container d94d8a4657e89f85b2b0ce4c1f0b1282e2fb80b4b3855c32095475c679e0c7e3: Status 404 returned error can't find the container with id d94d8a4657e89f85b2b0ce4c1f0b1282e2fb80b4b3855c32095475c679e0c7e3
Sep 19 05:41:48.508072 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:48.508053    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-b2b8g\""
Sep 19 05:41:48.516583 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:48.516561    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x5dk8"
Sep 19 05:41:48.797069 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:48.797036    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6a315987-3770-4424-8864-36e28a3fc43b-metrics-tls\") pod \"dns-default-656cq\" (UID: \"6a315987-3770-4424-8864-36e28a3fc43b\") " pod="openshift-dns/dns-default-656cq"
Sep 19 05:41:48.797309 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:48.797245    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7d39ddd1-8d0b-4dee-8c11-a67822704f32-registry-tls\") pod \"image-registry-55d7655b79-9sb7m\" (UID: \"7d39ddd1-8d0b-4dee-8c11-a67822704f32\") " pod="openshift-image-registry/image-registry-55d7655b79-9sb7m"
Sep 19 05:41:48.798604 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:48.798584    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-x5dk8"]
Sep 19 05:41:48.799711 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:48.799695    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6a315987-3770-4424-8864-36e28a3fc43b-metrics-tls\") pod \"dns-default-656cq\" (UID: \"6a315987-3770-4424-8864-36e28a3fc43b\") " pod="openshift-dns/dns-default-656cq"
Sep 19 05:41:48.799775 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:48.799745    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7d39ddd1-8d0b-4dee-8c11-a67822704f32-registry-tls\") pod \"image-registry-55d7655b79-9sb7m\" (UID: \"7d39ddd1-8d0b-4dee-8c11-a67822704f32\") " pod="openshift-image-registry/image-registry-55d7655b79-9sb7m"
Sep 19 05:41:48.801124 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:41:48.801098    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod379278e6_0c96_4ff2_ba60_71ab1d88ed6d.slice/crio-f8e897fe21e135250079f1bb7ffc8f821e8452164a2b9a49c870e24759b94d22 WatchSource:0}: Error finding container f8e897fe21e135250079f1bb7ffc8f821e8452164a2b9a49c870e24759b94d22: Status 404 returned error can't find the container with id f8e897fe21e135250079f1bb7ffc8f821e8452164a2b9a49c870e24759b94d22
Sep 19 05:41:48.927908 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:48.927873    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-x5dk8" event={"ID":"379278e6-0c96-4ff2-ba60-71ab1d88ed6d","Type":"ContainerStarted","Data":"9467ad05c073e373d3494b57bed2e77c52036c8ae0c264cd7f612b6d18e99542"}
Sep 19 05:41:48.928092 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:48.928014    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-x5dk8"
Sep 19 05:41:48.928092 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:48.928055    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-x5dk8" event={"ID":"379278e6-0c96-4ff2-ba60-71ab1d88ed6d","Type":"ContainerStarted","Data":"f8e897fe21e135250079f1bb7ffc8f821e8452164a2b9a49c870e24759b94d22"}
Sep 19 05:41:48.929271 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:48.929239    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-fpnw2" event={"ID":"4bd3176a-beea-424d-b4c2-dfc6f27298d8","Type":"ContainerStarted","Data":"d94d8a4657e89f85b2b0ce4c1f0b1282e2fb80b4b3855c32095475c679e0c7e3"}
Sep 19 05:41:49.026679 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:49.026643    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-sp8wq\""
Sep 19 05:41:49.035006 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:49.034975    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-55d7655b79-9sb7m"
Sep 19 05:41:49.045426 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:49.045265    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-mbwrd\""
Sep 19 05:41:49.053673 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:49.053610    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-656cq"
Sep 19 05:41:49.696722 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:49.696626    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-x5dk8" podStartSLOduration=66.696605275 podStartE2EDuration="1m6.696605275s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:40:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:41:48.950569815 +0000 UTC m=+67.609024374" watchObservedRunningTime="2026-09-19 05:41:49.696605275 +0000 UTC m=+68.355059830"
Sep 19 05:41:49.697182 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:49.697115    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-656cq"]
Sep 19 05:41:49.700989 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:41:49.700962    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a315987_3770_4424_8864_36e28a3fc43b.slice/crio-6daa013330749d5a528a05c4cc51c060e8d97120c7bef442af91d2e85b27656f WatchSource:0}: Error finding container 6daa013330749d5a528a05c4cc51c060e8d97120c7bef442af91d2e85b27656f: Status 404 returned error can't find the container with id 6daa013330749d5a528a05c4cc51c060e8d97120c7bef442af91d2e85b27656f
Sep 19 05:41:49.704840 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:49.704806    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-55d7655b79-9sb7m"]
Sep 19 05:41:49.707461 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:41:49.707440    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d39ddd1_8d0b_4dee_8c11_a67822704f32.slice/crio-cbe559ce56704411812df2b0423db9b55e904276dabe322ea8840b6acb572431 WatchSource:0}: Error finding container cbe559ce56704411812df2b0423db9b55e904276dabe322ea8840b6acb572431: Status 404 returned error can't find the container with id cbe559ce56704411812df2b0423db9b55e904276dabe322ea8840b6acb572431
Sep 19 05:41:49.935624 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:49.935582    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-55d7655b79-9sb7m" event={"ID":"7d39ddd1-8d0b-4dee-8c11-a67822704f32","Type":"ContainerStarted","Data":"169f61e1f9fc3a42dc2d12154471615d7be8bbde4c38b832636efd3f2cf9658b"}
Sep 19 05:41:49.935810 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:49.935647    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-55d7655b79-9sb7m" event={"ID":"7d39ddd1-8d0b-4dee-8c11-a67822704f32","Type":"ContainerStarted","Data":"cbe559ce56704411812df2b0423db9b55e904276dabe322ea8840b6acb572431"}
Sep 19 05:41:49.935879 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:49.935804    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-55d7655b79-9sb7m"
Sep 19 05:41:49.938350 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:49.938287    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-npzd4" event={"ID":"8b4a7ecf-a718-4241-a172-7334070ef48e","Type":"ContainerStarted","Data":"fe02e387e8b6c48f96e1dbfd7ed14941c361d70509f8f5f4f5cd2d5599655c15"}
Sep 19 05:41:49.938350 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:49.938338    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-npzd4" event={"ID":"8b4a7ecf-a718-4241-a172-7334070ef48e","Type":"ContainerStarted","Data":"13788540e4ba799dd89ad443d90009502c725cf5b2fe068b97ab6559aa9d97bf"}
Sep 19 05:41:49.939690 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:49.939661    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-656cq" event={"ID":"6a315987-3770-4424-8864-36e28a3fc43b","Type":"ContainerStarted","Data":"6daa013330749d5a528a05c4cc51c060e8d97120c7bef442af91d2e85b27656f"}
Sep 19 05:41:49.959667 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:49.959577    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-55d7655b79-9sb7m" podStartSLOduration=65.959558265 podStartE2EDuration="1m5.959558265s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:40:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:41:49.958475285 +0000 UTC m=+68.616929853" watchObservedRunningTime="2026-09-19 05:41:49.959558265 +0000 UTC m=+68.618012824"
Sep 19 05:41:49.975849 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:49.975798    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-npzd4" podStartSLOduration=17.307412865 podStartE2EDuration="18.975785028s" totalImagesPullingTime="1.668372163s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:41:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:41:49.974710405 +0000 UTC m=+68.633164964" watchObservedRunningTime="2026-09-19 05:41:49.975785028 +0000 UTC m=+68.634239651"
Sep 19 05:41:52.430340 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:52.430298    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/098cb613-a7e2-46de-a6c6-898a7ccb1a14-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jgkc7\" (UID: \"098cb613-a7e2-46de-a6c6-898a7ccb1a14\") " pod="openshift-insights/insights-runtime-extractor-jgkc7"
Sep 19 05:41:52.430340 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:52.430338    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/098cb613-a7e2-46de-a6c6-898a7ccb1a14-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jgkc7\" (UID: \"098cb613-a7e2-46de-a6c6-898a7ccb1a14\") " pod="openshift-insights/insights-runtime-extractor-jgkc7"
Sep 19 05:41:52.430936 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:52.430914    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/098cb613-a7e2-46de-a6c6-898a7ccb1a14-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jgkc7\" (UID: \"098cb613-a7e2-46de-a6c6-898a7ccb1a14\") " pod="openshift-insights/insights-runtime-extractor-jgkc7"
Sep 19 05:41:52.433028 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:52.433003    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/098cb613-a7e2-46de-a6c6-898a7ccb1a14-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jgkc7\" (UID: \"098cb613-a7e2-46de-a6c6-898a7ccb1a14\") " pod="openshift-insights/insights-runtime-extractor-jgkc7"
Sep 19 05:41:52.527103 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:52.527071    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-s598g\""
Sep 19 05:41:52.534883 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:52.534857    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-jgkc7"
Sep 19 05:41:52.953164 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:52.953113    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-fpnw2" event={"ID":"4bd3176a-beea-424d-b4c2-dfc6f27298d8","Type":"ContainerStarted","Data":"ac6a3fe643d8d7b86536a11a0205212f3925075b4d95afbcbcb945def1e2be36"}
Sep 19 05:41:52.955103 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:52.955081    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-656cq" event={"ID":"6a315987-3770-4424-8864-36e28a3fc43b","Type":"ContainerStarted","Data":"4791d9c5dfb913796ea8eee2b8b9e02d414e13777fb1be4aa192130ea64e18e1"}
Sep 19 05:41:52.955256 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:52.955140    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-656cq" event={"ID":"6a315987-3770-4424-8864-36e28a3fc43b","Type":"ContainerStarted","Data":"ebc3687433657211533d96dc793d0b833ec104beab3ced4de3ca0765ade7e6e1"}
Sep 19 05:41:52.955256 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:52.955164    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-656cq"
Sep 19 05:41:52.972382 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:52.972326    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-fpnw2" podStartSLOduration=65.73209762 podStartE2EDuration="1m9.972306995s" totalImagesPullingTime="4.240209375s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 05:40:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:41:52.97104105 +0000 UTC m=+71.629495609" watchObservedRunningTime="2026-09-19 05:41:52.972306995 +0000 UTC m=+71.630761554"
Sep 19 05:41:52.990664 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:52.990548    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-656cq" podStartSLOduration=33.966958206 podStartE2EDuration="36.990532513s" totalImagesPullingTime="3.023574307s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:41:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:41:52.989204481 +0000 UTC m=+71.647659031" watchObservedRunningTime="2026-09-19 05:41:52.990532513 +0000 UTC m=+71.648987070"
Sep 19 05:41:53.006849 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:53.006817    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-jgkc7"]
Sep 19 05:41:53.009707 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:41:53.009686    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod098cb613_a7e2_46de_a6c6_898a7ccb1a14.slice/crio-840c53caf7f88d11e5bfb1a1132b31e62e7a5266d17a2f58e2666c07213cc174 WatchSource:0}: Error finding container 840c53caf7f88d11e5bfb1a1132b31e62e7a5266d17a2f58e2666c07213cc174: Status 404 returned error can't find the container with id 840c53caf7f88d11e5bfb1a1132b31e62e7a5266d17a2f58e2666c07213cc174
Sep 19 05:41:53.959807 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:53.959721    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jgkc7" event={"ID":"098cb613-a7e2-46de-a6c6-898a7ccb1a14","Type":"ContainerStarted","Data":"1172fbc9685f05db4fa420078f59beea53b9b7982f3666d04705ca073a2409b8"}
Sep 19 05:41:53.960174 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:53.959809    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jgkc7" event={"ID":"098cb613-a7e2-46de-a6c6-898a7ccb1a14","Type":"ContainerStarted","Data":"1052bcea78dce3dd1451ec73457b0bbd3c368ec3695a5f6540cb6ea9b4bfa69c"}
Sep 19 05:41:53.960174 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:53.959832    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jgkc7" event={"ID":"098cb613-a7e2-46de-a6c6-898a7ccb1a14","Type":"ContainerStarted","Data":"840c53caf7f88d11e5bfb1a1132b31e62e7a5266d17a2f58e2666c07213cc174"}
Sep 19 05:41:55.967757 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:55.967620    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jgkc7" event={"ID":"098cb613-a7e2-46de-a6c6-898a7ccb1a14","Type":"ContainerStarted","Data":"c195fe57b2b9aaa6e96bdbc6a3302c842ff87590fb4b38a110bd72dac4c8d022"}
Sep 19 05:41:55.991197 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:41:55.991122    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-jgkc7" podStartSLOduration=18.184554704 podStartE2EDuration="19.991101415s" totalImagesPullingTime="1.806546711s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 05:41:36 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:41:55.991036199 +0000 UTC m=+74.649490756" watchObservedRunningTime="2026-09-19 05:41:55.991101415 +0000 UTC m=+74.649555973"
Sep 19 05:42:01.395904 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:01.395871    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6446c8d75d-zpk78"]
Sep 19 05:42:01.400210 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:01.400189    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6446c8d75d-zpk78"
Sep 19 05:42:01.403762 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:01.403735    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-9r899\""
Sep 19 05:42:01.403907 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:01.403765    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Sep 19 05:42:01.403907 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:01.403735    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Sep 19 05:42:01.412517 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:01.412493    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-774fb4d868-xbtz2"]
Sep 19 05:42:01.415537 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:01.415516    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6446c8d75d-zpk78"]
Sep 19 05:42:01.415648 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:01.415619    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-774fb4d868-xbtz2"
Sep 19 05:42:01.418868 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:01.418848    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Sep 19 05:42:01.419462 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:01.419443    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Sep 19 05:42:01.419700 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:01.419685    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Sep 19 05:42:01.419769 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:01.419755    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-5sgdf\""
Sep 19 05:42:01.421323 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:01.421307    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Sep 19 05:42:01.437349 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:01.437325    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-774fb4d868-xbtz2"]
Sep 19 05:42:01.499764 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:01.499731    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d76dc580-506e-47c0-81b7-9fa0e7db353c-networking-console-plugin-cert\") pod \"networking-console-plugin-6446c8d75d-zpk78\" (UID: \"d76dc580-506e-47c0-81b7-9fa0e7db353c\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-zpk78"
Sep 19 05:42:01.499940 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:01.499779    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dxkcd\" (UniqueName: \"kubernetes.io/projected/da068954-43bc-4038-9004-8c5ff890d761-kube-api-access-dxkcd\") pod \"managed-serviceaccount-addon-agent-774fb4d868-xbtz2\" (UID: \"da068954-43bc-4038-9004-8c5ff890d761\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-774fb4d868-xbtz2"
Sep 19 05:42:01.499940 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:01.499846    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/d76dc580-506e-47c0-81b7-9fa0e7db353c-nginx-conf\") pod \"networking-console-plugin-6446c8d75d-zpk78\" (UID: \"d76dc580-506e-47c0-81b7-9fa0e7db353c\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-zpk78"
Sep 19 05:42:01.499940 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:01.499880    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/da068954-43bc-4038-9004-8c5ff890d761-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-774fb4d868-xbtz2\" (UID: \"da068954-43bc-4038-9004-8c5ff890d761\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-774fb4d868-xbtz2"
Sep 19 05:42:01.502081 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:01.502056    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-6594f4c75-dnccb"]
Sep 19 05:42:01.505098 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:01.505079    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-6594f4c75-dnccb"
Sep 19 05:42:01.507902 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:01.507885    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 19 05:42:01.508201 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:01.508181    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 19 05:42:01.508293 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:01.508184    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-8qkmh\""
Sep 19 05:42:01.518606 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:01.518583    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-6594f4c75-dnccb"]
Sep 19 05:42:01.600964 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:01.600925    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dxkcd\" (UniqueName: \"kubernetes.io/projected/da068954-43bc-4038-9004-8c5ff890d761-kube-api-access-dxkcd\") pod \"managed-serviceaccount-addon-agent-774fb4d868-xbtz2\" (UID: \"da068954-43bc-4038-9004-8c5ff890d761\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-774fb4d868-xbtz2"
Sep 19 05:42:01.601194 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:01.600974    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/d76dc580-506e-47c0-81b7-9fa0e7db353c-nginx-conf\") pod \"networking-console-plugin-6446c8d75d-zpk78\" (UID: \"d76dc580-506e-47c0-81b7-9fa0e7db353c\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-zpk78"
Sep 19 05:42:01.601194 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:01.600995    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/da068954-43bc-4038-9004-8c5ff890d761-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-774fb4d868-xbtz2\" (UID: \"da068954-43bc-4038-9004-8c5ff890d761\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-774fb4d868-xbtz2"
Sep 19 05:42:01.601194 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:01.601035    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c79fs\" (UniqueName: \"kubernetes.io/projected/93fe0f3f-a6cc-4238-9492-fe1488f10bfd-kube-api-access-c79fs\") pod \"downloads-6594f4c75-dnccb\" (UID: \"93fe0f3f-a6cc-4238-9492-fe1488f10bfd\") " pod="openshift-console/downloads-6594f4c75-dnccb"
Sep 19 05:42:01.601371 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:01.601223    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d76dc580-506e-47c0-81b7-9fa0e7db353c-networking-console-plugin-cert\") pod \"networking-console-plugin-6446c8d75d-zpk78\" (UID: \"d76dc580-506e-47c0-81b7-9fa0e7db353c\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-zpk78"
Sep 19 05:42:01.601711 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:01.601682    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/d76dc580-506e-47c0-81b7-9fa0e7db353c-nginx-conf\") pod \"networking-console-plugin-6446c8d75d-zpk78\" (UID: \"d76dc580-506e-47c0-81b7-9fa0e7db353c\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-zpk78"
Sep 19 05:42:01.603490 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:01.603468    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d76dc580-506e-47c0-81b7-9fa0e7db353c-networking-console-plugin-cert\") pod \"networking-console-plugin-6446c8d75d-zpk78\" (UID: \"d76dc580-506e-47c0-81b7-9fa0e7db353c\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-zpk78"
Sep 19 05:42:01.603587 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:01.603534    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/da068954-43bc-4038-9004-8c5ff890d761-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-774fb4d868-xbtz2\" (UID: \"da068954-43bc-4038-9004-8c5ff890d761\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-774fb4d868-xbtz2"
Sep 19 05:42:01.614440 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:01.614415    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dxkcd\" (UniqueName: \"kubernetes.io/projected/da068954-43bc-4038-9004-8c5ff890d761-kube-api-access-dxkcd\") pod \"managed-serviceaccount-addon-agent-774fb4d868-xbtz2\" (UID: \"da068954-43bc-4038-9004-8c5ff890d761\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-774fb4d868-xbtz2"
Sep 19 05:42:01.702453 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:01.702387    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c79fs\" (UniqueName: \"kubernetes.io/projected/93fe0f3f-a6cc-4238-9492-fe1488f10bfd-kube-api-access-c79fs\") pod \"downloads-6594f4c75-dnccb\" (UID: \"93fe0f3f-a6cc-4238-9492-fe1488f10bfd\") " pod="openshift-console/downloads-6594f4c75-dnccb"
Sep 19 05:42:01.708353 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:01.708334    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6446c8d75d-zpk78"
Sep 19 05:42:01.717637 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:01.717609    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c79fs\" (UniqueName: \"kubernetes.io/projected/93fe0f3f-a6cc-4238-9492-fe1488f10bfd-kube-api-access-c79fs\") pod \"downloads-6594f4c75-dnccb\" (UID: \"93fe0f3f-a6cc-4238-9492-fe1488f10bfd\") " pod="openshift-console/downloads-6594f4c75-dnccb"
Sep 19 05:42:01.735417 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:01.735386    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-774fb4d868-xbtz2"
Sep 19 05:42:01.814262 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:01.814213    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-6594f4c75-dnccb"
Sep 19 05:42:01.980885 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:01.980811    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6446c8d75d-zpk78"]
Sep 19 05:42:01.983675 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:42:01.983654    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd76dc580_506e_47c0_81b7_9fa0e7db353c.slice/crio-db10058a97b6389c2ce990a678872880ae9498b790027028062a7013e5aff0cd WatchSource:0}: Error finding container db10058a97b6389c2ce990a678872880ae9498b790027028062a7013e5aff0cd: Status 404 returned error can't find the container with id db10058a97b6389c2ce990a678872880ae9498b790027028062a7013e5aff0cd
Sep 19 05:42:01.988518 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:01.988487    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6446c8d75d-zpk78" event={"ID":"d76dc580-506e-47c0-81b7-9fa0e7db353c","Type":"ContainerStarted","Data":"db10058a97b6389c2ce990a678872880ae9498b790027028062a7013e5aff0cd"}
Sep 19 05:42:02.007162 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:02.007140    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-774fb4d868-xbtz2"]
Sep 19 05:42:02.009397 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:42:02.009371    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podda068954_43bc_4038_9004_8c5ff890d761.slice/crio-4275a739ad4d258aaa4531a29806912b4bfd8d13bb30fac87ef14362f9cc4f58 WatchSource:0}: Error finding container 4275a739ad4d258aaa4531a29806912b4bfd8d13bb30fac87ef14362f9cc4f58: Status 404 returned error can't find the container with id 4275a739ad4d258aaa4531a29806912b4bfd8d13bb30fac87ef14362f9cc4f58
Sep 19 05:42:02.084734 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:02.084700    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-6594f4c75-dnccb"]
Sep 19 05:42:02.087526 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:42:02.087498    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93fe0f3f_a6cc_4238_9492_fe1488f10bfd.slice/crio-b4c6f769f0f6a321b1a6428bf4f80505369995f7ebb039fbb35a847363193d14 WatchSource:0}: Error finding container b4c6f769f0f6a321b1a6428bf4f80505369995f7ebb039fbb35a847363193d14: Status 404 returned error can't find the container with id b4c6f769f0f6a321b1a6428bf4f80505369995f7ebb039fbb35a847363193d14
Sep 19 05:42:02.962944 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:02.962912    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-656cq"
Sep 19 05:42:02.996971 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:02.996881    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-6594f4c75-dnccb" event={"ID":"93fe0f3f-a6cc-4238-9492-fe1488f10bfd","Type":"ContainerStarted","Data":"b4c6f769f0f6a321b1a6428bf4f80505369995f7ebb039fbb35a847363193d14"}
Sep 19 05:42:03.000044 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:02.999991    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-774fb4d868-xbtz2" event={"ID":"da068954-43bc-4038-9004-8c5ff890d761","Type":"ContainerStarted","Data":"4275a739ad4d258aaa4531a29806912b4bfd8d13bb30fac87ef14362f9cc4f58"}
Sep 19 05:42:03.316588 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:03.316499    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e3671a49-9883-42c5-b0cf-726774c62cd1-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-847b8bd8d7-9nkln\" (UID: \"e3671a49-9883-42c5-b0cf-726774c62cd1\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-9nkln"
Sep 19 05:42:03.321918 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:03.321842    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e3671a49-9883-42c5-b0cf-726774c62cd1-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-847b8bd8d7-9nkln\" (UID: \"e3671a49-9883-42c5-b0cf-726774c62cd1\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-9nkln"
Sep 19 05:42:03.517902 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:03.517871    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-gc5t7\""
Sep 19 05:42:03.526063 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:03.526032    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-9nkln"
Sep 19 05:42:05.010200 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:05.010159    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6446c8d75d-zpk78" event={"ID":"d76dc580-506e-47c0-81b7-9fa0e7db353c","Type":"ContainerStarted","Data":"a1131942b3f23b63b44293e99d1cad930fecb91fc0af273cf34bb619ad2fbdfa"}
Sep 19 05:42:05.049479 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:05.049426    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6446c8d75d-zpk78" podStartSLOduration=1.196730255 podStartE2EDuration="4.049410107s" totalImagesPullingTime="2.852679852s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:42:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:42:05.047663081 +0000 UTC m=+83.706117639" watchObservedRunningTime="2026-09-19 05:42:05.049410107 +0000 UTC m=+83.707864666"
Sep 19 05:42:05.131406 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:05.131368    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-9nkln"]
Sep 19 05:42:05.176770 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:42:05.176727    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3671a49_9883_42c5_b0cf_726774c62cd1.slice/crio-1dc24e27ccab3cc69f31cfc3d273064160c1c8ae1dc707fd58680c54a9bc37b2 WatchSource:0}: Error finding container 1dc24e27ccab3cc69f31cfc3d273064160c1c8ae1dc707fd58680c54a9bc37b2: Status 404 returned error can't find the container with id 1dc24e27ccab3cc69f31cfc3d273064160c1c8ae1dc707fd58680c54a9bc37b2
Sep 19 05:42:06.016989 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:06.016906    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-9nkln" event={"ID":"e3671a49-9883-42c5-b0cf-726774c62cd1","Type":"ContainerStarted","Data":"1dc24e27ccab3cc69f31cfc3d273064160c1c8ae1dc707fd58680c54a9bc37b2"}
Sep 19 05:42:06.018912 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:06.018883    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-774fb4d868-xbtz2" event={"ID":"da068954-43bc-4038-9004-8c5ff890d761","Type":"ContainerStarted","Data":"737a3ae9c404bb61a93962d0f49bf227dd86f581742ee7ed0ed330e51e63d70a"}
Sep 19 05:42:06.037467 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:06.037412    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-774fb4d868-xbtz2" podStartSLOduration=1.840953558 podStartE2EDuration="5.037392425s" totalImagesPullingTime="3.196438867s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:42:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:42:06.035816612 +0000 UTC m=+84.694271175" watchObservedRunningTime="2026-09-19 05:42:06.037392425 +0000 UTC m=+84.695846983"
Sep 19 05:42:08.028072 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:08.028032    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-9nkln" event={"ID":"e3671a49-9883-42c5-b0cf-726774c62cd1","Type":"ContainerStarted","Data":"0d7b715778744a312e2c8b7437c2fb5dfb879979608cabfa880551fa92e4790a"}
Sep 19 05:42:08.046339 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:08.046284    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-9nkln" podStartSLOduration=35.248981177 podStartE2EDuration="37.046268344s" totalImagesPullingTime="1.797287167s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:41:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:42:08.045318403 +0000 UTC m=+86.703772962" watchObservedRunningTime="2026-09-19 05:42:08.046268344 +0000 UTC m=+86.704722901"
Sep 19 05:42:09.041188 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:09.041149    2583 patch_prober.go:28] interesting pod/image-registry-55d7655b79-9sb7m container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Sep 19 05:42:09.041728 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:09.041215    2583 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-55d7655b79-9sb7m" podUID="7d39ddd1-8d0b-4dee-8c11-a67822704f32" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 19 05:42:10.949567 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:10.949536    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-55d7655b79-9sb7m"
Sep 19 05:42:19.072665 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:19.072075    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-6594f4c75-dnccb" event={"ID":"93fe0f3f-a6cc-4238-9492-fe1488f10bfd","Type":"ContainerStarted","Data":"8f31921aa44a4c5214411bb56781fc6cd703bc7fac90241e4089c480105cd7cd"}
Sep 19 05:42:19.072665 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:19.072379    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-6594f4c75-dnccb"
Sep 19 05:42:19.080422 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:19.080395    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-6594f4c75-dnccb"
Sep 19 05:42:19.100445 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:19.098705    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-6594f4c75-dnccb" podStartSLOduration=1.4905586739999999 podStartE2EDuration="18.098685229s" totalImagesPullingTime="16.608126555s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:42:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:42:19.096623571 +0000 UTC m=+97.755078130" watchObservedRunningTime="2026-09-19 05:42:19.098685229 +0000 UTC m=+97.757139787"
Sep 19 05:42:19.221065 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:19.219744    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-hvzsv"]
Sep 19 05:42:19.224650 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:19.224623    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-hvzsv"
Sep 19 05:42:19.228445 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:19.228419    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 19 05:42:19.229345 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:19.228714    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-x6wsd\""
Sep 19 05:42:19.229345 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:19.228895    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 19 05:42:19.229345 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:19.229221    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 19 05:42:19.232641 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:19.232274    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 19 05:42:19.357332 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:19.357240    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/6c4cd1ae-b085-41f9-bf91-6520dee8b101-root\") pod \"node-exporter-hvzsv\" (UID: \"6c4cd1ae-b085-41f9-bf91-6520dee8b101\") " pod="openshift-monitoring/node-exporter-hvzsv"
Sep 19 05:42:19.357603 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:19.357563    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zgc2t\" (UniqueName: \"kubernetes.io/projected/6c4cd1ae-b085-41f9-bf91-6520dee8b101-kube-api-access-zgc2t\") pod \"node-exporter-hvzsv\" (UID: \"6c4cd1ae-b085-41f9-bf91-6520dee8b101\") " pod="openshift-monitoring/node-exporter-hvzsv"
Sep 19 05:42:19.357803 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:19.357771    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/6c4cd1ae-b085-41f9-bf91-6520dee8b101-node-exporter-textfile\") pod \"node-exporter-hvzsv\" (UID: \"6c4cd1ae-b085-41f9-bf91-6520dee8b101\") " pod="openshift-monitoring/node-exporter-hvzsv"
Sep 19 05:42:19.357950 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:19.357933    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6c4cd1ae-b085-41f9-bf91-6520dee8b101-metrics-client-ca\") pod \"node-exporter-hvzsv\" (UID: \"6c4cd1ae-b085-41f9-bf91-6520dee8b101\") " pod="openshift-monitoring/node-exporter-hvzsv"
Sep 19 05:42:19.358096 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:19.358077    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/6c4cd1ae-b085-41f9-bf91-6520dee8b101-node-exporter-accelerators-collector-config\") pod \"node-exporter-hvzsv\" (UID: \"6c4cd1ae-b085-41f9-bf91-6520dee8b101\") " pod="openshift-monitoring/node-exporter-hvzsv"
Sep 19 05:42:19.358253 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:19.358237    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/6c4cd1ae-b085-41f9-bf91-6520dee8b101-node-exporter-wtmp\") pod \"node-exporter-hvzsv\" (UID: \"6c4cd1ae-b085-41f9-bf91-6520dee8b101\") " pod="openshift-monitoring/node-exporter-hvzsv"
Sep 19 05:42:19.358407 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:19.358392    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6c4cd1ae-b085-41f9-bf91-6520dee8b101-sys\") pod \"node-exporter-hvzsv\" (UID: \"6c4cd1ae-b085-41f9-bf91-6520dee8b101\") " pod="openshift-monitoring/node-exporter-hvzsv"
Sep 19 05:42:19.358557 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:19.358540    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6c4cd1ae-b085-41f9-bf91-6520dee8b101-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-hvzsv\" (UID: \"6c4cd1ae-b085-41f9-bf91-6520dee8b101\") " pod="openshift-monitoring/node-exporter-hvzsv"
Sep 19 05:42:19.358749 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:19.358732    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/6c4cd1ae-b085-41f9-bf91-6520dee8b101-node-exporter-tls\") pod \"node-exporter-hvzsv\" (UID: \"6c4cd1ae-b085-41f9-bf91-6520dee8b101\") " pod="openshift-monitoring/node-exporter-hvzsv"
Sep 19 05:42:19.459469 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:19.459430    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6c4cd1ae-b085-41f9-bf91-6520dee8b101-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-hvzsv\" (UID: \"6c4cd1ae-b085-41f9-bf91-6520dee8b101\") " pod="openshift-monitoring/node-exporter-hvzsv"
Sep 19 05:42:19.459668 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:19.459500    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/6c4cd1ae-b085-41f9-bf91-6520dee8b101-node-exporter-tls\") pod \"node-exporter-hvzsv\" (UID: \"6c4cd1ae-b085-41f9-bf91-6520dee8b101\") " pod="openshift-monitoring/node-exporter-hvzsv"
Sep 19 05:42:19.459668 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:19.459529    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/6c4cd1ae-b085-41f9-bf91-6520dee8b101-root\") pod \"node-exporter-hvzsv\" (UID: \"6c4cd1ae-b085-41f9-bf91-6520dee8b101\") " pod="openshift-monitoring/node-exporter-hvzsv"
Sep 19 05:42:19.459668 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:19.459557    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zgc2t\" (UniqueName: \"kubernetes.io/projected/6c4cd1ae-b085-41f9-bf91-6520dee8b101-kube-api-access-zgc2t\") pod \"node-exporter-hvzsv\" (UID: \"6c4cd1ae-b085-41f9-bf91-6520dee8b101\") " pod="openshift-monitoring/node-exporter-hvzsv"
Sep 19 05:42:19.459668 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:19.459598    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/6c4cd1ae-b085-41f9-bf91-6520dee8b101-node-exporter-textfile\") pod \"node-exporter-hvzsv\" (UID: \"6c4cd1ae-b085-41f9-bf91-6520dee8b101\") " pod="openshift-monitoring/node-exporter-hvzsv"
Sep 19 05:42:19.459668 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:19.459624    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6c4cd1ae-b085-41f9-bf91-6520dee8b101-metrics-client-ca\") pod \"node-exporter-hvzsv\" (UID: \"6c4cd1ae-b085-41f9-bf91-6520dee8b101\") " pod="openshift-monitoring/node-exporter-hvzsv"
Sep 19 05:42:19.459668 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:19.459653    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/6c4cd1ae-b085-41f9-bf91-6520dee8b101-node-exporter-accelerators-collector-config\") pod \"node-exporter-hvzsv\" (UID: \"6c4cd1ae-b085-41f9-bf91-6520dee8b101\") " pod="openshift-monitoring/node-exporter-hvzsv"
Sep 19 05:42:19.459971 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:19.459679    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/6c4cd1ae-b085-41f9-bf91-6520dee8b101-node-exporter-wtmp\") pod \"node-exporter-hvzsv\" (UID: \"6c4cd1ae-b085-41f9-bf91-6520dee8b101\") " pod="openshift-monitoring/node-exporter-hvzsv"
Sep 19 05:42:19.459971 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:19.459723    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6c4cd1ae-b085-41f9-bf91-6520dee8b101-sys\") pod \"node-exporter-hvzsv\" (UID: \"6c4cd1ae-b085-41f9-bf91-6520dee8b101\") " pod="openshift-monitoring/node-exporter-hvzsv"
Sep 19 05:42:19.459971 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:19.459805    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6c4cd1ae-b085-41f9-bf91-6520dee8b101-sys\") pod \"node-exporter-hvzsv\" (UID: \"6c4cd1ae-b085-41f9-bf91-6520dee8b101\") " pod="openshift-monitoring/node-exporter-hvzsv"
Sep 19 05:42:19.460521 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:19.460489    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/6c4cd1ae-b085-41f9-bf91-6520dee8b101-node-exporter-textfile\") pod \"node-exporter-hvzsv\" (UID: \"6c4cd1ae-b085-41f9-bf91-6520dee8b101\") " pod="openshift-monitoring/node-exporter-hvzsv"
Sep 19 05:42:19.460676 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:42:19.460597    2583 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Sep 19 05:42:19.460676 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:42:19.460653    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c4cd1ae-b085-41f9-bf91-6520dee8b101-node-exporter-tls podName:6c4cd1ae-b085-41f9-bf91-6520dee8b101 nodeName:}" failed. No retries permitted until 2026-09-19 05:42:19.960633153 +0000 UTC m=+98.619087703 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/6c4cd1ae-b085-41f9-bf91-6520dee8b101-node-exporter-tls") pod "node-exporter-hvzsv" (UID: "6c4cd1ae-b085-41f9-bf91-6520dee8b101") : secret "node-exporter-tls" not found
Sep 19 05:42:19.460808 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:19.460704    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/6c4cd1ae-b085-41f9-bf91-6520dee8b101-root\") pod \"node-exporter-hvzsv\" (UID: \"6c4cd1ae-b085-41f9-bf91-6520dee8b101\") " pod="openshift-monitoring/node-exporter-hvzsv"
Sep 19 05:42:19.460808 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:19.460755    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/6c4cd1ae-b085-41f9-bf91-6520dee8b101-node-exporter-accelerators-collector-config\") pod \"node-exporter-hvzsv\" (UID: \"6c4cd1ae-b085-41f9-bf91-6520dee8b101\") " pod="openshift-monitoring/node-exporter-hvzsv"
Sep 19 05:42:19.461215 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:19.461189    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6c4cd1ae-b085-41f9-bf91-6520dee8b101-metrics-client-ca\") pod \"node-exporter-hvzsv\" (UID: \"6c4cd1ae-b085-41f9-bf91-6520dee8b101\") " pod="openshift-monitoring/node-exporter-hvzsv"
Sep 19 05:42:19.461350 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:19.461219    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/6c4cd1ae-b085-41f9-bf91-6520dee8b101-node-exporter-wtmp\") pod \"node-exporter-hvzsv\" (UID: \"6c4cd1ae-b085-41f9-bf91-6520dee8b101\") " pod="openshift-monitoring/node-exporter-hvzsv"
Sep 19 05:42:19.463746 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:19.463589    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6c4cd1ae-b085-41f9-bf91-6520dee8b101-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-hvzsv\" (UID: \"6c4cd1ae-b085-41f9-bf91-6520dee8b101\") " pod="openshift-monitoring/node-exporter-hvzsv"
Sep 19 05:42:19.474898 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:19.474851    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zgc2t\" (UniqueName: \"kubernetes.io/projected/6c4cd1ae-b085-41f9-bf91-6520dee8b101-kube-api-access-zgc2t\") pod \"node-exporter-hvzsv\" (UID: \"6c4cd1ae-b085-41f9-bf91-6520dee8b101\") " pod="openshift-monitoring/node-exporter-hvzsv"
Sep 19 05:42:19.943071 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:19.942979    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-x5dk8"
Sep 19 05:42:19.965500 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:19.965457    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/6c4cd1ae-b085-41f9-bf91-6520dee8b101-node-exporter-tls\") pod \"node-exporter-hvzsv\" (UID: \"6c4cd1ae-b085-41f9-bf91-6520dee8b101\") " pod="openshift-monitoring/node-exporter-hvzsv"
Sep 19 05:42:19.965685 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:42:19.965634    2583 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Sep 19 05:42:19.965757 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:42:19.965701    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c4cd1ae-b085-41f9-bf91-6520dee8b101-node-exporter-tls podName:6c4cd1ae-b085-41f9-bf91-6520dee8b101 nodeName:}" failed. No retries permitted until 2026-09-19 05:42:20.965683301 +0000 UTC m=+99.624137846 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/6c4cd1ae-b085-41f9-bf91-6520dee8b101-node-exporter-tls") pod "node-exporter-hvzsv" (UID: "6c4cd1ae-b085-41f9-bf91-6520dee8b101") : secret "node-exporter-tls" not found
Sep 19 05:42:20.975036 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:20.974944    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/6c4cd1ae-b085-41f9-bf91-6520dee8b101-node-exporter-tls\") pod \"node-exporter-hvzsv\" (UID: \"6c4cd1ae-b085-41f9-bf91-6520dee8b101\") " pod="openshift-monitoring/node-exporter-hvzsv"
Sep 19 05:42:20.977776 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:20.977747    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/6c4cd1ae-b085-41f9-bf91-6520dee8b101-node-exporter-tls\") pod \"node-exporter-hvzsv\" (UID: \"6c4cd1ae-b085-41f9-bf91-6520dee8b101\") " pod="openshift-monitoring/node-exporter-hvzsv"
Sep 19 05:42:21.040852 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:21.040810    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-hvzsv"
Sep 19 05:42:21.072667 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:42:21.072623    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c4cd1ae_b085_41f9_bf91_6520dee8b101.slice/crio-b2839ad07eef2ab287071db4c439dcb3452de989c76bb6894593b8d5816c334a WatchSource:0}: Error finding container b2839ad07eef2ab287071db4c439dcb3452de989c76bb6894593b8d5816c334a: Status 404 returned error can't find the container with id b2839ad07eef2ab287071db4c439dcb3452de989c76bb6894593b8d5816c334a
Sep 19 05:42:21.082165 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:21.082122    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hvzsv" event={"ID":"6c4cd1ae-b085-41f9-bf91-6520dee8b101","Type":"ContainerStarted","Data":"b2839ad07eef2ab287071db4c439dcb3452de989c76bb6894593b8d5816c334a"}
Sep 19 05:42:23.093223 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:23.093183    2583 generic.go:356] "Generic (PLEG): container finished" podID="6c4cd1ae-b085-41f9-bf91-6520dee8b101" containerID="8ee6d34ee1c5499552f1d2c27e34b1fd03fac2050f17cc787aec32bff80bc91d" exitCode=0
Sep 19 05:42:23.093711 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:23.093282    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hvzsv" event={"ID":"6c4cd1ae-b085-41f9-bf91-6520dee8b101","Type":"ContainerDied","Data":"8ee6d34ee1c5499552f1d2c27e34b1fd03fac2050f17cc787aec32bff80bc91d"}
Sep 19 05:42:24.107446 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:24.107406    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hvzsv" event={"ID":"6c4cd1ae-b085-41f9-bf91-6520dee8b101","Type":"ContainerStarted","Data":"321375467a53dad81f9785f497576faec7819601bbcaebc745e69d317bc12496"}
Sep 19 05:42:24.107930 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:24.107492    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hvzsv" event={"ID":"6c4cd1ae-b085-41f9-bf91-6520dee8b101","Type":"ContainerStarted","Data":"d0f5c058c083ccb16449ca0673762c33f7427041389fba24cb589818482fccd3"}
Sep 19 05:42:24.131820 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:24.131760    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-hvzsv" podStartSLOduration=3.504829686 podStartE2EDuration="5.131739896s" totalImagesPullingTime="1.543933858s" totalInitContainerRuntime="82.976352ms" isStatefulPod=true podCreationTimestamp="2026-09-19 05:42:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:42:24.131378828 +0000 UTC m=+102.789833395" watchObservedRunningTime="2026-09-19 05:42:24.131739896 +0000 UTC m=+102.790194455"
Sep 19 05:42:24.925060 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:24.925026    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-55d7655b79-9sb7m"]
Sep 19 05:42:49.947606 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:49.947565    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-55d7655b79-9sb7m" podUID="7d39ddd1-8d0b-4dee-8c11-a67822704f32" containerName="registry" containerID="cri-o://169f61e1f9fc3a42dc2d12154471615d7be8bbde4c38b832636efd3f2cf9658b" gracePeriod=30
Sep 19 05:42:50.204544 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:50.204478    2583 generic.go:356] "Generic (PLEG): container finished" podID="7d39ddd1-8d0b-4dee-8c11-a67822704f32" containerID="169f61e1f9fc3a42dc2d12154471615d7be8bbde4c38b832636efd3f2cf9658b" exitCode=0
Sep 19 05:42:50.204544 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:50.204520    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-55d7655b79-9sb7m" event={"ID":"7d39ddd1-8d0b-4dee-8c11-a67822704f32","Type":"ContainerDied","Data":"169f61e1f9fc3a42dc2d12154471615d7be8bbde4c38b832636efd3f2cf9658b"}
Sep 19 05:42:50.204680 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:50.204554    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-55d7655b79-9sb7m" event={"ID":"7d39ddd1-8d0b-4dee-8c11-a67822704f32","Type":"ContainerDied","Data":"cbe559ce56704411812df2b0423db9b55e904276dabe322ea8840b6acb572431"}
Sep 19 05:42:50.204680 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:50.204569    2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cbe559ce56704411812df2b0423db9b55e904276dabe322ea8840b6acb572431"
Sep 19 05:42:50.216265 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:50.216237    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-55d7655b79-9sb7m"
Sep 19 05:42:50.299823 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:50.299788    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7d39ddd1-8d0b-4dee-8c11-a67822704f32-bound-sa-token\") pod \"7d39ddd1-8d0b-4dee-8c11-a67822704f32\" (UID: \"7d39ddd1-8d0b-4dee-8c11-a67822704f32\") "
Sep 19 05:42:50.299975 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:50.299829    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7d39ddd1-8d0b-4dee-8c11-a67822704f32-trusted-ca\") pod \"7d39ddd1-8d0b-4dee-8c11-a67822704f32\" (UID: \"7d39ddd1-8d0b-4dee-8c11-a67822704f32\") "
Sep 19 05:42:50.299975 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:50.299871    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7d39ddd1-8d0b-4dee-8c11-a67822704f32-image-registry-private-configuration\") pod \"7d39ddd1-8d0b-4dee-8c11-a67822704f32\" (UID: \"7d39ddd1-8d0b-4dee-8c11-a67822704f32\") "
Sep 19 05:42:50.299975 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:50.299899    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7d39ddd1-8d0b-4dee-8c11-a67822704f32-registry-certificates\") pod \"7d39ddd1-8d0b-4dee-8c11-a67822704f32\" (UID: \"7d39ddd1-8d0b-4dee-8c11-a67822704f32\") "
Sep 19 05:42:50.299975 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:50.299929    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7d39ddd1-8d0b-4dee-8c11-a67822704f32-ca-trust-extracted\") pod \"7d39ddd1-8d0b-4dee-8c11-a67822704f32\" (UID: \"7d39ddd1-8d0b-4dee-8c11-a67822704f32\") "
Sep 19 05:42:50.299975 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:50.299958    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7d39ddd1-8d0b-4dee-8c11-a67822704f32-installation-pull-secrets\") pod \"7d39ddd1-8d0b-4dee-8c11-a67822704f32\" (UID: \"7d39ddd1-8d0b-4dee-8c11-a67822704f32\") "
Sep 19 05:42:50.300583 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:50.299985    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7d39ddd1-8d0b-4dee-8c11-a67822704f32-registry-tls\") pod \"7d39ddd1-8d0b-4dee-8c11-a67822704f32\" (UID: \"7d39ddd1-8d0b-4dee-8c11-a67822704f32\") "
Sep 19 05:42:50.300583 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:50.300011    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8f5dj\" (UniqueName: \"kubernetes.io/projected/7d39ddd1-8d0b-4dee-8c11-a67822704f32-kube-api-access-8f5dj\") pod \"7d39ddd1-8d0b-4dee-8c11-a67822704f32\" (UID: \"7d39ddd1-8d0b-4dee-8c11-a67822704f32\") "
Sep 19 05:42:50.300583 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:50.300382    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7d39ddd1-8d0b-4dee-8c11-a67822704f32-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "7d39ddd1-8d0b-4dee-8c11-a67822704f32" (UID: "7d39ddd1-8d0b-4dee-8c11-a67822704f32"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 05:42:50.300583 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:50.300414    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7d39ddd1-8d0b-4dee-8c11-a67822704f32-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "7d39ddd1-8d0b-4dee-8c11-a67822704f32" (UID: "7d39ddd1-8d0b-4dee-8c11-a67822704f32"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 05:42:50.302721 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:50.302464    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d39ddd1-8d0b-4dee-8c11-a67822704f32-kube-api-access-8f5dj" (OuterVolumeSpecName: "kube-api-access-8f5dj") pod "7d39ddd1-8d0b-4dee-8c11-a67822704f32" (UID: "7d39ddd1-8d0b-4dee-8c11-a67822704f32"). InnerVolumeSpecName "kube-api-access-8f5dj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 05:42:50.302721 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:50.302652    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7d39ddd1-8d0b-4dee-8c11-a67822704f32-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "7d39ddd1-8d0b-4dee-8c11-a67822704f32" (UID: "7d39ddd1-8d0b-4dee-8c11-a67822704f32"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 05:42:50.302721 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:50.302693    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d39ddd1-8d0b-4dee-8c11-a67822704f32-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "7d39ddd1-8d0b-4dee-8c11-a67822704f32" (UID: "7d39ddd1-8d0b-4dee-8c11-a67822704f32"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 05:42:50.302721 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:50.302709    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7d39ddd1-8d0b-4dee-8c11-a67822704f32-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "7d39ddd1-8d0b-4dee-8c11-a67822704f32" (UID: "7d39ddd1-8d0b-4dee-8c11-a67822704f32"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 05:42:50.302918 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:50.302743    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d39ddd1-8d0b-4dee-8c11-a67822704f32-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "7d39ddd1-8d0b-4dee-8c11-a67822704f32" (UID: "7d39ddd1-8d0b-4dee-8c11-a67822704f32"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 05:42:50.310882 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:50.310852    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d39ddd1-8d0b-4dee-8c11-a67822704f32-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "7d39ddd1-8d0b-4dee-8c11-a67822704f32" (UID: "7d39ddd1-8d0b-4dee-8c11-a67822704f32"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 05:42:50.400537 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:50.400501    2583 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7d39ddd1-8d0b-4dee-8c11-a67822704f32-bound-sa-token\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 05:42:50.400537 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:50.400534    2583 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7d39ddd1-8d0b-4dee-8c11-a67822704f32-trusted-ca\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 05:42:50.400537 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:50.400544    2583 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7d39ddd1-8d0b-4dee-8c11-a67822704f32-image-registry-private-configuration\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 05:42:50.400733 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:50.400553    2583 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7d39ddd1-8d0b-4dee-8c11-a67822704f32-registry-certificates\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 05:42:50.400733 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:50.400564    2583 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7d39ddd1-8d0b-4dee-8c11-a67822704f32-ca-trust-extracted\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 05:42:50.400733 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:50.400572    2583 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7d39ddd1-8d0b-4dee-8c11-a67822704f32-installation-pull-secrets\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 05:42:50.400733 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:50.400580    2583 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7d39ddd1-8d0b-4dee-8c11-a67822704f32-registry-tls\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 05:42:50.400733 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:50.400588    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8f5dj\" (UniqueName: \"kubernetes.io/projected/7d39ddd1-8d0b-4dee-8c11-a67822704f32-kube-api-access-8f5dj\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 05:42:51.209113 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:51.209079    2583 generic.go:356] "Generic (PLEG): container finished" podID="18abc165-1181-4294-965e-b444f51b576c" containerID="53bf208e82fdc97f904d1337b5435d7eec08d85204b7fdfc038c055b188853f6" exitCode=0
Sep 19 05:42:51.209486 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:51.209156    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-t949s" event={"ID":"18abc165-1181-4294-965e-b444f51b576c","Type":"ContainerDied","Data":"53bf208e82fdc97f904d1337b5435d7eec08d85204b7fdfc038c055b188853f6"}
Sep 19 05:42:51.209486 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:51.209195    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-55d7655b79-9sb7m"
Sep 19 05:42:51.209713 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:51.209694    2583 scope.go:117] "RemoveContainer" containerID="53bf208e82fdc97f904d1337b5435d7eec08d85204b7fdfc038c055b188853f6"
Sep 19 05:42:51.244324 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:51.244292    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-55d7655b79-9sb7m"]
Sep 19 05:42:51.250486 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:51.250463    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-55d7655b79-9sb7m"]
Sep 19 05:42:51.497086 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:51.497005    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7d39ddd1-8d0b-4dee-8c11-a67822704f32" path="/var/lib/kubelet/pods/7d39ddd1-8d0b-4dee-8c11-a67822704f32/volumes"
Sep 19 05:42:52.212864 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:52.212830    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b291ff5e-1667-4066-98d5-21495cd76311-metrics-certs\") pod \"network-metrics-daemon-nrtjk\" (UID: \"b291ff5e-1667-4066-98d5-21495cd76311\") " pod="openshift-multus/network-metrics-daemon-nrtjk"
Sep 19 05:42:52.214709 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:52.214676    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-t949s" event={"ID":"18abc165-1181-4294-965e-b444f51b576c","Type":"ContainerStarted","Data":"e52d4e14373136dc2da757dd4e0526449332029ea62725a6b4d2e494d1c4ffaf"}
Sep 19 05:42:52.215286 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:52.215266    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b291ff5e-1667-4066-98d5-21495cd76311-metrics-certs\") pod \"network-metrics-daemon-nrtjk\" (UID: \"b291ff5e-1667-4066-98d5-21495cd76311\") " pod="openshift-multus/network-metrics-daemon-nrtjk"
Sep 19 05:42:52.415931 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:52.415898    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-tjt75\""
Sep 19 05:42:52.423257 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:52.423229    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nrtjk"
Sep 19 05:42:52.705848 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:52.705816    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-nrtjk"]
Sep 19 05:42:52.709280 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:42:52.709256    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb291ff5e_1667_4066_98d5_21495cd76311.slice/crio-3c82c17e465392b9717d60c4faf067051c27e7a117153d3d7285797cf1a7148d WatchSource:0}: Error finding container 3c82c17e465392b9717d60c4faf067051c27e7a117153d3d7285797cf1a7148d: Status 404 returned error can't find the container with id 3c82c17e465392b9717d60c4faf067051c27e7a117153d3d7285797cf1a7148d
Sep 19 05:42:53.220076 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:53.220034    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-nrtjk" event={"ID":"b291ff5e-1667-4066-98d5-21495cd76311","Type":"ContainerStarted","Data":"3c82c17e465392b9717d60c4faf067051c27e7a117153d3d7285797cf1a7148d"}
Sep 19 05:42:55.228286 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:55.228244    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-nrtjk" event={"ID":"b291ff5e-1667-4066-98d5-21495cd76311","Type":"ContainerStarted","Data":"92777a6ecbd6db8d01c9d5ce2773d38fe43f3886c07256f624f9286c76c95eb4"}
Sep 19 05:42:55.228664 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:55.228309    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-nrtjk" event={"ID":"b291ff5e-1667-4066-98d5-21495cd76311","Type":"ContainerStarted","Data":"45cd3e698ec8d7590e96379dea1b213a50deac6726acfd02d59f2f1f312a5310"}
Sep 19 05:42:55.247117 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:55.247069    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-nrtjk" podStartSLOduration=130.716126681 podStartE2EDuration="2m12.24705518s" totalImagesPullingTime="1.530928499s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:40:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:42:55.245333089 +0000 UTC m=+133.903787644" watchObservedRunningTime="2026-09-19 05:42:55.24705518 +0000 UTC m=+133.905509741"
Sep 19 05:42:56.233623 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:56.233542    2583 generic.go:356] "Generic (PLEG): container finished" podID="965fac7e-ec07-48bb-8d5b-202ddd313625" containerID="fb9d7f3d50cbe98f3890594288d3bcded9e737c1bf92ab014b72ffcec2ba2827" exitCode=0
Sep 19 05:42:56.234050 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:56.233621    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-6f9c644dd9-k9mzm" event={"ID":"965fac7e-ec07-48bb-8d5b-202ddd313625","Type":"ContainerDied","Data":"fb9d7f3d50cbe98f3890594288d3bcded9e737c1bf92ab014b72ffcec2ba2827"}
Sep 19 05:42:56.234050 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:56.234038    2583 scope.go:117] "RemoveContainer" containerID="fb9d7f3d50cbe98f3890594288d3bcded9e737c1bf92ab014b72ffcec2ba2827"
Sep 19 05:42:57.244630 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:42:57.244348    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-6f9c644dd9-k9mzm" event={"ID":"965fac7e-ec07-48bb-8d5b-202ddd313625","Type":"ContainerStarted","Data":"e73383fed818ddf25f2ddf8a8138c014956289bead68bdbc126bbefa8795e3f1"}
Sep 19 05:44:17.271448 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:17.271412    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qdwlw"]
Sep 19 05:44:17.271952 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:17.271699    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d39ddd1-8d0b-4dee-8c11-a67822704f32" containerName="registry"
Sep 19 05:44:17.271952 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:17.271712    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d39ddd1-8d0b-4dee-8c11-a67822704f32" containerName="registry"
Sep 19 05:44:17.271952 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:17.271777    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="7d39ddd1-8d0b-4dee-8c11-a67822704f32" containerName="registry"
Sep 19 05:44:17.273598 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:17.273580    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qdwlw"
Sep 19 05:44:17.276346 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:17.276323    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 19 05:44:17.276466 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:17.276387    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 19 05:44:17.276466 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:17.276399    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-vqxsl\""
Sep 19 05:44:17.287186 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:17.287162    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qdwlw"]
Sep 19 05:44:17.324209 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:17.324177    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h7fx8\" (UniqueName: \"kubernetes.io/projected/4f044bdb-c7d2-4084-ba7a-a23f7a261e36-kube-api-access-h7fx8\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qdwlw\" (UID: \"4f044bdb-c7d2-4084-ba7a-a23f7a261e36\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qdwlw"
Sep 19 05:44:17.324399 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:17.324228    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4f044bdb-c7d2-4084-ba7a-a23f7a261e36-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qdwlw\" (UID: \"4f044bdb-c7d2-4084-ba7a-a23f7a261e36\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qdwlw"
Sep 19 05:44:17.324399 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:17.324251    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4f044bdb-c7d2-4084-ba7a-a23f7a261e36-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qdwlw\" (UID: \"4f044bdb-c7d2-4084-ba7a-a23f7a261e36\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qdwlw"
Sep 19 05:44:17.425533 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:17.425493    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4f044bdb-c7d2-4084-ba7a-a23f7a261e36-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qdwlw\" (UID: \"4f044bdb-c7d2-4084-ba7a-a23f7a261e36\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qdwlw"
Sep 19 05:44:17.425533 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:17.425536    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4f044bdb-c7d2-4084-ba7a-a23f7a261e36-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qdwlw\" (UID: \"4f044bdb-c7d2-4084-ba7a-a23f7a261e36\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qdwlw"
Sep 19 05:44:17.425764 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:17.425609    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h7fx8\" (UniqueName: \"kubernetes.io/projected/4f044bdb-c7d2-4084-ba7a-a23f7a261e36-kube-api-access-h7fx8\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qdwlw\" (UID: \"4f044bdb-c7d2-4084-ba7a-a23f7a261e36\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qdwlw"
Sep 19 05:44:17.425882 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:17.425860    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4f044bdb-c7d2-4084-ba7a-a23f7a261e36-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qdwlw\" (UID: \"4f044bdb-c7d2-4084-ba7a-a23f7a261e36\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qdwlw"
Sep 19 05:44:17.425947 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:17.425887    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4f044bdb-c7d2-4084-ba7a-a23f7a261e36-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qdwlw\" (UID: \"4f044bdb-c7d2-4084-ba7a-a23f7a261e36\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qdwlw"
Sep 19 05:44:17.434987 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:17.434963    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h7fx8\" (UniqueName: \"kubernetes.io/projected/4f044bdb-c7d2-4084-ba7a-a23f7a261e36-kube-api-access-h7fx8\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qdwlw\" (UID: \"4f044bdb-c7d2-4084-ba7a-a23f7a261e36\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qdwlw"
Sep 19 05:44:17.583047 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:17.583011    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qdwlw"
Sep 19 05:44:17.857248 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:17.857220    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qdwlw"]
Sep 19 05:44:17.859640 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:44:17.859612    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f044bdb_c7d2_4084_ba7a_a23f7a261e36.slice/crio-6d0c7a757210c179e582b7959a8a5968c4ce2fdd7fa2ead9d8f166762b47cac1 WatchSource:0}: Error finding container 6d0c7a757210c179e582b7959a8a5968c4ce2fdd7fa2ead9d8f166762b47cac1: Status 404 returned error can't find the container with id 6d0c7a757210c179e582b7959a8a5968c4ce2fdd7fa2ead9d8f166762b47cac1
Sep 19 05:44:18.531385 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:18.531343    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qdwlw" event={"ID":"4f044bdb-c7d2-4084-ba7a-a23f7a261e36","Type":"ContainerStarted","Data":"6d0c7a757210c179e582b7959a8a5968c4ce2fdd7fa2ead9d8f166762b47cac1"}
Sep 19 05:44:23.551822 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:23.551785    2583 generic.go:356] "Generic (PLEG): container finished" podID="4f044bdb-c7d2-4084-ba7a-a23f7a261e36" containerID="54dbfed4477eb281728ec707a2528d99063e4d06f1842725c53d4f1e6d80c24c" exitCode=0
Sep 19 05:44:23.552274 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:23.551869    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qdwlw" event={"ID":"4f044bdb-c7d2-4084-ba7a-a23f7a261e36","Type":"ContainerDied","Data":"54dbfed4477eb281728ec707a2528d99063e4d06f1842725c53d4f1e6d80c24c"}
Sep 19 05:44:25.561344 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:25.561266    2583 generic.go:356] "Generic (PLEG): container finished" podID="4f044bdb-c7d2-4084-ba7a-a23f7a261e36" containerID="44e7be98ec88315725df5d421dc76f3e0a4b168f381038856633833cd5faf92a" exitCode=0
Sep 19 05:44:25.561685 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:25.561348    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qdwlw" event={"ID":"4f044bdb-c7d2-4084-ba7a-a23f7a261e36","Type":"ContainerDied","Data":"44e7be98ec88315725df5d421dc76f3e0a4b168f381038856633833cd5faf92a"}
Sep 19 05:44:32.589216 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:32.589181    2583 generic.go:356] "Generic (PLEG): container finished" podID="4f044bdb-c7d2-4084-ba7a-a23f7a261e36" containerID="9936e86546a235fa9030c93497221092460ed45e27a5cc369afe3a0660343498" exitCode=0
Sep 19 05:44:32.589669 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:32.589265    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qdwlw" event={"ID":"4f044bdb-c7d2-4084-ba7a-a23f7a261e36","Type":"ContainerDied","Data":"9936e86546a235fa9030c93497221092460ed45e27a5cc369afe3a0660343498"}
Sep 19 05:44:33.713369 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:33.713343    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qdwlw"
Sep 19 05:44:33.865044 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:33.864964    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4f044bdb-c7d2-4084-ba7a-a23f7a261e36-util\") pod \"4f044bdb-c7d2-4084-ba7a-a23f7a261e36\" (UID: \"4f044bdb-c7d2-4084-ba7a-a23f7a261e36\") "
Sep 19 05:44:33.865044 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:33.865026    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4f044bdb-c7d2-4084-ba7a-a23f7a261e36-bundle\") pod \"4f044bdb-c7d2-4084-ba7a-a23f7a261e36\" (UID: \"4f044bdb-c7d2-4084-ba7a-a23f7a261e36\") "
Sep 19 05:44:33.865301 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:33.865092    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h7fx8\" (UniqueName: \"kubernetes.io/projected/4f044bdb-c7d2-4084-ba7a-a23f7a261e36-kube-api-access-h7fx8\") pod \"4f044bdb-c7d2-4084-ba7a-a23f7a261e36\" (UID: \"4f044bdb-c7d2-4084-ba7a-a23f7a261e36\") "
Sep 19 05:44:33.865637 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:33.865602    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f044bdb-c7d2-4084-ba7a-a23f7a261e36-bundle" (OuterVolumeSpecName: "bundle") pod "4f044bdb-c7d2-4084-ba7a-a23f7a261e36" (UID: "4f044bdb-c7d2-4084-ba7a-a23f7a261e36"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 05:44:33.867383 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:33.867350    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4f044bdb-c7d2-4084-ba7a-a23f7a261e36-kube-api-access-h7fx8" (OuterVolumeSpecName: "kube-api-access-h7fx8") pod "4f044bdb-c7d2-4084-ba7a-a23f7a261e36" (UID: "4f044bdb-c7d2-4084-ba7a-a23f7a261e36"). InnerVolumeSpecName "kube-api-access-h7fx8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 05:44:33.869605 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:33.869584    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f044bdb-c7d2-4084-ba7a-a23f7a261e36-util" (OuterVolumeSpecName: "util") pod "4f044bdb-c7d2-4084-ba7a-a23f7a261e36" (UID: "4f044bdb-c7d2-4084-ba7a-a23f7a261e36"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 05:44:33.965829 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:33.965792    2583 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4f044bdb-c7d2-4084-ba7a-a23f7a261e36-util\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 05:44:33.965829 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:33.965821    2583 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4f044bdb-c7d2-4084-ba7a-a23f7a261e36-bundle\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 05:44:33.965829 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:33.965831    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h7fx8\" (UniqueName: \"kubernetes.io/projected/4f044bdb-c7d2-4084-ba7a-a23f7a261e36-kube-api-access-h7fx8\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 05:44:34.598227 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:34.598187    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qdwlw" event={"ID":"4f044bdb-c7d2-4084-ba7a-a23f7a261e36","Type":"ContainerDied","Data":"6d0c7a757210c179e582b7959a8a5968c4ce2fdd7fa2ead9d8f166762b47cac1"}
Sep 19 05:44:34.598227 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:34.598215    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qdwlw"
Sep 19 05:44:34.598227 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:34.598233    2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6d0c7a757210c179e582b7959a8a5968c4ce2fdd7fa2ead9d8f166762b47cac1"
Sep 19 05:44:40.246113 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:40.246082    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-gbzq9"]
Sep 19 05:44:40.246598 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:40.246354    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f044bdb-c7d2-4084-ba7a-a23f7a261e36" containerName="pull"
Sep 19 05:44:40.246598 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:40.246365    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f044bdb-c7d2-4084-ba7a-a23f7a261e36" containerName="pull"
Sep 19 05:44:40.246598 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:40.246375    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f044bdb-c7d2-4084-ba7a-a23f7a261e36" containerName="util"
Sep 19 05:44:40.246598 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:40.246380    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f044bdb-c7d2-4084-ba7a-a23f7a261e36" containerName="util"
Sep 19 05:44:40.246598 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:40.246386    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f044bdb-c7d2-4084-ba7a-a23f7a261e36" containerName="extract"
Sep 19 05:44:40.246598 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:40.246391    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f044bdb-c7d2-4084-ba7a-a23f7a261e36" containerName="extract"
Sep 19 05:44:40.246598 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:40.246443    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="4f044bdb-c7d2-4084-ba7a-a23f7a261e36" containerName="extract"
Sep 19 05:44:40.300378 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:40.300348    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-gbzq9"]
Sep 19 05:44:40.300524 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:40.300440    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-gbzq9"
Sep 19 05:44:40.303059 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:40.303032    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Sep 19 05:44:40.303245 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:40.303226    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Sep 19 05:44:40.303446 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:40.303427    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-jk4xb\""
Sep 19 05:44:40.412880 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:40.412849    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q8s97\" (UniqueName: \"kubernetes.io/projected/8fa4dd92-dcd0-4e07-b1ce-1f5e5b2def11-kube-api-access-q8s97\") pod \"cert-manager-operator-controller-manager-55c444bf86-gbzq9\" (UID: \"8fa4dd92-dcd0-4e07-b1ce-1f5e5b2def11\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-gbzq9"
Sep 19 05:44:40.413026 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:40.412893    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8fa4dd92-dcd0-4e07-b1ce-1f5e5b2def11-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-gbzq9\" (UID: \"8fa4dd92-dcd0-4e07-b1ce-1f5e5b2def11\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-gbzq9"
Sep 19 05:44:40.513473 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:40.513394    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q8s97\" (UniqueName: \"kubernetes.io/projected/8fa4dd92-dcd0-4e07-b1ce-1f5e5b2def11-kube-api-access-q8s97\") pod \"cert-manager-operator-controller-manager-55c444bf86-gbzq9\" (UID: \"8fa4dd92-dcd0-4e07-b1ce-1f5e5b2def11\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-gbzq9"
Sep 19 05:44:40.513473 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:40.513445    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8fa4dd92-dcd0-4e07-b1ce-1f5e5b2def11-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-gbzq9\" (UID: \"8fa4dd92-dcd0-4e07-b1ce-1f5e5b2def11\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-gbzq9"
Sep 19 05:44:40.513846 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:40.513824    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8fa4dd92-dcd0-4e07-b1ce-1f5e5b2def11-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-gbzq9\" (UID: \"8fa4dd92-dcd0-4e07-b1ce-1f5e5b2def11\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-gbzq9"
Sep 19 05:44:40.524966 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:40.524933    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q8s97\" (UniqueName: \"kubernetes.io/projected/8fa4dd92-dcd0-4e07-b1ce-1f5e5b2def11-kube-api-access-q8s97\") pod \"cert-manager-operator-controller-manager-55c444bf86-gbzq9\" (UID: \"8fa4dd92-dcd0-4e07-b1ce-1f5e5b2def11\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-gbzq9"
Sep 19 05:44:40.609877 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:40.609854    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-gbzq9"
Sep 19 05:44:40.907615 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:40.907590    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-gbzq9"]
Sep 19 05:44:40.910417 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:44:40.910391    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8fa4dd92_dcd0_4e07_b1ce_1f5e5b2def11.slice/crio-ac3a3b7c7c87610a62cee95ab6b90aa547a7eec787d4ccda08bad041e6e3a11f WatchSource:0}: Error finding container ac3a3b7c7c87610a62cee95ab6b90aa547a7eec787d4ccda08bad041e6e3a11f: Status 404 returned error can't find the container with id ac3a3b7c7c87610a62cee95ab6b90aa547a7eec787d4ccda08bad041e6e3a11f
Sep 19 05:44:41.626309 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:41.626269    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-gbzq9" event={"ID":"8fa4dd92-dcd0-4e07-b1ce-1f5e5b2def11","Type":"ContainerStarted","Data":"ac3a3b7c7c87610a62cee95ab6b90aa547a7eec787d4ccda08bad041e6e3a11f"}
Sep 19 05:44:43.637244 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:43.637205    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-gbzq9" event={"ID":"8fa4dd92-dcd0-4e07-b1ce-1f5e5b2def11","Type":"ContainerStarted","Data":"4610df482f637bd5d8d17be1f5ac40dc4c84ca108eeabca2a3594f85f24397bf"}
Sep 19 05:44:43.664425 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:43.664377    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-gbzq9" podStartSLOduration=1.7355008029999999 podStartE2EDuration="3.664360916s" totalImagesPullingTime="1.928860113s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:44:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:44:43.663114125 +0000 UTC m=+242.321568694" watchObservedRunningTime="2026-09-19 05:44:43.664360916 +0000 UTC m=+242.322815473"
Sep 19 05:44:44.877273 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:44.877236    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fg7k48"]
Sep 19 05:44:44.895062 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:44.895038    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fg7k48"]
Sep 19 05:44:44.895190 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:44.895155    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fg7k48"
Sep 19 05:44:44.897774 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:44.897744    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 19 05:44:44.897896 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:44.897801    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-vqxsl\""
Sep 19 05:44:44.898876 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:44.898856    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 19 05:44:45.045427 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:45.045391    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4d52936d-f73a-4bf0-b76b-f48ed87239ad-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fg7k48\" (UID: \"4d52936d-f73a-4bf0-b76b-f48ed87239ad\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fg7k48"
Sep 19 05:44:45.045595 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:45.045431    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4d52936d-f73a-4bf0-b76b-f48ed87239ad-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fg7k48\" (UID: \"4d52936d-f73a-4bf0-b76b-f48ed87239ad\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fg7k48"
Sep 19 05:44:45.045595 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:45.045510    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mmr4c\" (UniqueName: \"kubernetes.io/projected/4d52936d-f73a-4bf0-b76b-f48ed87239ad-kube-api-access-mmr4c\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fg7k48\" (UID: \"4d52936d-f73a-4bf0-b76b-f48ed87239ad\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fg7k48"
Sep 19 05:44:45.146801 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:45.146714    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4d52936d-f73a-4bf0-b76b-f48ed87239ad-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fg7k48\" (UID: \"4d52936d-f73a-4bf0-b76b-f48ed87239ad\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fg7k48"
Sep 19 05:44:45.146801 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:45.146753    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4d52936d-f73a-4bf0-b76b-f48ed87239ad-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fg7k48\" (UID: \"4d52936d-f73a-4bf0-b76b-f48ed87239ad\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fg7k48"
Sep 19 05:44:45.146801 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:45.146782    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mmr4c\" (UniqueName: \"kubernetes.io/projected/4d52936d-f73a-4bf0-b76b-f48ed87239ad-kube-api-access-mmr4c\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fg7k48\" (UID: \"4d52936d-f73a-4bf0-b76b-f48ed87239ad\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fg7k48"
Sep 19 05:44:45.147151 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:45.147109    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4d52936d-f73a-4bf0-b76b-f48ed87239ad-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fg7k48\" (UID: \"4d52936d-f73a-4bf0-b76b-f48ed87239ad\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fg7k48"
Sep 19 05:44:45.147235 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:45.147171    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4d52936d-f73a-4bf0-b76b-f48ed87239ad-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fg7k48\" (UID: \"4d52936d-f73a-4bf0-b76b-f48ed87239ad\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fg7k48"
Sep 19 05:44:45.156045 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:45.156022    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mmr4c\" (UniqueName: \"kubernetes.io/projected/4d52936d-f73a-4bf0-b76b-f48ed87239ad-kube-api-access-mmr4c\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fg7k48\" (UID: \"4d52936d-f73a-4bf0-b76b-f48ed87239ad\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fg7k48"
Sep 19 05:44:45.204097 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:45.204068    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fg7k48"
Sep 19 05:44:45.496227 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:44:45.496196    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d52936d_f73a_4bf0_b76b_f48ed87239ad.slice/crio-c188a1aa546581c7f6b192b5f15c33bc46d90f3f60f3b278a8676f5411ab9576 WatchSource:0}: Error finding container c188a1aa546581c7f6b192b5f15c33bc46d90f3f60f3b278a8676f5411ab9576: Status 404 returned error can't find the container with id c188a1aa546581c7f6b192b5f15c33bc46d90f3f60f3b278a8676f5411ab9576
Sep 19 05:44:45.497954 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:45.497929    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fg7k48"]
Sep 19 05:44:45.646827 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:45.646792    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fg7k48" event={"ID":"4d52936d-f73a-4bf0-b76b-f48ed87239ad","Type":"ContainerStarted","Data":"522cc74151a0ebba1c07ada2ab3925f1df214c1db83d6b03acc84019e8b597e2"}
Sep 19 05:44:45.646967 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:45.646849    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fg7k48" event={"ID":"4d52936d-f73a-4bf0-b76b-f48ed87239ad","Type":"ContainerStarted","Data":"c188a1aa546581c7f6b192b5f15c33bc46d90f3f60f3b278a8676f5411ab9576"}
Sep 19 05:44:46.226665 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:46.226630    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-hh7ft"]
Sep 19 05:44:46.243212 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:46.243188    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-hh7ft"]
Sep 19 05:44:46.243342 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:46.243269    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-hh7ft"
Sep 19 05:44:46.245563 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:46.245544    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Sep 19 05:44:46.245672 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:46.245625    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Sep 19 05:44:46.246660 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:46.246644    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-7w8kh\""
Sep 19 05:44:46.356145 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:46.356031    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6f269498-e648-4772-9371-38b36f61be01-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-hh7ft\" (UID: \"6f269498-e648-4772-9371-38b36f61be01\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-hh7ft"
Sep 19 05:44:46.356300 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:46.356198    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-smm8g\" (UniqueName: \"kubernetes.io/projected/6f269498-e648-4772-9371-38b36f61be01-kube-api-access-smm8g\") pod \"cert-manager-cainjector-559cdd8c87-hh7ft\" (UID: \"6f269498-e648-4772-9371-38b36f61be01\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-hh7ft"
Sep 19 05:44:46.457583 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:46.457506    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6f269498-e648-4772-9371-38b36f61be01-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-hh7ft\" (UID: \"6f269498-e648-4772-9371-38b36f61be01\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-hh7ft"
Sep 19 05:44:46.457583 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:46.457568    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-smm8g\" (UniqueName: \"kubernetes.io/projected/6f269498-e648-4772-9371-38b36f61be01-kube-api-access-smm8g\") pod \"cert-manager-cainjector-559cdd8c87-hh7ft\" (UID: \"6f269498-e648-4772-9371-38b36f61be01\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-hh7ft"
Sep 19 05:44:46.466688 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:46.466654    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6f269498-e648-4772-9371-38b36f61be01-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-hh7ft\" (UID: \"6f269498-e648-4772-9371-38b36f61be01\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-hh7ft"
Sep 19 05:44:46.466792 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:46.466723    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-smm8g\" (UniqueName: \"kubernetes.io/projected/6f269498-e648-4772-9371-38b36f61be01-kube-api-access-smm8g\") pod \"cert-manager-cainjector-559cdd8c87-hh7ft\" (UID: \"6f269498-e648-4772-9371-38b36f61be01\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-hh7ft"
Sep 19 05:44:46.553738 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:46.553703    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-hh7ft"
Sep 19 05:44:46.652352 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:46.652317    2583 generic.go:356] "Generic (PLEG): container finished" podID="4d52936d-f73a-4bf0-b76b-f48ed87239ad" containerID="522cc74151a0ebba1c07ada2ab3925f1df214c1db83d6b03acc84019e8b597e2" exitCode=0
Sep 19 05:44:46.652480 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:46.652372    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fg7k48" event={"ID":"4d52936d-f73a-4bf0-b76b-f48ed87239ad","Type":"ContainerDied","Data":"522cc74151a0ebba1c07ada2ab3925f1df214c1db83d6b03acc84019e8b597e2"}
Sep 19 05:44:46.820486 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:46.820461    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-hh7ft"]
Sep 19 05:44:46.822705 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:44:46.822679    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6f269498_e648_4772_9371_38b36f61be01.slice/crio-0f91ed5352826b9acc7f5b130e26a860eacabbba6bf23eccddb4d7648a135e22 WatchSource:0}: Error finding container 0f91ed5352826b9acc7f5b130e26a860eacabbba6bf23eccddb4d7648a135e22: Status 404 returned error can't find the container with id 0f91ed5352826b9acc7f5b130e26a860eacabbba6bf23eccddb4d7648a135e22
Sep 19 05:44:47.664628 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:47.664518    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-hh7ft" event={"ID":"6f269498-e648-4772-9371-38b36f61be01","Type":"ContainerStarted","Data":"0f91ed5352826b9acc7f5b130e26a860eacabbba6bf23eccddb4d7648a135e22"}
Sep 19 05:44:50.620590 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:50.620554    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-dr4ch"]
Sep 19 05:44:50.622517 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:50.622500    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-dr4ch"
Sep 19 05:44:50.625009 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:50.624988    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-8gsld\""
Sep 19 05:44:50.632020 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:50.631992    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-dr4ch"]
Sep 19 05:44:50.678694 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:50.678660    2583 generic.go:356] "Generic (PLEG): container finished" podID="4d52936d-f73a-4bf0-b76b-f48ed87239ad" containerID="8ca886df644be0d50652de3c390c6d5ccd77c1da23b0f2a30389ee639b2b41c1" exitCode=0
Sep 19 05:44:50.678841 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:50.678747    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fg7k48" event={"ID":"4d52936d-f73a-4bf0-b76b-f48ed87239ad","Type":"ContainerDied","Data":"8ca886df644be0d50652de3c390c6d5ccd77c1da23b0f2a30389ee639b2b41c1"}
Sep 19 05:44:50.680414 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:50.680383    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-hh7ft" event={"ID":"6f269498-e648-4772-9371-38b36f61be01","Type":"ContainerStarted","Data":"3a853ee5a2563bbed52ad47839a03237e2b8f9ba6d4af1343a0d2abcd6128a84"}
Sep 19 05:44:50.714491 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:50.714440    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-hh7ft" podStartSLOduration=1.7168195590000002 podStartE2EDuration="4.714420578s" totalImagesPullingTime="2.997601019s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:44:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:44:50.71414341 +0000 UTC m=+249.372597961" watchObservedRunningTime="2026-09-19 05:44:50.714420578 +0000 UTC m=+249.372875138"
Sep 19 05:44:50.794396 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:50.794349    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r8lsz\" (UniqueName: \"kubernetes.io/projected/4d2ce863-958d-49c0-87a6-44d023c22b56-kube-api-access-r8lsz\") pod \"cert-manager-webhook-6486ffd997-dr4ch\" (UID: \"4d2ce863-958d-49c0-87a6-44d023c22b56\") " pod="cert-manager/cert-manager-webhook-6486ffd997-dr4ch"
Sep 19 05:44:50.794542 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:50.794519    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4d2ce863-958d-49c0-87a6-44d023c22b56-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-dr4ch\" (UID: \"4d2ce863-958d-49c0-87a6-44d023c22b56\") " pod="cert-manager/cert-manager-webhook-6486ffd997-dr4ch"
Sep 19 05:44:50.895691 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:50.895607    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r8lsz\" (UniqueName: \"kubernetes.io/projected/4d2ce863-958d-49c0-87a6-44d023c22b56-kube-api-access-r8lsz\") pod \"cert-manager-webhook-6486ffd997-dr4ch\" (UID: \"4d2ce863-958d-49c0-87a6-44d023c22b56\") " pod="cert-manager/cert-manager-webhook-6486ffd997-dr4ch"
Sep 19 05:44:50.895691 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:50.895660    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4d2ce863-958d-49c0-87a6-44d023c22b56-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-dr4ch\" (UID: \"4d2ce863-958d-49c0-87a6-44d023c22b56\") " pod="cert-manager/cert-manager-webhook-6486ffd997-dr4ch"
Sep 19 05:44:50.906894 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:50.906864    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4d2ce863-958d-49c0-87a6-44d023c22b56-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-dr4ch\" (UID: \"4d2ce863-958d-49c0-87a6-44d023c22b56\") " pod="cert-manager/cert-manager-webhook-6486ffd997-dr4ch"
Sep 19 05:44:50.906894 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:50.906887    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r8lsz\" (UniqueName: \"kubernetes.io/projected/4d2ce863-958d-49c0-87a6-44d023c22b56-kube-api-access-r8lsz\") pod \"cert-manager-webhook-6486ffd997-dr4ch\" (UID: \"4d2ce863-958d-49c0-87a6-44d023c22b56\") " pod="cert-manager/cert-manager-webhook-6486ffd997-dr4ch"
Sep 19 05:44:50.931917 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:50.931891    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-dr4ch"
Sep 19 05:44:51.215683 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:51.215607    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-dr4ch"]
Sep 19 05:44:51.218688 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:44:51.218649    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4d2ce863_958d_49c0_87a6_44d023c22b56.slice/crio-438775b5b867739b3df30000a3ef3d33467b549814d7bb18030b8d9b43b931c5 WatchSource:0}: Error finding container 438775b5b867739b3df30000a3ef3d33467b549814d7bb18030b8d9b43b931c5: Status 404 returned error can't find the container with id 438775b5b867739b3df30000a3ef3d33467b549814d7bb18030b8d9b43b931c5
Sep 19 05:44:51.686937 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:51.686906    2583 generic.go:356] "Generic (PLEG): container finished" podID="4d52936d-f73a-4bf0-b76b-f48ed87239ad" containerID="d8972bc3a2ae3a306efb1582c85ae43fcceb8edb4fd61b20de2f2063441fa98b" exitCode=0
Sep 19 05:44:51.687352 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:51.687005    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fg7k48" event={"ID":"4d52936d-f73a-4bf0-b76b-f48ed87239ad","Type":"ContainerDied","Data":"d8972bc3a2ae3a306efb1582c85ae43fcceb8edb4fd61b20de2f2063441fa98b"}
Sep 19 05:44:51.688505 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:51.688483    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-dr4ch" event={"ID":"4d2ce863-958d-49c0-87a6-44d023c22b56","Type":"ContainerStarted","Data":"e73a116b62e94a9dc5238f1876ea940730143d45c64c8efb2c03e94c863c7fb8"}
Sep 19 05:44:51.688551 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:51.688515    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-dr4ch" event={"ID":"4d2ce863-958d-49c0-87a6-44d023c22b56","Type":"ContainerStarted","Data":"438775b5b867739b3df30000a3ef3d33467b549814d7bb18030b8d9b43b931c5"}
Sep 19 05:44:52.692974 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:52.692946    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-dr4ch"
Sep 19 05:44:52.816686 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:52.816662    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fg7k48"
Sep 19 05:44:52.851259 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:52.851203    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-dr4ch" podStartSLOduration=2.851185202 podStartE2EDuration="2.851185202s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:44:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:44:51.769303283 +0000 UTC m=+250.427757842" watchObservedRunningTime="2026-09-19 05:44:52.851185202 +0000 UTC m=+251.509639760"
Sep 19 05:44:52.910574 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:52.910534    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mmr4c\" (UniqueName: \"kubernetes.io/projected/4d52936d-f73a-4bf0-b76b-f48ed87239ad-kube-api-access-mmr4c\") pod \"4d52936d-f73a-4bf0-b76b-f48ed87239ad\" (UID: \"4d52936d-f73a-4bf0-b76b-f48ed87239ad\") "
Sep 19 05:44:52.910761 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:52.910597    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4d52936d-f73a-4bf0-b76b-f48ed87239ad-util\") pod \"4d52936d-f73a-4bf0-b76b-f48ed87239ad\" (UID: \"4d52936d-f73a-4bf0-b76b-f48ed87239ad\") "
Sep 19 05:44:52.910761 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:52.910618    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4d52936d-f73a-4bf0-b76b-f48ed87239ad-bundle\") pod \"4d52936d-f73a-4bf0-b76b-f48ed87239ad\" (UID: \"4d52936d-f73a-4bf0-b76b-f48ed87239ad\") "
Sep 19 05:44:52.911054 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:52.911029    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d52936d-f73a-4bf0-b76b-f48ed87239ad-bundle" (OuterVolumeSpecName: "bundle") pod "4d52936d-f73a-4bf0-b76b-f48ed87239ad" (UID: "4d52936d-f73a-4bf0-b76b-f48ed87239ad"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 05:44:52.912788 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:52.912750    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4d52936d-f73a-4bf0-b76b-f48ed87239ad-kube-api-access-mmr4c" (OuterVolumeSpecName: "kube-api-access-mmr4c") pod "4d52936d-f73a-4bf0-b76b-f48ed87239ad" (UID: "4d52936d-f73a-4bf0-b76b-f48ed87239ad"). InnerVolumeSpecName "kube-api-access-mmr4c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 05:44:52.915117 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:52.915088    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d52936d-f73a-4bf0-b76b-f48ed87239ad-util" (OuterVolumeSpecName: "util") pod "4d52936d-f73a-4bf0-b76b-f48ed87239ad" (UID: "4d52936d-f73a-4bf0-b76b-f48ed87239ad"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 05:44:53.011364 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:53.011284    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mmr4c\" (UniqueName: \"kubernetes.io/projected/4d52936d-f73a-4bf0-b76b-f48ed87239ad-kube-api-access-mmr4c\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 05:44:53.011364 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:53.011315    2583 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4d52936d-f73a-4bf0-b76b-f48ed87239ad-util\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 05:44:53.011364 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:53.011324    2583 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4d52936d-f73a-4bf0-b76b-f48ed87239ad-bundle\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 05:44:53.698748 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:53.698715    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fg7k48"
Sep 19 05:44:53.698748 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:53.698719    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fg7k48" event={"ID":"4d52936d-f73a-4bf0-b76b-f48ed87239ad","Type":"ContainerDied","Data":"c188a1aa546581c7f6b192b5f15c33bc46d90f3f60f3b278a8676f5411ab9576"}
Sep 19 05:44:53.699302 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:53.698775    2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c188a1aa546581c7f6b192b5f15c33bc46d90f3f60f3b278a8676f5411ab9576"
Sep 19 05:44:58.700914 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:44:58.700834    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-dr4ch"
Sep 19 05:45:04.030090 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:04.030049    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6b2rc"]
Sep 19 05:45:04.030588 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:04.030458    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d52936d-f73a-4bf0-b76b-f48ed87239ad" containerName="util"
Sep 19 05:45:04.030588 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:04.030476    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d52936d-f73a-4bf0-b76b-f48ed87239ad" containerName="util"
Sep 19 05:45:04.030588 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:04.030491    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d52936d-f73a-4bf0-b76b-f48ed87239ad" containerName="extract"
Sep 19 05:45:04.030588 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:04.030499    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d52936d-f73a-4bf0-b76b-f48ed87239ad" containerName="extract"
Sep 19 05:45:04.030588 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:04.030511    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d52936d-f73a-4bf0-b76b-f48ed87239ad" containerName="pull"
Sep 19 05:45:04.030588 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:04.030518    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d52936d-f73a-4bf0-b76b-f48ed87239ad" containerName="pull"
Sep 19 05:45:04.030885 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:04.030593    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="4d52936d-f73a-4bf0-b76b-f48ed87239ad" containerName="extract"
Sep 19 05:45:04.032579 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:04.032557    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6b2rc"
Sep 19 05:45:04.035256 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:04.035233    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-vqxsl\""
Sep 19 05:45:04.035329 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:04.035289    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 19 05:45:04.036174 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:04.036154    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 19 05:45:04.046576 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:04.046552    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6b2rc"]
Sep 19 05:45:04.193419 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:04.193384    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4ee6b0e2-6c39-4935-bf51-f555cdc01636-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6b2rc\" (UID: \"4ee6b0e2-6c39-4935-bf51-f555cdc01636\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6b2rc"
Sep 19 05:45:04.193419 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:04.193421    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4ee6b0e2-6c39-4935-bf51-f555cdc01636-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6b2rc\" (UID: \"4ee6b0e2-6c39-4935-bf51-f555cdc01636\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6b2rc"
Sep 19 05:45:04.193608 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:04.193448    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xpmn8\" (UniqueName: \"kubernetes.io/projected/4ee6b0e2-6c39-4935-bf51-f555cdc01636-kube-api-access-xpmn8\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6b2rc\" (UID: \"4ee6b0e2-6c39-4935-bf51-f555cdc01636\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6b2rc"
Sep 19 05:45:04.294456 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:04.294373    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4ee6b0e2-6c39-4935-bf51-f555cdc01636-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6b2rc\" (UID: \"4ee6b0e2-6c39-4935-bf51-f555cdc01636\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6b2rc"
Sep 19 05:45:04.294456 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:04.294415    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4ee6b0e2-6c39-4935-bf51-f555cdc01636-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6b2rc\" (UID: \"4ee6b0e2-6c39-4935-bf51-f555cdc01636\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6b2rc"
Sep 19 05:45:04.294456 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:04.294443    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xpmn8\" (UniqueName: \"kubernetes.io/projected/4ee6b0e2-6c39-4935-bf51-f555cdc01636-kube-api-access-xpmn8\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6b2rc\" (UID: \"4ee6b0e2-6c39-4935-bf51-f555cdc01636\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6b2rc"
Sep 19 05:45:04.294843 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:04.294820    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4ee6b0e2-6c39-4935-bf51-f555cdc01636-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6b2rc\" (UID: \"4ee6b0e2-6c39-4935-bf51-f555cdc01636\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6b2rc"
Sep 19 05:45:04.294878 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:04.294827    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4ee6b0e2-6c39-4935-bf51-f555cdc01636-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6b2rc\" (UID: \"4ee6b0e2-6c39-4935-bf51-f555cdc01636\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6b2rc"
Sep 19 05:45:04.303532 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:04.303503    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xpmn8\" (UniqueName: \"kubernetes.io/projected/4ee6b0e2-6c39-4935-bf51-f555cdc01636-kube-api-access-xpmn8\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6b2rc\" (UID: \"4ee6b0e2-6c39-4935-bf51-f555cdc01636\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6b2rc"
Sep 19 05:45:04.341956 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:04.341934    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6b2rc"
Sep 19 05:45:04.613943 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:04.613916    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6b2rc"]
Sep 19 05:45:04.616321 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:45:04.616293    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ee6b0e2_6c39_4935_bf51_f555cdc01636.slice/crio-ce3b6b34f1bb93c56b4c3aa7ba9f3f1e3c244b4f183166b95617bb654519d2b8 WatchSource:0}: Error finding container ce3b6b34f1bb93c56b4c3aa7ba9f3f1e3c244b4f183166b95617bb654519d2b8: Status 404 returned error can't find the container with id ce3b6b34f1bb93c56b4c3aa7ba9f3f1e3c244b4f183166b95617bb654519d2b8
Sep 19 05:45:04.744855 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:04.744818    2583 generic.go:356] "Generic (PLEG): container finished" podID="4ee6b0e2-6c39-4935-bf51-f555cdc01636" containerID="1d65f4afc6c8bf001f8f79749c412be22499685bf86256c926d52f9350ba13f1" exitCode=0
Sep 19 05:45:04.745002 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:04.744908    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6b2rc" event={"ID":"4ee6b0e2-6c39-4935-bf51-f555cdc01636","Type":"ContainerDied","Data":"1d65f4afc6c8bf001f8f79749c412be22499685bf86256c926d52f9350ba13f1"}
Sep 19 05:45:04.745002 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:04.744970    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6b2rc" event={"ID":"4ee6b0e2-6c39-4935-bf51-f555cdc01636","Type":"ContainerStarted","Data":"ce3b6b34f1bb93c56b4c3aa7ba9f3f1e3c244b4f183166b95617bb654519d2b8"}
Sep 19 05:45:05.750741 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:05.750707    2583 generic.go:356] "Generic (PLEG): container finished" podID="4ee6b0e2-6c39-4935-bf51-f555cdc01636" containerID="607ecc8cdb32f9a653e9ce0f1926fcf327008d76b247bd3ab1227c89cd2657ca" exitCode=0
Sep 19 05:45:05.751091 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:05.750790    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6b2rc" event={"ID":"4ee6b0e2-6c39-4935-bf51-f555cdc01636","Type":"ContainerDied","Data":"607ecc8cdb32f9a653e9ce0f1926fcf327008d76b247bd3ab1227c89cd2657ca"}
Sep 19 05:45:06.757278 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:06.757246    2583 generic.go:356] "Generic (PLEG): container finished" podID="4ee6b0e2-6c39-4935-bf51-f555cdc01636" containerID="ea0f7b2066e7544b2a8c31502c7bd8b5df2f0245b7181a5076f2a5896fd60806" exitCode=0
Sep 19 05:45:06.757668 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:06.757311    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6b2rc" event={"ID":"4ee6b0e2-6c39-4935-bf51-f555cdc01636","Type":"ContainerDied","Data":"ea0f7b2066e7544b2a8c31502c7bd8b5df2f0245b7181a5076f2a5896fd60806"}
Sep 19 05:45:07.881623 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:07.881601    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6b2rc"
Sep 19 05:45:07.921174 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:07.921148    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xpmn8\" (UniqueName: \"kubernetes.io/projected/4ee6b0e2-6c39-4935-bf51-f555cdc01636-kube-api-access-xpmn8\") pod \"4ee6b0e2-6c39-4935-bf51-f555cdc01636\" (UID: \"4ee6b0e2-6c39-4935-bf51-f555cdc01636\") "
Sep 19 05:45:07.921304 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:07.921193    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4ee6b0e2-6c39-4935-bf51-f555cdc01636-util\") pod \"4ee6b0e2-6c39-4935-bf51-f555cdc01636\" (UID: \"4ee6b0e2-6c39-4935-bf51-f555cdc01636\") "
Sep 19 05:45:07.921304 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:07.921210    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4ee6b0e2-6c39-4935-bf51-f555cdc01636-bundle\") pod \"4ee6b0e2-6c39-4935-bf51-f555cdc01636\" (UID: \"4ee6b0e2-6c39-4935-bf51-f555cdc01636\") "
Sep 19 05:45:07.921892 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:07.921870    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4ee6b0e2-6c39-4935-bf51-f555cdc01636-bundle" (OuterVolumeSpecName: "bundle") pod "4ee6b0e2-6c39-4935-bf51-f555cdc01636" (UID: "4ee6b0e2-6c39-4935-bf51-f555cdc01636"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 05:45:07.923121 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:07.923098    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4ee6b0e2-6c39-4935-bf51-f555cdc01636-kube-api-access-xpmn8" (OuterVolumeSpecName: "kube-api-access-xpmn8") pod "4ee6b0e2-6c39-4935-bf51-f555cdc01636" (UID: "4ee6b0e2-6c39-4935-bf51-f555cdc01636"). InnerVolumeSpecName "kube-api-access-xpmn8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 05:45:07.927071 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:07.927019    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4ee6b0e2-6c39-4935-bf51-f555cdc01636-util" (OuterVolumeSpecName: "util") pod "4ee6b0e2-6c39-4935-bf51-f555cdc01636" (UID: "4ee6b0e2-6c39-4935-bf51-f555cdc01636"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 05:45:08.022155 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:08.022036    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xpmn8\" (UniqueName: \"kubernetes.io/projected/4ee6b0e2-6c39-4935-bf51-f555cdc01636-kube-api-access-xpmn8\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 05:45:08.022155 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:08.022079    2583 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4ee6b0e2-6c39-4935-bf51-f555cdc01636-util\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 05:45:08.022155 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:08.022094    2583 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4ee6b0e2-6c39-4935-bf51-f555cdc01636-bundle\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 05:45:08.768054 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:08.768015    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6b2rc" event={"ID":"4ee6b0e2-6c39-4935-bf51-f555cdc01636","Type":"ContainerDied","Data":"ce3b6b34f1bb93c56b4c3aa7ba9f3f1e3c244b4f183166b95617bb654519d2b8"}
Sep 19 05:45:08.768217 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:08.768058    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6b2rc"
Sep 19 05:45:08.768217 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:08.768068    2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ce3b6b34f1bb93c56b4c3aa7ba9f3f1e3c244b4f183166b95617bb654519d2b8"
Sep 19 05:45:12.250971 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:12.250936    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-58cf5d9b8c-jsq77"]
Sep 19 05:45:12.251582 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:12.251346    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4ee6b0e2-6c39-4935-bf51-f555cdc01636" containerName="util"
Sep 19 05:45:12.251582 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:12.251364    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="4ee6b0e2-6c39-4935-bf51-f555cdc01636" containerName="util"
Sep 19 05:45:12.251582 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:12.251386    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4ee6b0e2-6c39-4935-bf51-f555cdc01636" containerName="pull"
Sep 19 05:45:12.251582 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:12.251394    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="4ee6b0e2-6c39-4935-bf51-f555cdc01636" containerName="pull"
Sep 19 05:45:12.251582 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:12.251412    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4ee6b0e2-6c39-4935-bf51-f555cdc01636" containerName="extract"
Sep 19 05:45:12.251582 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:12.251421    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="4ee6b0e2-6c39-4935-bf51-f555cdc01636" containerName="extract"
Sep 19 05:45:12.251582 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:12.251495    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="4ee6b0e2-6c39-4935-bf51-f555cdc01636" containerName="extract"
Sep 19 05:45:12.254865 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:12.254843    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-58cf5d9b8c-jsq77"
Sep 19 05:45:12.259358 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:12.259334    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 19 05:45:12.259478 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:12.259441    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Sep 19 05:45:12.259574 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:12.259555    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Sep 19 05:45:12.260400 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:12.260382    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 19 05:45:12.260517 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:12.260499    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Sep 19 05:45:12.260583 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:12.260544    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-g7d5s\""
Sep 19 05:45:12.267324 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:12.267305    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-58cf5d9b8c-jsq77"]
Sep 19 05:45:12.354847 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:12.354815    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/32b98bc1-5450-4de6-a3a4-0bce0a8e9371-manager-config\") pod \"lws-controller-manager-58cf5d9b8c-jsq77\" (UID: \"32b98bc1-5450-4de6-a3a4-0bce0a8e9371\") " pod="openshift-lws-operator/lws-controller-manager-58cf5d9b8c-jsq77"
Sep 19 05:45:12.355027 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:12.354854    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/32b98bc1-5450-4de6-a3a4-0bce0a8e9371-cert\") pod \"lws-controller-manager-58cf5d9b8c-jsq77\" (UID: \"32b98bc1-5450-4de6-a3a4-0bce0a8e9371\") " pod="openshift-lws-operator/lws-controller-manager-58cf5d9b8c-jsq77"
Sep 19 05:45:12.355027 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:12.354961    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vbzzb\" (UniqueName: \"kubernetes.io/projected/32b98bc1-5450-4de6-a3a4-0bce0a8e9371-kube-api-access-vbzzb\") pod \"lws-controller-manager-58cf5d9b8c-jsq77\" (UID: \"32b98bc1-5450-4de6-a3a4-0bce0a8e9371\") " pod="openshift-lws-operator/lws-controller-manager-58cf5d9b8c-jsq77"
Sep 19 05:45:12.355027 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:12.354988    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/32b98bc1-5450-4de6-a3a4-0bce0a8e9371-metrics-cert\") pod \"lws-controller-manager-58cf5d9b8c-jsq77\" (UID: \"32b98bc1-5450-4de6-a3a4-0bce0a8e9371\") " pod="openshift-lws-operator/lws-controller-manager-58cf5d9b8c-jsq77"
Sep 19 05:45:12.456328 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:12.456289    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/32b98bc1-5450-4de6-a3a4-0bce0a8e9371-manager-config\") pod \"lws-controller-manager-58cf5d9b8c-jsq77\" (UID: \"32b98bc1-5450-4de6-a3a4-0bce0a8e9371\") " pod="openshift-lws-operator/lws-controller-manager-58cf5d9b8c-jsq77"
Sep 19 05:45:12.456328 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:12.456330    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/32b98bc1-5450-4de6-a3a4-0bce0a8e9371-cert\") pod \"lws-controller-manager-58cf5d9b8c-jsq77\" (UID: \"32b98bc1-5450-4de6-a3a4-0bce0a8e9371\") " pod="openshift-lws-operator/lws-controller-manager-58cf5d9b8c-jsq77"
Sep 19 05:45:12.456576 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:12.456400    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vbzzb\" (UniqueName: \"kubernetes.io/projected/32b98bc1-5450-4de6-a3a4-0bce0a8e9371-kube-api-access-vbzzb\") pod \"lws-controller-manager-58cf5d9b8c-jsq77\" (UID: \"32b98bc1-5450-4de6-a3a4-0bce0a8e9371\") " pod="openshift-lws-operator/lws-controller-manager-58cf5d9b8c-jsq77"
Sep 19 05:45:12.456576 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:12.456424    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/32b98bc1-5450-4de6-a3a4-0bce0a8e9371-metrics-cert\") pod \"lws-controller-manager-58cf5d9b8c-jsq77\" (UID: \"32b98bc1-5450-4de6-a3a4-0bce0a8e9371\") " pod="openshift-lws-operator/lws-controller-manager-58cf5d9b8c-jsq77"
Sep 19 05:45:12.456953 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:12.456920    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/32b98bc1-5450-4de6-a3a4-0bce0a8e9371-manager-config\") pod \"lws-controller-manager-58cf5d9b8c-jsq77\" (UID: \"32b98bc1-5450-4de6-a3a4-0bce0a8e9371\") " pod="openshift-lws-operator/lws-controller-manager-58cf5d9b8c-jsq77"
Sep 19 05:45:12.458866 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:12.458847    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/32b98bc1-5450-4de6-a3a4-0bce0a8e9371-metrics-cert\") pod \"lws-controller-manager-58cf5d9b8c-jsq77\" (UID: \"32b98bc1-5450-4de6-a3a4-0bce0a8e9371\") " pod="openshift-lws-operator/lws-controller-manager-58cf5d9b8c-jsq77"
Sep 19 05:45:12.459046 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:12.459023    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/32b98bc1-5450-4de6-a3a4-0bce0a8e9371-cert\") pod \"lws-controller-manager-58cf5d9b8c-jsq77\" (UID: \"32b98bc1-5450-4de6-a3a4-0bce0a8e9371\") " pod="openshift-lws-operator/lws-controller-manager-58cf5d9b8c-jsq77"
Sep 19 05:45:12.464883 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:12.464854    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vbzzb\" (UniqueName: \"kubernetes.io/projected/32b98bc1-5450-4de6-a3a4-0bce0a8e9371-kube-api-access-vbzzb\") pod \"lws-controller-manager-58cf5d9b8c-jsq77\" (UID: \"32b98bc1-5450-4de6-a3a4-0bce0a8e9371\") " pod="openshift-lws-operator/lws-controller-manager-58cf5d9b8c-jsq77"
Sep 19 05:45:12.564394 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:12.564362    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-58cf5d9b8c-jsq77"
Sep 19 05:45:12.848892 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:12.848814    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-58cf5d9b8c-jsq77"]
Sep 19 05:45:12.851866 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:45:12.851834    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32b98bc1_5450_4de6_a3a4_0bce0a8e9371.slice/crio-20a4cb5a399fffd93a31e2b3aa04039d52ec22300568647d45a4ae922894d2db WatchSource:0}: Error finding container 20a4cb5a399fffd93a31e2b3aa04039d52ec22300568647d45a4ae922894d2db: Status 404 returned error can't find the container with id 20a4cb5a399fffd93a31e2b3aa04039d52ec22300568647d45a4ae922894d2db
Sep 19 05:45:13.792852 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:13.792774    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-58cf5d9b8c-jsq77" event={"ID":"32b98bc1-5450-4de6-a3a4-0bce0a8e9371","Type":"ContainerStarted","Data":"20a4cb5a399fffd93a31e2b3aa04039d52ec22300568647d45a4ae922894d2db"}
Sep 19 05:45:14.799213 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:14.799176    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-58cf5d9b8c-jsq77" event={"ID":"32b98bc1-5450-4de6-a3a4-0bce0a8e9371","Type":"ContainerStarted","Data":"a02778a3114826b14c23f746de220100d57105d2988c5ae6d9eff87449e987d4"}
Sep 19 05:45:14.799613 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:14.799277    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-58cf5d9b8c-jsq77"
Sep 19 05:45:14.833641 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:14.833594    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-58cf5d9b8c-jsq77" podStartSLOduration=1.670424704 podStartE2EDuration="2.833579078s" totalImagesPullingTime="1.163154374s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:45:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:45:14.831270613 +0000 UTC m=+273.489725173" watchObservedRunningTime="2026-09-19 05:45:14.833579078 +0000 UTC m=+273.492033640"
Sep 19 05:45:21.262966 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:21.262924    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-fd6794c5b-bptml"]
Sep 19 05:45:21.266572 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:21.266550    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-fd6794c5b-bptml"
Sep 19 05:45:21.278006 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:21.277985    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-s27qb\""
Sep 19 05:45:21.278108 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:21.277985    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 19 05:45:21.278313 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:21.278297    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 19 05:45:21.278734 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:21.278721    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 19 05:45:21.278917 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:21.278905    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 19 05:45:21.320025 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:21.320001    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-fd6794c5b-bptml"]
Sep 19 05:45:21.326319 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:21.326297    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/41b05291-5cbe-4649-9cf3-355202e312d1-webhook-cert\") pod \"opendatahub-operator-controller-manager-fd6794c5b-bptml\" (UID: \"41b05291-5cbe-4649-9cf3-355202e312d1\") " pod="opendatahub/opendatahub-operator-controller-manager-fd6794c5b-bptml"
Sep 19 05:45:21.326405 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:21.326330    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m7b8n\" (UniqueName: \"kubernetes.io/projected/41b05291-5cbe-4649-9cf3-355202e312d1-kube-api-access-m7b8n\") pod \"opendatahub-operator-controller-manager-fd6794c5b-bptml\" (UID: \"41b05291-5cbe-4649-9cf3-355202e312d1\") " pod="opendatahub/opendatahub-operator-controller-manager-fd6794c5b-bptml"
Sep 19 05:45:21.326405 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:21.326351    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/41b05291-5cbe-4649-9cf3-355202e312d1-manifests\") pod \"opendatahub-operator-controller-manager-fd6794c5b-bptml\" (UID: \"41b05291-5cbe-4649-9cf3-355202e312d1\") " pod="opendatahub/opendatahub-operator-controller-manager-fd6794c5b-bptml"
Sep 19 05:45:21.326405 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:21.326368    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/41b05291-5cbe-4649-9cf3-355202e312d1-tmp\") pod \"opendatahub-operator-controller-manager-fd6794c5b-bptml\" (UID: \"41b05291-5cbe-4649-9cf3-355202e312d1\") " pod="opendatahub/opendatahub-operator-controller-manager-fd6794c5b-bptml"
Sep 19 05:45:21.326516 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:21.326431    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/41b05291-5cbe-4649-9cf3-355202e312d1-apiservice-cert\") pod \"opendatahub-operator-controller-manager-fd6794c5b-bptml\" (UID: \"41b05291-5cbe-4649-9cf3-355202e312d1\") " pod="opendatahub/opendatahub-operator-controller-manager-fd6794c5b-bptml"
Sep 19 05:45:21.427474 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:21.427435    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/41b05291-5cbe-4649-9cf3-355202e312d1-webhook-cert\") pod \"opendatahub-operator-controller-manager-fd6794c5b-bptml\" (UID: \"41b05291-5cbe-4649-9cf3-355202e312d1\") " pod="opendatahub/opendatahub-operator-controller-manager-fd6794c5b-bptml"
Sep 19 05:45:21.427474 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:21.427473    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m7b8n\" (UniqueName: \"kubernetes.io/projected/41b05291-5cbe-4649-9cf3-355202e312d1-kube-api-access-m7b8n\") pod \"opendatahub-operator-controller-manager-fd6794c5b-bptml\" (UID: \"41b05291-5cbe-4649-9cf3-355202e312d1\") " pod="opendatahub/opendatahub-operator-controller-manager-fd6794c5b-bptml"
Sep 19 05:45:21.427672 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:21.427494    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/41b05291-5cbe-4649-9cf3-355202e312d1-manifests\") pod \"opendatahub-operator-controller-manager-fd6794c5b-bptml\" (UID: \"41b05291-5cbe-4649-9cf3-355202e312d1\") " pod="opendatahub/opendatahub-operator-controller-manager-fd6794c5b-bptml"
Sep 19 05:45:21.427672 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:21.427509    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/41b05291-5cbe-4649-9cf3-355202e312d1-tmp\") pod \"opendatahub-operator-controller-manager-fd6794c5b-bptml\" (UID: \"41b05291-5cbe-4649-9cf3-355202e312d1\") " pod="opendatahub/opendatahub-operator-controller-manager-fd6794c5b-bptml"
Sep 19 05:45:21.427672 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:21.427543    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/41b05291-5cbe-4649-9cf3-355202e312d1-apiservice-cert\") pod \"opendatahub-operator-controller-manager-fd6794c5b-bptml\" (UID: \"41b05291-5cbe-4649-9cf3-355202e312d1\") " pod="opendatahub/opendatahub-operator-controller-manager-fd6794c5b-bptml"
Sep 19 05:45:21.427956 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:21.427925    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/41b05291-5cbe-4649-9cf3-355202e312d1-manifests\") pod \"opendatahub-operator-controller-manager-fd6794c5b-bptml\" (UID: \"41b05291-5cbe-4649-9cf3-355202e312d1\") " pod="opendatahub/opendatahub-operator-controller-manager-fd6794c5b-bptml"
Sep 19 05:45:21.429938 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:21.429909    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/41b05291-5cbe-4649-9cf3-355202e312d1-tmp\") pod \"opendatahub-operator-controller-manager-fd6794c5b-bptml\" (UID: \"41b05291-5cbe-4649-9cf3-355202e312d1\") " pod="opendatahub/opendatahub-operator-controller-manager-fd6794c5b-bptml"
Sep 19 05:45:21.430184 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:21.430166    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/41b05291-5cbe-4649-9cf3-355202e312d1-webhook-cert\") pod \"opendatahub-operator-controller-manager-fd6794c5b-bptml\" (UID: \"41b05291-5cbe-4649-9cf3-355202e312d1\") " pod="opendatahub/opendatahub-operator-controller-manager-fd6794c5b-bptml"
Sep 19 05:45:21.430234 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:21.430185    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/41b05291-5cbe-4649-9cf3-355202e312d1-apiservice-cert\") pod \"opendatahub-operator-controller-manager-fd6794c5b-bptml\" (UID: \"41b05291-5cbe-4649-9cf3-355202e312d1\") " pod="opendatahub/opendatahub-operator-controller-manager-fd6794c5b-bptml"
Sep 19 05:45:21.437355 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:21.437325    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m7b8n\" (UniqueName: \"kubernetes.io/projected/41b05291-5cbe-4649-9cf3-355202e312d1-kube-api-access-m7b8n\") pod \"opendatahub-operator-controller-manager-fd6794c5b-bptml\" (UID: \"41b05291-5cbe-4649-9cf3-355202e312d1\") " pod="opendatahub/opendatahub-operator-controller-manager-fd6794c5b-bptml"
Sep 19 05:45:21.484846 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:21.484815    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0fbqlj"]
Sep 19 05:45:21.488292 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:21.488277    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0fbqlj"
Sep 19 05:45:21.490911 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:21.490891    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 19 05:45:21.491072 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:21.490895    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-vqxsl\""
Sep 19 05:45:21.491256 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:21.491240    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 19 05:45:21.500275 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:21.500247    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0fbqlj"]
Sep 19 05:45:21.528405 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:21.528383    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fc639673-c5c1-4bf0-996d-64370a296a9c-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0fbqlj\" (UID: \"fc639673-c5c1-4bf0-996d-64370a296a9c\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0fbqlj"
Sep 19 05:45:21.528523 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:21.528428    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9bcrz\" (UniqueName: \"kubernetes.io/projected/fc639673-c5c1-4bf0-996d-64370a296a9c-kube-api-access-9bcrz\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0fbqlj\" (UID: \"fc639673-c5c1-4bf0-996d-64370a296a9c\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0fbqlj"
Sep 19 05:45:21.528523 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:21.528461    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fc639673-c5c1-4bf0-996d-64370a296a9c-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0fbqlj\" (UID: \"fc639673-c5c1-4bf0-996d-64370a296a9c\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0fbqlj"
Sep 19 05:45:21.576457 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:21.576427    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-fd6794c5b-bptml"
Sep 19 05:45:21.629664 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:21.629629    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9bcrz\" (UniqueName: \"kubernetes.io/projected/fc639673-c5c1-4bf0-996d-64370a296a9c-kube-api-access-9bcrz\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0fbqlj\" (UID: \"fc639673-c5c1-4bf0-996d-64370a296a9c\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0fbqlj"
Sep 19 05:45:21.629812 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:21.629697    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fc639673-c5c1-4bf0-996d-64370a296a9c-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0fbqlj\" (UID: \"fc639673-c5c1-4bf0-996d-64370a296a9c\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0fbqlj"
Sep 19 05:45:21.629812 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:21.629788    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fc639673-c5c1-4bf0-996d-64370a296a9c-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0fbqlj\" (UID: \"fc639673-c5c1-4bf0-996d-64370a296a9c\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0fbqlj"
Sep 19 05:45:21.630522 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:21.630496    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fc639673-c5c1-4bf0-996d-64370a296a9c-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0fbqlj\" (UID: \"fc639673-c5c1-4bf0-996d-64370a296a9c\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0fbqlj"
Sep 19 05:45:21.630522 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:21.630510    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fc639673-c5c1-4bf0-996d-64370a296a9c-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0fbqlj\" (UID: \"fc639673-c5c1-4bf0-996d-64370a296a9c\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0fbqlj"
Sep 19 05:45:21.657323 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:21.657291    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9bcrz\" (UniqueName: \"kubernetes.io/projected/fc639673-c5c1-4bf0-996d-64370a296a9c-kube-api-access-9bcrz\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0fbqlj\" (UID: \"fc639673-c5c1-4bf0-996d-64370a296a9c\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0fbqlj"
Sep 19 05:45:21.797908 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:21.797829    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0fbqlj"
Sep 19 05:45:21.865704 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:21.865681    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-fd6794c5b-bptml"]
Sep 19 05:45:21.868405 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:45:21.868376    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41b05291_5cbe_4649_9cf3_355202e312d1.slice/crio-b2e7adc9ee08615c16fbcfd96267a212d8afc225452e78d159449921c2335a6b WatchSource:0}: Error finding container b2e7adc9ee08615c16fbcfd96267a212d8afc225452e78d159449921c2335a6b: Status 404 returned error can't find the container with id b2e7adc9ee08615c16fbcfd96267a212d8afc225452e78d159449921c2335a6b
Sep 19 05:45:22.084109 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:22.084026    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0fbqlj"]
Sep 19 05:45:22.087637 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:45:22.087608    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc639673_c5c1_4bf0_996d_64370a296a9c.slice/crio-00e1cdca703e2fe12b750436752278c575f10e16d7865d744d51acaed6034918 WatchSource:0}: Error finding container 00e1cdca703e2fe12b750436752278c575f10e16d7865d744d51acaed6034918: Status 404 returned error can't find the container with id 00e1cdca703e2fe12b750436752278c575f10e16d7865d744d51acaed6034918
Sep 19 05:45:22.835853 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:22.835770    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-fd6794c5b-bptml" event={"ID":"41b05291-5cbe-4649-9cf3-355202e312d1","Type":"ContainerStarted","Data":"b2e7adc9ee08615c16fbcfd96267a212d8afc225452e78d159449921c2335a6b"}
Sep 19 05:45:22.838056 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:22.838020    2583 generic.go:356] "Generic (PLEG): container finished" podID="fc639673-c5c1-4bf0-996d-64370a296a9c" containerID="d7256a927faa7f8719c22a9d8963567b0c93134e398e23b555cfee6ff9f4b037" exitCode=0
Sep 19 05:45:22.838218 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:22.838076    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0fbqlj" event={"ID":"fc639673-c5c1-4bf0-996d-64370a296a9c","Type":"ContainerDied","Data":"d7256a927faa7f8719c22a9d8963567b0c93134e398e23b555cfee6ff9f4b037"}
Sep 19 05:45:22.838218 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:22.838109    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0fbqlj" event={"ID":"fc639673-c5c1-4bf0-996d-64370a296a9c","Type":"ContainerStarted","Data":"00e1cdca703e2fe12b750436752278c575f10e16d7865d744d51acaed6034918"}
Sep 19 05:45:25.805654 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:25.805624    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-58cf5d9b8c-jsq77"
Sep 19 05:45:25.852572 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:25.852543    2583 generic.go:356] "Generic (PLEG): container finished" podID="fc639673-c5c1-4bf0-996d-64370a296a9c" containerID="8e4378f233b8f68bcb818b6dda9ba98cc4c953d5167e3df1b6ee7637a5f9e01b" exitCode=0
Sep 19 05:45:25.852754 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:25.852633    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0fbqlj" event={"ID":"fc639673-c5c1-4bf0-996d-64370a296a9c","Type":"ContainerDied","Data":"8e4378f233b8f68bcb818b6dda9ba98cc4c953d5167e3df1b6ee7637a5f9e01b"}
Sep 19 05:45:25.854654 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:25.854629    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-fd6794c5b-bptml" event={"ID":"41b05291-5cbe-4649-9cf3-355202e312d1","Type":"ContainerStarted","Data":"4fb85e6270160fa9369f8ad96791872401cacc9f948a67e13f65ae3982c64830"}
Sep 19 05:45:26.860684 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:26.860649    2583 generic.go:356] "Generic (PLEG): container finished" podID="41b05291-5cbe-4649-9cf3-355202e312d1" containerID="4fb85e6270160fa9369f8ad96791872401cacc9f948a67e13f65ae3982c64830" exitCode=0
Sep 19 05:45:26.861195 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:26.860739    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-fd6794c5b-bptml" event={"ID":"41b05291-5cbe-4649-9cf3-355202e312d1","Type":"ContainerDied","Data":"4fb85e6270160fa9369f8ad96791872401cacc9f948a67e13f65ae3982c64830"}
Sep 19 05:45:26.862990 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:26.862967    2583 generic.go:356] "Generic (PLEG): container finished" podID="fc639673-c5c1-4bf0-996d-64370a296a9c" containerID="593efcfd6021dfc5430dcc8afc162ebf52b23c508e46cf6206f4ed57a9a3caa2" exitCode=0
Sep 19 05:45:26.863087 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:26.863003    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0fbqlj" event={"ID":"fc639673-c5c1-4bf0-996d-64370a296a9c","Type":"ContainerDied","Data":"593efcfd6021dfc5430dcc8afc162ebf52b23c508e46cf6206f4ed57a9a3caa2"}
Sep 19 05:45:27.869238 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:27.869200    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-fd6794c5b-bptml" event={"ID":"41b05291-5cbe-4649-9cf3-355202e312d1","Type":"ContainerStarted","Data":"007d2337c7ed562ceac78ed4a33ec05a298e6ca63e1c4b202f0c9faea2ebe3e3"}
Sep 19 05:45:27.899341 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:27.899284    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-fd6794c5b-bptml" podStartSLOduration=2.585428776 podStartE2EDuration="6.89926748s" totalImagesPullingTime="3.298918483s" totalInitContainerRuntime="1.014920221s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:45:21 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:45:27.896323141 +0000 UTC m=+286.554777726" watchObservedRunningTime="2026-09-19 05:45:27.89926748 +0000 UTC m=+286.557722037"
Sep 19 05:45:27.993779 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:27.993753    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0fbqlj"
Sep 19 05:45:28.085079 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:28.085048    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9bcrz\" (UniqueName: \"kubernetes.io/projected/fc639673-c5c1-4bf0-996d-64370a296a9c-kube-api-access-9bcrz\") pod \"fc639673-c5c1-4bf0-996d-64370a296a9c\" (UID: \"fc639673-c5c1-4bf0-996d-64370a296a9c\") "
Sep 19 05:45:28.085244 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:28.085089    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fc639673-c5c1-4bf0-996d-64370a296a9c-util\") pod \"fc639673-c5c1-4bf0-996d-64370a296a9c\" (UID: \"fc639673-c5c1-4bf0-996d-64370a296a9c\") "
Sep 19 05:45:28.085244 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:28.085116    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fc639673-c5c1-4bf0-996d-64370a296a9c-bundle\") pod \"fc639673-c5c1-4bf0-996d-64370a296a9c\" (UID: \"fc639673-c5c1-4bf0-996d-64370a296a9c\") "
Sep 19 05:45:28.085833 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:28.085808    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc639673-c5c1-4bf0-996d-64370a296a9c-bundle" (OuterVolumeSpecName: "bundle") pod "fc639673-c5c1-4bf0-996d-64370a296a9c" (UID: "fc639673-c5c1-4bf0-996d-64370a296a9c"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 05:45:28.087199 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:28.087162    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc639673-c5c1-4bf0-996d-64370a296a9c-kube-api-access-9bcrz" (OuterVolumeSpecName: "kube-api-access-9bcrz") pod "fc639673-c5c1-4bf0-996d-64370a296a9c" (UID: "fc639673-c5c1-4bf0-996d-64370a296a9c"). InnerVolumeSpecName "kube-api-access-9bcrz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 05:45:28.089300 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:28.089280    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc639673-c5c1-4bf0-996d-64370a296a9c-util" (OuterVolumeSpecName: "util") pod "fc639673-c5c1-4bf0-996d-64370a296a9c" (UID: "fc639673-c5c1-4bf0-996d-64370a296a9c"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 05:45:28.186473 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:28.186413    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9bcrz\" (UniqueName: \"kubernetes.io/projected/fc639673-c5c1-4bf0-996d-64370a296a9c-kube-api-access-9bcrz\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 05:45:28.186473 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:28.186436    2583 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fc639673-c5c1-4bf0-996d-64370a296a9c-util\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 05:45:28.186473 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:28.186444    2583 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fc639673-c5c1-4bf0-996d-64370a296a9c-bundle\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 05:45:28.875077 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:28.875041    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0fbqlj" event={"ID":"fc639673-c5c1-4bf0-996d-64370a296a9c","Type":"ContainerDied","Data":"00e1cdca703e2fe12b750436752278c575f10e16d7865d744d51acaed6034918"}
Sep 19 05:45:28.875077 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:28.875081    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0fbqlj"
Sep 19 05:45:28.875512 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:28.875096    2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="00e1cdca703e2fe12b750436752278c575f10e16d7865d744d51acaed6034918"
Sep 19 05:45:28.875512 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:28.875227    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-fd6794c5b-bptml"
Sep 19 05:45:39.882082 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:39.882045    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-fd6794c5b-bptml"
Sep 19 05:45:43.423146 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:43.423103    2583 kubelet.go:1648] "Image garbage collection succeeded"
Sep 19 05:45:53.261586 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:53.261546    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-djvnv"]
Sep 19 05:45:53.265242 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:53.261844    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fc639673-c5c1-4bf0-996d-64370a296a9c" containerName="pull"
Sep 19 05:45:53.265242 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:53.261856    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="fc639673-c5c1-4bf0-996d-64370a296a9c" containerName="pull"
Sep 19 05:45:53.265242 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:53.261868    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fc639673-c5c1-4bf0-996d-64370a296a9c" containerName="extract"
Sep 19 05:45:53.265242 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:53.261873    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="fc639673-c5c1-4bf0-996d-64370a296a9c" containerName="extract"
Sep 19 05:45:53.265242 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:53.261892    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fc639673-c5c1-4bf0-996d-64370a296a9c" containerName="util"
Sep 19 05:45:53.265242 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:53.261898    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="fc639673-c5c1-4bf0-996d-64370a296a9c" containerName="util"
Sep 19 05:45:53.265242 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:53.261939    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="fc639673-c5c1-4bf0-996d-64370a296a9c" containerName="extract"
Sep 19 05:45:53.266312 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:53.266295    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-djvnv"
Sep 19 05:45:53.269087 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:53.269062    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-fzhbf\""
Sep 19 05:45:53.270504 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:53.270487    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Sep 19 05:45:53.299973 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:53.299942    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-djvnv"]
Sep 19 05:45:53.393180 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:53.393115    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f188d5a4-6946-4fb2-88ad-c5c3ec50cfa3-cert\") pod \"odh-model-controller-7998c67dc4-djvnv\" (UID: \"f188d5a4-6946-4fb2-88ad-c5c3ec50cfa3\") " pod="opendatahub/odh-model-controller-7998c67dc4-djvnv"
Sep 19 05:45:53.393361 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:53.393214    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6s5c5\" (UniqueName: \"kubernetes.io/projected/f188d5a4-6946-4fb2-88ad-c5c3ec50cfa3-kube-api-access-6s5c5\") pod \"odh-model-controller-7998c67dc4-djvnv\" (UID: \"f188d5a4-6946-4fb2-88ad-c5c3ec50cfa3\") " pod="opendatahub/odh-model-controller-7998c67dc4-djvnv"
Sep 19 05:45:53.494367 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:53.494326    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6s5c5\" (UniqueName: \"kubernetes.io/projected/f188d5a4-6946-4fb2-88ad-c5c3ec50cfa3-kube-api-access-6s5c5\") pod \"odh-model-controller-7998c67dc4-djvnv\" (UID: \"f188d5a4-6946-4fb2-88ad-c5c3ec50cfa3\") " pod="opendatahub/odh-model-controller-7998c67dc4-djvnv"
Sep 19 05:45:53.494541 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:53.494426    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f188d5a4-6946-4fb2-88ad-c5c3ec50cfa3-cert\") pod \"odh-model-controller-7998c67dc4-djvnv\" (UID: \"f188d5a4-6946-4fb2-88ad-c5c3ec50cfa3\") " pod="opendatahub/odh-model-controller-7998c67dc4-djvnv"
Sep 19 05:45:53.494595 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:45:53.494550    2583 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Sep 19 05:45:53.494631 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:45:53.494619    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f188d5a4-6946-4fb2-88ad-c5c3ec50cfa3-cert podName:f188d5a4-6946-4fb2-88ad-c5c3ec50cfa3 nodeName:}" failed. No retries permitted until 2026-09-19 05:45:53.994597583 +0000 UTC m=+312.653052133 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/f188d5a4-6946-4fb2-88ad-c5c3ec50cfa3-cert") pod "odh-model-controller-7998c67dc4-djvnv" (UID: "f188d5a4-6946-4fb2-88ad-c5c3ec50cfa3") : secret "odh-model-controller-webhook-cert" not found
Sep 19 05:45:53.509943 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:53.509912    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6s5c5\" (UniqueName: \"kubernetes.io/projected/f188d5a4-6946-4fb2-88ad-c5c3ec50cfa3-kube-api-access-6s5c5\") pod \"odh-model-controller-7998c67dc4-djvnv\" (UID: \"f188d5a4-6946-4fb2-88ad-c5c3ec50cfa3\") " pod="opendatahub/odh-model-controller-7998c67dc4-djvnv"
Sep 19 05:45:53.998197 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:53.998161    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f188d5a4-6946-4fb2-88ad-c5c3ec50cfa3-cert\") pod \"odh-model-controller-7998c67dc4-djvnv\" (UID: \"f188d5a4-6946-4fb2-88ad-c5c3ec50cfa3\") " pod="opendatahub/odh-model-controller-7998c67dc4-djvnv"
Sep 19 05:45:54.000608 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:54.000582    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f188d5a4-6946-4fb2-88ad-c5c3ec50cfa3-cert\") pod \"odh-model-controller-7998c67dc4-djvnv\" (UID: \"f188d5a4-6946-4fb2-88ad-c5c3ec50cfa3\") " pod="opendatahub/odh-model-controller-7998c67dc4-djvnv"
Sep 19 05:45:54.021198 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:54.021166    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-pqsfk"]
Sep 19 05:45:54.029276 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:54.029251    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-pqsfk"
Sep 19 05:45:54.032277 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:54.032254    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Sep 19 05:45:54.057223 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:54.057185    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-pqsfk"]
Sep 19 05:45:54.176807 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:54.176768    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-djvnv"
Sep 19 05:45:54.200541 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:54.200507    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/043ddfe7-5d27-4ec7-a278-3ba5df899d67-tls-certs\") pod \"model-serving-api-54d89bd5fc-pqsfk\" (UID: \"043ddfe7-5d27-4ec7-a278-3ba5df899d67\") " pod="opendatahub/model-serving-api-54d89bd5fc-pqsfk"
Sep 19 05:45:54.200689 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:54.200562    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9slt9\" (UniqueName: \"kubernetes.io/projected/043ddfe7-5d27-4ec7-a278-3ba5df899d67-kube-api-access-9slt9\") pod \"model-serving-api-54d89bd5fc-pqsfk\" (UID: \"043ddfe7-5d27-4ec7-a278-3ba5df899d67\") " pod="opendatahub/model-serving-api-54d89bd5fc-pqsfk"
Sep 19 05:45:54.301081 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:54.301054    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/043ddfe7-5d27-4ec7-a278-3ba5df899d67-tls-certs\") pod \"model-serving-api-54d89bd5fc-pqsfk\" (UID: \"043ddfe7-5d27-4ec7-a278-3ba5df899d67\") " pod="opendatahub/model-serving-api-54d89bd5fc-pqsfk"
Sep 19 05:45:54.301477 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:54.301101    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9slt9\" (UniqueName: \"kubernetes.io/projected/043ddfe7-5d27-4ec7-a278-3ba5df899d67-kube-api-access-9slt9\") pod \"model-serving-api-54d89bd5fc-pqsfk\" (UID: \"043ddfe7-5d27-4ec7-a278-3ba5df899d67\") " pod="opendatahub/model-serving-api-54d89bd5fc-pqsfk"
Sep 19 05:45:54.303544 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:54.303524    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/043ddfe7-5d27-4ec7-a278-3ba5df899d67-tls-certs\") pod \"model-serving-api-54d89bd5fc-pqsfk\" (UID: \"043ddfe7-5d27-4ec7-a278-3ba5df899d67\") " pod="opendatahub/model-serving-api-54d89bd5fc-pqsfk"
Sep 19 05:45:54.320561 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:54.320529    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9slt9\" (UniqueName: \"kubernetes.io/projected/043ddfe7-5d27-4ec7-a278-3ba5df899d67-kube-api-access-9slt9\") pod \"model-serving-api-54d89bd5fc-pqsfk\" (UID: \"043ddfe7-5d27-4ec7-a278-3ba5df899d67\") " pod="opendatahub/model-serving-api-54d89bd5fc-pqsfk"
Sep 19 05:45:54.340049 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:54.340022    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-pqsfk"
Sep 19 05:45:54.460635 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:54.460606    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-djvnv"]
Sep 19 05:45:54.463216 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:45:54.463178    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf188d5a4_6946_4fb2_88ad_c5c3ec50cfa3.slice/crio-13a1933c2d62e21b53606ff32fb1583d96097cd6b14afe6307cac7d6660a111b WatchSource:0}: Error finding container 13a1933c2d62e21b53606ff32fb1583d96097cd6b14afe6307cac7d6660a111b: Status 404 returned error can't find the container with id 13a1933c2d62e21b53606ff32fb1583d96097cd6b14afe6307cac7d6660a111b
Sep 19 05:45:54.467203 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:54.467184    2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 19 05:45:54.618677 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:54.618649    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-pqsfk"]
Sep 19 05:45:54.621082 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:45:54.621054    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod043ddfe7_5d27_4ec7_a278_3ba5df899d67.slice/crio-4c93b4b576fb3842fdd78eb6688f4a52b50fb24551afeb9e34f01db421643c6f WatchSource:0}: Error finding container 4c93b4b576fb3842fdd78eb6688f4a52b50fb24551afeb9e34f01db421643c6f: Status 404 returned error can't find the container with id 4c93b4b576fb3842fdd78eb6688f4a52b50fb24551afeb9e34f01db421643c6f
Sep 19 05:45:55.000476 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:55.000378    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-pqsfk" event={"ID":"043ddfe7-5d27-4ec7-a278-3ba5df899d67","Type":"ContainerStarted","Data":"4c93b4b576fb3842fdd78eb6688f4a52b50fb24551afeb9e34f01db421643c6f"}
Sep 19 05:45:55.001508 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:55.001480    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-djvnv" event={"ID":"f188d5a4-6946-4fb2-88ad-c5c3ec50cfa3","Type":"ContainerStarted","Data":"13a1933c2d62e21b53606ff32fb1583d96097cd6b14afe6307cac7d6660a111b"}
Sep 19 05:45:55.719915 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:55.719876    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-8whhg"]
Sep 19 05:45:55.723692 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:55.723669    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-8whhg"
Sep 19 05:45:55.729723 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:55.729287    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 19 05:45:55.729723 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:55.729299    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 19 05:45:55.729723 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:55.729542    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-tctm5\""
Sep 19 05:45:55.729723 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:55.729548    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 19 05:45:55.744099 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:55.743952    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-8whhg"]
Sep 19 05:45:55.811681 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:55.811626    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bc5086e2-363c-404c-9b76-b11cf86d9071-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-8whhg\" (UID: \"bc5086e2-363c-404c-9b76-b11cf86d9071\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-8whhg"
Sep 19 05:45:55.811860 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:55.811683    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bc5086e2-363c-404c-9b76-b11cf86d9071-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-8whhg\" (UID: \"bc5086e2-363c-404c-9b76-b11cf86d9071\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-8whhg"
Sep 19 05:45:55.811860 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:55.811725    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc5086e2-363c-404c-9b76-b11cf86d9071-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-8whhg\" (UID: \"bc5086e2-363c-404c-9b76-b11cf86d9071\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-8whhg"
Sep 19 05:45:55.811860 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:55.811754    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f2qd9\" (UniqueName: \"kubernetes.io/projected/bc5086e2-363c-404c-9b76-b11cf86d9071-kube-api-access-f2qd9\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-8whhg\" (UID: \"bc5086e2-363c-404c-9b76-b11cf86d9071\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-8whhg"
Sep 19 05:45:55.811860 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:55.811801    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc5086e2-363c-404c-9b76-b11cf86d9071-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-8whhg\" (UID: \"bc5086e2-363c-404c-9b76-b11cf86d9071\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-8whhg"
Sep 19 05:45:55.811860 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:55.811827    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bc5086e2-363c-404c-9b76-b11cf86d9071-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-8whhg\" (UID: \"bc5086e2-363c-404c-9b76-b11cf86d9071\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-8whhg"
Sep 19 05:45:55.812141 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:55.811871    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bc5086e2-363c-404c-9b76-b11cf86d9071-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-8whhg\" (UID: \"bc5086e2-363c-404c-9b76-b11cf86d9071\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-8whhg"
Sep 19 05:45:55.812141 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:55.811897    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bc5086e2-363c-404c-9b76-b11cf86d9071-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-8whhg\" (UID: \"bc5086e2-363c-404c-9b76-b11cf86d9071\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-8whhg"
Sep 19 05:45:55.812141 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:55.811936    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bc5086e2-363c-404c-9b76-b11cf86d9071-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-8whhg\" (UID: \"bc5086e2-363c-404c-9b76-b11cf86d9071\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-8whhg"
Sep 19 05:45:55.912694 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:55.912653    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bc5086e2-363c-404c-9b76-b11cf86d9071-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-8whhg\" (UID: \"bc5086e2-363c-404c-9b76-b11cf86d9071\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-8whhg"
Sep 19 05:45:55.912872 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:55.912720    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc5086e2-363c-404c-9b76-b11cf86d9071-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-8whhg\" (UID: \"bc5086e2-363c-404c-9b76-b11cf86d9071\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-8whhg"
Sep 19 05:45:55.912872 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:55.912749    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f2qd9\" (UniqueName: \"kubernetes.io/projected/bc5086e2-363c-404c-9b76-b11cf86d9071-kube-api-access-f2qd9\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-8whhg\" (UID: \"bc5086e2-363c-404c-9b76-b11cf86d9071\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-8whhg"
Sep 19 05:45:55.912872 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:55.912801    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc5086e2-363c-404c-9b76-b11cf86d9071-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-8whhg\" (UID: \"bc5086e2-363c-404c-9b76-b11cf86d9071\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-8whhg"
Sep 19 05:45:55.912872 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:55.912832    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bc5086e2-363c-404c-9b76-b11cf86d9071-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-8whhg\" (UID: \"bc5086e2-363c-404c-9b76-b11cf86d9071\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-8whhg"
Sep 19 05:45:55.913076 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:55.912874    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bc5086e2-363c-404c-9b76-b11cf86d9071-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-8whhg\" (UID: \"bc5086e2-363c-404c-9b76-b11cf86d9071\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-8whhg"
Sep 19 05:45:55.913076 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:55.912902    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bc5086e2-363c-404c-9b76-b11cf86d9071-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-8whhg\" (UID: \"bc5086e2-363c-404c-9b76-b11cf86d9071\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-8whhg"
Sep 19 05:45:55.913076 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:55.912941    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bc5086e2-363c-404c-9b76-b11cf86d9071-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-8whhg\" (UID: \"bc5086e2-363c-404c-9b76-b11cf86d9071\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-8whhg"
Sep 19 05:45:55.913076 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:55.912983    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bc5086e2-363c-404c-9b76-b11cf86d9071-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-8whhg\" (UID: \"bc5086e2-363c-404c-9b76-b11cf86d9071\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-8whhg"
Sep 19 05:45:55.913302 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:55.913112    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc5086e2-363c-404c-9b76-b11cf86d9071-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-8whhg\" (UID: \"bc5086e2-363c-404c-9b76-b11cf86d9071\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-8whhg"
Sep 19 05:45:55.913430 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:55.913404    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bc5086e2-363c-404c-9b76-b11cf86d9071-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-8whhg\" (UID: \"bc5086e2-363c-404c-9b76-b11cf86d9071\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-8whhg"
Sep 19 05:45:55.913494 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:55.913470    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc5086e2-363c-404c-9b76-b11cf86d9071-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-8whhg\" (UID: \"bc5086e2-363c-404c-9b76-b11cf86d9071\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-8whhg"
Sep 19 05:45:55.913593 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:55.913565    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bc5086e2-363c-404c-9b76-b11cf86d9071-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-8whhg\" (UID: \"bc5086e2-363c-404c-9b76-b11cf86d9071\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-8whhg"
Sep 19 05:45:55.914000 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:55.913977    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bc5086e2-363c-404c-9b76-b11cf86d9071-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-8whhg\" (UID: \"bc5086e2-363c-404c-9b76-b11cf86d9071\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-8whhg"
Sep 19 05:45:55.917656 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:55.916212    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bc5086e2-363c-404c-9b76-b11cf86d9071-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-8whhg\" (UID: \"bc5086e2-363c-404c-9b76-b11cf86d9071\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-8whhg"
Sep 19 05:45:55.917656 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:55.917619    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bc5086e2-363c-404c-9b76-b11cf86d9071-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-8whhg\" (UID: \"bc5086e2-363c-404c-9b76-b11cf86d9071\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-8whhg"
Sep 19 05:45:55.925265 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:55.925239    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f2qd9\" (UniqueName: \"kubernetes.io/projected/bc5086e2-363c-404c-9b76-b11cf86d9071-kube-api-access-f2qd9\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-8whhg\" (UID: \"bc5086e2-363c-404c-9b76-b11cf86d9071\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-8whhg"
Sep 19 05:45:55.927251 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:55.927226    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bc5086e2-363c-404c-9b76-b11cf86d9071-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-8whhg\" (UID: \"bc5086e2-363c-404c-9b76-b11cf86d9071\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-8whhg"
Sep 19 05:45:56.045197 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:56.045154    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-8whhg"
Sep 19 05:45:58.741925 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:58.741838    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-8whhg"]
Sep 19 05:45:58.744863 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:45:58.744834    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc5086e2_363c_404c_9b76_b11cf86d9071.slice/crio-ffabf37a5a5a1c882fc1039feaadafba34c6995f570c2dac6b98a405159a7472 WatchSource:0}: Error finding container ffabf37a5a5a1c882fc1039feaadafba34c6995f570c2dac6b98a405159a7472: Status 404 returned error can't find the container with id ffabf37a5a5a1c882fc1039feaadafba34c6995f570c2dac6b98a405159a7472
Sep 19 05:45:59.026006 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:59.025918    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-8whhg" event={"ID":"bc5086e2-363c-404c-9b76-b11cf86d9071","Type":"ContainerStarted","Data":"ffabf37a5a5a1c882fc1039feaadafba34c6995f570c2dac6b98a405159a7472"}
Sep 19 05:45:59.027731 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:59.027698    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-pqsfk" event={"ID":"043ddfe7-5d27-4ec7-a278-3ba5df899d67","Type":"ContainerStarted","Data":"106079c368dcd04c71b2e8aa29dab9e297aea613042e77222c7f8e20ec628de3"}
Sep 19 05:45:59.027881 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:59.027806    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-pqsfk"
Sep 19 05:45:59.029453 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:59.029432    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-djvnv" event={"ID":"f188d5a4-6946-4fb2-88ad-c5c3ec50cfa3","Type":"ContainerStarted","Data":"b2f1e6af19682a04313e4a3d7b570a643da834cc169c49b3c3a48cd536524a2f"}
Sep 19 05:45:59.029564 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:59.029492    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-djvnv"
Sep 19 05:45:59.047993 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:59.047952    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-pqsfk" podStartSLOduration=2.170248968 podStartE2EDuration="6.047938536s" totalImagesPullingTime="3.877689568s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:45:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:45:59.046458554 +0000 UTC m=+317.704913128" watchObservedRunningTime="2026-09-19 05:45:59.047938536 +0000 UTC m=+317.706393094"
Sep 19 05:45:59.064921 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:45:59.064877    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-djvnv" podStartSLOduration=2.041879829 podStartE2EDuration="6.064862108s" totalImagesPullingTime="4.022982279s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:45:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:45:59.064289298 +0000 UTC m=+317.722743872" watchObservedRunningTime="2026-09-19 05:45:59.064862108 +0000 UTC m=+317.723316667"
Sep 19 05:46:00.801834 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:46:00.801803    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-cgpct"]
Sep 19 05:46:00.808679 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:46:00.808564    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cgpct"
Sep 19 05:46:00.811623 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:46:00.811600    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Sep 19 05:46:00.811764 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:46:00.811724    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-gvr8h\""
Sep 19 05:46:00.811875 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:46:00.811856    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Sep 19 05:46:00.811999 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:46:00.811980    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Sep 19 05:46:00.839698 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:46:00.839669    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-cgpct"]
Sep 19 05:46:00.957781 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:46:00.957742    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/0a797d1e-2b5c-4511-8823-346eeb6c8596-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-cgpct\" (UID: \"0a797d1e-2b5c-4511-8823-346eeb6c8596\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cgpct"
Sep 19 05:46:00.957953 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:46:00.957829    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/0a797d1e-2b5c-4511-8823-346eeb6c8596-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-cgpct\" (UID: \"0a797d1e-2b5c-4511-8823-346eeb6c8596\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cgpct"
Sep 19 05:46:00.957953 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:46:00.957895    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0a797d1e-2b5c-4511-8823-346eeb6c8596-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-cgpct\" (UID: \"0a797d1e-2b5c-4511-8823-346eeb6c8596\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cgpct"
Sep 19 05:46:00.957953 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:46:00.957951    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hkhmp\" (UniqueName: \"kubernetes.io/projected/0a797d1e-2b5c-4511-8823-346eeb6c8596-kube-api-access-hkhmp\") pod \"istiod-openshift-gateway-5677ccb49b-cgpct\" (UID: \"0a797d1e-2b5c-4511-8823-346eeb6c8596\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cgpct"
Sep 19 05:46:00.958114 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:46:00.958024    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/0a797d1e-2b5c-4511-8823-346eeb6c8596-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-cgpct\" (UID: \"0a797d1e-2b5c-4511-8823-346eeb6c8596\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cgpct"
Sep 19 05:46:00.958114 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:46:00.958068    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0a797d1e-2b5c-4511-8823-346eeb6c8596-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-cgpct\" (UID: \"0a797d1e-2b5c-4511-8823-346eeb6c8596\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cgpct"
Sep 19 05:46:00.958114 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:46:00.958093    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/0a797d1e-2b5c-4511-8823-346eeb6c8596-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-cgpct\" (UID: \"0a797d1e-2b5c-4511-8823-346eeb6c8596\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cgpct"
Sep 19 05:46:01.059343 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:46:01.059252    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0a797d1e-2b5c-4511-8823-346eeb6c8596-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-cgpct\" (UID: \"0a797d1e-2b5c-4511-8823-346eeb6c8596\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cgpct"
Sep 19 05:46:01.059343 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:46:01.059317    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hkhmp\" (UniqueName: \"kubernetes.io/projected/0a797d1e-2b5c-4511-8823-346eeb6c8596-kube-api-access-hkhmp\") pod \"istiod-openshift-gateway-5677ccb49b-cgpct\" (UID: \"0a797d1e-2b5c-4511-8823-346eeb6c8596\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cgpct"
Sep 19 05:46:01.059566 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:46:01.059355    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/0a797d1e-2b5c-4511-8823-346eeb6c8596-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-cgpct\" (UID: \"0a797d1e-2b5c-4511-8823-346eeb6c8596\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cgpct"
Sep 19 05:46:01.059566 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:46:01.059382    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0a797d1e-2b5c-4511-8823-346eeb6c8596-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-cgpct\" (UID: \"0a797d1e-2b5c-4511-8823-346eeb6c8596\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cgpct"
Sep 19 05:46:01.059566 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:46:01.059405    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/0a797d1e-2b5c-4511-8823-346eeb6c8596-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-cgpct\" (UID: \"0a797d1e-2b5c-4511-8823-346eeb6c8596\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cgpct"
Sep 19 05:46:01.059566 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:46:01.059438    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/0a797d1e-2b5c-4511-8823-346eeb6c8596-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-cgpct\" (UID: \"0a797d1e-2b5c-4511-8823-346eeb6c8596\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cgpct"
Sep 19 05:46:01.059566 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:46:01.059474    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/0a797d1e-2b5c-4511-8823-346eeb6c8596-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-cgpct\" (UID: \"0a797d1e-2b5c-4511-8823-346eeb6c8596\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cgpct"
Sep 19 05:46:01.060332 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:46:01.060297    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/0a797d1e-2b5c-4511-8823-346eeb6c8596-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-cgpct\" (UID: \"0a797d1e-2b5c-4511-8823-346eeb6c8596\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cgpct"
Sep 19 05:46:01.062521 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:46:01.062454    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/0a797d1e-2b5c-4511-8823-346eeb6c8596-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-cgpct\" (UID: \"0a797d1e-2b5c-4511-8823-346eeb6c8596\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cgpct"
Sep 19 05:46:01.062755 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:46:01.062725    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/0a797d1e-2b5c-4511-8823-346eeb6c8596-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-cgpct\" (UID: \"0a797d1e-2b5c-4511-8823-346eeb6c8596\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cgpct"
Sep 19 05:46:01.062815 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:46:01.062788    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0a797d1e-2b5c-4511-8823-346eeb6c8596-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-cgpct\" (UID: \"0a797d1e-2b5c-4511-8823-346eeb6c8596\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cgpct"
Sep 19 05:46:01.062886 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:46:01.062863    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/0a797d1e-2b5c-4511-8823-346eeb6c8596-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-cgpct\" (UID: \"0a797d1e-2b5c-4511-8823-346eeb6c8596\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cgpct"
Sep 19 05:46:01.070107 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:46:01.070080    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0a797d1e-2b5c-4511-8823-346eeb6c8596-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-cgpct\" (UID: \"0a797d1e-2b5c-4511-8823-346eeb6c8596\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cgpct"
Sep 19 05:46:01.070525 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:46:01.070503    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hkhmp\" (UniqueName: \"kubernetes.io/projected/0a797d1e-2b5c-4511-8823-346eeb6c8596-kube-api-access-hkhmp\") pod \"istiod-openshift-gateway-5677ccb49b-cgpct\" (UID: \"0a797d1e-2b5c-4511-8823-346eeb6c8596\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cgpct"
Sep 19 05:46:01.119701 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:46:01.119652    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cgpct"
Sep 19 05:46:01.301967 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:46:01.301924    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 19 05:46:01.302063 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:46:01.302008    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 19 05:46:01.302063 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:46:01.302034    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 19 05:46:01.514750 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:46:01.514716    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-cgpct"]
Sep 19 05:46:01.518869 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:46:01.518788    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a797d1e_2b5c_4511_8823_346eeb6c8596.slice/crio-10f5df7e4c433c77f797bbe151a6f9a4cc7dfa214b822a2baeb20e1aaa02c3a6 WatchSource:0}: Error finding container 10f5df7e4c433c77f797bbe151a6f9a4cc7dfa214b822a2baeb20e1aaa02c3a6: Status 404 returned error can't find the container with id 10f5df7e4c433c77f797bbe151a6f9a4cc7dfa214b822a2baeb20e1aaa02c3a6
Sep 19 05:46:02.046614 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:46:02.046581    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cgpct" event={"ID":"0a797d1e-2b5c-4511-8823-346eeb6c8596","Type":"ContainerStarted","Data":"10f5df7e4c433c77f797bbe151a6f9a4cc7dfa214b822a2baeb20e1aaa02c3a6"}
Sep 19 05:46:02.048231 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:46:02.048198    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-8whhg" event={"ID":"bc5086e2-363c-404c-9b76-b11cf86d9071","Type":"ContainerStarted","Data":"71ab5e99003f25848322dea887d259768710507b4a72fe8beb55de48b608a714"}
Sep 19 05:46:02.069967 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:46:02.069922    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-8whhg" podStartSLOduration=4.51529281 podStartE2EDuration="7.069907625s" totalImagesPullingTime="2.554614815s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:45:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:46:02.069281817 +0000 UTC m=+320.727736376" watchObservedRunningTime="2026-09-19 05:46:02.069907625 +0000 UTC m=+320.728362188"
Sep 19 05:46:03.046125 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:46:03.046078    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-8whhg"
Sep 19 05:46:03.051470 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:46:03.051444    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-8whhg"
Sep 19 05:46:03.054064 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:46:03.054037    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-8whhg"
Sep 19 05:46:03.054783 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:46:03.054766    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-8whhg"
Sep 19 05:46:03.893249 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:46:03.893213    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 19 05:46:03.893375 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:46:03.893292    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 19 05:46:04.060926 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:46:04.060779    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cgpct" event={"ID":"0a797d1e-2b5c-4511-8823-346eeb6c8596","Type":"ContainerStarted","Data":"65109d4df8a8714ad555bee0a4c6a3e45fbcddec0d8ba3f2808414cb4b868dde"}
Sep 19 05:46:04.061491 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:46:04.060979    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cgpct"
Sep 19 05:46:04.062380 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:46:04.062354    2583 patch_prober.go:28] interesting pod/istiod-openshift-gateway-5677ccb49b-cgpct container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 19 05:46:04.062483 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:46:04.062406    2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cgpct" podUID="0a797d1e-2b5c-4511-8823-346eeb6c8596" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 19 05:46:04.184949 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:46:04.184847    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cgpct" podStartSLOduration=1.813201269 podStartE2EDuration="4.184831306s" totalImagesPullingTime="2.371630037s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:46:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:46:04.184694477 +0000 UTC m=+322.843149036" watchObservedRunningTime="2026-09-19 05:46:04.184831306 +0000 UTC m=+322.843285864"
Sep 19 05:46:05.066330 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:46:05.066296    2583 patch_prober.go:28] interesting pod/istiod-openshift-gateway-5677ccb49b-cgpct container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 19 05:46:05.066702 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:46:05.066360    2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cgpct" podUID="0a797d1e-2b5c-4511-8823-346eeb6c8596" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 19 05:46:08.066676 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:46:08.066640    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cgpct"
Sep 19 05:46:10.037414 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:46:10.037384    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-djvnv"
Sep 19 05:46:10.039336 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:46:10.039315    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-pqsfk"
Sep 19 05:46:59.112751 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:46:59.112670    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6x2nsj"]
Sep 19 05:46:59.115607 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:46:59.115590    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6x2nsj"
Sep 19 05:46:59.118310 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:46:59.118291    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 19 05:46:59.118422 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:46:59.118294    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 19 05:46:59.119213 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:46:59.119198    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-vqxsl\""
Sep 19 05:46:59.125692 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:46:59.125671    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6x2nsj"]
Sep 19 05:46:59.214945 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:46:59.214912    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7hnbbl"]
Sep 19 05:46:59.217934 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:46:59.217913    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7hnbbl"
Sep 19 05:46:59.226668 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:46:59.226644    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7hnbbl"]
Sep 19 05:46:59.241916 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:46:59.241895    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/09873dc7-416a-4214-8de2-4ffccdd25785-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6x2nsj\" (UID: \"09873dc7-416a-4214-8de2-4ffccdd25785\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6x2nsj"
Sep 19 05:46:59.242036 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:46:59.241939    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/09873dc7-416a-4214-8de2-4ffccdd25785-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6x2nsj\" (UID: \"09873dc7-416a-4214-8de2-4ffccdd25785\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6x2nsj"
Sep 19 05:46:59.242101 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:46:59.242050    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-555bf\" (UniqueName: \"kubernetes.io/projected/09873dc7-416a-4214-8de2-4ffccdd25785-kube-api-access-555bf\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6x2nsj\" (UID: \"09873dc7-416a-4214-8de2-4ffccdd25785\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6x2nsj"
Sep 19 05:46:59.343290 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:46:59.343264    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/533d5517-8284-4b47-8dc4-5603ceb9be28-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7hnbbl\" (UID: \"533d5517-8284-4b47-8dc4-5603ceb9be28\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7hnbbl"
Sep 19 05:46:59.343423 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:46:59.343300    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-555bf\" (UniqueName: \"kubernetes.io/projected/09873dc7-416a-4214-8de2-4ffccdd25785-kube-api-access-555bf\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6x2nsj\" (UID: \"09873dc7-416a-4214-8de2-4ffccdd25785\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6x2nsj"
Sep 19 05:46:59.343423 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:46:59.343358    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/09873dc7-416a-4214-8de2-4ffccdd25785-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6x2nsj\" (UID: \"09873dc7-416a-4214-8de2-4ffccdd25785\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6x2nsj"
Sep 19 05:46:59.343423 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:46:59.343403    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kfdg6\" (UniqueName: \"kubernetes.io/projected/533d5517-8284-4b47-8dc4-5603ceb9be28-kube-api-access-kfdg6\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7hnbbl\" (UID: \"533d5517-8284-4b47-8dc4-5603ceb9be28\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7hnbbl"
Sep 19 05:46:59.343533 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:46:59.343424    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/09873dc7-416a-4214-8de2-4ffccdd25785-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6x2nsj\" (UID: \"09873dc7-416a-4214-8de2-4ffccdd25785\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6x2nsj"
Sep 19 05:46:59.343533 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:46:59.343482    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/533d5517-8284-4b47-8dc4-5603ceb9be28-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7hnbbl\" (UID: \"533d5517-8284-4b47-8dc4-5603ceb9be28\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7hnbbl"
Sep 19 05:46:59.343804 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:46:59.343787    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/09873dc7-416a-4214-8de2-4ffccdd25785-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6x2nsj\" (UID: \"09873dc7-416a-4214-8de2-4ffccdd25785\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6x2nsj"
Sep 19 05:46:59.343841 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:46:59.343820    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/09873dc7-416a-4214-8de2-4ffccdd25785-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6x2nsj\" (UID: \"09873dc7-416a-4214-8de2-4ffccdd25785\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6x2nsj"
Sep 19 05:46:59.361259 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:46:59.361230    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-555bf\" (UniqueName: \"kubernetes.io/projected/09873dc7-416a-4214-8de2-4ffccdd25785-kube-api-access-555bf\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6x2nsj\" (UID: \"09873dc7-416a-4214-8de2-4ffccdd25785\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6x2nsj"
Sep 19 05:46:59.361508 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:46:59.361493    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847az68j5"]
Sep 19 05:46:59.364633 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:46:59.364595    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847az68j5"
Sep 19 05:46:59.382637 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:46:59.382613    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847az68j5"]
Sep 19 05:46:59.425450 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:46:59.425428    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6x2nsj"
Sep 19 05:46:59.444025 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:46:59.444005    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/533d5517-8284-4b47-8dc4-5603ceb9be28-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7hnbbl\" (UID: \"533d5517-8284-4b47-8dc4-5603ceb9be28\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7hnbbl"
Sep 19 05:46:59.444165 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:46:59.444055    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/533d5517-8284-4b47-8dc4-5603ceb9be28-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7hnbbl\" (UID: \"533d5517-8284-4b47-8dc4-5603ceb9be28\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7hnbbl"
Sep 19 05:46:59.444165 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:46:59.444124    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kfdg6\" (UniqueName: \"kubernetes.io/projected/533d5517-8284-4b47-8dc4-5603ceb9be28-kube-api-access-kfdg6\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7hnbbl\" (UID: \"533d5517-8284-4b47-8dc4-5603ceb9be28\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7hnbbl"
Sep 19 05:46:59.444369 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:46:59.444348    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/533d5517-8284-4b47-8dc4-5603ceb9be28-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7hnbbl\" (UID: \"533d5517-8284-4b47-8dc4-5603ceb9be28\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7hnbbl"
Sep 19 05:46:59.444463 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:46:59.444442    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/533d5517-8284-4b47-8dc4-5603ceb9be28-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7hnbbl\" (UID: \"533d5517-8284-4b47-8dc4-5603ceb9be28\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7hnbbl"
Sep 19 05:46:59.461026 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:46:59.461002    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kfdg6\" (UniqueName: \"kubernetes.io/projected/533d5517-8284-4b47-8dc4-5603ceb9be28-kube-api-access-kfdg6\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7hnbbl\" (UID: \"533d5517-8284-4b47-8dc4-5603ceb9be28\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7hnbbl"
Sep 19 05:46:59.511995 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:46:59.511966    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cx9prb"]
Sep 19 05:46:59.516157 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:46:59.516120    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cx9prb"
Sep 19 05:46:59.528148 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:46:59.528115    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7hnbbl"
Sep 19 05:46:59.531207 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:46:59.531185    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cx9prb"]
Sep 19 05:46:59.544999 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:46:59.544973    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/35da0a63-f8b4-459e-91f5-35af8747783e-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847az68j5\" (UID: \"35da0a63-f8b4-459e-91f5-35af8747783e\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847az68j5"
Sep 19 05:46:59.545120 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:46:59.545007    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6knlz\" (UniqueName: \"kubernetes.io/projected/35da0a63-f8b4-459e-91f5-35af8747783e-kube-api-access-6knlz\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847az68j5\" (UID: \"35da0a63-f8b4-459e-91f5-35af8747783e\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847az68j5"
Sep 19 05:46:59.545226 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:46:59.545146    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/35da0a63-f8b4-459e-91f5-35af8747783e-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847az68j5\" (UID: \"35da0a63-f8b4-459e-91f5-35af8747783e\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847az68j5"
Sep 19 05:46:59.646483 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:46:59.646416    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0ecb4201-8707-480a-b5fd-d254ff6011ec-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cx9prb\" (UID: \"0ecb4201-8707-480a-b5fd-d254ff6011ec\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cx9prb"
Sep 19 05:46:59.646483 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:46:59.646456    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mclk2\" (UniqueName: \"kubernetes.io/projected/0ecb4201-8707-480a-b5fd-d254ff6011ec-kube-api-access-mclk2\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cx9prb\" (UID: \"0ecb4201-8707-480a-b5fd-d254ff6011ec\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cx9prb"
Sep 19 05:46:59.646652 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:46:59.646494    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0ecb4201-8707-480a-b5fd-d254ff6011ec-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cx9prb\" (UID: \"0ecb4201-8707-480a-b5fd-d254ff6011ec\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cx9prb"
Sep 19 05:46:59.646652 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:46:59.646524    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/35da0a63-f8b4-459e-91f5-35af8747783e-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847az68j5\" (UID: \"35da0a63-f8b4-459e-91f5-35af8747783e\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847az68j5"
Sep 19 05:46:59.646652 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:46:59.646544    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6knlz\" (UniqueName: \"kubernetes.io/projected/35da0a63-f8b4-459e-91f5-35af8747783e-kube-api-access-6knlz\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847az68j5\" (UID: \"35da0a63-f8b4-459e-91f5-35af8747783e\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847az68j5"
Sep 19 05:46:59.646652 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:46:59.646638    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/35da0a63-f8b4-459e-91f5-35af8747783e-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847az68j5\" (UID: \"35da0a63-f8b4-459e-91f5-35af8747783e\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847az68j5"
Sep 19 05:46:59.646869 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:46:59.646850    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/35da0a63-f8b4-459e-91f5-35af8747783e-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847az68j5\" (UID: \"35da0a63-f8b4-459e-91f5-35af8747783e\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847az68j5"
Sep 19 05:46:59.646977 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:46:59.646960    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/35da0a63-f8b4-459e-91f5-35af8747783e-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847az68j5\" (UID: \"35da0a63-f8b4-459e-91f5-35af8747783e\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847az68j5"
Sep 19 05:46:59.656445 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:46:59.656416    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6knlz\" (UniqueName: \"kubernetes.io/projected/35da0a63-f8b4-459e-91f5-35af8747783e-kube-api-access-6knlz\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847az68j5\" (UID: \"35da0a63-f8b4-459e-91f5-35af8747783e\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847az68j5"
Sep 19 05:46:59.675365 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:46:59.675346    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847az68j5"
Sep 19 05:46:59.717808 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:46:59.717781    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6x2nsj"]
Sep 19 05:46:59.720162 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:46:59.720110    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09873dc7_416a_4214_8de2_4ffccdd25785.slice/crio-0c7ad62378eabbcb1afd9d23b04bea69613bae37e2da66f7b2439999522c0c61 WatchSource:0}: Error finding container 0c7ad62378eabbcb1afd9d23b04bea69613bae37e2da66f7b2439999522c0c61: Status 404 returned error can't find the container with id 0c7ad62378eabbcb1afd9d23b04bea69613bae37e2da66f7b2439999522c0c61
Sep 19 05:46:59.747968 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:46:59.747944    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0ecb4201-8707-480a-b5fd-d254ff6011ec-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cx9prb\" (UID: \"0ecb4201-8707-480a-b5fd-d254ff6011ec\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cx9prb"
Sep 19 05:46:59.748068 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:46:59.748056    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0ecb4201-8707-480a-b5fd-d254ff6011ec-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cx9prb\" (UID: \"0ecb4201-8707-480a-b5fd-d254ff6011ec\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cx9prb"
Sep 19 05:46:59.748142 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:46:59.748082    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mclk2\" (UniqueName: \"kubernetes.io/projected/0ecb4201-8707-480a-b5fd-d254ff6011ec-kube-api-access-mclk2\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cx9prb\" (UID: \"0ecb4201-8707-480a-b5fd-d254ff6011ec\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cx9prb"
Sep 19 05:46:59.748458 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:46:59.748432    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0ecb4201-8707-480a-b5fd-d254ff6011ec-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cx9prb\" (UID: \"0ecb4201-8707-480a-b5fd-d254ff6011ec\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cx9prb"
Sep 19 05:46:59.748525 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:46:59.748444    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0ecb4201-8707-480a-b5fd-d254ff6011ec-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cx9prb\" (UID: \"0ecb4201-8707-480a-b5fd-d254ff6011ec\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cx9prb"
Sep 19 05:46:59.771065 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:46:59.771042    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mclk2\" (UniqueName: \"kubernetes.io/projected/0ecb4201-8707-480a-b5fd-d254ff6011ec-kube-api-access-mclk2\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cx9prb\" (UID: \"0ecb4201-8707-480a-b5fd-d254ff6011ec\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cx9prb"
Sep 19 05:46:59.816466 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:46:59.814246    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7hnbbl"]
Sep 19 05:46:59.817097 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:46:59.817069    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod533d5517_8284_4b47_8dc4_5603ceb9be28.slice/crio-b4f67fe2b7ebd8e9b328681f6a74c08e07f9dc19d78e74da62317359e16515fd WatchSource:0}: Error finding container b4f67fe2b7ebd8e9b328681f6a74c08e07f9dc19d78e74da62317359e16515fd: Status 404 returned error can't find the container with id b4f67fe2b7ebd8e9b328681f6a74c08e07f9dc19d78e74da62317359e16515fd
Sep 19 05:46:59.826218 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:46:59.826196    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cx9prb"
Sep 19 05:46:59.956692 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:46:59.956669    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847az68j5"]
Sep 19 05:46:59.993850 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:46:59.993824    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35da0a63_f8b4_459e_91f5_35af8747783e.slice/crio-543268c2119d43c3207a121082807a06dba9da84fbf60d09465d8f125481b74b WatchSource:0}: Error finding container 543268c2119d43c3207a121082807a06dba9da84fbf60d09465d8f125481b74b: Status 404 returned error can't find the container with id 543268c2119d43c3207a121082807a06dba9da84fbf60d09465d8f125481b74b
Sep 19 05:47:00.099759 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:47:00.099708    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cx9prb"]
Sep 19 05:47:00.136907 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:47:00.136879    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ecb4201_8707_480a_b5fd_d254ff6011ec.slice/crio-bdac2ae5efdbc5d1b41602d420d772becfd3b30aa45a6884daefcf56dc684d94 WatchSource:0}: Error finding container bdac2ae5efdbc5d1b41602d420d772becfd3b30aa45a6884daefcf56dc684d94: Status 404 returned error can't find the container with id bdac2ae5efdbc5d1b41602d420d772becfd3b30aa45a6884daefcf56dc684d94
Sep 19 05:47:00.326237 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:47:00.326208    2583 generic.go:356] "Generic (PLEG): container finished" podID="0ecb4201-8707-480a-b5fd-d254ff6011ec" containerID="fa9989bf2dbcf3e11289cc3570520ce9a1c1cdc1088eabffcfaccb5cea2291d7" exitCode=0
Sep 19 05:47:00.326406 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:47:00.326289    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cx9prb" event={"ID":"0ecb4201-8707-480a-b5fd-d254ff6011ec","Type":"ContainerDied","Data":"fa9989bf2dbcf3e11289cc3570520ce9a1c1cdc1088eabffcfaccb5cea2291d7"}
Sep 19 05:47:00.326406 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:47:00.326348    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cx9prb" event={"ID":"0ecb4201-8707-480a-b5fd-d254ff6011ec","Type":"ContainerStarted","Data":"bdac2ae5efdbc5d1b41602d420d772becfd3b30aa45a6884daefcf56dc684d94"}
Sep 19 05:47:00.327891 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:47:00.327871    2583 generic.go:356] "Generic (PLEG): container finished" podID="533d5517-8284-4b47-8dc4-5603ceb9be28" containerID="3f0341bc23f63b4ff37049369ff529202f6783ef5a2f46c52949cb58475d0558" exitCode=0
Sep 19 05:47:00.328024 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:47:00.327998    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7hnbbl" event={"ID":"533d5517-8284-4b47-8dc4-5603ceb9be28","Type":"ContainerDied","Data":"3f0341bc23f63b4ff37049369ff529202f6783ef5a2f46c52949cb58475d0558"}
Sep 19 05:47:00.328091 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:47:00.328051    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7hnbbl" event={"ID":"533d5517-8284-4b47-8dc4-5603ceb9be28","Type":"ContainerStarted","Data":"b4f67fe2b7ebd8e9b328681f6a74c08e07f9dc19d78e74da62317359e16515fd"}
Sep 19 05:47:00.329626 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:47:00.329600    2583 generic.go:356] "Generic (PLEG): container finished" podID="35da0a63-f8b4-459e-91f5-35af8747783e" containerID="f0f01fb01bb915cad4dba2b82dae105d6a93c5e532d74ef153e4847be93e5e31" exitCode=0
Sep 19 05:47:00.329730 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:47:00.329680    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847az68j5" event={"ID":"35da0a63-f8b4-459e-91f5-35af8747783e","Type":"ContainerDied","Data":"f0f01fb01bb915cad4dba2b82dae105d6a93c5e532d74ef153e4847be93e5e31"}
Sep 19 05:47:00.329730 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:47:00.329725    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847az68j5" event={"ID":"35da0a63-f8b4-459e-91f5-35af8747783e","Type":"ContainerStarted","Data":"543268c2119d43c3207a121082807a06dba9da84fbf60d09465d8f125481b74b"}
Sep 19 05:47:00.331372 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:47:00.331352    2583 generic.go:356] "Generic (PLEG): container finished" podID="09873dc7-416a-4214-8de2-4ffccdd25785" containerID="65e236812d084242d97df570819ab47264eb0708dbcd8ad259ae4aa26376ebbe" exitCode=0
Sep 19 05:47:00.331459 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:47:00.331380    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6x2nsj" event={"ID":"09873dc7-416a-4214-8de2-4ffccdd25785","Type":"ContainerDied","Data":"65e236812d084242d97df570819ab47264eb0708dbcd8ad259ae4aa26376ebbe"}
Sep 19 05:47:00.331459 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:47:00.331416    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6x2nsj" event={"ID":"09873dc7-416a-4214-8de2-4ffccdd25785","Type":"ContainerStarted","Data":"0c7ad62378eabbcb1afd9d23b04bea69613bae37e2da66f7b2439999522c0c61"}
Sep 19 05:47:01.338263 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:47:01.338226    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847az68j5" event={"ID":"35da0a63-f8b4-459e-91f5-35af8747783e","Type":"ContainerStarted","Data":"142b6d6c147281c8035df8fd81e6d0ba03e5135bcbbc640bf728aa3c3b1f762b"}
Sep 19 05:47:01.339985 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:47:01.339951    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cx9prb" event={"ID":"0ecb4201-8707-480a-b5fd-d254ff6011ec","Type":"ContainerStarted","Data":"829efe1cfa7652a4e8531e39cae28e906aca49b3342c21871d7dfcb205d8ede1"}
Sep 19 05:47:02.346814 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:47:02.346783    2583 generic.go:356] "Generic (PLEG): container finished" podID="533d5517-8284-4b47-8dc4-5603ceb9be28" containerID="9eaaa026c917fd9218245dbd3d73b2ec426c7611a6c9ff692a0c2aa17b288891" exitCode=0
Sep 19 05:47:02.347293 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:47:02.346874    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7hnbbl" event={"ID":"533d5517-8284-4b47-8dc4-5603ceb9be28","Type":"ContainerDied","Data":"9eaaa026c917fd9218245dbd3d73b2ec426c7611a6c9ff692a0c2aa17b288891"}
Sep 19 05:47:02.348846 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:47:02.348827    2583 generic.go:356] "Generic (PLEG): container finished" podID="35da0a63-f8b4-459e-91f5-35af8747783e" containerID="142b6d6c147281c8035df8fd81e6d0ba03e5135bcbbc640bf728aa3c3b1f762b" exitCode=0
Sep 19 05:47:02.348939 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:47:02.348857    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847az68j5" event={"ID":"35da0a63-f8b4-459e-91f5-35af8747783e","Type":"ContainerDied","Data":"142b6d6c147281c8035df8fd81e6d0ba03e5135bcbbc640bf728aa3c3b1f762b"}
Sep 19 05:47:02.350818 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:47:02.350798    2583 generic.go:356] "Generic (PLEG): container finished" podID="09873dc7-416a-4214-8de2-4ffccdd25785" containerID="c6576313b63a7a088af8c15e041191cd39ebf3b13848b0ce84a472b1f735b14d" exitCode=0
Sep 19 05:47:02.350895 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:47:02.350877    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6x2nsj" event={"ID":"09873dc7-416a-4214-8de2-4ffccdd25785","Type":"ContainerDied","Data":"c6576313b63a7a088af8c15e041191cd39ebf3b13848b0ce84a472b1f735b14d"}
Sep 19 05:47:02.352909 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:47:02.352887    2583 generic.go:356] "Generic (PLEG): container finished" podID="0ecb4201-8707-480a-b5fd-d254ff6011ec" containerID="829efe1cfa7652a4e8531e39cae28e906aca49b3342c21871d7dfcb205d8ede1" exitCode=0
Sep 19 05:47:02.352994 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:47:02.352965    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cx9prb" event={"ID":"0ecb4201-8707-480a-b5fd-d254ff6011ec","Type":"ContainerDied","Data":"829efe1cfa7652a4e8531e39cae28e906aca49b3342c21871d7dfcb205d8ede1"}
Sep 19 05:47:03.360046 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:47:03.360010    2583 generic.go:356] "Generic (PLEG): container finished" podID="35da0a63-f8b4-459e-91f5-35af8747783e" containerID="a8878369b478e61eea4ab92a68591739aa04efae1dd7501cee00361ea73d98da" exitCode=0
Sep 19 05:47:03.360461 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:47:03.360095    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847az68j5" event={"ID":"35da0a63-f8b4-459e-91f5-35af8747783e","Type":"ContainerDied","Data":"a8878369b478e61eea4ab92a68591739aa04efae1dd7501cee00361ea73d98da"}
Sep 19 05:47:03.362246 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:47:03.362220    2583 generic.go:356] "Generic (PLEG): container finished" podID="09873dc7-416a-4214-8de2-4ffccdd25785" containerID="1f2a150ae41da8ff57a051ae98c4be675d0bf3e9fa37829d1b61d609e650e474" exitCode=0
Sep 19 05:47:03.362372 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:47:03.362298    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6x2nsj" event={"ID":"09873dc7-416a-4214-8de2-4ffccdd25785","Type":"ContainerDied","Data":"1f2a150ae41da8ff57a051ae98c4be675d0bf3e9fa37829d1b61d609e650e474"}
Sep 19 05:47:03.364432 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:47:03.364411    2583 generic.go:356] "Generic (PLEG): container finished" podID="0ecb4201-8707-480a-b5fd-d254ff6011ec" containerID="788c7f5a2df638fdd3e8efed0cd73bbb0100c20ad54683d33a7a25fd1c8f5aa8" exitCode=0
Sep 19 05:47:03.364523 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:47:03.364489    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cx9prb" event={"ID":"0ecb4201-8707-480a-b5fd-d254ff6011ec","Type":"ContainerDied","Data":"788c7f5a2df638fdd3e8efed0cd73bbb0100c20ad54683d33a7a25fd1c8f5aa8"}
Sep 19 05:47:03.366697 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:47:03.366678    2583 generic.go:356] "Generic (PLEG): container finished" podID="533d5517-8284-4b47-8dc4-5603ceb9be28" containerID="9af292933cf6d3998c29a031892027be33dcc05f6e8028f70d8536e12959286a" exitCode=0
Sep 19 05:47:03.366777 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:47:03.366705    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7hnbbl" event={"ID":"533d5517-8284-4b47-8dc4-5603ceb9be28","Type":"ContainerDied","Data":"9af292933cf6d3998c29a031892027be33dcc05f6e8028f70d8536e12959286a"}
Sep 19 05:47:04.519536 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:47:04.519511    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cx9prb"
Sep 19 05:47:04.559784 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:47:04.559753    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6x2nsj"
Sep 19 05:47:04.580188 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:47:04.580170    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847az68j5"
Sep 19 05:47:04.583582 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:47:04.583557    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7hnbbl"
Sep 19 05:47:04.690568 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:47:04.690491    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/35da0a63-f8b4-459e-91f5-35af8747783e-bundle\") pod \"35da0a63-f8b4-459e-91f5-35af8747783e\" (UID: \"35da0a63-f8b4-459e-91f5-35af8747783e\") "
Sep 19 05:47:04.690568 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:47:04.690526    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0ecb4201-8707-480a-b5fd-d254ff6011ec-bundle\") pod \"0ecb4201-8707-480a-b5fd-d254ff6011ec\" (UID: \"0ecb4201-8707-480a-b5fd-d254ff6011ec\") "
Sep 19 05:47:04.690568 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:47:04.690557    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mclk2\" (UniqueName: \"kubernetes.io/projected/0ecb4201-8707-480a-b5fd-d254ff6011ec-kube-api-access-mclk2\") pod \"0ecb4201-8707-480a-b5fd-d254ff6011ec\" (UID: \"0ecb4201-8707-480a-b5fd-d254ff6011ec\") "
Sep 19 05:47:04.690847 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:47:04.690602    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kfdg6\" (UniqueName: \"kubernetes.io/projected/533d5517-8284-4b47-8dc4-5603ceb9be28-kube-api-access-kfdg6\") pod \"533d5517-8284-4b47-8dc4-5603ceb9be28\" (UID: \"533d5517-8284-4b47-8dc4-5603ceb9be28\") "
Sep 19 05:47:04.690847 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:47:04.690637    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/533d5517-8284-4b47-8dc4-5603ceb9be28-util\") pod \"533d5517-8284-4b47-8dc4-5603ceb9be28\" (UID: \"533d5517-8284-4b47-8dc4-5603ceb9be28\") "
Sep 19 05:47:04.690847 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:47:04.690664    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/09873dc7-416a-4214-8de2-4ffccdd25785-bundle\") pod \"09873dc7-416a-4214-8de2-4ffccdd25785\" (UID: \"09873dc7-416a-4214-8de2-4ffccdd25785\") "
Sep 19 05:47:04.690847 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:47:04.690687    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-555bf\" (UniqueName: \"kubernetes.io/projected/09873dc7-416a-4214-8de2-4ffccdd25785-kube-api-access-555bf\") pod \"09873dc7-416a-4214-8de2-4ffccdd25785\" (UID: \"09873dc7-416a-4214-8de2-4ffccdd25785\") "
Sep 19 05:47:04.690847 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:47:04.690723    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6knlz\" (UniqueName: \"kubernetes.io/projected/35da0a63-f8b4-459e-91f5-35af8747783e-kube-api-access-6knlz\") pod \"35da0a63-f8b4-459e-91f5-35af8747783e\" (UID: \"35da0a63-f8b4-459e-91f5-35af8747783e\") "
Sep 19 05:47:04.690847 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:47:04.690759    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0ecb4201-8707-480a-b5fd-d254ff6011ec-util\") pod \"0ecb4201-8707-480a-b5fd-d254ff6011ec\" (UID: \"0ecb4201-8707-480a-b5fd-d254ff6011ec\") "
Sep 19 05:47:04.690847 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:47:04.690814    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/533d5517-8284-4b47-8dc4-5603ceb9be28-bundle\") pod \"533d5517-8284-4b47-8dc4-5603ceb9be28\" (UID: \"533d5517-8284-4b47-8dc4-5603ceb9be28\") "
Sep 19 05:47:04.690847 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:47:04.690845    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/09873dc7-416a-4214-8de2-4ffccdd25785-util\") pod \"09873dc7-416a-4214-8de2-4ffccdd25785\" (UID: \"09873dc7-416a-4214-8de2-4ffccdd25785\") "
Sep 19 05:47:04.691295 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:47:04.690903    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/35da0a63-f8b4-459e-91f5-35af8747783e-util\") pod \"35da0a63-f8b4-459e-91f5-35af8747783e\" (UID: \"35da0a63-f8b4-459e-91f5-35af8747783e\") "
Sep 19 05:47:04.691295 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:47:04.691190    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ecb4201-8707-480a-b5fd-d254ff6011ec-bundle" (OuterVolumeSpecName: "bundle") pod "0ecb4201-8707-480a-b5fd-d254ff6011ec" (UID: "0ecb4201-8707-480a-b5fd-d254ff6011ec"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 05:47:04.691395 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:47:04.691368    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/35da0a63-f8b4-459e-91f5-35af8747783e-bundle" (OuterVolumeSpecName: "bundle") pod "35da0a63-f8b4-459e-91f5-35af8747783e" (UID: "35da0a63-f8b4-459e-91f5-35af8747783e"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 05:47:04.691706 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:47:04.691672    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09873dc7-416a-4214-8de2-4ffccdd25785-bundle" (OuterVolumeSpecName: "bundle") pod "09873dc7-416a-4214-8de2-4ffccdd25785" (UID: "09873dc7-416a-4214-8de2-4ffccdd25785"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 05:47:04.692061 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:47:04.692033    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/533d5517-8284-4b47-8dc4-5603ceb9be28-bundle" (OuterVolumeSpecName: "bundle") pod "533d5517-8284-4b47-8dc4-5603ceb9be28" (UID: "533d5517-8284-4b47-8dc4-5603ceb9be28"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 05:47:04.693449 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:47:04.693420    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/533d5517-8284-4b47-8dc4-5603ceb9be28-kube-api-access-kfdg6" (OuterVolumeSpecName: "kube-api-access-kfdg6") pod "533d5517-8284-4b47-8dc4-5603ceb9be28" (UID: "533d5517-8284-4b47-8dc4-5603ceb9be28"). InnerVolumeSpecName "kube-api-access-kfdg6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 05:47:04.693558 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:47:04.693457    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0ecb4201-8707-480a-b5fd-d254ff6011ec-kube-api-access-mclk2" (OuterVolumeSpecName: "kube-api-access-mclk2") pod "0ecb4201-8707-480a-b5fd-d254ff6011ec" (UID: "0ecb4201-8707-480a-b5fd-d254ff6011ec"). InnerVolumeSpecName "kube-api-access-mclk2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 05:47:04.693558 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:47:04.693513    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/09873dc7-416a-4214-8de2-4ffccdd25785-kube-api-access-555bf" (OuterVolumeSpecName: "kube-api-access-555bf") pod "09873dc7-416a-4214-8de2-4ffccdd25785" (UID: "09873dc7-416a-4214-8de2-4ffccdd25785"). InnerVolumeSpecName "kube-api-access-555bf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 05:47:04.694946 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:47:04.694923    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/35da0a63-f8b4-459e-91f5-35af8747783e-kube-api-access-6knlz" (OuterVolumeSpecName: "kube-api-access-6knlz") pod "35da0a63-f8b4-459e-91f5-35af8747783e" (UID: "35da0a63-f8b4-459e-91f5-35af8747783e"). InnerVolumeSpecName "kube-api-access-6knlz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 05:47:04.697979 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:47:04.697944    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/533d5517-8284-4b47-8dc4-5603ceb9be28-util" (OuterVolumeSpecName: "util") pod "533d5517-8284-4b47-8dc4-5603ceb9be28" (UID: "533d5517-8284-4b47-8dc4-5603ceb9be28"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 05:47:04.698586 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:47:04.698565    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09873dc7-416a-4214-8de2-4ffccdd25785-util" (OuterVolumeSpecName: "util") pod "09873dc7-416a-4214-8de2-4ffccdd25785" (UID: "09873dc7-416a-4214-8de2-4ffccdd25785"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 05:47:04.700348 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:47:04.700323    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/35da0a63-f8b4-459e-91f5-35af8747783e-util" (OuterVolumeSpecName: "util") pod "35da0a63-f8b4-459e-91f5-35af8747783e" (UID: "35da0a63-f8b4-459e-91f5-35af8747783e"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 05:47:04.700481 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:47:04.700465    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ecb4201-8707-480a-b5fd-d254ff6011ec-util" (OuterVolumeSpecName: "util") pod "0ecb4201-8707-480a-b5fd-d254ff6011ec" (UID: "0ecb4201-8707-480a-b5fd-d254ff6011ec"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 05:47:04.791844 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:47:04.791818    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kfdg6\" (UniqueName: \"kubernetes.io/projected/533d5517-8284-4b47-8dc4-5603ceb9be28-kube-api-access-kfdg6\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 05:47:04.791844 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:47:04.791840    2583 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/533d5517-8284-4b47-8dc4-5603ceb9be28-util\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 05:47:04.791844 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:47:04.791849    2583 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/09873dc7-416a-4214-8de2-4ffccdd25785-bundle\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 05:47:04.792012 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:47:04.791857    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-555bf\" (UniqueName: \"kubernetes.io/projected/09873dc7-416a-4214-8de2-4ffccdd25785-kube-api-access-555bf\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 05:47:04.792012 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:47:04.791865    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6knlz\" (UniqueName: \"kubernetes.io/projected/35da0a63-f8b4-459e-91f5-35af8747783e-kube-api-access-6knlz\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 05:47:04.792012 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:47:04.791874    2583 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0ecb4201-8707-480a-b5fd-d254ff6011ec-util\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 05:47:04.792012 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:47:04.791884    2583 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/533d5517-8284-4b47-8dc4-5603ceb9be28-bundle\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 05:47:04.792012 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:47:04.791892    2583 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/09873dc7-416a-4214-8de2-4ffccdd25785-util\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 05:47:04.792012 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:47:04.791899    2583 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/35da0a63-f8b4-459e-91f5-35af8747783e-util\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 05:47:04.792012 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:47:04.791906    2583 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/35da0a63-f8b4-459e-91f5-35af8747783e-bundle\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 05:47:04.792012 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:47:04.791913    2583 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0ecb4201-8707-480a-b5fd-d254ff6011ec-bundle\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 05:47:04.792012 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:47:04.791920    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mclk2\" (UniqueName: \"kubernetes.io/projected/0ecb4201-8707-480a-b5fd-d254ff6011ec-kube-api-access-mclk2\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 05:47:05.379009 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:47:05.378978    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7hnbbl"
Sep 19 05:47:05.379218 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:47:05.379024    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7hnbbl" event={"ID":"533d5517-8284-4b47-8dc4-5603ceb9be28","Type":"ContainerDied","Data":"b4f67fe2b7ebd8e9b328681f6a74c08e07f9dc19d78e74da62317359e16515fd"}
Sep 19 05:47:05.379218 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:47:05.379070    2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b4f67fe2b7ebd8e9b328681f6a74c08e07f9dc19d78e74da62317359e16515fd"
Sep 19 05:47:05.381039 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:47:05.381016    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847az68j5"
Sep 19 05:47:05.381039 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:47:05.381025    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847az68j5" event={"ID":"35da0a63-f8b4-459e-91f5-35af8747783e","Type":"ContainerDied","Data":"543268c2119d43c3207a121082807a06dba9da84fbf60d09465d8f125481b74b"}
Sep 19 05:47:05.381269 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:47:05.381062    2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="543268c2119d43c3207a121082807a06dba9da84fbf60d09465d8f125481b74b"
Sep 19 05:47:05.390221 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:47:05.390198    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6x2nsj"
Sep 19 05:47:05.390352 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:47:05.390219    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6x2nsj" event={"ID":"09873dc7-416a-4214-8de2-4ffccdd25785","Type":"ContainerDied","Data":"0c7ad62378eabbcb1afd9d23b04bea69613bae37e2da66f7b2439999522c0c61"}
Sep 19 05:47:05.390352 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:47:05.390289    2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0c7ad62378eabbcb1afd9d23b04bea69613bae37e2da66f7b2439999522c0c61"
Sep 19 05:47:05.392712 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:47:05.392693    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cx9prb"
Sep 19 05:47:05.392830 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:47:05.392692    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cx9prb" event={"ID":"0ecb4201-8707-480a-b5fd-d254ff6011ec","Type":"ContainerDied","Data":"bdac2ae5efdbc5d1b41602d420d772becfd3b30aa45a6884daefcf56dc684d94"}
Sep 19 05:47:05.392830 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:47:05.392804    2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bdac2ae5efdbc5d1b41602d420d772becfd3b30aa45a6884daefcf56dc684d94"
Sep 19 05:48:10.699498 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:48:10.699465    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5trhr"]
Sep 19 05:48:10.700068 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:48:10.699831    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="09873dc7-416a-4214-8de2-4ffccdd25785" containerName="util"
Sep 19 05:48:10.700068 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:48:10.699845    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="09873dc7-416a-4214-8de2-4ffccdd25785" containerName="util"
Sep 19 05:48:10.700068 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:48:10.699856    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="09873dc7-416a-4214-8de2-4ffccdd25785" containerName="pull"
Sep 19 05:48:10.700068 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:48:10.699861    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="09873dc7-416a-4214-8de2-4ffccdd25785" containerName="pull"
Sep 19 05:48:10.700068 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:48:10.699867    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="533d5517-8284-4b47-8dc4-5603ceb9be28" containerName="extract"
Sep 19 05:48:10.700068 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:48:10.699872    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="533d5517-8284-4b47-8dc4-5603ceb9be28" containerName="extract"
Sep 19 05:48:10.700068 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:48:10.699880    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0ecb4201-8707-480a-b5fd-d254ff6011ec" containerName="util"
Sep 19 05:48:10.700068 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:48:10.699884    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="0ecb4201-8707-480a-b5fd-d254ff6011ec" containerName="util"
Sep 19 05:48:10.700068 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:48:10.699890    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="35da0a63-f8b4-459e-91f5-35af8747783e" containerName="pull"
Sep 19 05:48:10.700068 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:48:10.699895    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="35da0a63-f8b4-459e-91f5-35af8747783e" containerName="pull"
Sep 19 05:48:10.700068 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:48:10.699904    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="533d5517-8284-4b47-8dc4-5603ceb9be28" containerName="pull"
Sep 19 05:48:10.700068 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:48:10.699909    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="533d5517-8284-4b47-8dc4-5603ceb9be28" containerName="pull"
Sep 19 05:48:10.700068 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:48:10.699917    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="533d5517-8284-4b47-8dc4-5603ceb9be28" containerName="util"
Sep 19 05:48:10.700068 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:48:10.699922    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="533d5517-8284-4b47-8dc4-5603ceb9be28" containerName="util"
Sep 19 05:48:10.700068 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:48:10.699931    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="35da0a63-f8b4-459e-91f5-35af8747783e" containerName="util"
Sep 19 05:48:10.700068 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:48:10.699936    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="35da0a63-f8b4-459e-91f5-35af8747783e" containerName="util"
Sep 19 05:48:10.700068 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:48:10.699942    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0ecb4201-8707-480a-b5fd-d254ff6011ec" containerName="extract"
Sep 19 05:48:10.700068 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:48:10.699947    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="0ecb4201-8707-480a-b5fd-d254ff6011ec" containerName="extract"
Sep 19 05:48:10.700068 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:48:10.699954    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0ecb4201-8707-480a-b5fd-d254ff6011ec" containerName="pull"
Sep 19 05:48:10.700068 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:48:10.699959    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="0ecb4201-8707-480a-b5fd-d254ff6011ec" containerName="pull"
Sep 19 05:48:10.700068 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:48:10.699970    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="09873dc7-416a-4214-8de2-4ffccdd25785" containerName="extract"
Sep 19 05:48:10.700068 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:48:10.699975    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="09873dc7-416a-4214-8de2-4ffccdd25785" containerName="extract"
Sep 19 05:48:10.700068 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:48:10.699980    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="35da0a63-f8b4-459e-91f5-35af8747783e" containerName="extract"
Sep 19 05:48:10.700068 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:48:10.699988    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="35da0a63-f8b4-459e-91f5-35af8747783e" containerName="extract"
Sep 19 05:48:10.700068 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:48:10.700037    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="35da0a63-f8b4-459e-91f5-35af8747783e" containerName="extract"
Sep 19 05:48:10.700068 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:48:10.700046    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="0ecb4201-8707-480a-b5fd-d254ff6011ec" containerName="extract"
Sep 19 05:48:10.700068 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:48:10.700054    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="533d5517-8284-4b47-8dc4-5603ceb9be28" containerName="extract"
Sep 19 05:48:10.700068 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:48:10.700061    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="09873dc7-416a-4214-8de2-4ffccdd25785" containerName="extract"
Sep 19 05:48:10.708111 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:48:10.708080    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5trhr"
Sep 19 05:48:10.710623 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:48:10.710605    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-h2prp\""
Sep 19 05:48:10.720235 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:48:10.720209    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5trhr"]
Sep 19 05:48:10.727551 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:48:10.727527    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f43550bd-3864-4d97-90c6-2f2a283b088a-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-5trhr\" (UID: \"f43550bd-3864-4d97-90c6-2f2a283b088a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5trhr"
Sep 19 05:48:10.727663 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:48:10.727584    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f43550bd-3864-4d97-90c6-2f2a283b088a-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-5trhr\" (UID: \"f43550bd-3864-4d97-90c6-2f2a283b088a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5trhr"
Sep 19 05:48:10.727663 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:48:10.727627    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f43550bd-3864-4d97-90c6-2f2a283b088a-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-5trhr\" (UID: \"f43550bd-3864-4d97-90c6-2f2a283b088a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5trhr"
Sep 19 05:48:10.727663 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:48:10.727656    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f43550bd-3864-4d97-90c6-2f2a283b088a-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-5trhr\" (UID: \"f43550bd-3864-4d97-90c6-2f2a283b088a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5trhr"
Sep 19 05:48:10.727783 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:48:10.727709    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tmfrq\" (UniqueName: \"kubernetes.io/projected/f43550bd-3864-4d97-90c6-2f2a283b088a-kube-api-access-tmfrq\") pod \"maas-default-gateway-openshift-default-78bb78ff78-5trhr\" (UID: \"f43550bd-3864-4d97-90c6-2f2a283b088a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5trhr"
Sep 19 05:48:10.727783 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:48:10.727755    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f43550bd-3864-4d97-90c6-2f2a283b088a-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-5trhr\" (UID: \"f43550bd-3864-4d97-90c6-2f2a283b088a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5trhr"
Sep 19 05:48:10.727856 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:48:10.727784    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f43550bd-3864-4d97-90c6-2f2a283b088a-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-5trhr\" (UID: \"f43550bd-3864-4d97-90c6-2f2a283b088a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5trhr"
Sep 19 05:48:10.727856 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:48:10.727831    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f43550bd-3864-4d97-90c6-2f2a283b088a-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-5trhr\" (UID: \"f43550bd-3864-4d97-90c6-2f2a283b088a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5trhr"
Sep 19 05:48:10.727931 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:48:10.727855    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f43550bd-3864-4d97-90c6-2f2a283b088a-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-5trhr\" (UID: \"f43550bd-3864-4d97-90c6-2f2a283b088a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5trhr"
Sep 19 05:48:10.829207 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:48:10.829169    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f43550bd-3864-4d97-90c6-2f2a283b088a-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-5trhr\" (UID: \"f43550bd-3864-4d97-90c6-2f2a283b088a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5trhr"
Sep 19 05:48:10.829353 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:48:10.829215    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tmfrq\" (UniqueName: \"kubernetes.io/projected/f43550bd-3864-4d97-90c6-2f2a283b088a-kube-api-access-tmfrq\") pod \"maas-default-gateway-openshift-default-78bb78ff78-5trhr\" (UID: \"f43550bd-3864-4d97-90c6-2f2a283b088a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5trhr"
Sep 19 05:48:10.829353 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:48:10.829261    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f43550bd-3864-4d97-90c6-2f2a283b088a-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-5trhr\" (UID: \"f43550bd-3864-4d97-90c6-2f2a283b088a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5trhr"
Sep 19 05:48:10.829353 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:48:10.829286    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f43550bd-3864-4d97-90c6-2f2a283b088a-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-5trhr\" (UID: \"f43550bd-3864-4d97-90c6-2f2a283b088a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5trhr"
Sep 19 05:48:10.829353 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:48:10.829315    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f43550bd-3864-4d97-90c6-2f2a283b088a-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-5trhr\" (UID: \"f43550bd-3864-4d97-90c6-2f2a283b088a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5trhr"
Sep 19 05:48:10.829572 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:48:10.829441    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f43550bd-3864-4d97-90c6-2f2a283b088a-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-5trhr\" (UID: \"f43550bd-3864-4d97-90c6-2f2a283b088a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5trhr"
Sep 19 05:48:10.829572 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:48:10.829531    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f43550bd-3864-4d97-90c6-2f2a283b088a-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-5trhr\" (UID: \"f43550bd-3864-4d97-90c6-2f2a283b088a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5trhr"
Sep 19 05:48:10.829677 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:48:10.829595    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f43550bd-3864-4d97-90c6-2f2a283b088a-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-5trhr\" (UID: \"f43550bd-3864-4d97-90c6-2f2a283b088a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5trhr"
Sep 19 05:48:10.829677 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:48:10.829625    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f43550bd-3864-4d97-90c6-2f2a283b088a-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-5trhr\" (UID: \"f43550bd-3864-4d97-90c6-2f2a283b088a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5trhr"
Sep 19 05:48:10.829677 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:48:10.829668    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f43550bd-3864-4d97-90c6-2f2a283b088a-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-5trhr\" (UID: \"f43550bd-3864-4d97-90c6-2f2a283b088a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5trhr"
Sep 19 05:48:10.829809 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:48:10.829693    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f43550bd-3864-4d97-90c6-2f2a283b088a-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-5trhr\" (UID: \"f43550bd-3864-4d97-90c6-2f2a283b088a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5trhr"
Sep 19 05:48:10.829809 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:48:10.829763    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f43550bd-3864-4d97-90c6-2f2a283b088a-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-5trhr\" (UID: \"f43550bd-3864-4d97-90c6-2f2a283b088a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5trhr"
Sep 19 05:48:10.830140 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:48:10.830097    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f43550bd-3864-4d97-90c6-2f2a283b088a-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-5trhr\" (UID: \"f43550bd-3864-4d97-90c6-2f2a283b088a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5trhr"
Sep 19 05:48:10.830362 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:48:10.830341    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f43550bd-3864-4d97-90c6-2f2a283b088a-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-5trhr\" (UID: \"f43550bd-3864-4d97-90c6-2f2a283b088a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5trhr"
Sep 19 05:48:10.831918 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:48:10.831894    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f43550bd-3864-4d97-90c6-2f2a283b088a-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-5trhr\" (UID: \"f43550bd-3864-4d97-90c6-2f2a283b088a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5trhr"
Sep 19 05:48:10.832205 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:48:10.832185    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f43550bd-3864-4d97-90c6-2f2a283b088a-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-5trhr\" (UID: \"f43550bd-3864-4d97-90c6-2f2a283b088a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5trhr"
Sep 19 05:48:10.837842 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:48:10.837818    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f43550bd-3864-4d97-90c6-2f2a283b088a-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-5trhr\" (UID: \"f43550bd-3864-4d97-90c6-2f2a283b088a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5trhr"
Sep 19 05:48:10.838444 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:48:10.838422    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tmfrq\" (UniqueName: \"kubernetes.io/projected/f43550bd-3864-4d97-90c6-2f2a283b088a-kube-api-access-tmfrq\") pod \"maas-default-gateway-openshift-default-78bb78ff78-5trhr\" (UID: \"f43550bd-3864-4d97-90c6-2f2a283b088a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5trhr"
Sep 19 05:48:11.019998 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:48:11.019924    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5trhr"
Sep 19 05:48:11.296740 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:48:11.296715    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5trhr"]
Sep 19 05:48:11.298859 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:48:11.298833    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf43550bd_3864_4d97_90c6_2f2a283b088a.slice/crio-5be77dcfc226129622e9095b87095bb5c81c87604c6101743fc4fa7eb170a1e3 WatchSource:0}: Error finding container 5be77dcfc226129622e9095b87095bb5c81c87604c6101743fc4fa7eb170a1e3: Status 404 returned error can't find the container with id 5be77dcfc226129622e9095b87095bb5c81c87604c6101743fc4fa7eb170a1e3
Sep 19 05:48:11.301121 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:48:11.301087    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 19 05:48:11.301233 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:48:11.301195    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 19 05:48:11.301233 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:48:11.301225    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 19 05:48:11.728335 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:48:11.728232    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5trhr" event={"ID":"f43550bd-3864-4d97-90c6-2f2a283b088a","Type":"ContainerStarted","Data":"a099de22f8b994555cbcb22c105205fe43401408195105b872548444877fbbd6"}
Sep 19 05:48:11.728335 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:48:11.728300    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5trhr" event={"ID":"f43550bd-3864-4d97-90c6-2f2a283b088a","Type":"ContainerStarted","Data":"5be77dcfc226129622e9095b87095bb5c81c87604c6101743fc4fa7eb170a1e3"}
Sep 19 05:48:11.751624 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:48:11.751574    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5trhr" podStartSLOduration=1.751559586 podStartE2EDuration="1.751559586s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:48:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:48:11.749302489 +0000 UTC m=+450.407757061" watchObservedRunningTime="2026-09-19 05:48:11.751559586 +0000 UTC m=+450.410014143"
Sep 19 05:48:12.020638 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:48:12.020557    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5trhr"
Sep 19 05:48:12.024984 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:48:12.024960    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5trhr"
Sep 19 05:48:12.733818 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:48:12.733780    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5trhr"
Sep 19 05:48:12.739869 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:48:12.739838    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5trhr"
Sep 19 05:48:43.432672 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:48:43.432643    2583 scope.go:117] "RemoveContainer" containerID="169f61e1f9fc3a42dc2d12154471615d7be8bbde4c38b832636efd3f2cf9658b"
Sep 19 05:49:26.164891 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:49:26.164812    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-5598d8c9fb-wjwvk"]
Sep 19 05:49:26.168088 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:49:26.168072    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-wjwvk"
Sep 19 05:49:26.171113 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:49:26.171091    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 19 05:49:26.171261 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:49:26.171146    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 19 05:49:26.172376 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:49:26.172357    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-nkngb\""
Sep 19 05:49:26.172535 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:49:26.172521    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 19 05:49:26.172608 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:49:26.172524    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 19 05:49:26.184967 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:49:26.184944    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5598d8c9fb-wjwvk"]
Sep 19 05:49:26.255165 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:49:26.255115    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bfd333e1-9918-4a1b-aa17-a6f889847c46-metrics-certs\") pod \"maas-api-5598d8c9fb-wjwvk\" (UID: \"bfd333e1-9918-4a1b-aa17-a6f889847c46\") " pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-wjwvk"
Sep 19 05:49:26.255165 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:49:26.255172    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7nv6t\" (UniqueName: \"kubernetes.io/projected/bfd333e1-9918-4a1b-aa17-a6f889847c46-kube-api-access-7nv6t\") pod \"maas-api-5598d8c9fb-wjwvk\" (UID: \"bfd333e1-9918-4a1b-aa17-a6f889847c46\") " pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-wjwvk"
Sep 19 05:49:26.255363 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:49:26.255261    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bfd333e1-9918-4a1b-aa17-a6f889847c46-maas-api-tls\") pod \"maas-api-5598d8c9fb-wjwvk\" (UID: \"bfd333e1-9918-4a1b-aa17-a6f889847c46\") " pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-wjwvk"
Sep 19 05:49:26.255363 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:49:26.255324    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bfd333e1-9918-4a1b-aa17-a6f889847c46-openshift-service-ca\") pod \"maas-api-5598d8c9fb-wjwvk\" (UID: \"bfd333e1-9918-4a1b-aa17-a6f889847c46\") " pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-wjwvk"
Sep 19 05:49:26.356532 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:49:26.356503    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bfd333e1-9918-4a1b-aa17-a6f889847c46-openshift-service-ca\") pod \"maas-api-5598d8c9fb-wjwvk\" (UID: \"bfd333e1-9918-4a1b-aa17-a6f889847c46\") " pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-wjwvk"
Sep 19 05:49:26.356719 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:49:26.356552    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bfd333e1-9918-4a1b-aa17-a6f889847c46-metrics-certs\") pod \"maas-api-5598d8c9fb-wjwvk\" (UID: \"bfd333e1-9918-4a1b-aa17-a6f889847c46\") " pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-wjwvk"
Sep 19 05:49:26.356719 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:49:26.356569    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7nv6t\" (UniqueName: \"kubernetes.io/projected/bfd333e1-9918-4a1b-aa17-a6f889847c46-kube-api-access-7nv6t\") pod \"maas-api-5598d8c9fb-wjwvk\" (UID: \"bfd333e1-9918-4a1b-aa17-a6f889847c46\") " pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-wjwvk"
Sep 19 05:49:26.356719 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:49:26.356613    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bfd333e1-9918-4a1b-aa17-a6f889847c46-maas-api-tls\") pod \"maas-api-5598d8c9fb-wjwvk\" (UID: \"bfd333e1-9918-4a1b-aa17-a6f889847c46\") " pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-wjwvk"
Sep 19 05:49:26.357372 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:49:26.357348    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bfd333e1-9918-4a1b-aa17-a6f889847c46-openshift-service-ca\") pod \"maas-api-5598d8c9fb-wjwvk\" (UID: \"bfd333e1-9918-4a1b-aa17-a6f889847c46\") " pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-wjwvk"
Sep 19 05:49:26.358985 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:49:26.358966    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bfd333e1-9918-4a1b-aa17-a6f889847c46-maas-api-tls\") pod \"maas-api-5598d8c9fb-wjwvk\" (UID: \"bfd333e1-9918-4a1b-aa17-a6f889847c46\") " pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-wjwvk"
Sep 19 05:49:26.359094 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:49:26.358999    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bfd333e1-9918-4a1b-aa17-a6f889847c46-metrics-certs\") pod \"maas-api-5598d8c9fb-wjwvk\" (UID: \"bfd333e1-9918-4a1b-aa17-a6f889847c46\") " pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-wjwvk"
Sep 19 05:49:26.364818 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:49:26.364798    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7nv6t\" (UniqueName: \"kubernetes.io/projected/bfd333e1-9918-4a1b-aa17-a6f889847c46-kube-api-access-7nv6t\") pod \"maas-api-5598d8c9fb-wjwvk\" (UID: \"bfd333e1-9918-4a1b-aa17-a6f889847c46\") " pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-wjwvk"
Sep 19 05:49:26.478769 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:49:26.478697    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-wjwvk"
Sep 19 05:49:26.757940 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:49:26.757910    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5598d8c9fb-wjwvk"]
Sep 19 05:49:26.760112 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:49:26.760081    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbfd333e1_9918_4a1b_aa17_a6f889847c46.slice/crio-7f73376b256672680ca407bbaf941dd0fcb82e6215c5f11c7004c68a1d3988ad WatchSource:0}: Error finding container 7f73376b256672680ca407bbaf941dd0fcb82e6215c5f11c7004c68a1d3988ad: Status 404 returned error can't find the container with id 7f73376b256672680ca407bbaf941dd0fcb82e6215c5f11c7004c68a1d3988ad
Sep 19 05:49:27.100899 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:49:27.100860    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-wjwvk" event={"ID":"bfd333e1-9918-4a1b-aa17-a6f889847c46","Type":"ContainerStarted","Data":"7f73376b256672680ca407bbaf941dd0fcb82e6215c5f11c7004c68a1d3988ad"}
Sep 19 05:49:30.120603 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:49:30.120554    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-wjwvk" event={"ID":"bfd333e1-9918-4a1b-aa17-a6f889847c46","Type":"ContainerStarted","Data":"66fa0cae40cbda3fa074f5be80cf4cc775f35dbb513136f8805df186b4bb5610"}
Sep 19 05:49:30.121429 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:49:30.121399    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-wjwvk"
Sep 19 05:49:30.140597 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:49:30.140530    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-wjwvk" podStartSLOduration=1.28702166 podStartE2EDuration="4.140495053s" totalImagesPullingTime="2.853473393s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:49:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:49:30.13867535 +0000 UTC m=+528.797129910" watchObservedRunningTime="2026-09-19 05:49:30.140495053 +0000 UTC m=+528.798949612"
Sep 19 05:49:36.158913 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:49:36.158828    2583 generic.go:356] "Generic (PLEG): container finished" podID="f43550bd-3864-4d97-90c6-2f2a283b088a" containerID="a099de22f8b994555cbcb22c105205fe43401408195105b872548444877fbbd6" exitCode=0
Sep 19 05:49:36.158913 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:49:36.158896    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5trhr" event={"ID":"f43550bd-3864-4d97-90c6-2f2a283b088a","Type":"ContainerDied","Data":"a099de22f8b994555cbcb22c105205fe43401408195105b872548444877fbbd6"}
Sep 19 05:49:36.159369 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:49:36.159262    2583 scope.go:117] "RemoveContainer" containerID="a099de22f8b994555cbcb22c105205fe43401408195105b872548444877fbbd6"
Sep 19 05:49:36.160268 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:49:36.160230    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 19 05:49:36.160387 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:49:36.160314    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 19 05:49:36.160387 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:49:36.160360    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 19 05:49:37.144827 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:49:37.144795    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-wjwvk"
Sep 19 05:49:37.166521 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:49:37.166491    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5trhr" event={"ID":"f43550bd-3864-4d97-90c6-2f2a283b088a","Type":"ContainerStarted","Data":"ca693d3e41cc0ebc069891acb3287facd38b72fd13a5ed94a166491245692b32"}
Sep 19 05:49:38.020865 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:49:38.020780    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5trhr"
Sep 19 05:49:38.021095 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:49:38.021062    2583 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-5trhr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body=
Sep 19 05:49:38.021208 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:49:38.021158    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5trhr" podUID="f43550bd-3864-4d97-90c6-2f2a283b088a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused"
Sep 19 05:49:39.021440 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:49:39.021396    2583 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-5trhr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body=
Sep 19 05:49:39.021937 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:49:39.021466    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5trhr" podUID="f43550bd-3864-4d97-90c6-2f2a283b088a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused"
Sep 19 05:49:40.020808 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:49:40.020769    2583 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-5trhr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body=
Sep 19 05:49:40.021002 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:49:40.020839    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5trhr" podUID="f43550bd-3864-4d97-90c6-2f2a283b088a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused"
Sep 19 05:49:41.021123 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:49:41.021080    2583 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-5trhr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body=
Sep 19 05:49:41.021754 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:49:41.021719    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5trhr" podUID="f43550bd-3864-4d97-90c6-2f2a283b088a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused"
Sep 19 05:49:42.021068 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:49:42.021023    2583 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-5trhr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body=
Sep 19 05:49:42.021300 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:49:42.021116    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5trhr" podUID="f43550bd-3864-4d97-90c6-2f2a283b088a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused"
Sep 19 05:49:42.734683 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:49:42.734648    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5trhr"
Sep 19 05:49:43.024963 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:49:43.024883    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5trhr"
Sep 19 05:49:43.200273 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:49:43.200243    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5trhr"
Sep 19 05:50:08.342369 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:50:08.342338    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-qnggd"]
Sep 19 05:50:08.345546 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:50:08.345529    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qnggd"
Sep 19 05:50:08.348937 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:50:08.348905    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 19 05:50:08.348937 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:50:08.348918    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 19 05:50:08.348937 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:50:08.348909    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Sep 19 05:50:08.349189 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:50:08.348908    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-pg7v7\""
Sep 19 05:50:08.354855 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:50:08.354828    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-qnggd"]
Sep 19 05:50:08.407709 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:50:08.407675    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/873e444d-8837-4170-bbbe-ce6ff40e745d-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qnggd\" (UID: \"873e444d-8837-4170-bbbe-ce6ff40e745d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qnggd"
Sep 19 05:50:08.407709 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:50:08.407708    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/873e444d-8837-4170-bbbe-ce6ff40e745d-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qnggd\" (UID: \"873e444d-8837-4170-bbbe-ce6ff40e745d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qnggd"
Sep 19 05:50:08.407900 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:50:08.407790    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/873e444d-8837-4170-bbbe-ce6ff40e745d-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qnggd\" (UID: \"873e444d-8837-4170-bbbe-ce6ff40e745d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qnggd"
Sep 19 05:50:08.407900 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:50:08.407824    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s4r9x\" (UniqueName: \"kubernetes.io/projected/873e444d-8837-4170-bbbe-ce6ff40e745d-kube-api-access-s4r9x\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qnggd\" (UID: \"873e444d-8837-4170-bbbe-ce6ff40e745d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qnggd"
Sep 19 05:50:08.407900 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:50:08.407870    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/873e444d-8837-4170-bbbe-ce6ff40e745d-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qnggd\" (UID: \"873e444d-8837-4170-bbbe-ce6ff40e745d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qnggd"
Sep 19 05:50:08.407900 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:50:08.407891    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/873e444d-8837-4170-bbbe-ce6ff40e745d-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qnggd\" (UID: \"873e444d-8837-4170-bbbe-ce6ff40e745d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qnggd"
Sep 19 05:50:08.508452 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:50:08.508416    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s4r9x\" (UniqueName: \"kubernetes.io/projected/873e444d-8837-4170-bbbe-ce6ff40e745d-kube-api-access-s4r9x\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qnggd\" (UID: \"873e444d-8837-4170-bbbe-ce6ff40e745d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qnggd"
Sep 19 05:50:08.508673 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:50:08.508652    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/873e444d-8837-4170-bbbe-ce6ff40e745d-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qnggd\" (UID: \"873e444d-8837-4170-bbbe-ce6ff40e745d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qnggd"
Sep 19 05:50:08.508719 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:50:08.508687    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/873e444d-8837-4170-bbbe-ce6ff40e745d-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qnggd\" (UID: \"873e444d-8837-4170-bbbe-ce6ff40e745d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qnggd"
Sep 19 05:50:08.508774 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:50:08.508753    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/873e444d-8837-4170-bbbe-ce6ff40e745d-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qnggd\" (UID: \"873e444d-8837-4170-bbbe-ce6ff40e745d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qnggd"
Sep 19 05:50:08.508827 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:50:08.508796    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/873e444d-8837-4170-bbbe-ce6ff40e745d-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qnggd\" (UID: \"873e444d-8837-4170-bbbe-ce6ff40e745d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qnggd"
Sep 19 05:50:08.508921 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:50:08.508899    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/873e444d-8837-4170-bbbe-ce6ff40e745d-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qnggd\" (UID: \"873e444d-8837-4170-bbbe-ce6ff40e745d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qnggd"
Sep 19 05:50:08.509115 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:50:08.509088    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/873e444d-8837-4170-bbbe-ce6ff40e745d-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qnggd\" (UID: \"873e444d-8837-4170-bbbe-ce6ff40e745d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qnggd"
Sep 19 05:50:08.509299 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:50:08.509122    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/873e444d-8837-4170-bbbe-ce6ff40e745d-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qnggd\" (UID: \"873e444d-8837-4170-bbbe-ce6ff40e745d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qnggd"
Sep 19 05:50:08.509299 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:50:08.509194    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/873e444d-8837-4170-bbbe-ce6ff40e745d-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qnggd\" (UID: \"873e444d-8837-4170-bbbe-ce6ff40e745d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qnggd"
Sep 19 05:50:08.511023 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:50:08.510992    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/873e444d-8837-4170-bbbe-ce6ff40e745d-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qnggd\" (UID: \"873e444d-8837-4170-bbbe-ce6ff40e745d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qnggd"
Sep 19 05:50:08.511333 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:50:08.511316    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/873e444d-8837-4170-bbbe-ce6ff40e745d-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qnggd\" (UID: \"873e444d-8837-4170-bbbe-ce6ff40e745d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qnggd"
Sep 19 05:50:08.520343 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:50:08.520323    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s4r9x\" (UniqueName: \"kubernetes.io/projected/873e444d-8837-4170-bbbe-ce6ff40e745d-kube-api-access-s4r9x\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qnggd\" (UID: \"873e444d-8837-4170-bbbe-ce6ff40e745d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qnggd"
Sep 19 05:50:08.658106 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:50:08.658017    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qnggd"
Sep 19 05:50:08.927391 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:50:08.927365    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-qnggd"]
Sep 19 05:50:08.929362 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:50:08.929326    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod873e444d_8837_4170_bbbe_ce6ff40e745d.slice/crio-9a93faa60c406d8fce48facae64ce3e4baca8cbb7b28da4140408ec76bfc2f2a WatchSource:0}: Error finding container 9a93faa60c406d8fce48facae64ce3e4baca8cbb7b28da4140408ec76bfc2f2a: Status 404 returned error can't find the container with id 9a93faa60c406d8fce48facae64ce3e4baca8cbb7b28da4140408ec76bfc2f2a
Sep 19 05:50:09.353437 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:50:09.353392    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qnggd" event={"ID":"873e444d-8837-4170-bbbe-ce6ff40e745d","Type":"ContainerStarted","Data":"9a93faa60c406d8fce48facae64ce3e4baca8cbb7b28da4140408ec76bfc2f2a"}
Sep 19 05:50:10.361176 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:50:10.361082    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qnggd" event={"ID":"873e444d-8837-4170-bbbe-ce6ff40e745d","Type":"ContainerStarted","Data":"a31f6689214a4a8d92294e9e9f92b22a00edd88b979740223267d270f9e39055"}
Sep 19 05:50:10.378959 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:50:10.378909    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qnggd" podStartSLOduration=1.268854442 podStartE2EDuration="2.378895005s" totalImagesPullingTime="1.110040563s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:50:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:50:10.376780053 +0000 UTC m=+569.035234623" watchObservedRunningTime="2026-09-19 05:50:10.378895005 +0000 UTC m=+569.037349563"
Sep 19 05:50:18.658542 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:50:18.658505    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qnggd"
Sep 19 05:50:18.659038 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:50:18.658577    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qnggd"
Sep 19 05:50:18.670903 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:50:18.670877    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qnggd"
Sep 19 05:50:19.422144 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:50:19.422099    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qnggd"
Sep 19 05:50:51.245546 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:50:51.245515    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-s6jvb"]
Sep 19 05:50:51.250025 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:50:51.250007    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-s6jvb"
Sep 19 05:50:51.252557 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:50:51.252536    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Sep 19 05:50:51.258605 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:50:51.258265    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-s6jvb"]
Sep 19 05:50:51.377393 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:50:51.377356    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/84cf7fac-b867-4a9b-9d68-d74950714d2a-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-s6jvb\" (UID: \"84cf7fac-b867-4a9b-9d68-d74950714d2a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-s6jvb"
Sep 19 05:50:51.377568 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:50:51.377403    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/84cf7fac-b867-4a9b-9d68-d74950714d2a-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-s6jvb\" (UID: \"84cf7fac-b867-4a9b-9d68-d74950714d2a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-s6jvb"
Sep 19 05:50:51.377568 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:50:51.377500    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x7s85\" (UniqueName: \"kubernetes.io/projected/84cf7fac-b867-4a9b-9d68-d74950714d2a-kube-api-access-x7s85\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-s6jvb\" (UID: \"84cf7fac-b867-4a9b-9d68-d74950714d2a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-s6jvb"
Sep 19 05:50:51.377568 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:50:51.377536    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/84cf7fac-b867-4a9b-9d68-d74950714d2a-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-s6jvb\" (UID: \"84cf7fac-b867-4a9b-9d68-d74950714d2a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-s6jvb"
Sep 19 05:50:51.377568 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:50:51.377559    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/84cf7fac-b867-4a9b-9d68-d74950714d2a-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-s6jvb\" (UID: \"84cf7fac-b867-4a9b-9d68-d74950714d2a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-s6jvb"
Sep 19 05:50:51.377718 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:50:51.377588    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/84cf7fac-b867-4a9b-9d68-d74950714d2a-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-s6jvb\" (UID: \"84cf7fac-b867-4a9b-9d68-d74950714d2a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-s6jvb"
Sep 19 05:50:51.478116 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:50:51.478081    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/84cf7fac-b867-4a9b-9d68-d74950714d2a-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-s6jvb\" (UID: \"84cf7fac-b867-4a9b-9d68-d74950714d2a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-s6jvb"
Sep 19 05:50:51.478116 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:50:51.478119    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/84cf7fac-b867-4a9b-9d68-d74950714d2a-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-s6jvb\" (UID: \"84cf7fac-b867-4a9b-9d68-d74950714d2a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-s6jvb"
Sep 19 05:50:51.478375 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:50:51.478149    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/84cf7fac-b867-4a9b-9d68-d74950714d2a-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-s6jvb\" (UID: \"84cf7fac-b867-4a9b-9d68-d74950714d2a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-s6jvb"
Sep 19 05:50:51.478375 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:50:51.478188    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/84cf7fac-b867-4a9b-9d68-d74950714d2a-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-s6jvb\" (UID: \"84cf7fac-b867-4a9b-9d68-d74950714d2a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-s6jvb"
Sep 19 05:50:51.478375 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:50:51.478214    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/84cf7fac-b867-4a9b-9d68-d74950714d2a-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-s6jvb\" (UID: \"84cf7fac-b867-4a9b-9d68-d74950714d2a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-s6jvb"
Sep 19 05:50:51.478375 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:50:51.478264    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x7s85\" (UniqueName: \"kubernetes.io/projected/84cf7fac-b867-4a9b-9d68-d74950714d2a-kube-api-access-x7s85\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-s6jvb\" (UID: \"84cf7fac-b867-4a9b-9d68-d74950714d2a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-s6jvb"
Sep 19 05:50:51.478608 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:50:51.478530    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/84cf7fac-b867-4a9b-9d68-d74950714d2a-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-s6jvb\" (UID: \"84cf7fac-b867-4a9b-9d68-d74950714d2a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-s6jvb"
Sep 19 05:50:51.478608 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:50:51.478594    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/84cf7fac-b867-4a9b-9d68-d74950714d2a-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-s6jvb\" (UID: \"84cf7fac-b867-4a9b-9d68-d74950714d2a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-s6jvb"
Sep 19 05:50:51.478716 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:50:51.478640    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/84cf7fac-b867-4a9b-9d68-d74950714d2a-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-s6jvb\" (UID: \"84cf7fac-b867-4a9b-9d68-d74950714d2a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-s6jvb"
Sep 19 05:50:51.480408 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:50:51.480380    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/84cf7fac-b867-4a9b-9d68-d74950714d2a-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-s6jvb\" (UID: \"84cf7fac-b867-4a9b-9d68-d74950714d2a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-s6jvb"
Sep 19 05:50:51.480823 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:50:51.480803    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/84cf7fac-b867-4a9b-9d68-d74950714d2a-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-s6jvb\" (UID: \"84cf7fac-b867-4a9b-9d68-d74950714d2a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-s6jvb"
Sep 19 05:50:51.485480 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:50:51.485462    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x7s85\" (UniqueName: \"kubernetes.io/projected/84cf7fac-b867-4a9b-9d68-d74950714d2a-kube-api-access-x7s85\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-s6jvb\" (UID: \"84cf7fac-b867-4a9b-9d68-d74950714d2a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-s6jvb"
Sep 19 05:50:51.562650 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:50:51.562606    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-s6jvb"
Sep 19 05:50:51.846787 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:50:51.846755    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-s6jvb"]
Sep 19 05:50:51.847908 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:50:51.847872    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84cf7fac_b867_4a9b_9d68_d74950714d2a.slice/crio-dc723714e2c7f1f802243750f2254c021806d844e89473c9bdcd7a259b5c0eda WatchSource:0}: Error finding container dc723714e2c7f1f802243750f2254c021806d844e89473c9bdcd7a259b5c0eda: Status 404 returned error can't find the container with id dc723714e2c7f1f802243750f2254c021806d844e89473c9bdcd7a259b5c0eda
Sep 19 05:50:52.578786 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:50:52.578753    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-s6jvb" event={"ID":"84cf7fac-b867-4a9b-9d68-d74950714d2a","Type":"ContainerStarted","Data":"83f5220d8ad309c40a388818cb09d69a5808c1786ef555e98ca783112a7b76f8"}
Sep 19 05:50:52.579201 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:50:52.578812    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-s6jvb" event={"ID":"84cf7fac-b867-4a9b-9d68-d74950714d2a","Type":"ContainerStarted","Data":"dc723714e2c7f1f802243750f2254c021806d844e89473c9bdcd7a259b5c0eda"}
Sep 19 05:50:52.603637 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:50:52.600519    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-s6jvb" podStartSLOduration=1.433156587 podStartE2EDuration="1.600493395s" totalImagesPullingTime="167.336808ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:50:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:50:52.596526312 +0000 UTC m=+611.254980881" watchObservedRunningTime="2026-09-19 05:50:52.600493395 +0000 UTC m=+611.258947954"
Sep 19 05:51:01.563476 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:51:01.563376    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-s6jvb"
Sep 19 05:51:01.563910 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:51:01.563529    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-s6jvb"
Sep 19 05:51:01.576445 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:51:01.576415    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-s6jvb"
Sep 19 05:51:01.637689 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:51:01.637659    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-s6jvb"
Sep 19 05:53:24.962113 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:24.962080    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-4b78e2-openshift-default-7b7664c598-rz5lx"]
Sep 19 05:53:24.970142 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:24.970104    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-4b78e2-openshift-default-7b7664c598-rz5lx"
Sep 19 05:53:24.972811 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:24.972791    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-4b78e2-openshift-default-dockercfg-48l8f\""
Sep 19 05:53:24.980139 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:24.980095    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-4b78e2-openshift-default-7b7664c598-rz5lx"]
Sep 19 05:53:25.025483 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:25.025454    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/57450d37-8096-4ff3-a23f-5ce98d6cbc22-istio-envoy\") pod \"e2e-inf-a-4b78e2-openshift-default-7b7664c598-rz5lx\" (UID: \"57450d37-8096-4ff3-a23f-5ce98d6cbc22\") " pod="openshift-ingress/e2e-inf-a-4b78e2-openshift-default-7b7664c598-rz5lx"
Sep 19 05:53:25.025647 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:25.025490    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/57450d37-8096-4ff3-a23f-5ce98d6cbc22-istiod-ca-cert\") pod \"e2e-inf-a-4b78e2-openshift-default-7b7664c598-rz5lx\" (UID: \"57450d37-8096-4ff3-a23f-5ce98d6cbc22\") " pod="openshift-ingress/e2e-inf-a-4b78e2-openshift-default-7b7664c598-rz5lx"
Sep 19 05:53:25.025647 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:25.025516    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/57450d37-8096-4ff3-a23f-5ce98d6cbc22-credential-socket\") pod \"e2e-inf-a-4b78e2-openshift-default-7b7664c598-rz5lx\" (UID: \"57450d37-8096-4ff3-a23f-5ce98d6cbc22\") " pod="openshift-ingress/e2e-inf-a-4b78e2-openshift-default-7b7664c598-rz5lx"
Sep 19 05:53:25.025647 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:25.025574    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/57450d37-8096-4ff3-a23f-5ce98d6cbc22-workload-certs\") pod \"e2e-inf-a-4b78e2-openshift-default-7b7664c598-rz5lx\" (UID: \"57450d37-8096-4ff3-a23f-5ce98d6cbc22\") " pod="openshift-ingress/e2e-inf-a-4b78e2-openshift-default-7b7664c598-rz5lx"
Sep 19 05:53:25.025806 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:25.025655    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x57rl\" (UniqueName: \"kubernetes.io/projected/57450d37-8096-4ff3-a23f-5ce98d6cbc22-kube-api-access-x57rl\") pod \"e2e-inf-a-4b78e2-openshift-default-7b7664c598-rz5lx\" (UID: \"57450d37-8096-4ff3-a23f-5ce98d6cbc22\") " pod="openshift-ingress/e2e-inf-a-4b78e2-openshift-default-7b7664c598-rz5lx"
Sep 19 05:53:25.025806 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:25.025691    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/57450d37-8096-4ff3-a23f-5ce98d6cbc22-istio-podinfo\") pod \"e2e-inf-a-4b78e2-openshift-default-7b7664c598-rz5lx\" (UID: \"57450d37-8096-4ff3-a23f-5ce98d6cbc22\") " pod="openshift-ingress/e2e-inf-a-4b78e2-openshift-default-7b7664c598-rz5lx"
Sep 19 05:53:25.025806 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:25.025750    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/57450d37-8096-4ff3-a23f-5ce98d6cbc22-istio-token\") pod \"e2e-inf-a-4b78e2-openshift-default-7b7664c598-rz5lx\" (UID: \"57450d37-8096-4ff3-a23f-5ce98d6cbc22\") " pod="openshift-ingress/e2e-inf-a-4b78e2-openshift-default-7b7664c598-rz5lx"
Sep 19 05:53:25.025806 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:25.025766    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/57450d37-8096-4ff3-a23f-5ce98d6cbc22-istio-data\") pod \"e2e-inf-a-4b78e2-openshift-default-7b7664c598-rz5lx\" (UID: \"57450d37-8096-4ff3-a23f-5ce98d6cbc22\") " pod="openshift-ingress/e2e-inf-a-4b78e2-openshift-default-7b7664c598-rz5lx"
Sep 19 05:53:25.026009 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:25.025813    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/57450d37-8096-4ff3-a23f-5ce98d6cbc22-workload-socket\") pod \"e2e-inf-a-4b78e2-openshift-default-7b7664c598-rz5lx\" (UID: \"57450d37-8096-4ff3-a23f-5ce98d6cbc22\") " pod="openshift-ingress/e2e-inf-a-4b78e2-openshift-default-7b7664c598-rz5lx"
Sep 19 05:53:25.113179 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:25.113121    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-3182b5-openshift-default-76dd9fb6bb-b5p7s"]
Sep 19 05:53:25.116555 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:25.116533    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-3182b5-openshift-default-76dd9fb6bb-b5p7s"
Sep 19 05:53:25.119200 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:25.119174    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w4-3182b5-openshift-default-dockercfg-6pqvn\""
Sep 19 05:53:25.126394 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:25.126357    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/57450d37-8096-4ff3-a23f-5ce98d6cbc22-istio-token\") pod \"e2e-inf-a-4b78e2-openshift-default-7b7664c598-rz5lx\" (UID: \"57450d37-8096-4ff3-a23f-5ce98d6cbc22\") " pod="openshift-ingress/e2e-inf-a-4b78e2-openshift-default-7b7664c598-rz5lx"
Sep 19 05:53:25.126510 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:25.126402    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/57450d37-8096-4ff3-a23f-5ce98d6cbc22-istio-data\") pod \"e2e-inf-a-4b78e2-openshift-default-7b7664c598-rz5lx\" (UID: \"57450d37-8096-4ff3-a23f-5ce98d6cbc22\") " pod="openshift-ingress/e2e-inf-a-4b78e2-openshift-default-7b7664c598-rz5lx"
Sep 19 05:53:25.126510 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:25.126437    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/57450d37-8096-4ff3-a23f-5ce98d6cbc22-workload-socket\") pod \"e2e-inf-a-4b78e2-openshift-default-7b7664c598-rz5lx\" (UID: \"57450d37-8096-4ff3-a23f-5ce98d6cbc22\") " pod="openshift-ingress/e2e-inf-a-4b78e2-openshift-default-7b7664c598-rz5lx"
Sep 19 05:53:25.126510 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:25.126473    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/57450d37-8096-4ff3-a23f-5ce98d6cbc22-istio-envoy\") pod \"e2e-inf-a-4b78e2-openshift-default-7b7664c598-rz5lx\" (UID: \"57450d37-8096-4ff3-a23f-5ce98d6cbc22\") " pod="openshift-ingress/e2e-inf-a-4b78e2-openshift-default-7b7664c598-rz5lx"
Sep 19 05:53:25.126510 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:25.126496    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/57450d37-8096-4ff3-a23f-5ce98d6cbc22-istiod-ca-cert\") pod \"e2e-inf-a-4b78e2-openshift-default-7b7664c598-rz5lx\" (UID: \"57450d37-8096-4ff3-a23f-5ce98d6cbc22\") " pod="openshift-ingress/e2e-inf-a-4b78e2-openshift-default-7b7664c598-rz5lx"
Sep 19 05:53:25.126713 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:25.126524    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/57450d37-8096-4ff3-a23f-5ce98d6cbc22-credential-socket\") pod \"e2e-inf-a-4b78e2-openshift-default-7b7664c598-rz5lx\" (UID: \"57450d37-8096-4ff3-a23f-5ce98d6cbc22\") " pod="openshift-ingress/e2e-inf-a-4b78e2-openshift-default-7b7664c598-rz5lx"
Sep 19 05:53:25.126713 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:25.126561    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/57450d37-8096-4ff3-a23f-5ce98d6cbc22-workload-certs\") pod \"e2e-inf-a-4b78e2-openshift-default-7b7664c598-rz5lx\" (UID: \"57450d37-8096-4ff3-a23f-5ce98d6cbc22\") " pod="openshift-ingress/e2e-inf-a-4b78e2-openshift-default-7b7664c598-rz5lx"
Sep 19 05:53:25.126713 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:25.126618    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x57rl\" (UniqueName: \"kubernetes.io/projected/57450d37-8096-4ff3-a23f-5ce98d6cbc22-kube-api-access-x57rl\") pod \"e2e-inf-a-4b78e2-openshift-default-7b7664c598-rz5lx\" (UID: \"57450d37-8096-4ff3-a23f-5ce98d6cbc22\") " pod="openshift-ingress/e2e-inf-a-4b78e2-openshift-default-7b7664c598-rz5lx"
Sep 19 05:53:25.126713 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:25.126664    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/57450d37-8096-4ff3-a23f-5ce98d6cbc22-istio-podinfo\") pod \"e2e-inf-a-4b78e2-openshift-default-7b7664c598-rz5lx\" (UID: \"57450d37-8096-4ff3-a23f-5ce98d6cbc22\") " pod="openshift-ingress/e2e-inf-a-4b78e2-openshift-default-7b7664c598-rz5lx"
Sep 19 05:53:25.128419 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:25.127984    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/57450d37-8096-4ff3-a23f-5ce98d6cbc22-credential-socket\") pod \"e2e-inf-a-4b78e2-openshift-default-7b7664c598-rz5lx\" (UID: \"57450d37-8096-4ff3-a23f-5ce98d6cbc22\") " pod="openshift-ingress/e2e-inf-a-4b78e2-openshift-default-7b7664c598-rz5lx"
Sep 19 05:53:25.132255 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:25.129341    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/57450d37-8096-4ff3-a23f-5ce98d6cbc22-istiod-ca-cert\") pod \"e2e-inf-a-4b78e2-openshift-default-7b7664c598-rz5lx\" (UID: \"57450d37-8096-4ff3-a23f-5ce98d6cbc22\") " pod="openshift-ingress/e2e-inf-a-4b78e2-openshift-default-7b7664c598-rz5lx"
Sep 19 05:53:25.149509 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:25.149477    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/57450d37-8096-4ff3-a23f-5ce98d6cbc22-istio-podinfo\") pod \"e2e-inf-a-4b78e2-openshift-default-7b7664c598-rz5lx\" (UID: \"57450d37-8096-4ff3-a23f-5ce98d6cbc22\") " pod="openshift-ingress/e2e-inf-a-4b78e2-openshift-default-7b7664c598-rz5lx"
Sep 19 05:53:25.153228 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:25.153202    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/57450d37-8096-4ff3-a23f-5ce98d6cbc22-istio-data\") pod \"e2e-inf-a-4b78e2-openshift-default-7b7664c598-rz5lx\" (UID: \"57450d37-8096-4ff3-a23f-5ce98d6cbc22\") " pod="openshift-ingress/e2e-inf-a-4b78e2-openshift-default-7b7664c598-rz5lx"
Sep 19 05:53:25.153340 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:25.153260    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-3182b5-openshift-default-76dd9fb6bb-b5p7s"]
Sep 19 05:53:25.153580 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:25.153508    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/57450d37-8096-4ff3-a23f-5ce98d6cbc22-workload-certs\") pod \"e2e-inf-a-4b78e2-openshift-default-7b7664c598-rz5lx\" (UID: \"57450d37-8096-4ff3-a23f-5ce98d6cbc22\") " pod="openshift-ingress/e2e-inf-a-4b78e2-openshift-default-7b7664c598-rz5lx"
Sep 19 05:53:25.154161 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:25.153746    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/57450d37-8096-4ff3-a23f-5ce98d6cbc22-workload-socket\") pod \"e2e-inf-a-4b78e2-openshift-default-7b7664c598-rz5lx\" (UID: \"57450d37-8096-4ff3-a23f-5ce98d6cbc22\") " pod="openshift-ingress/e2e-inf-a-4b78e2-openshift-default-7b7664c598-rz5lx"
Sep 19 05:53:25.154738 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:25.154717    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/57450d37-8096-4ff3-a23f-5ce98d6cbc22-istio-envoy\") pod \"e2e-inf-a-4b78e2-openshift-default-7b7664c598-rz5lx\" (UID: \"57450d37-8096-4ff3-a23f-5ce98d6cbc22\") " pod="openshift-ingress/e2e-inf-a-4b78e2-openshift-default-7b7664c598-rz5lx"
Sep 19 05:53:25.158628 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:25.158601    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/57450d37-8096-4ff3-a23f-5ce98d6cbc22-istio-token\") pod \"e2e-inf-a-4b78e2-openshift-default-7b7664c598-rz5lx\" (UID: \"57450d37-8096-4ff3-a23f-5ce98d6cbc22\") " pod="openshift-ingress/e2e-inf-a-4b78e2-openshift-default-7b7664c598-rz5lx"
Sep 19 05:53:25.159234 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:25.159211    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x57rl\" (UniqueName: \"kubernetes.io/projected/57450d37-8096-4ff3-a23f-5ce98d6cbc22-kube-api-access-x57rl\") pod \"e2e-inf-a-4b78e2-openshift-default-7b7664c598-rz5lx\" (UID: \"57450d37-8096-4ff3-a23f-5ce98d6cbc22\") " pod="openshift-ingress/e2e-inf-a-4b78e2-openshift-default-7b7664c598-rz5lx"
Sep 19 05:53:25.227980 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:25.227825    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b57edf34-4a41-4cb6-bc1f-f73dae8adad1-workload-socket\") pod \"e2e-worker-w4-3182b5-openshift-default-76dd9fb6bb-b5p7s\" (UID: \"b57edf34-4a41-4cb6-bc1f-f73dae8adad1\") " pod="openshift-ingress/e2e-worker-w4-3182b5-openshift-default-76dd9fb6bb-b5p7s"
Sep 19 05:53:25.229173 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:25.227912    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b57edf34-4a41-4cb6-bc1f-f73dae8adad1-istio-data\") pod \"e2e-worker-w4-3182b5-openshift-default-76dd9fb6bb-b5p7s\" (UID: \"b57edf34-4a41-4cb6-bc1f-f73dae8adad1\") " pod="openshift-ingress/e2e-worker-w4-3182b5-openshift-default-76dd9fb6bb-b5p7s"
Sep 19 05:53:25.229173 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:25.228848    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b57edf34-4a41-4cb6-bc1f-f73dae8adad1-istio-podinfo\") pod \"e2e-worker-w4-3182b5-openshift-default-76dd9fb6bb-b5p7s\" (UID: \"b57edf34-4a41-4cb6-bc1f-f73dae8adad1\") " pod="openshift-ingress/e2e-worker-w4-3182b5-openshift-default-76dd9fb6bb-b5p7s"
Sep 19 05:53:25.229173 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:25.228894    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b57edf34-4a41-4cb6-bc1f-f73dae8adad1-istio-token\") pod \"e2e-worker-w4-3182b5-openshift-default-76dd9fb6bb-b5p7s\" (UID: \"b57edf34-4a41-4cb6-bc1f-f73dae8adad1\") " pod="openshift-ingress/e2e-worker-w4-3182b5-openshift-default-76dd9fb6bb-b5p7s"
Sep 19 05:53:25.229173 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:25.228924    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dn57q\" (UniqueName: \"kubernetes.io/projected/b57edf34-4a41-4cb6-bc1f-f73dae8adad1-kube-api-access-dn57q\") pod \"e2e-worker-w4-3182b5-openshift-default-76dd9fb6bb-b5p7s\" (UID: \"b57edf34-4a41-4cb6-bc1f-f73dae8adad1\") " pod="openshift-ingress/e2e-worker-w4-3182b5-openshift-default-76dd9fb6bb-b5p7s"
Sep 19 05:53:25.229173 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:25.228970    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b57edf34-4a41-4cb6-bc1f-f73dae8adad1-istio-envoy\") pod \"e2e-worker-w4-3182b5-openshift-default-76dd9fb6bb-b5p7s\" (UID: \"b57edf34-4a41-4cb6-bc1f-f73dae8adad1\") " pod="openshift-ingress/e2e-worker-w4-3182b5-openshift-default-76dd9fb6bb-b5p7s"
Sep 19 05:53:25.229173 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:25.229007    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b57edf34-4a41-4cb6-bc1f-f73dae8adad1-credential-socket\") pod \"e2e-worker-w4-3182b5-openshift-default-76dd9fb6bb-b5p7s\" (UID: \"b57edf34-4a41-4cb6-bc1f-f73dae8adad1\") " pod="openshift-ingress/e2e-worker-w4-3182b5-openshift-default-76dd9fb6bb-b5p7s"
Sep 19 05:53:25.229173 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:25.229033    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b57edf34-4a41-4cb6-bc1f-f73dae8adad1-workload-certs\") pod \"e2e-worker-w4-3182b5-openshift-default-76dd9fb6bb-b5p7s\" (UID: \"b57edf34-4a41-4cb6-bc1f-f73dae8adad1\") " pod="openshift-ingress/e2e-worker-w4-3182b5-openshift-default-76dd9fb6bb-b5p7s"
Sep 19 05:53:25.229173 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:25.229056    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b57edf34-4a41-4cb6-bc1f-f73dae8adad1-istiod-ca-cert\") pod \"e2e-worker-w4-3182b5-openshift-default-76dd9fb6bb-b5p7s\" (UID: \"b57edf34-4a41-4cb6-bc1f-f73dae8adad1\") " pod="openshift-ingress/e2e-worker-w4-3182b5-openshift-default-76dd9fb6bb-b5p7s"
Sep 19 05:53:25.283040 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:25.283002    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-4b78e2-openshift-default-7b7664c598-rz5lx"
Sep 19 05:53:25.330386 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:25.330352    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b57edf34-4a41-4cb6-bc1f-f73dae8adad1-istio-token\") pod \"e2e-worker-w4-3182b5-openshift-default-76dd9fb6bb-b5p7s\" (UID: \"b57edf34-4a41-4cb6-bc1f-f73dae8adad1\") " pod="openshift-ingress/e2e-worker-w4-3182b5-openshift-default-76dd9fb6bb-b5p7s"
Sep 19 05:53:25.330561 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:25.330390    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dn57q\" (UniqueName: \"kubernetes.io/projected/b57edf34-4a41-4cb6-bc1f-f73dae8adad1-kube-api-access-dn57q\") pod \"e2e-worker-w4-3182b5-openshift-default-76dd9fb6bb-b5p7s\" (UID: \"b57edf34-4a41-4cb6-bc1f-f73dae8adad1\") " pod="openshift-ingress/e2e-worker-w4-3182b5-openshift-default-76dd9fb6bb-b5p7s"
Sep 19 05:53:25.330626 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:25.330579    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b57edf34-4a41-4cb6-bc1f-f73dae8adad1-istio-envoy\") pod \"e2e-worker-w4-3182b5-openshift-default-76dd9fb6bb-b5p7s\" (UID: \"b57edf34-4a41-4cb6-bc1f-f73dae8adad1\") " pod="openshift-ingress/e2e-worker-w4-3182b5-openshift-default-76dd9fb6bb-b5p7s"
Sep 19 05:53:25.330681 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:25.330629    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b57edf34-4a41-4cb6-bc1f-f73dae8adad1-credential-socket\") pod \"e2e-worker-w4-3182b5-openshift-default-76dd9fb6bb-b5p7s\" (UID: \"b57edf34-4a41-4cb6-bc1f-f73dae8adad1\") " pod="openshift-ingress/e2e-worker-w4-3182b5-openshift-default-76dd9fb6bb-b5p7s"
Sep 19 05:53:25.330681 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:25.330652    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b57edf34-4a41-4cb6-bc1f-f73dae8adad1-workload-certs\") pod \"e2e-worker-w4-3182b5-openshift-default-76dd9fb6bb-b5p7s\" (UID: \"b57edf34-4a41-4cb6-bc1f-f73dae8adad1\") " pod="openshift-ingress/e2e-worker-w4-3182b5-openshift-default-76dd9fb6bb-b5p7s"
Sep 19 05:53:25.330681 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:25.330668    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b57edf34-4a41-4cb6-bc1f-f73dae8adad1-istiod-ca-cert\") pod \"e2e-worker-w4-3182b5-openshift-default-76dd9fb6bb-b5p7s\" (UID: \"b57edf34-4a41-4cb6-bc1f-f73dae8adad1\") " pod="openshift-ingress/e2e-worker-w4-3182b5-openshift-default-76dd9fb6bb-b5p7s"
Sep 19 05:53:25.330829 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:25.330742    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b57edf34-4a41-4cb6-bc1f-f73dae8adad1-workload-socket\") pod \"e2e-worker-w4-3182b5-openshift-default-76dd9fb6bb-b5p7s\" (UID: \"b57edf34-4a41-4cb6-bc1f-f73dae8adad1\") " pod="openshift-ingress/e2e-worker-w4-3182b5-openshift-default-76dd9fb6bb-b5p7s"
Sep 19 05:53:25.330829 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:25.330799    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b57edf34-4a41-4cb6-bc1f-f73dae8adad1-istio-data\") pod \"e2e-worker-w4-3182b5-openshift-default-76dd9fb6bb-b5p7s\" (UID: \"b57edf34-4a41-4cb6-bc1f-f73dae8adad1\") " pod="openshift-ingress/e2e-worker-w4-3182b5-openshift-default-76dd9fb6bb-b5p7s"
Sep 19 05:53:25.330829 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:25.330815    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b57edf34-4a41-4cb6-bc1f-f73dae8adad1-istio-podinfo\") pod \"e2e-worker-w4-3182b5-openshift-default-76dd9fb6bb-b5p7s\" (UID: \"b57edf34-4a41-4cb6-bc1f-f73dae8adad1\") " pod="openshift-ingress/e2e-worker-w4-3182b5-openshift-default-76dd9fb6bb-b5p7s"
Sep 19 05:53:25.331362 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:25.331162    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b57edf34-4a41-4cb6-bc1f-f73dae8adad1-workload-certs\") pod \"e2e-worker-w4-3182b5-openshift-default-76dd9fb6bb-b5p7s\" (UID: \"b57edf34-4a41-4cb6-bc1f-f73dae8adad1\") " pod="openshift-ingress/e2e-worker-w4-3182b5-openshift-default-76dd9fb6bb-b5p7s"
Sep 19 05:53:25.331362 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:25.331306    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b57edf34-4a41-4cb6-bc1f-f73dae8adad1-istio-data\") pod \"e2e-worker-w4-3182b5-openshift-default-76dd9fb6bb-b5p7s\" (UID: \"b57edf34-4a41-4cb6-bc1f-f73dae8adad1\") " pod="openshift-ingress/e2e-worker-w4-3182b5-openshift-default-76dd9fb6bb-b5p7s"
Sep 19 05:53:25.331532 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:25.331503    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b57edf34-4a41-4cb6-bc1f-f73dae8adad1-workload-socket\") pod \"e2e-worker-w4-3182b5-openshift-default-76dd9fb6bb-b5p7s\" (UID: \"b57edf34-4a41-4cb6-bc1f-f73dae8adad1\") " pod="openshift-ingress/e2e-worker-w4-3182b5-openshift-default-76dd9fb6bb-b5p7s"
Sep 19 05:53:25.331532 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:25.331524    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b57edf34-4a41-4cb6-bc1f-f73dae8adad1-istiod-ca-cert\") pod \"e2e-worker-w4-3182b5-openshift-default-76dd9fb6bb-b5p7s\" (UID: \"b57edf34-4a41-4cb6-bc1f-f73dae8adad1\") " pod="openshift-ingress/e2e-worker-w4-3182b5-openshift-default-76dd9fb6bb-b5p7s"
Sep 19 05:53:25.331667 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:25.331650    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b57edf34-4a41-4cb6-bc1f-f73dae8adad1-credential-socket\") pod \"e2e-worker-w4-3182b5-openshift-default-76dd9fb6bb-b5p7s\" (UID: \"b57edf34-4a41-4cb6-bc1f-f73dae8adad1\") " pod="openshift-ingress/e2e-worker-w4-3182b5-openshift-default-76dd9fb6bb-b5p7s"
Sep 19 05:53:25.334417 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:25.333070    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b57edf34-4a41-4cb6-bc1f-f73dae8adad1-istio-envoy\") pod \"e2e-worker-w4-3182b5-openshift-default-76dd9fb6bb-b5p7s\" (UID: \"b57edf34-4a41-4cb6-bc1f-f73dae8adad1\") " pod="openshift-ingress/e2e-worker-w4-3182b5-openshift-default-76dd9fb6bb-b5p7s"
Sep 19 05:53:25.335293 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:25.335191    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b57edf34-4a41-4cb6-bc1f-f73dae8adad1-istio-podinfo\") pod \"e2e-worker-w4-3182b5-openshift-default-76dd9fb6bb-b5p7s\" (UID: \"b57edf34-4a41-4cb6-bc1f-f73dae8adad1\") " pod="openshift-ingress/e2e-worker-w4-3182b5-openshift-default-76dd9fb6bb-b5p7s"
Sep 19 05:53:25.341255 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:25.341178    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dn57q\" (UniqueName: \"kubernetes.io/projected/b57edf34-4a41-4cb6-bc1f-f73dae8adad1-kube-api-access-dn57q\") pod \"e2e-worker-w4-3182b5-openshift-default-76dd9fb6bb-b5p7s\" (UID: \"b57edf34-4a41-4cb6-bc1f-f73dae8adad1\") " pod="openshift-ingress/e2e-worker-w4-3182b5-openshift-default-76dd9fb6bb-b5p7s"
Sep 19 05:53:25.342718 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:25.342699    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b57edf34-4a41-4cb6-bc1f-f73dae8adad1-istio-token\") pod \"e2e-worker-w4-3182b5-openshift-default-76dd9fb6bb-b5p7s\" (UID: \"b57edf34-4a41-4cb6-bc1f-f73dae8adad1\") " pod="openshift-ingress/e2e-worker-w4-3182b5-openshift-default-76dd9fb6bb-b5p7s"
Sep 19 05:53:25.454648 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:25.454612    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-3182b5-openshift-default-76dd9fb6bb-b5p7s"
Sep 19 05:53:25.565831 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:25.565802    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-4b78e2-openshift-default-7b7664c598-rz5lx"]
Sep 19 05:53:25.567096 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:53:25.567067    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57450d37_8096_4ff3_a23f_5ce98d6cbc22.slice/crio-5b874ef4ee9dab004826779b4707e7f00d1e4328fe7f331c8b46c8db04b60a17 WatchSource:0}: Error finding container 5b874ef4ee9dab004826779b4707e7f00d1e4328fe7f331c8b46c8db04b60a17: Status 404 returned error can't find the container with id 5b874ef4ee9dab004826779b4707e7f00d1e4328fe7f331c8b46c8db04b60a17
Sep 19 05:53:25.572577 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:25.572540    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 19 05:53:25.572661 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:25.572642    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 19 05:53:25.572711 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:25.572674    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 19 05:53:25.730117 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:25.730090    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-3182b5-openshift-default-76dd9fb6bb-b5p7s"]
Sep 19 05:53:25.731615 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:53:25.731588    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb57edf34_4a41_4cb6_bc1f_f73dae8adad1.slice/crio-1cf10e3ed82925e97de55f0e82102d5b297ce34566c4560e174216ca5a389484 WatchSource:0}: Error finding container 1cf10e3ed82925e97de55f0e82102d5b297ce34566c4560e174216ca5a389484: Status 404 returned error can't find the container with id 1cf10e3ed82925e97de55f0e82102d5b297ce34566c4560e174216ca5a389484
Sep 19 05:53:25.733970 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:25.733938    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 19 05:53:25.734036 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:25.734002    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 19 05:53:25.734036 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:25.734033    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 19 05:53:26.396215 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:26.396176    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-3182b5-openshift-default-76dd9fb6bb-b5p7s" event={"ID":"b57edf34-4a41-4cb6-bc1f-f73dae8adad1","Type":"ContainerStarted","Data":"718372ca6c871216a911c583987ab7a96ce5d8ea38e760b37b4a66ec0d50da8e"}
Sep 19 05:53:26.396631 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:26.396241    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-3182b5-openshift-default-76dd9fb6bb-b5p7s" event={"ID":"b57edf34-4a41-4cb6-bc1f-f73dae8adad1","Type":"ContainerStarted","Data":"1cf10e3ed82925e97de55f0e82102d5b297ce34566c4560e174216ca5a389484"}
Sep 19 05:53:26.397850 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:26.397823    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-4b78e2-openshift-default-7b7664c598-rz5lx" event={"ID":"57450d37-8096-4ff3-a23f-5ce98d6cbc22","Type":"ContainerStarted","Data":"e0ed9229f52bc20a9a8c8b116f2575ff5abaeb42445d4ef500940a278d7a0bf8"}
Sep 19 05:53:26.397953 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:26.397875    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-4b78e2-openshift-default-7b7664c598-rz5lx" event={"ID":"57450d37-8096-4ff3-a23f-5ce98d6cbc22","Type":"ContainerStarted","Data":"5b874ef4ee9dab004826779b4707e7f00d1e4328fe7f331c8b46c8db04b60a17"}
Sep 19 05:53:26.419611 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:26.419566    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-3182b5-openshift-default-76dd9fb6bb-b5p7s" podStartSLOduration=1.419550687 podStartE2EDuration="1.419550687s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:53:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:53:26.418199365 +0000 UTC m=+765.076653924" watchObservedRunningTime="2026-09-19 05:53:26.419550687 +0000 UTC m=+765.078005245"
Sep 19 05:53:26.439432 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:26.439388    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-4b78e2-openshift-default-7b7664c598-rz5lx" podStartSLOduration=2.439374013 podStartE2EDuration="2.439374013s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:53:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:53:26.436907231 +0000 UTC m=+765.095361790" watchObservedRunningTime="2026-09-19 05:53:26.439374013 +0000 UTC m=+765.097828572"
Sep 19 05:53:26.454829 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:26.454803    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-3182b5-openshift-default-76dd9fb6bb-b5p7s"
Sep 19 05:53:26.459552 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:26.459530    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-3182b5-openshift-default-76dd9fb6bb-b5p7s"
Sep 19 05:53:27.283534 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:27.283497    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-4b78e2-openshift-default-7b7664c598-rz5lx"
Sep 19 05:53:27.288193 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:27.288168    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-4b78e2-openshift-default-7b7664c598-rz5lx"
Sep 19 05:53:27.402957 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:27.402919    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-4b78e2-openshift-default-7b7664c598-rz5lx"
Sep 19 05:53:27.403408 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:27.402985    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-3182b5-openshift-default-76dd9fb6bb-b5p7s"
Sep 19 05:53:27.403730 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:27.403710    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-3182b5-openshift-default-76dd9fb6bb-b5p7s"
Sep 19 05:53:27.403793 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:27.403771    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-4b78e2-openshift-default-7b7664c598-rz5lx"
Sep 19 05:53:30.172676 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:30.172646    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-3182b5-d6c9696c6-jwzw7"]
Sep 19 05:53:30.175977 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:30.175961    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-3182b5-d6c9696c6-jwzw7"
Sep 19 05:53:30.178369 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:30.178345    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w4-3182b5-serving-cert\""
Sep 19 05:53:30.187649 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:30.187621    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-3182b5-d6c9696c6-jwzw7"]
Sep 19 05:53:30.280850 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:30.280813    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1980603d-cfad-4f61-959c-e9b7507da10f-maas-api-tls\") pod \"maas-api-e2e-worker-w4-3182b5-d6c9696c6-jwzw7\" (UID: \"1980603d-cfad-4f61-959c-e9b7507da10f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-3182b5-d6c9696c6-jwzw7"
Sep 19 05:53:30.280997 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:30.280873    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1980603d-cfad-4f61-959c-e9b7507da10f-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-3182b5-d6c9696c6-jwzw7\" (UID: \"1980603d-cfad-4f61-959c-e9b7507da10f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-3182b5-d6c9696c6-jwzw7"
Sep 19 05:53:30.281077 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:30.280990    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-swnm2\" (UniqueName: \"kubernetes.io/projected/1980603d-cfad-4f61-959c-e9b7507da10f-kube-api-access-swnm2\") pod \"maas-api-e2e-worker-w4-3182b5-d6c9696c6-jwzw7\" (UID: \"1980603d-cfad-4f61-959c-e9b7507da10f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-3182b5-d6c9696c6-jwzw7"
Sep 19 05:53:30.281171 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:30.281113    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1980603d-cfad-4f61-959c-e9b7507da10f-metrics-certs\") pod \"maas-api-e2e-worker-w4-3182b5-d6c9696c6-jwzw7\" (UID: \"1980603d-cfad-4f61-959c-e9b7507da10f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-3182b5-d6c9696c6-jwzw7"
Sep 19 05:53:30.381630 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:30.381590    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1980603d-cfad-4f61-959c-e9b7507da10f-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-3182b5-d6c9696c6-jwzw7\" (UID: \"1980603d-cfad-4f61-959c-e9b7507da10f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-3182b5-d6c9696c6-jwzw7"
Sep 19 05:53:30.381825 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:30.381663    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-swnm2\" (UniqueName: \"kubernetes.io/projected/1980603d-cfad-4f61-959c-e9b7507da10f-kube-api-access-swnm2\") pod \"maas-api-e2e-worker-w4-3182b5-d6c9696c6-jwzw7\" (UID: \"1980603d-cfad-4f61-959c-e9b7507da10f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-3182b5-d6c9696c6-jwzw7"
Sep 19 05:53:30.381825 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:30.381686    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1980603d-cfad-4f61-959c-e9b7507da10f-metrics-certs\") pod \"maas-api-e2e-worker-w4-3182b5-d6c9696c6-jwzw7\" (UID: \"1980603d-cfad-4f61-959c-e9b7507da10f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-3182b5-d6c9696c6-jwzw7"
Sep 19 05:53:30.381825 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:30.381730    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1980603d-cfad-4f61-959c-e9b7507da10f-maas-api-tls\") pod \"maas-api-e2e-worker-w4-3182b5-d6c9696c6-jwzw7\" (UID: \"1980603d-cfad-4f61-959c-e9b7507da10f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-3182b5-d6c9696c6-jwzw7"
Sep 19 05:53:30.381825 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:53:30.381826    2583 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w4-3182b5-serving-cert: secret "maas-api-e2e-worker-w4-3182b5-serving-cert" not found
Sep 19 05:53:30.382047 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:53:30.381885    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1980603d-cfad-4f61-959c-e9b7507da10f-maas-api-tls podName:1980603d-cfad-4f61-959c-e9b7507da10f nodeName:}" failed. No retries permitted until 2026-09-19 05:53:30.88186652 +0000 UTC m=+769.540321061 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/1980603d-cfad-4f61-959c-e9b7507da10f-maas-api-tls") pod "maas-api-e2e-worker-w4-3182b5-d6c9696c6-jwzw7" (UID: "1980603d-cfad-4f61-959c-e9b7507da10f") : secret "maas-api-e2e-worker-w4-3182b5-serving-cert" not found
Sep 19 05:53:30.382387 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:30.382364    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1980603d-cfad-4f61-959c-e9b7507da10f-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-3182b5-d6c9696c6-jwzw7\" (UID: \"1980603d-cfad-4f61-959c-e9b7507da10f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-3182b5-d6c9696c6-jwzw7"
Sep 19 05:53:30.384362 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:30.384330    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1980603d-cfad-4f61-959c-e9b7507da10f-metrics-certs\") pod \"maas-api-e2e-worker-w4-3182b5-d6c9696c6-jwzw7\" (UID: \"1980603d-cfad-4f61-959c-e9b7507da10f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-3182b5-d6c9696c6-jwzw7"
Sep 19 05:53:30.392029 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:30.392006    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-swnm2\" (UniqueName: \"kubernetes.io/projected/1980603d-cfad-4f61-959c-e9b7507da10f-kube-api-access-swnm2\") pod \"maas-api-e2e-worker-w4-3182b5-d6c9696c6-jwzw7\" (UID: \"1980603d-cfad-4f61-959c-e9b7507da10f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-3182b5-d6c9696c6-jwzw7"
Sep 19 05:53:30.887614 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:30.887577    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1980603d-cfad-4f61-959c-e9b7507da10f-maas-api-tls\") pod \"maas-api-e2e-worker-w4-3182b5-d6c9696c6-jwzw7\" (UID: \"1980603d-cfad-4f61-959c-e9b7507da10f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-3182b5-d6c9696c6-jwzw7"
Sep 19 05:53:30.890078 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:30.890050    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1980603d-cfad-4f61-959c-e9b7507da10f-maas-api-tls\") pod \"maas-api-e2e-worker-w4-3182b5-d6c9696c6-jwzw7\" (UID: \"1980603d-cfad-4f61-959c-e9b7507da10f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-3182b5-d6c9696c6-jwzw7"
Sep 19 05:53:31.088437 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:31.088404    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-3182b5-d6c9696c6-jwzw7"
Sep 19 05:53:31.349763 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:31.349716    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-4b78e2-6bf5c5f97b-75ldh"]
Sep 19 05:53:31.353632 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:31.353608    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4b78e2-6bf5c5f97b-75ldh"
Sep 19 05:53:31.356034 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:31.356006    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-4b78e2-serving-cert\""
Sep 19 05:53:31.365143 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:31.365109    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-4b78e2-6bf5c5f97b-75ldh"]
Sep 19 05:53:31.387770 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:31.387743    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-3182b5-d6c9696c6-jwzw7"]
Sep 19 05:53:31.388994 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:53:31.388967    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1980603d_cfad_4f61_959c_e9b7507da10f.slice/crio-ec0972eb601f5d8dec482e0537334b019e75a20c43f8dcd55032313b8147ff52 WatchSource:0}: Error finding container ec0972eb601f5d8dec482e0537334b019e75a20c43f8dcd55032313b8147ff52: Status 404 returned error can't find the container with id ec0972eb601f5d8dec482e0537334b019e75a20c43f8dcd55032313b8147ff52
Sep 19 05:53:31.390739 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:31.390714    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9b70e086-50fc-498a-9e22-ff7fa70d4738-maas-api-tls\") pod \"maas-api-e2e-inf-a-4b78e2-6bf5c5f97b-75ldh\" (UID: \"9b70e086-50fc-498a-9e22-ff7fa70d4738\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4b78e2-6bf5c5f97b-75ldh"
Sep 19 05:53:31.390831 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:31.390770    2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 19 05:53:31.390831 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:31.390776    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b70e086-50fc-498a-9e22-ff7fa70d4738-openshift-service-ca\") pod \"maas-api-e2e-inf-a-4b78e2-6bf5c5f97b-75ldh\" (UID: \"9b70e086-50fc-498a-9e22-ff7fa70d4738\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4b78e2-6bf5c5f97b-75ldh"
Sep 19 05:53:31.390940 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:31.390893    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nnccl\" (UniqueName: \"kubernetes.io/projected/9b70e086-50fc-498a-9e22-ff7fa70d4738-kube-api-access-nnccl\") pod \"maas-api-e2e-inf-a-4b78e2-6bf5c5f97b-75ldh\" (UID: \"9b70e086-50fc-498a-9e22-ff7fa70d4738\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4b78e2-6bf5c5f97b-75ldh"
Sep 19 05:53:31.390994 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:31.390940    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9b70e086-50fc-498a-9e22-ff7fa70d4738-metrics-certs\") pod \"maas-api-e2e-inf-a-4b78e2-6bf5c5f97b-75ldh\" (UID: \"9b70e086-50fc-498a-9e22-ff7fa70d4738\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4b78e2-6bf5c5f97b-75ldh"
Sep 19 05:53:31.424234 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:31.424165    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-3182b5-d6c9696c6-jwzw7" event={"ID":"1980603d-cfad-4f61-959c-e9b7507da10f","Type":"ContainerStarted","Data":"ec0972eb601f5d8dec482e0537334b019e75a20c43f8dcd55032313b8147ff52"}
Sep 19 05:53:31.492708 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:31.492655    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b70e086-50fc-498a-9e22-ff7fa70d4738-openshift-service-ca\") pod \"maas-api-e2e-inf-a-4b78e2-6bf5c5f97b-75ldh\" (UID: \"9b70e086-50fc-498a-9e22-ff7fa70d4738\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4b78e2-6bf5c5f97b-75ldh"
Sep 19 05:53:31.492917 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:31.492803    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nnccl\" (UniqueName: \"kubernetes.io/projected/9b70e086-50fc-498a-9e22-ff7fa70d4738-kube-api-access-nnccl\") pod \"maas-api-e2e-inf-a-4b78e2-6bf5c5f97b-75ldh\" (UID: \"9b70e086-50fc-498a-9e22-ff7fa70d4738\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4b78e2-6bf5c5f97b-75ldh"
Sep 19 05:53:31.492917 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:31.492858    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9b70e086-50fc-498a-9e22-ff7fa70d4738-metrics-certs\") pod \"maas-api-e2e-inf-a-4b78e2-6bf5c5f97b-75ldh\" (UID: \"9b70e086-50fc-498a-9e22-ff7fa70d4738\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4b78e2-6bf5c5f97b-75ldh"
Sep 19 05:53:31.492917 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:31.492895    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9b70e086-50fc-498a-9e22-ff7fa70d4738-maas-api-tls\") pod \"maas-api-e2e-inf-a-4b78e2-6bf5c5f97b-75ldh\" (UID: \"9b70e086-50fc-498a-9e22-ff7fa70d4738\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4b78e2-6bf5c5f97b-75ldh"
Sep 19 05:53:31.493067 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:53:31.493034    2583 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-4b78e2-serving-cert: secret "maas-api-e2e-inf-a-4b78e2-serving-cert" not found
Sep 19 05:53:31.493115 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:53:31.493097    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9b70e086-50fc-498a-9e22-ff7fa70d4738-maas-api-tls podName:9b70e086-50fc-498a-9e22-ff7fa70d4738 nodeName:}" failed. No retries permitted until 2026-09-19 05:53:31.993074949 +0000 UTC m=+770.651529487 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/9b70e086-50fc-498a-9e22-ff7fa70d4738-maas-api-tls") pod "maas-api-e2e-inf-a-4b78e2-6bf5c5f97b-75ldh" (UID: "9b70e086-50fc-498a-9e22-ff7fa70d4738") : secret "maas-api-e2e-inf-a-4b78e2-serving-cert" not found
Sep 19 05:53:31.493952 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:31.493919    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b70e086-50fc-498a-9e22-ff7fa70d4738-openshift-service-ca\") pod \"maas-api-e2e-inf-a-4b78e2-6bf5c5f97b-75ldh\" (UID: \"9b70e086-50fc-498a-9e22-ff7fa70d4738\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4b78e2-6bf5c5f97b-75ldh"
Sep 19 05:53:31.497171 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:31.497146    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9b70e086-50fc-498a-9e22-ff7fa70d4738-metrics-certs\") pod \"maas-api-e2e-inf-a-4b78e2-6bf5c5f97b-75ldh\" (UID: \"9b70e086-50fc-498a-9e22-ff7fa70d4738\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4b78e2-6bf5c5f97b-75ldh"
Sep 19 05:53:31.503010 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:31.502989    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nnccl\" (UniqueName: \"kubernetes.io/projected/9b70e086-50fc-498a-9e22-ff7fa70d4738-kube-api-access-nnccl\") pod \"maas-api-e2e-inf-a-4b78e2-6bf5c5f97b-75ldh\" (UID: \"9b70e086-50fc-498a-9e22-ff7fa70d4738\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4b78e2-6bf5c5f97b-75ldh"
Sep 19 05:53:31.997594 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:31.997558    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9b70e086-50fc-498a-9e22-ff7fa70d4738-maas-api-tls\") pod \"maas-api-e2e-inf-a-4b78e2-6bf5c5f97b-75ldh\" (UID: \"9b70e086-50fc-498a-9e22-ff7fa70d4738\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4b78e2-6bf5c5f97b-75ldh"
Sep 19 05:53:32.000210 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:32.000178    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9b70e086-50fc-498a-9e22-ff7fa70d4738-maas-api-tls\") pod \"maas-api-e2e-inf-a-4b78e2-6bf5c5f97b-75ldh\" (UID: \"9b70e086-50fc-498a-9e22-ff7fa70d4738\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4b78e2-6bf5c5f97b-75ldh"
Sep 19 05:53:32.270376 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:32.270283    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4b78e2-6bf5c5f97b-75ldh"
Sep 19 05:53:32.436939 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:32.436899    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-3182b5-d6c9696c6-jwzw7" event={"ID":"1980603d-cfad-4f61-959c-e9b7507da10f","Type":"ContainerStarted","Data":"5bc6d41a7bb6e4fed869ab419b59e36e25d438ff862af39e09dd81b41419d2f9"}
Sep 19 05:53:32.437443 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:32.437006    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-3182b5-d6c9696c6-jwzw7"
Sep 19 05:53:32.454885 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:32.454839    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-3182b5-d6c9696c6-jwzw7" podStartSLOduration=2.039503892 podStartE2EDuration="2.454825079s" totalImagesPullingTime="415.321187ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:53:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:53:32.453415831 +0000 UTC m=+771.111870403" watchObservedRunningTime="2026-09-19 05:53:32.454825079 +0000 UTC m=+771.113279631"
Sep 19 05:53:32.550739 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:32.550712    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-4b78e2-6bf5c5f97b-75ldh"]
Sep 19 05:53:32.552401 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:53:32.552376    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b70e086_50fc_498a_9e22_ff7fa70d4738.slice/crio-ffb5966f2d2a1d14797f69cc590881ec25bfe9d5baf885f2f8564b0815318a6b WatchSource:0}: Error finding container ffb5966f2d2a1d14797f69cc590881ec25bfe9d5baf885f2f8564b0815318a6b: Status 404 returned error can't find the container with id ffb5966f2d2a1d14797f69cc590881ec25bfe9d5baf885f2f8564b0815318a6b
Sep 19 05:53:33.444845 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:33.444760    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4b78e2-6bf5c5f97b-75ldh" event={"ID":"9b70e086-50fc-498a-9e22-ff7fa70d4738","Type":"ContainerStarted","Data":"5579c2f7804809dda888ea44e6bd483413620b630a38d1250091650e279f3827"}
Sep 19 05:53:33.445223 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:33.445010    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4b78e2-6bf5c5f97b-75ldh"
Sep 19 05:53:33.445223 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:33.445046    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4b78e2-6bf5c5f97b-75ldh" event={"ID":"9b70e086-50fc-498a-9e22-ff7fa70d4738","Type":"ContainerStarted","Data":"ffb5966f2d2a1d14797f69cc590881ec25bfe9d5baf885f2f8564b0815318a6b"}
Sep 19 05:53:33.468778 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:33.468728    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4b78e2-6bf5c5f97b-75ldh" podStartSLOduration=1.982703498 podStartE2EDuration="2.46871397s" totalImagesPullingTime="486.010472ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:53:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:53:33.46607639 +0000 UTC m=+772.124530997" watchObservedRunningTime="2026-09-19 05:53:33.46871397 +0000 UTC m=+772.127168567"
Sep 19 05:53:34.500169 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:34.500117    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-bd4c1118-openshift-default-565f6989b4-xjv76"]
Sep 19 05:53:34.503737 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:34.503716    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-bd4c1118-openshift-default-565f6989b4-xjv76"
Sep 19 05:53:34.507284 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:34.507259    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-bd4c1118-openshift-default-dockercfg-95q5f\""
Sep 19 05:53:34.522560 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:34.522536    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-bd4c1118-openshift-default-565f6989b4-xjv76"]
Sep 19 05:53:34.621822 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:34.621791    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/465e9b26-ea7c-4957-baae-d4dda4827e78-istio-podinfo\") pod \"e2e-ait-bd4c1118-openshift-default-565f6989b4-xjv76\" (UID: \"465e9b26-ea7c-4957-baae-d4dda4827e78\") " pod="openshift-ingress/e2e-ait-bd4c1118-openshift-default-565f6989b4-xjv76"
Sep 19 05:53:34.622014 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:34.621884    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/465e9b26-ea7c-4957-baae-d4dda4827e78-istio-data\") pod \"e2e-ait-bd4c1118-openshift-default-565f6989b4-xjv76\" (UID: \"465e9b26-ea7c-4957-baae-d4dda4827e78\") " pod="openshift-ingress/e2e-ait-bd4c1118-openshift-default-565f6989b4-xjv76"
Sep 19 05:53:34.622014 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:34.621913    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n4j8k\" (UniqueName: \"kubernetes.io/projected/465e9b26-ea7c-4957-baae-d4dda4827e78-kube-api-access-n4j8k\") pod \"e2e-ait-bd4c1118-openshift-default-565f6989b4-xjv76\" (UID: \"465e9b26-ea7c-4957-baae-d4dda4827e78\") " pod="openshift-ingress/e2e-ait-bd4c1118-openshift-default-565f6989b4-xjv76"
Sep 19 05:53:34.622014 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:34.621947    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/465e9b26-ea7c-4957-baae-d4dda4827e78-credential-socket\") pod \"e2e-ait-bd4c1118-openshift-default-565f6989b4-xjv76\" (UID: \"465e9b26-ea7c-4957-baae-d4dda4827e78\") " pod="openshift-ingress/e2e-ait-bd4c1118-openshift-default-565f6989b4-xjv76"
Sep 19 05:53:34.622014 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:34.621982    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/465e9b26-ea7c-4957-baae-d4dda4827e78-workload-certs\") pod \"e2e-ait-bd4c1118-openshift-default-565f6989b4-xjv76\" (UID: \"465e9b26-ea7c-4957-baae-d4dda4827e78\") " pod="openshift-ingress/e2e-ait-bd4c1118-openshift-default-565f6989b4-xjv76"
Sep 19 05:53:34.622258 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:34.622029    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/465e9b26-ea7c-4957-baae-d4dda4827e78-istio-token\") pod \"e2e-ait-bd4c1118-openshift-default-565f6989b4-xjv76\" (UID: \"465e9b26-ea7c-4957-baae-d4dda4827e78\") " pod="openshift-ingress/e2e-ait-bd4c1118-openshift-default-565f6989b4-xjv76"
Sep 19 05:53:34.622258 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:34.622054    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/465e9b26-ea7c-4957-baae-d4dda4827e78-istio-envoy\") pod \"e2e-ait-bd4c1118-openshift-default-565f6989b4-xjv76\" (UID: \"465e9b26-ea7c-4957-baae-d4dda4827e78\") " pod="openshift-ingress/e2e-ait-bd4c1118-openshift-default-565f6989b4-xjv76"
Sep 19 05:53:34.622258 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:34.622068    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/465e9b26-ea7c-4957-baae-d4dda4827e78-istiod-ca-cert\") pod \"e2e-ait-bd4c1118-openshift-default-565f6989b4-xjv76\" (UID: \"465e9b26-ea7c-4957-baae-d4dda4827e78\") " pod="openshift-ingress/e2e-ait-bd4c1118-openshift-default-565f6989b4-xjv76"
Sep 19 05:53:34.622258 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:34.622110    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/465e9b26-ea7c-4957-baae-d4dda4827e78-workload-socket\") pod \"e2e-ait-bd4c1118-openshift-default-565f6989b4-xjv76\" (UID: \"465e9b26-ea7c-4957-baae-d4dda4827e78\") " pod="openshift-ingress/e2e-ait-bd4c1118-openshift-default-565f6989b4-xjv76"
Sep 19 05:53:34.722731 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:34.722692    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/465e9b26-ea7c-4957-baae-d4dda4827e78-workload-socket\") pod \"e2e-ait-bd4c1118-openshift-default-565f6989b4-xjv76\" (UID: \"465e9b26-ea7c-4957-baae-d4dda4827e78\") " pod="openshift-ingress/e2e-ait-bd4c1118-openshift-default-565f6989b4-xjv76"
Sep 19 05:53:34.722901 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:34.722745    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/465e9b26-ea7c-4957-baae-d4dda4827e78-istio-podinfo\") pod \"e2e-ait-bd4c1118-openshift-default-565f6989b4-xjv76\" (UID: \"465e9b26-ea7c-4957-baae-d4dda4827e78\") " pod="openshift-ingress/e2e-ait-bd4c1118-openshift-default-565f6989b4-xjv76"
Sep 19 05:53:34.722901 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:34.722786    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/465e9b26-ea7c-4957-baae-d4dda4827e78-istio-data\") pod \"e2e-ait-bd4c1118-openshift-default-565f6989b4-xjv76\" (UID: \"465e9b26-ea7c-4957-baae-d4dda4827e78\") " pod="openshift-ingress/e2e-ait-bd4c1118-openshift-default-565f6989b4-xjv76"
Sep 19 05:53:34.722901 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:34.722805    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n4j8k\" (UniqueName: \"kubernetes.io/projected/465e9b26-ea7c-4957-baae-d4dda4827e78-kube-api-access-n4j8k\") pod \"e2e-ait-bd4c1118-openshift-default-565f6989b4-xjv76\" (UID: \"465e9b26-ea7c-4957-baae-d4dda4827e78\") " pod="openshift-ingress/e2e-ait-bd4c1118-openshift-default-565f6989b4-xjv76"
Sep 19 05:53:34.722901 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:34.722821    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/465e9b26-ea7c-4957-baae-d4dda4827e78-credential-socket\") pod \"e2e-ait-bd4c1118-openshift-default-565f6989b4-xjv76\" (UID: \"465e9b26-ea7c-4957-baae-d4dda4827e78\") " pod="openshift-ingress/e2e-ait-bd4c1118-openshift-default-565f6989b4-xjv76"
Sep 19 05:53:34.722901 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:34.722844    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/465e9b26-ea7c-4957-baae-d4dda4827e78-workload-certs\") pod \"e2e-ait-bd4c1118-openshift-default-565f6989b4-xjv76\" (UID: \"465e9b26-ea7c-4957-baae-d4dda4827e78\") " pod="openshift-ingress/e2e-ait-bd4c1118-openshift-default-565f6989b4-xjv76"
Sep 19 05:53:34.723209 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:34.722885    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/465e9b26-ea7c-4957-baae-d4dda4827e78-istio-token\") pod \"e2e-ait-bd4c1118-openshift-default-565f6989b4-xjv76\" (UID: \"465e9b26-ea7c-4957-baae-d4dda4827e78\") " pod="openshift-ingress/e2e-ait-bd4c1118-openshift-default-565f6989b4-xjv76"
Sep 19 05:53:34.723209 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:34.723182    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/465e9b26-ea7c-4957-baae-d4dda4827e78-workload-socket\") pod \"e2e-ait-bd4c1118-openshift-default-565f6989b4-xjv76\" (UID: \"465e9b26-ea7c-4957-baae-d4dda4827e78\") " pod="openshift-ingress/e2e-ait-bd4c1118-openshift-default-565f6989b4-xjv76"
Sep 19 05:53:34.723209 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:34.723197    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/465e9b26-ea7c-4957-baae-d4dda4827e78-istio-data\") pod \"e2e-ait-bd4c1118-openshift-default-565f6989b4-xjv76\" (UID: \"465e9b26-ea7c-4957-baae-d4dda4827e78\") " pod="openshift-ingress/e2e-ait-bd4c1118-openshift-default-565f6989b4-xjv76"
Sep 19 05:53:34.723364 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:34.723264    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/465e9b26-ea7c-4957-baae-d4dda4827e78-workload-certs\") pod \"e2e-ait-bd4c1118-openshift-default-565f6989b4-xjv76\" (UID: \"465e9b26-ea7c-4957-baae-d4dda4827e78\") " pod="openshift-ingress/e2e-ait-bd4c1118-openshift-default-565f6989b4-xjv76"
Sep 19 05:53:34.723364 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:34.722962    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/465e9b26-ea7c-4957-baae-d4dda4827e78-istio-envoy\") pod \"e2e-ait-bd4c1118-openshift-default-565f6989b4-xjv76\" (UID: \"465e9b26-ea7c-4957-baae-d4dda4827e78\") " pod="openshift-ingress/e2e-ait-bd4c1118-openshift-default-565f6989b4-xjv76"
Sep 19 05:53:34.723364 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:34.723336    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/465e9b26-ea7c-4957-baae-d4dda4827e78-istiod-ca-cert\") pod \"e2e-ait-bd4c1118-openshift-default-565f6989b4-xjv76\" (UID: \"465e9b26-ea7c-4957-baae-d4dda4827e78\") " pod="openshift-ingress/e2e-ait-bd4c1118-openshift-default-565f6989b4-xjv76"
Sep 19 05:53:34.723540 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:34.723525    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/465e9b26-ea7c-4957-baae-d4dda4827e78-credential-socket\") pod \"e2e-ait-bd4c1118-openshift-default-565f6989b4-xjv76\" (UID: \"465e9b26-ea7c-4957-baae-d4dda4827e78\") " pod="openshift-ingress/e2e-ait-bd4c1118-openshift-default-565f6989b4-xjv76"
Sep 19 05:53:34.724064 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:34.724042    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/465e9b26-ea7c-4957-baae-d4dda4827e78-istiod-ca-cert\") pod \"e2e-ait-bd4c1118-openshift-default-565f6989b4-xjv76\" (UID: \"465e9b26-ea7c-4957-baae-d4dda4827e78\") " pod="openshift-ingress/e2e-ait-bd4c1118-openshift-default-565f6989b4-xjv76"
Sep 19 05:53:34.725362 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:34.725341    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/465e9b26-ea7c-4957-baae-d4dda4827e78-istio-envoy\") pod \"e2e-ait-bd4c1118-openshift-default-565f6989b4-xjv76\" (UID: \"465e9b26-ea7c-4957-baae-d4dda4827e78\") " pod="openshift-ingress/e2e-ait-bd4c1118-openshift-default-565f6989b4-xjv76"
Sep 19 05:53:34.725550 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:34.725530    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/465e9b26-ea7c-4957-baae-d4dda4827e78-istio-podinfo\") pod \"e2e-ait-bd4c1118-openshift-default-565f6989b4-xjv76\" (UID: \"465e9b26-ea7c-4957-baae-d4dda4827e78\") " pod="openshift-ingress/e2e-ait-bd4c1118-openshift-default-565f6989b4-xjv76"
Sep 19 05:53:34.737787 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:34.737762    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n4j8k\" (UniqueName: \"kubernetes.io/projected/465e9b26-ea7c-4957-baae-d4dda4827e78-kube-api-access-n4j8k\") pod \"e2e-ait-bd4c1118-openshift-default-565f6989b4-xjv76\" (UID: \"465e9b26-ea7c-4957-baae-d4dda4827e78\") " pod="openshift-ingress/e2e-ait-bd4c1118-openshift-default-565f6989b4-xjv76"
Sep 19 05:53:34.740637 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:34.740619    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/465e9b26-ea7c-4957-baae-d4dda4827e78-istio-token\") pod \"e2e-ait-bd4c1118-openshift-default-565f6989b4-xjv76\" (UID: \"465e9b26-ea7c-4957-baae-d4dda4827e78\") " pod="openshift-ingress/e2e-ait-bd4c1118-openshift-default-565f6989b4-xjv76"
Sep 19 05:53:34.816409 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:34.816386    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-bd4c1118-openshift-default-565f6989b4-xjv76"
Sep 19 05:53:35.101476 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:35.101444    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-bd4c1118-openshift-default-565f6989b4-xjv76"]
Sep 19 05:53:35.102549 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:53:35.102526    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod465e9b26_ea7c_4957_baae_d4dda4827e78.slice/crio-819fd4476eec913a2f8f53fad0af256bc16db9d410e38b96bfee1c21bcad9c2e WatchSource:0}: Error finding container 819fd4476eec913a2f8f53fad0af256bc16db9d410e38b96bfee1c21bcad9c2e: Status 404 returned error can't find the container with id 819fd4476eec913a2f8f53fad0af256bc16db9d410e38b96bfee1c21bcad9c2e
Sep 19 05:53:35.104908 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:35.104877    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 19 05:53:35.104966 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:35.104958    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 19 05:53:35.105008 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:35.104984    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 19 05:53:35.458076 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:35.457986    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-bd4c1118-openshift-default-565f6989b4-xjv76" event={"ID":"465e9b26-ea7c-4957-baae-d4dda4827e78","Type":"ContainerStarted","Data":"adf10e4a3bf211dce9ea3ab105f04464d8c734130f9c74ca3f005ad9c863894b"}
Sep 19 05:53:35.458076 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:35.458043    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-bd4c1118-openshift-default-565f6989b4-xjv76" event={"ID":"465e9b26-ea7c-4957-baae-d4dda4827e78","Type":"ContainerStarted","Data":"819fd4476eec913a2f8f53fad0af256bc16db9d410e38b96bfee1c21bcad9c2e"}
Sep 19 05:53:35.477043 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:35.476977    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-bd4c1118-openshift-default-565f6989b4-xjv76" podStartSLOduration=1.476957043 podStartE2EDuration="1.476957043s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:53:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:53:35.475787178 +0000 UTC m=+774.134241730" watchObservedRunningTime="2026-09-19 05:53:35.476957043 +0000 UTC m=+774.135411604"
Sep 19 05:53:35.816637 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:35.816609    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-bd4c1118-openshift-default-565f6989b4-xjv76"
Sep 19 05:53:36.822690 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:36.822654    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-bd4c1118-openshift-default-565f6989b4-xjv76"
Sep 19 05:53:37.472405 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:37.472372    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-bd4c1118-openshift-default-565f6989b4-xjv76"
Sep 19 05:53:37.475746 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:37.475721    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-bd4c1118-openshift-default-565f6989b4-xjv76"
Sep 19 05:53:38.449954 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:38.449926    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-3182b5-d6c9696c6-jwzw7"
Sep 19 05:53:38.472542 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:38.472512    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-bd4c1118-85c6d65957-fgxm5"]
Sep 19 05:53:38.476803 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:38.476778    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bd4c1118-85c6d65957-fgxm5"
Sep 19 05:53:38.479396 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:38.479376    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-bd4c1118-serving-cert\""
Sep 19 05:53:38.496437 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:38.496410    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-bd4c1118-85c6d65957-fgxm5"]
Sep 19 05:53:38.562568 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:38.562536    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1ddb803f-d353-4008-b572-6ff3d22c9156-metrics-certs\") pod \"maas-api-e2e-ait-bd4c1118-85c6d65957-fgxm5\" (UID: \"1ddb803f-d353-4008-b572-6ff3d22c9156\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bd4c1118-85c6d65957-fgxm5"
Sep 19 05:53:38.562568 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:38.562566    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nzrwd\" (UniqueName: \"kubernetes.io/projected/1ddb803f-d353-4008-b572-6ff3d22c9156-kube-api-access-nzrwd\") pod \"maas-api-e2e-ait-bd4c1118-85c6d65957-fgxm5\" (UID: \"1ddb803f-d353-4008-b572-6ff3d22c9156\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bd4c1118-85c6d65957-fgxm5"
Sep 19 05:53:38.562747 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:38.562618    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1ddb803f-d353-4008-b572-6ff3d22c9156-openshift-service-ca\") pod \"maas-api-e2e-ait-bd4c1118-85c6d65957-fgxm5\" (UID: \"1ddb803f-d353-4008-b572-6ff3d22c9156\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bd4c1118-85c6d65957-fgxm5"
Sep 19 05:53:38.562814 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:38.562799    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1ddb803f-d353-4008-b572-6ff3d22c9156-maas-api-tls\") pod \"maas-api-e2e-ait-bd4c1118-85c6d65957-fgxm5\" (UID: \"1ddb803f-d353-4008-b572-6ff3d22c9156\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bd4c1118-85c6d65957-fgxm5"
Sep 19 05:53:38.665226 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:38.664573    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1ddb803f-d353-4008-b572-6ff3d22c9156-metrics-certs\") pod \"maas-api-e2e-ait-bd4c1118-85c6d65957-fgxm5\" (UID: \"1ddb803f-d353-4008-b572-6ff3d22c9156\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bd4c1118-85c6d65957-fgxm5"
Sep 19 05:53:38.665226 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:38.664628    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nzrwd\" (UniqueName: \"kubernetes.io/projected/1ddb803f-d353-4008-b572-6ff3d22c9156-kube-api-access-nzrwd\") pod \"maas-api-e2e-ait-bd4c1118-85c6d65957-fgxm5\" (UID: \"1ddb803f-d353-4008-b572-6ff3d22c9156\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bd4c1118-85c6d65957-fgxm5"
Sep 19 05:53:38.665226 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:38.664669    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1ddb803f-d353-4008-b572-6ff3d22c9156-openshift-service-ca\") pod \"maas-api-e2e-ait-bd4c1118-85c6d65957-fgxm5\" (UID: \"1ddb803f-d353-4008-b572-6ff3d22c9156\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bd4c1118-85c6d65957-fgxm5"
Sep 19 05:53:38.665226 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:38.664761    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1ddb803f-d353-4008-b572-6ff3d22c9156-maas-api-tls\") pod \"maas-api-e2e-ait-bd4c1118-85c6d65957-fgxm5\" (UID: \"1ddb803f-d353-4008-b572-6ff3d22c9156\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bd4c1118-85c6d65957-fgxm5"
Sep 19 05:53:38.665226 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:53:38.664885    2583 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-bd4c1118-serving-cert: secret "maas-api-e2e-ait-bd4c1118-serving-cert" not found
Sep 19 05:53:38.665226 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:53:38.664955    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1ddb803f-d353-4008-b572-6ff3d22c9156-maas-api-tls podName:1ddb803f-d353-4008-b572-6ff3d22c9156 nodeName:}" failed. No retries permitted until 2026-09-19 05:53:39.164935824 +0000 UTC m=+777.823390360 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/1ddb803f-d353-4008-b572-6ff3d22c9156-maas-api-tls") pod "maas-api-e2e-ait-bd4c1118-85c6d65957-fgxm5" (UID: "1ddb803f-d353-4008-b572-6ff3d22c9156") : secret "maas-api-e2e-ait-bd4c1118-serving-cert" not found
Sep 19 05:53:38.665668 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:38.665511    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1ddb803f-d353-4008-b572-6ff3d22c9156-openshift-service-ca\") pod \"maas-api-e2e-ait-bd4c1118-85c6d65957-fgxm5\" (UID: \"1ddb803f-d353-4008-b572-6ff3d22c9156\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bd4c1118-85c6d65957-fgxm5"
Sep 19 05:53:38.667555 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:38.667493    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1ddb803f-d353-4008-b572-6ff3d22c9156-metrics-certs\") pod \"maas-api-e2e-ait-bd4c1118-85c6d65957-fgxm5\" (UID: \"1ddb803f-d353-4008-b572-6ff3d22c9156\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bd4c1118-85c6d65957-fgxm5"
Sep 19 05:53:38.675577 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:38.675553    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nzrwd\" (UniqueName: \"kubernetes.io/projected/1ddb803f-d353-4008-b572-6ff3d22c9156-kube-api-access-nzrwd\") pod \"maas-api-e2e-ait-bd4c1118-85c6d65957-fgxm5\" (UID: \"1ddb803f-d353-4008-b572-6ff3d22c9156\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bd4c1118-85c6d65957-fgxm5"
Sep 19 05:53:39.169616 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:39.169574    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1ddb803f-d353-4008-b572-6ff3d22c9156-maas-api-tls\") pod \"maas-api-e2e-ait-bd4c1118-85c6d65957-fgxm5\" (UID: \"1ddb803f-d353-4008-b572-6ff3d22c9156\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bd4c1118-85c6d65957-fgxm5"
Sep 19 05:53:39.172857 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:39.172823    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1ddb803f-d353-4008-b572-6ff3d22c9156-maas-api-tls\") pod \"maas-api-e2e-ait-bd4c1118-85c6d65957-fgxm5\" (UID: \"1ddb803f-d353-4008-b572-6ff3d22c9156\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bd4c1118-85c6d65957-fgxm5"
Sep 19 05:53:39.393475 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:39.393422    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bd4c1118-85c6d65957-fgxm5"
Sep 19 05:53:39.468032 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:39.464304    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4b78e2-6bf5c5f97b-75ldh"
Sep 19 05:53:39.731461 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:39.731375    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-bd4c1118-85c6d65957-fgxm5"]
Sep 19 05:53:39.750847 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:53:39.750810    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ddb803f_d353_4008_b572_6ff3d22c9156.slice/crio-632d9254ac50545d9d1c281acf87dcb66682cb83d9f8aaded151f642dff07a66 WatchSource:0}: Error finding container 632d9254ac50545d9d1c281acf87dcb66682cb83d9f8aaded151f642dff07a66: Status 404 returned error can't find the container with id 632d9254ac50545d9d1c281acf87dcb66682cb83d9f8aaded151f642dff07a66
Sep 19 05:53:40.545513 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:40.545464    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bd4c1118-85c6d65957-fgxm5" event={"ID":"1ddb803f-d353-4008-b572-6ff3d22c9156","Type":"ContainerStarted","Data":"7567513dd32352be5fdeb7ada49d9d3b7bb55f87bde60cd641f6b1cf08cc27e3"}
Sep 19 05:53:40.546000 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:40.545542    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bd4c1118-85c6d65957-fgxm5" event={"ID":"1ddb803f-d353-4008-b572-6ff3d22c9156","Type":"ContainerStarted","Data":"632d9254ac50545d9d1c281acf87dcb66682cb83d9f8aaded151f642dff07a66"}
Sep 19 05:53:40.546412 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:40.546384    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bd4c1118-85c6d65957-fgxm5"
Sep 19 05:53:43.866159 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:43.865728    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bd4c1118-85c6d65957-fgxm5" podStartSLOduration=5.47598302 podStartE2EDuration="5.865703185s" totalImagesPullingTime="389.720165ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:53:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:53:40.567928812 +0000 UTC m=+779.226383371" watchObservedRunningTime="2026-09-19 05:53:43.865703185 +0000 UTC m=+782.524157746"
Sep 19 05:53:43.878168 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:43.876596    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-c85316-openshift-default-57bfb64984-xgk69"]
Sep 19 05:53:43.881845 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:43.881815    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-c85316-openshift-default-57bfb64984-xgk69"
Sep 19 05:53:43.886047 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:43.886020    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-c85316-openshift-default-dockercfg-csm79\""
Sep 19 05:53:43.891914 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:43.891885    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-c85316-openshift-default-57bfb64984-xgk69"]
Sep 19 05:53:44.068154 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:44.067624    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3b2632bb-bcca-400d-835f-197701450eeb-istio-podinfo\") pod \"e2e-inf-b-c85316-openshift-default-57bfb64984-xgk69\" (UID: \"3b2632bb-bcca-400d-835f-197701450eeb\") " pod="openshift-ingress/e2e-inf-b-c85316-openshift-default-57bfb64984-xgk69"
Sep 19 05:53:44.068154 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:44.067682    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3b2632bb-bcca-400d-835f-197701450eeb-istiod-ca-cert\") pod \"e2e-inf-b-c85316-openshift-default-57bfb64984-xgk69\" (UID: \"3b2632bb-bcca-400d-835f-197701450eeb\") " pod="openshift-ingress/e2e-inf-b-c85316-openshift-default-57bfb64984-xgk69"
Sep 19 05:53:44.068154 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:44.067717    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b2632bb-bcca-400d-835f-197701450eeb-workload-socket\") pod \"e2e-inf-b-c85316-openshift-default-57bfb64984-xgk69\" (UID: \"3b2632bb-bcca-400d-835f-197701450eeb\") " pod="openshift-ingress/e2e-inf-b-c85316-openshift-default-57bfb64984-xgk69"
Sep 19 05:53:44.068154 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:44.067761    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3b2632bb-bcca-400d-835f-197701450eeb-istio-token\") pod \"e2e-inf-b-c85316-openshift-default-57bfb64984-xgk69\" (UID: \"3b2632bb-bcca-400d-835f-197701450eeb\") " pod="openshift-ingress/e2e-inf-b-c85316-openshift-default-57bfb64984-xgk69"
Sep 19 05:53:44.068154 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:44.067811    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3b2632bb-bcca-400d-835f-197701450eeb-istio-envoy\") pod \"e2e-inf-b-c85316-openshift-default-57bfb64984-xgk69\" (UID: \"3b2632bb-bcca-400d-835f-197701450eeb\") " pod="openshift-ingress/e2e-inf-b-c85316-openshift-default-57bfb64984-xgk69"
Sep 19 05:53:44.068154 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:44.067860    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bzckb\" (UniqueName: \"kubernetes.io/projected/3b2632bb-bcca-400d-835f-197701450eeb-kube-api-access-bzckb\") pod \"e2e-inf-b-c85316-openshift-default-57bfb64984-xgk69\" (UID: \"3b2632bb-bcca-400d-835f-197701450eeb\") " pod="openshift-ingress/e2e-inf-b-c85316-openshift-default-57bfb64984-xgk69"
Sep 19 05:53:44.068154 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:44.067909    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3b2632bb-bcca-400d-835f-197701450eeb-istio-data\") pod \"e2e-inf-b-c85316-openshift-default-57bfb64984-xgk69\" (UID: \"3b2632bb-bcca-400d-835f-197701450eeb\") " pod="openshift-ingress/e2e-inf-b-c85316-openshift-default-57bfb64984-xgk69"
Sep 19 05:53:44.068154 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:44.067979    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b2632bb-bcca-400d-835f-197701450eeb-credential-socket\") pod \"e2e-inf-b-c85316-openshift-default-57bfb64984-xgk69\" (UID: \"3b2632bb-bcca-400d-835f-197701450eeb\") " pod="openshift-ingress/e2e-inf-b-c85316-openshift-default-57bfb64984-xgk69"
Sep 19 05:53:44.068154 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:44.068003    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3b2632bb-bcca-400d-835f-197701450eeb-workload-certs\") pod \"e2e-inf-b-c85316-openshift-default-57bfb64984-xgk69\" (UID: \"3b2632bb-bcca-400d-835f-197701450eeb\") " pod="openshift-ingress/e2e-inf-b-c85316-openshift-default-57bfb64984-xgk69"
Sep 19 05:53:44.169036 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:44.168924    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b2632bb-bcca-400d-835f-197701450eeb-credential-socket\") pod \"e2e-inf-b-c85316-openshift-default-57bfb64984-xgk69\" (UID: \"3b2632bb-bcca-400d-835f-197701450eeb\") " pod="openshift-ingress/e2e-inf-b-c85316-openshift-default-57bfb64984-xgk69"
Sep 19 05:53:44.169036 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:44.168966    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3b2632bb-bcca-400d-835f-197701450eeb-workload-certs\") pod \"e2e-inf-b-c85316-openshift-default-57bfb64984-xgk69\" (UID: \"3b2632bb-bcca-400d-835f-197701450eeb\") " pod="openshift-ingress/e2e-inf-b-c85316-openshift-default-57bfb64984-xgk69"
Sep 19 05:53:44.169036 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:44.169001    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3b2632bb-bcca-400d-835f-197701450eeb-istio-podinfo\") pod \"e2e-inf-b-c85316-openshift-default-57bfb64984-xgk69\" (UID: \"3b2632bb-bcca-400d-835f-197701450eeb\") " pod="openshift-ingress/e2e-inf-b-c85316-openshift-default-57bfb64984-xgk69"
Sep 19 05:53:44.169036 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:44.169023    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3b2632bb-bcca-400d-835f-197701450eeb-istiod-ca-cert\") pod \"e2e-inf-b-c85316-openshift-default-57bfb64984-xgk69\" (UID: \"3b2632bb-bcca-400d-835f-197701450eeb\") " pod="openshift-ingress/e2e-inf-b-c85316-openshift-default-57bfb64984-xgk69"
Sep 19 05:53:44.169440 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:44.169051    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b2632bb-bcca-400d-835f-197701450eeb-workload-socket\") pod \"e2e-inf-b-c85316-openshift-default-57bfb64984-xgk69\" (UID: \"3b2632bb-bcca-400d-835f-197701450eeb\") " pod="openshift-ingress/e2e-inf-b-c85316-openshift-default-57bfb64984-xgk69"
Sep 19 05:53:44.169440 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:44.169089    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3b2632bb-bcca-400d-835f-197701450eeb-istio-token\") pod \"e2e-inf-b-c85316-openshift-default-57bfb64984-xgk69\" (UID: \"3b2632bb-bcca-400d-835f-197701450eeb\") " pod="openshift-ingress/e2e-inf-b-c85316-openshift-default-57bfb64984-xgk69"
Sep 19 05:53:44.169440 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:44.169147    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3b2632bb-bcca-400d-835f-197701450eeb-istio-envoy\") pod \"e2e-inf-b-c85316-openshift-default-57bfb64984-xgk69\" (UID: \"3b2632bb-bcca-400d-835f-197701450eeb\") " pod="openshift-ingress/e2e-inf-b-c85316-openshift-default-57bfb64984-xgk69"
Sep 19 05:53:44.169440 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:44.169196    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bzckb\" (UniqueName: \"kubernetes.io/projected/3b2632bb-bcca-400d-835f-197701450eeb-kube-api-access-bzckb\") pod \"e2e-inf-b-c85316-openshift-default-57bfb64984-xgk69\" (UID: \"3b2632bb-bcca-400d-835f-197701450eeb\") " pod="openshift-ingress/e2e-inf-b-c85316-openshift-default-57bfb64984-xgk69"
Sep 19 05:53:44.169440 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:44.169247    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3b2632bb-bcca-400d-835f-197701450eeb-istio-data\") pod \"e2e-inf-b-c85316-openshift-default-57bfb64984-xgk69\" (UID: \"3b2632bb-bcca-400d-835f-197701450eeb\") " pod="openshift-ingress/e2e-inf-b-c85316-openshift-default-57bfb64984-xgk69"
Sep 19 05:53:44.169711 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:44.169479    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3b2632bb-bcca-400d-835f-197701450eeb-workload-certs\") pod \"e2e-inf-b-c85316-openshift-default-57bfb64984-xgk69\" (UID: \"3b2632bb-bcca-400d-835f-197701450eeb\") " pod="openshift-ingress/e2e-inf-b-c85316-openshift-default-57bfb64984-xgk69"
Sep 19 05:53:44.169711 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:44.169497    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b2632bb-bcca-400d-835f-197701450eeb-workload-socket\") pod \"e2e-inf-b-c85316-openshift-default-57bfb64984-xgk69\" (UID: \"3b2632bb-bcca-400d-835f-197701450eeb\") " pod="openshift-ingress/e2e-inf-b-c85316-openshift-default-57bfb64984-xgk69"
Sep 19 05:53:44.169711 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:44.169643    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3b2632bb-bcca-400d-835f-197701450eeb-istio-data\") pod \"e2e-inf-b-c85316-openshift-default-57bfb64984-xgk69\" (UID: \"3b2632bb-bcca-400d-835f-197701450eeb\") " pod="openshift-ingress/e2e-inf-b-c85316-openshift-default-57bfb64984-xgk69"
Sep 19 05:53:44.169969 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:44.169946    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b2632bb-bcca-400d-835f-197701450eeb-credential-socket\") pod \"e2e-inf-b-c85316-openshift-default-57bfb64984-xgk69\" (UID: \"3b2632bb-bcca-400d-835f-197701450eeb\") " pod="openshift-ingress/e2e-inf-b-c85316-openshift-default-57bfb64984-xgk69"
Sep 19 05:53:44.170481 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:44.170454    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3b2632bb-bcca-400d-835f-197701450eeb-istiod-ca-cert\") pod \"e2e-inf-b-c85316-openshift-default-57bfb64984-xgk69\" (UID: \"3b2632bb-bcca-400d-835f-197701450eeb\") " pod="openshift-ingress/e2e-inf-b-c85316-openshift-default-57bfb64984-xgk69"
Sep 19 05:53:44.171835 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:44.171807    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3b2632bb-bcca-400d-835f-197701450eeb-istio-envoy\") pod \"e2e-inf-b-c85316-openshift-default-57bfb64984-xgk69\" (UID: \"3b2632bb-bcca-400d-835f-197701450eeb\") " pod="openshift-ingress/e2e-inf-b-c85316-openshift-default-57bfb64984-xgk69"
Sep 19 05:53:44.171964 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:44.171946    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3b2632bb-bcca-400d-835f-197701450eeb-istio-podinfo\") pod \"e2e-inf-b-c85316-openshift-default-57bfb64984-xgk69\" (UID: \"3b2632bb-bcca-400d-835f-197701450eeb\") " pod="openshift-ingress/e2e-inf-b-c85316-openshift-default-57bfb64984-xgk69"
Sep 19 05:53:44.178248 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:44.178227    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3b2632bb-bcca-400d-835f-197701450eeb-istio-token\") pod \"e2e-inf-b-c85316-openshift-default-57bfb64984-xgk69\" (UID: \"3b2632bb-bcca-400d-835f-197701450eeb\") " pod="openshift-ingress/e2e-inf-b-c85316-openshift-default-57bfb64984-xgk69"
Sep 19 05:53:44.178472 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:44.178451    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bzckb\" (UniqueName: \"kubernetes.io/projected/3b2632bb-bcca-400d-835f-197701450eeb-kube-api-access-bzckb\") pod \"e2e-inf-b-c85316-openshift-default-57bfb64984-xgk69\" (UID: \"3b2632bb-bcca-400d-835f-197701450eeb\") " pod="openshift-ingress/e2e-inf-b-c85316-openshift-default-57bfb64984-xgk69"
Sep 19 05:53:44.199122 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:44.199093    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-c85316-openshift-default-57bfb64984-xgk69"
Sep 19 05:53:44.535148 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:44.527903    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-c85316-openshift-default-57bfb64984-xgk69"]
Sep 19 05:53:44.544302 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:53:44.544251    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b2632bb_bcca_400d_835f_197701450eeb.slice/crio-56f646bbecad89ac3d30f994991c68c60f6072456632735bc1091abc3b16846a WatchSource:0}: Error finding container 56f646bbecad89ac3d30f994991c68c60f6072456632735bc1091abc3b16846a: Status 404 returned error can't find the container with id 56f646bbecad89ac3d30f994991c68c60f6072456632735bc1091abc3b16846a
Sep 19 05:53:44.547559 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:44.546811    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 19 05:53:44.547559 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:44.546896    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 19 05:53:44.547559 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:44.546943    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 19 05:53:44.580624 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:44.580583    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-c85316-openshift-default-57bfb64984-xgk69" event={"ID":"3b2632bb-bcca-400d-835f-197701450eeb","Type":"ContainerStarted","Data":"56f646bbecad89ac3d30f994991c68c60f6072456632735bc1091abc3b16846a"}
Sep 19 05:53:45.589866 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:45.589832    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-c85316-openshift-default-57bfb64984-xgk69" event={"ID":"3b2632bb-bcca-400d-835f-197701450eeb","Type":"ContainerStarted","Data":"589b669105a8f4fc5c1ef39d7c000ffc9fcae2006395787366059c24a0d1c43b"}
Sep 19 05:53:45.617054 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:45.616995    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-c85316-openshift-default-57bfb64984-xgk69" podStartSLOduration=2.6169750609999998 podStartE2EDuration="2.616975061s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:53:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:53:45.614459529 +0000 UTC m=+784.272914083" watchObservedRunningTime="2026-09-19 05:53:45.616975061 +0000 UTC m=+784.275429620"
Sep 19 05:53:46.200354 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:46.200310    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-c85316-openshift-default-57bfb64984-xgk69"
Sep 19 05:53:46.206166 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:46.206117    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-c85316-openshift-default-57bfb64984-xgk69"
Sep 19 05:53:46.597683 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:46.597646    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-c85316-openshift-default-57bfb64984-xgk69"
Sep 19 05:53:46.598764 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:46.598740    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-c85316-openshift-default-57bfb64984-xgk69"
Sep 19 05:53:47.568267 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:47.568239    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bd4c1118-85c6d65957-fgxm5"
Sep 19 05:53:48.613392 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:48.613357    2583 generic.go:356] "Generic (PLEG): container finished" podID="b57edf34-4a41-4cb6-bc1f-f73dae8adad1" containerID="718372ca6c871216a911c583987ab7a96ce5d8ea38e760b37b4a66ec0d50da8e" exitCode=0
Sep 19 05:53:48.613817 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:48.613430    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-3182b5-openshift-default-76dd9fb6bb-b5p7s" event={"ID":"b57edf34-4a41-4cb6-bc1f-f73dae8adad1","Type":"ContainerDied","Data":"718372ca6c871216a911c583987ab7a96ce5d8ea38e760b37b4a66ec0d50da8e"}
Sep 19 05:53:48.613905 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:48.613887    2583 scope.go:117] "RemoveContainer" containerID="718372ca6c871216a911c583987ab7a96ce5d8ea38e760b37b4a66ec0d50da8e"
Sep 19 05:53:48.615105 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:48.615070    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 19 05:53:48.615232 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:48.615177    2583 generic.go:356] "Generic (PLEG): container finished" podID="57450d37-8096-4ff3-a23f-5ce98d6cbc22" containerID="e0ed9229f52bc20a9a8c8b116f2575ff5abaeb42445d4ef500940a278d7a0bf8" exitCode=0
Sep 19 05:53:48.615232 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:48.615190    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 19 05:53:48.615232 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:48.615225    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 19 05:53:48.615338 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:48.615250    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-4b78e2-openshift-default-7b7664c598-rz5lx" event={"ID":"57450d37-8096-4ff3-a23f-5ce98d6cbc22","Type":"ContainerDied","Data":"e0ed9229f52bc20a9a8c8b116f2575ff5abaeb42445d4ef500940a278d7a0bf8"}
Sep 19 05:53:48.615688 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:48.615671    2583 scope.go:117] "RemoveContainer" containerID="e0ed9229f52bc20a9a8c8b116f2575ff5abaeb42445d4ef500940a278d7a0bf8"
Sep 19 05:53:48.616665 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:48.616643    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 19 05:53:48.616731 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:48.616705    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 19 05:53:48.616767 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:48.616737    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 19 05:53:49.627486 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:49.627441    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-3182b5-openshift-default-76dd9fb6bb-b5p7s" event={"ID":"b57edf34-4a41-4cb6-bc1f-f73dae8adad1","Type":"ContainerStarted","Data":"e69dfe65342b7667dfb020cee99016f94a0570281307afc79f6487d15004b2e3"}
Sep 19 05:53:49.635146 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:49.635088    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-4b78e2-openshift-default-7b7664c598-rz5lx" event={"ID":"57450d37-8096-4ff3-a23f-5ce98d6cbc22","Type":"ContainerStarted","Data":"c33be8bfb63320a231e5d408ec225e19cd71ebf3c73acbf7d68ce98b29804fcf"}
Sep 19 05:53:50.283510 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:50.283468    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-4b78e2-openshift-default-7b7664c598-rz5lx"
Sep 19 05:53:50.283730 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:50.283603    2583 patch_prober.go:28] interesting pod/e2e-inf-a-4b78e2-openshift-default-7b7664c598-rz5lx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body=
Sep 19 05:53:50.283730 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:50.283688    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-4b78e2-openshift-default-7b7664c598-rz5lx" podUID="57450d37-8096-4ff3-a23f-5ce98d6cbc22" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused"
Sep 19 05:53:50.455768 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:50.455726    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-3182b5-openshift-default-76dd9fb6bb-b5p7s"
Sep 19 05:53:50.456023 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:50.455989    2583 patch_prober.go:28] interesting pod/e2e-worker-w4-3182b5-openshift-default-76dd9fb6bb-b5p7s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body=
Sep 19 05:53:50.456104 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:50.456057    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-3182b5-openshift-default-76dd9fb6bb-b5p7s" podUID="b57edf34-4a41-4cb6-bc1f-f73dae8adad1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused"
Sep 19 05:53:51.284293 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:51.284252    2583 patch_prober.go:28] interesting pod/e2e-inf-a-4b78e2-openshift-default-7b7664c598-rz5lx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body=
Sep 19 05:53:51.285001 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:51.284962    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-4b78e2-openshift-default-7b7664c598-rz5lx" podUID="57450d37-8096-4ff3-a23f-5ce98d6cbc22" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused"
Sep 19 05:53:51.456247 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:51.455769    2583 patch_prober.go:28] interesting pod/e2e-worker-w4-3182b5-openshift-default-76dd9fb6bb-b5p7s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body=
Sep 19 05:53:51.456247 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:51.455836    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-3182b5-openshift-default-76dd9fb6bb-b5p7s" podUID="b57edf34-4a41-4cb6-bc1f-f73dae8adad1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused"
Sep 19 05:53:51.821794 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:51.821747    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-c85316-7f7d79c99-2tz2f"]
Sep 19 05:53:51.825100 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:51.825064    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c85316-7f7d79c99-2tz2f"
Sep 19 05:53:51.828023 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:51.827999    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-c85316-serving-cert\""
Sep 19 05:53:51.836292 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:51.836212    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-c85316-7f7d79c99-2tz2f"]
Sep 19 05:53:51.856700 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:51.856662    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d1e19012-831d-430e-b711-c2b0a4665df8-metrics-certs\") pod \"maas-api-e2e-inf-b-c85316-7f7d79c99-2tz2f\" (UID: \"d1e19012-831d-430e-b711-c2b0a4665df8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c85316-7f7d79c99-2tz2f"
Sep 19 05:53:51.856961 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:51.856938    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d1e19012-831d-430e-b711-c2b0a4665df8-maas-api-tls\") pod \"maas-api-e2e-inf-b-c85316-7f7d79c99-2tz2f\" (UID: \"d1e19012-831d-430e-b711-c2b0a4665df8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c85316-7f7d79c99-2tz2f"
Sep 19 05:53:51.857292 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:51.857234    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d1e19012-831d-430e-b711-c2b0a4665df8-openshift-service-ca\") pod \"maas-api-e2e-inf-b-c85316-7f7d79c99-2tz2f\" (UID: \"d1e19012-831d-430e-b711-c2b0a4665df8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c85316-7f7d79c99-2tz2f"
Sep 19 05:53:51.857457 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:51.857438    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7ftjc\" (UniqueName: \"kubernetes.io/projected/d1e19012-831d-430e-b711-c2b0a4665df8-kube-api-access-7ftjc\") pod \"maas-api-e2e-inf-b-c85316-7f7d79c99-2tz2f\" (UID: \"d1e19012-831d-430e-b711-c2b0a4665df8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c85316-7f7d79c99-2tz2f"
Sep 19 05:53:51.970688 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:51.970649    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d1e19012-831d-430e-b711-c2b0a4665df8-openshift-service-ca\") pod \"maas-api-e2e-inf-b-c85316-7f7d79c99-2tz2f\" (UID: \"d1e19012-831d-430e-b711-c2b0a4665df8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c85316-7f7d79c99-2tz2f"
Sep 19 05:53:51.971741 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:51.971721    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7ftjc\" (UniqueName: \"kubernetes.io/projected/d1e19012-831d-430e-b711-c2b0a4665df8-kube-api-access-7ftjc\") pod \"maas-api-e2e-inf-b-c85316-7f7d79c99-2tz2f\" (UID: \"d1e19012-831d-430e-b711-c2b0a4665df8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c85316-7f7d79c99-2tz2f"
Sep 19 05:53:51.972348 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:51.972323    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d1e19012-831d-430e-b711-c2b0a4665df8-metrics-certs\") pod \"maas-api-e2e-inf-b-c85316-7f7d79c99-2tz2f\" (UID: \"d1e19012-831d-430e-b711-c2b0a4665df8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c85316-7f7d79c99-2tz2f"
Sep 19 05:53:51.972948 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:51.972930    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d1e19012-831d-430e-b711-c2b0a4665df8-maas-api-tls\") pod \"maas-api-e2e-inf-b-c85316-7f7d79c99-2tz2f\" (UID: \"d1e19012-831d-430e-b711-c2b0a4665df8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c85316-7f7d79c99-2tz2f"
Sep 19 05:53:51.973275 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:51.971646    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d1e19012-831d-430e-b711-c2b0a4665df8-openshift-service-ca\") pod \"maas-api-e2e-inf-b-c85316-7f7d79c99-2tz2f\" (UID: \"d1e19012-831d-430e-b711-c2b0a4665df8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c85316-7f7d79c99-2tz2f"
Sep 19 05:53:51.973486 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:53:51.973155    2583 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-c85316-serving-cert: secret "maas-api-e2e-inf-b-c85316-serving-cert" not found
Sep 19 05:53:51.973650 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:53:51.973638    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1e19012-831d-430e-b711-c2b0a4665df8-maas-api-tls podName:d1e19012-831d-430e-b711-c2b0a4665df8 nodeName:}" failed. No retries permitted until 2026-09-19 05:53:52.473606972 +0000 UTC m=+791.132061509 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d1e19012-831d-430e-b711-c2b0a4665df8-maas-api-tls") pod "maas-api-e2e-inf-b-c85316-7f7d79c99-2tz2f" (UID: "d1e19012-831d-430e-b711-c2b0a4665df8") : secret "maas-api-e2e-inf-b-c85316-serving-cert" not found
Sep 19 05:53:51.984675 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:51.984651    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7ftjc\" (UniqueName: \"kubernetes.io/projected/d1e19012-831d-430e-b711-c2b0a4665df8-kube-api-access-7ftjc\") pod \"maas-api-e2e-inf-b-c85316-7f7d79c99-2tz2f\" (UID: \"d1e19012-831d-430e-b711-c2b0a4665df8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c85316-7f7d79c99-2tz2f"
Sep 19 05:53:51.984968 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:51.984687    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d1e19012-831d-430e-b711-c2b0a4665df8-metrics-certs\") pod \"maas-api-e2e-inf-b-c85316-7f7d79c99-2tz2f\" (UID: \"d1e19012-831d-430e-b711-c2b0a4665df8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c85316-7f7d79c99-2tz2f"
Sep 19 05:53:52.284116 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:52.284068    2583 patch_prober.go:28] interesting pod/e2e-inf-a-4b78e2-openshift-default-7b7664c598-rz5lx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body=
Sep 19 05:53:52.284353 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:52.284162    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-4b78e2-openshift-default-7b7664c598-rz5lx" podUID="57450d37-8096-4ff3-a23f-5ce98d6cbc22" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused"
Sep 19 05:53:52.455515 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:52.455471    2583 patch_prober.go:28] interesting pod/e2e-worker-w4-3182b5-openshift-default-76dd9fb6bb-b5p7s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body=
Sep 19 05:53:52.455730 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:52.455546    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-3182b5-openshift-default-76dd9fb6bb-b5p7s" podUID="b57edf34-4a41-4cb6-bc1f-f73dae8adad1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused"
Sep 19 05:53:52.479876 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:52.479829    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d1e19012-831d-430e-b711-c2b0a4665df8-maas-api-tls\") pod \"maas-api-e2e-inf-b-c85316-7f7d79c99-2tz2f\" (UID: \"d1e19012-831d-430e-b711-c2b0a4665df8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c85316-7f7d79c99-2tz2f"
Sep 19 05:53:52.480083 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:53:52.480011    2583 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-c85316-serving-cert: secret "maas-api-e2e-inf-b-c85316-serving-cert" not found
Sep 19 05:53:52.480083 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:53:52.480080    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1e19012-831d-430e-b711-c2b0a4665df8-maas-api-tls podName:d1e19012-831d-430e-b711-c2b0a4665df8 nodeName:}" failed. No retries permitted until 2026-09-19 05:53:53.480057823 +0000 UTC m=+792.138512376 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d1e19012-831d-430e-b711-c2b0a4665df8-maas-api-tls") pod "maas-api-e2e-inf-b-c85316-7f7d79c99-2tz2f" (UID: "d1e19012-831d-430e-b711-c2b0a4665df8") : secret "maas-api-e2e-inf-b-c85316-serving-cert" not found
Sep 19 05:53:53.285351 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:53.285296    2583 patch_prober.go:28] interesting pod/e2e-inf-a-4b78e2-openshift-default-7b7664c598-rz5lx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body=
Sep 19 05:53:53.285882 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:53.285376    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-4b78e2-openshift-default-7b7664c598-rz5lx" podUID="57450d37-8096-4ff3-a23f-5ce98d6cbc22" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused"
Sep 19 05:53:53.455684 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:53.455243    2583 patch_prober.go:28] interesting pod/e2e-worker-w4-3182b5-openshift-default-76dd9fb6bb-b5p7s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body=
Sep 19 05:53:53.455684 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:53.455316    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-3182b5-openshift-default-76dd9fb6bb-b5p7s" podUID="b57edf34-4a41-4cb6-bc1f-f73dae8adad1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused"
Sep 19 05:53:53.490537 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:53.490497    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d1e19012-831d-430e-b711-c2b0a4665df8-maas-api-tls\") pod \"maas-api-e2e-inf-b-c85316-7f7d79c99-2tz2f\" (UID: \"d1e19012-831d-430e-b711-c2b0a4665df8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c85316-7f7d79c99-2tz2f"
Sep 19 05:53:53.493744 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:53.493718    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d1e19012-831d-430e-b711-c2b0a4665df8-maas-api-tls\") pod \"maas-api-e2e-inf-b-c85316-7f7d79c99-2tz2f\" (UID: \"d1e19012-831d-430e-b711-c2b0a4665df8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c85316-7f7d79c99-2tz2f"
Sep 19 05:53:53.649450 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:53.649353    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c85316-7f7d79c99-2tz2f"
Sep 19 05:53:54.041908 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:54.041856    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-c85316-7f7d79c99-2tz2f"]
Sep 19 05:53:54.067724 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:53:54.065078    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1e19012_831d_430e_b711_c2b0a4665df8.slice/crio-502118f2c0fba9edac334652dd77c05d6fcad3efe0b489807f2548d5dcb030a6 WatchSource:0}: Error finding container 502118f2c0fba9edac334652dd77c05d6fcad3efe0b489807f2548d5dcb030a6: Status 404 returned error can't find the container with id 502118f2c0fba9edac334652dd77c05d6fcad3efe0b489807f2548d5dcb030a6
Sep 19 05:53:54.284538 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:54.284481    2583 patch_prober.go:28] interesting pod/e2e-inf-a-4b78e2-openshift-default-7b7664c598-rz5lx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body=
Sep 19 05:53:54.284742 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:54.284554    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-4b78e2-openshift-default-7b7664c598-rz5lx" podUID="57450d37-8096-4ff3-a23f-5ce98d6cbc22" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused"
Sep 19 05:53:54.458153 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:54.455066    2583 patch_prober.go:28] interesting pod/e2e-worker-w4-3182b5-openshift-default-76dd9fb6bb-b5p7s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body=
Sep 19 05:53:54.458153 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:54.455145    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-3182b5-openshift-default-76dd9fb6bb-b5p7s" podUID="b57edf34-4a41-4cb6-bc1f-f73dae8adad1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused"
Sep 19 05:53:54.700438 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:54.700336    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c85316-7f7d79c99-2tz2f" event={"ID":"d1e19012-831d-430e-b711-c2b0a4665df8","Type":"ContainerStarted","Data":"00f3f973469ff192cd8cb1d83568e64aba1e1425b25bd40badaebff372daa9da"}
Sep 19 05:53:54.700438 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:54.700406    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c85316-7f7d79c99-2tz2f" event={"ID":"d1e19012-831d-430e-b711-c2b0a4665df8","Type":"ContainerStarted","Data":"502118f2c0fba9edac334652dd77c05d6fcad3efe0b489807f2548d5dcb030a6"}
Sep 19 05:53:54.700644 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:54.700459    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c85316-7f7d79c99-2tz2f"
Sep 19 05:53:54.725512 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:54.725438    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c85316-7f7d79c99-2tz2f" podStartSLOduration=3.4058039190000002 podStartE2EDuration="3.725414347s" totalImagesPullingTime="319.610428ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:53:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:53:54.722875472 +0000 UTC m=+793.381330031" watchObservedRunningTime="2026-09-19 05:53:54.725414347 +0000 UTC m=+793.383868907"
Sep 19 05:53:55.284107 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:55.284057    2583 patch_prober.go:28] interesting pod/e2e-inf-a-4b78e2-openshift-default-7b7664c598-rz5lx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body=
Sep 19 05:53:55.284307 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:55.284153    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-4b78e2-openshift-default-7b7664c598-rz5lx" podUID="57450d37-8096-4ff3-a23f-5ce98d6cbc22" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused"
Sep 19 05:53:55.474328 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:55.474236    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-3182b5-openshift-default-76dd9fb6bb-b5p7s"
Sep 19 05:53:55.475418 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:55.475397    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-3182b5-openshift-default-76dd9fb6bb-b5p7s"
Sep 19 05:53:55.476636 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:55.476615    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-3182b5-openshift-default-76dd9fb6bb-b5p7s"
Sep 19 05:53:56.287700 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:56.287670    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-4b78e2-openshift-default-7b7664c598-rz5lx"
Sep 19 05:53:56.287954 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:56.287929    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-4b78e2-openshift-default-7b7664c598-rz5lx"
Sep 19 05:53:56.288676 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:53:56.288651    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-4b78e2-openshift-default-7b7664c598-rz5lx"
Sep 19 05:54:00.711765 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:00.711736    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c85316-7f7d79c99-2tz2f"
Sep 19 05:54:00.740070 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:00.740038    2583 generic.go:356] "Generic (PLEG): container finished" podID="465e9b26-ea7c-4957-baae-d4dda4827e78" containerID="adf10e4a3bf211dce9ea3ab105f04464d8c734130f9c74ca3f005ad9c863894b" exitCode=0
Sep 19 05:54:00.740260 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:00.740085    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-bd4c1118-openshift-default-565f6989b4-xjv76" event={"ID":"465e9b26-ea7c-4957-baae-d4dda4827e78","Type":"ContainerDied","Data":"adf10e4a3bf211dce9ea3ab105f04464d8c734130f9c74ca3f005ad9c863894b"}
Sep 19 05:54:00.740569 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:00.740549    2583 scope.go:117] "RemoveContainer" containerID="adf10e4a3bf211dce9ea3ab105f04464d8c734130f9c74ca3f005ad9c863894b"
Sep 19 05:54:00.741816 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:00.741774    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 19 05:54:00.741919 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:00.741854    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 19 05:54:00.741919 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:00.741911    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 19 05:54:01.748511 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:01.748475    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-bd4c1118-openshift-default-565f6989b4-xjv76" event={"ID":"465e9b26-ea7c-4957-baae-d4dda4827e78","Type":"ContainerStarted","Data":"300e85c5cd323e1ea731a6c523d328e7545cd0158ec702163298b876426e85c5"}
Sep 19 05:54:01.817451 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:01.817415    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-bd4c1118-openshift-default-565f6989b4-xjv76"
Sep 19 05:54:01.817801 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:01.817773    2583 patch_prober.go:28] interesting pod/e2e-ait-bd4c1118-openshift-default-565f6989b4-xjv76 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body=
Sep 19 05:54:01.817934 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:01.817828    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-bd4c1118-openshift-default-565f6989b4-xjv76" podUID="465e9b26-ea7c-4957-baae-d4dda4827e78" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused"
Sep 19 05:54:02.817363 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:02.817319    2583 patch_prober.go:28] interesting pod/e2e-ait-bd4c1118-openshift-default-565f6989b4-xjv76 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body=
Sep 19 05:54:02.817883 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:02.817408    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-bd4c1118-openshift-default-565f6989b4-xjv76" podUID="465e9b26-ea7c-4957-baae-d4dda4827e78" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused"
Sep 19 05:54:03.817071 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:03.817033    2583 patch_prober.go:28] interesting pod/e2e-ait-bd4c1118-openshift-default-565f6989b4-xjv76 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body=
Sep 19 05:54:03.817311 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:03.817101    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-bd4c1118-openshift-default-565f6989b4-xjv76" podUID="465e9b26-ea7c-4957-baae-d4dda4827e78" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused"
Sep 19 05:54:04.817028 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:04.816991    2583 patch_prober.go:28] interesting pod/e2e-ait-bd4c1118-openshift-default-565f6989b4-xjv76 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body=
Sep 19 05:54:04.817639 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:04.817609    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-bd4c1118-openshift-default-565f6989b4-xjv76" podUID="465e9b26-ea7c-4957-baae-d4dda4827e78" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused"
Sep 19 05:54:05.817586 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:05.817539    2583 patch_prober.go:28] interesting pod/e2e-ait-bd4c1118-openshift-default-565f6989b4-xjv76 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body=
Sep 19 05:54:05.818069 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:05.817621    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-bd4c1118-openshift-default-565f6989b4-xjv76" podUID="465e9b26-ea7c-4957-baae-d4dda4827e78" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused"
Sep 19 05:54:06.817677 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:06.817634    2583 patch_prober.go:28] interesting pod/e2e-ait-bd4c1118-openshift-default-565f6989b4-xjv76 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body=
Sep 19 05:54:06.818203 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:06.817709    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-bd4c1118-openshift-default-565f6989b4-xjv76" podUID="465e9b26-ea7c-4957-baae-d4dda4827e78" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused"
Sep 19 05:54:07.472605 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:07.472574    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-bd4c1118-openshift-default-565f6989b4-xjv76"
Sep 19 05:54:07.828661 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:07.828630    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-bd4c1118-openshift-default-565f6989b4-xjv76"
Sep 19 05:54:07.833894 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:07.833869    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-bd4c1118-openshift-default-565f6989b4-xjv76"
Sep 19 05:54:08.210882 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:08.210803    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-bd4c1118-85c6d65957-fgxm5"]
Sep 19 05:54:08.211062 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:08.211017    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bd4c1118-85c6d65957-fgxm5" podUID="1ddb803f-d353-4008-b572-6ff3d22c9156" containerName="maas-api" containerID="cri-o://7567513dd32352be5fdeb7ada49d9d3b7bb55f87bde60cd641f6b1cf08cc27e3" gracePeriod=30
Sep 19 05:54:08.500713 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:08.500692    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bd4c1118-85c6d65957-fgxm5"
Sep 19 05:54:08.573592 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:08.573561    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1ddb803f-d353-4008-b572-6ff3d22c9156-maas-api-tls\") pod \"1ddb803f-d353-4008-b572-6ff3d22c9156\" (UID: \"1ddb803f-d353-4008-b572-6ff3d22c9156\") "
Sep 19 05:54:08.573771 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:08.573645    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nzrwd\" (UniqueName: \"kubernetes.io/projected/1ddb803f-d353-4008-b572-6ff3d22c9156-kube-api-access-nzrwd\") pod \"1ddb803f-d353-4008-b572-6ff3d22c9156\" (UID: \"1ddb803f-d353-4008-b572-6ff3d22c9156\") "
Sep 19 05:54:08.573771 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:08.573701    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1ddb803f-d353-4008-b572-6ff3d22c9156-metrics-certs\") pod \"1ddb803f-d353-4008-b572-6ff3d22c9156\" (UID: \"1ddb803f-d353-4008-b572-6ff3d22c9156\") "
Sep 19 05:54:08.573771 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:08.573759    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1ddb803f-d353-4008-b572-6ff3d22c9156-openshift-service-ca\") pod \"1ddb803f-d353-4008-b572-6ff3d22c9156\" (UID: \"1ddb803f-d353-4008-b572-6ff3d22c9156\") "
Sep 19 05:54:08.574163 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:08.574112    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1ddb803f-d353-4008-b572-6ff3d22c9156-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "1ddb803f-d353-4008-b572-6ff3d22c9156" (UID: "1ddb803f-d353-4008-b572-6ff3d22c9156"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 05:54:08.575931 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:08.575905    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1ddb803f-d353-4008-b572-6ff3d22c9156-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "1ddb803f-d353-4008-b572-6ff3d22c9156" (UID: "1ddb803f-d353-4008-b572-6ff3d22c9156"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 05:54:08.576083 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:08.575957    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1ddb803f-d353-4008-b572-6ff3d22c9156-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "1ddb803f-d353-4008-b572-6ff3d22c9156" (UID: "1ddb803f-d353-4008-b572-6ff3d22c9156"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 05:54:08.576244 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:08.576220    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1ddb803f-d353-4008-b572-6ff3d22c9156-kube-api-access-nzrwd" (OuterVolumeSpecName: "kube-api-access-nzrwd") pod "1ddb803f-d353-4008-b572-6ff3d22c9156" (UID: "1ddb803f-d353-4008-b572-6ff3d22c9156"). InnerVolumeSpecName "kube-api-access-nzrwd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 05:54:08.613549 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:08.613520    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-bd4c1118-openshift-default-565f6989b4-xjv76"]
Sep 19 05:54:08.675158 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:08.675103    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1ddb803f-d353-4008-b572-6ff3d22c9156-maas-api-tls\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 05:54:08.675325 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:08.675164    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nzrwd\" (UniqueName: \"kubernetes.io/projected/1ddb803f-d353-4008-b572-6ff3d22c9156-kube-api-access-nzrwd\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 05:54:08.675325 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:08.675178    2583 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1ddb803f-d353-4008-b572-6ff3d22c9156-metrics-certs\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 05:54:08.675325 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:08.675187    2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1ddb803f-d353-4008-b572-6ff3d22c9156-openshift-service-ca\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 05:54:08.794378 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:08.794342    2583 generic.go:356] "Generic (PLEG): container finished" podID="1ddb803f-d353-4008-b572-6ff3d22c9156" containerID="7567513dd32352be5fdeb7ada49d9d3b7bb55f87bde60cd641f6b1cf08cc27e3" exitCode=0
Sep 19 05:54:08.794573 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:08.794395    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bd4c1118-85c6d65957-fgxm5"
Sep 19 05:54:08.794573 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:08.794444    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bd4c1118-85c6d65957-fgxm5" event={"ID":"1ddb803f-d353-4008-b572-6ff3d22c9156","Type":"ContainerDied","Data":"7567513dd32352be5fdeb7ada49d9d3b7bb55f87bde60cd641f6b1cf08cc27e3"}
Sep 19 05:54:08.794573 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:08.794514    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bd4c1118-85c6d65957-fgxm5" event={"ID":"1ddb803f-d353-4008-b572-6ff3d22c9156","Type":"ContainerDied","Data":"632d9254ac50545d9d1c281acf87dcb66682cb83d9f8aaded151f642dff07a66"}
Sep 19 05:54:08.794573 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:08.794548    2583 scope.go:117] "RemoveContainer" containerID="7567513dd32352be5fdeb7ada49d9d3b7bb55f87bde60cd641f6b1cf08cc27e3"
Sep 19 05:54:08.794909 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:08.794889    2583 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/e2e-ait-bd4c1118-openshift-default-565f6989b4-xjv76" secret="" err="secret \"e2e-ait-bd4c1118-openshift-default-dockercfg-95q5f\" not found"
Sep 19 05:54:08.804912 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:08.804895    2583 scope.go:117] "RemoveContainer" containerID="7567513dd32352be5fdeb7ada49d9d3b7bb55f87bde60cd641f6b1cf08cc27e3"
Sep 19 05:54:08.805215 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:54:08.805189    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7567513dd32352be5fdeb7ada49d9d3b7bb55f87bde60cd641f6b1cf08cc27e3\": container with ID starting with 7567513dd32352be5fdeb7ada49d9d3b7bb55f87bde60cd641f6b1cf08cc27e3 not found: ID does not exist" containerID="7567513dd32352be5fdeb7ada49d9d3b7bb55f87bde60cd641f6b1cf08cc27e3"
Sep 19 05:54:08.805294 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:08.805228    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7567513dd32352be5fdeb7ada49d9d3b7bb55f87bde60cd641f6b1cf08cc27e3"} err="failed to get container status \"7567513dd32352be5fdeb7ada49d9d3b7bb55f87bde60cd641f6b1cf08cc27e3\": rpc error: code = NotFound desc = could not find container \"7567513dd32352be5fdeb7ada49d9d3b7bb55f87bde60cd641f6b1cf08cc27e3\": container with ID starting with 7567513dd32352be5fdeb7ada49d9d3b7bb55f87bde60cd641f6b1cf08cc27e3 not found: ID does not exist"
Sep 19 05:54:08.815885 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:08.815862    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-bd4c1118-85c6d65957-fgxm5"]
Sep 19 05:54:08.828856 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:08.828824    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-bd4c1118-85c6d65957-fgxm5"]
Sep 19 05:54:09.499364 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:09.499332    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1ddb803f-d353-4008-b572-6ff3d22c9156" path="/var/lib/kubelet/pods/1ddb803f-d353-4008-b572-6ff3d22c9156/volumes"
Sep 19 05:54:09.802848 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:09.802771    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-bd4c1118-openshift-default-565f6989b4-xjv76" podUID="465e9b26-ea7c-4957-baae-d4dda4827e78" containerName="istio-proxy" containerID="cri-o://300e85c5cd323e1ea731a6c523d328e7545cd0158ec702163298b876426e85c5" gracePeriod=30
Sep 19 05:54:14.837302 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:14.837271    2583 generic.go:356] "Generic (PLEG): container finished" podID="3b2632bb-bcca-400d-835f-197701450eeb" containerID="589b669105a8f4fc5c1ef39d7c000ffc9fcae2006395787366059c24a0d1c43b" exitCode=0
Sep 19 05:54:14.837723 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:14.837338    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-c85316-openshift-default-57bfb64984-xgk69" event={"ID":"3b2632bb-bcca-400d-835f-197701450eeb","Type":"ContainerDied","Data":"589b669105a8f4fc5c1ef39d7c000ffc9fcae2006395787366059c24a0d1c43b"}
Sep 19 05:54:14.838038 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:14.837983    2583 scope.go:117] "RemoveContainer" containerID="589b669105a8f4fc5c1ef39d7c000ffc9fcae2006395787366059c24a0d1c43b"
Sep 19 05:54:14.839873 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:14.839553    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 19 05:54:14.839873 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:14.839642    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 19 05:54:14.839873 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:14.839686    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 19 05:54:15.074645 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:15.074618    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-bd4c1118-openshift-default-565f6989b4-xjv76"
Sep 19 05:54:15.238934 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:15.238856    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n4j8k\" (UniqueName: \"kubernetes.io/projected/465e9b26-ea7c-4957-baae-d4dda4827e78-kube-api-access-n4j8k\") pod \"465e9b26-ea7c-4957-baae-d4dda4827e78\" (UID: \"465e9b26-ea7c-4957-baae-d4dda4827e78\") "
Sep 19 05:54:15.238934 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:15.238901    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/465e9b26-ea7c-4957-baae-d4dda4827e78-workload-certs\") pod \"465e9b26-ea7c-4957-baae-d4dda4827e78\" (UID: \"465e9b26-ea7c-4957-baae-d4dda4827e78\") "
Sep 19 05:54:15.238934 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:15.238927    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/465e9b26-ea7c-4957-baae-d4dda4827e78-istio-data\") pod \"465e9b26-ea7c-4957-baae-d4dda4827e78\" (UID: \"465e9b26-ea7c-4957-baae-d4dda4827e78\") "
Sep 19 05:54:15.239200 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:15.238950    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/465e9b26-ea7c-4957-baae-d4dda4827e78-istiod-ca-cert\") pod \"465e9b26-ea7c-4957-baae-d4dda4827e78\" (UID: \"465e9b26-ea7c-4957-baae-d4dda4827e78\") "
Sep 19 05:54:15.239200 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:15.239007    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/465e9b26-ea7c-4957-baae-d4dda4827e78-istio-token\") pod \"465e9b26-ea7c-4957-baae-d4dda4827e78\" (UID: \"465e9b26-ea7c-4957-baae-d4dda4827e78\") "
Sep 19 05:54:15.239200 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:15.239069    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/465e9b26-ea7c-4957-baae-d4dda4827e78-istio-podinfo\") pod \"465e9b26-ea7c-4957-baae-d4dda4827e78\" (UID: \"465e9b26-ea7c-4957-baae-d4dda4827e78\") "
Sep 19 05:54:15.239200 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:15.239104    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/465e9b26-ea7c-4957-baae-d4dda4827e78-istio-envoy\") pod \"465e9b26-ea7c-4957-baae-d4dda4827e78\" (UID: \"465e9b26-ea7c-4957-baae-d4dda4827e78\") "
Sep 19 05:54:15.239200 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:15.239176    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/465e9b26-ea7c-4957-baae-d4dda4827e78-credential-socket\") pod \"465e9b26-ea7c-4957-baae-d4dda4827e78\" (UID: \"465e9b26-ea7c-4957-baae-d4dda4827e78\") "
Sep 19 05:54:15.239446 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:15.239196    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/465e9b26-ea7c-4957-baae-d4dda4827e78-istio-data" (OuterVolumeSpecName: "istio-data") pod "465e9b26-ea7c-4957-baae-d4dda4827e78" (UID: "465e9b26-ea7c-4957-baae-d4dda4827e78"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 05:54:15.239446 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:15.239211    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/465e9b26-ea7c-4957-baae-d4dda4827e78-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "465e9b26-ea7c-4957-baae-d4dda4827e78" (UID: "465e9b26-ea7c-4957-baae-d4dda4827e78"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 05:54:15.239446 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:15.239244    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/465e9b26-ea7c-4957-baae-d4dda4827e78-workload-socket\") pod \"465e9b26-ea7c-4957-baae-d4dda4827e78\" (UID: \"465e9b26-ea7c-4957-baae-d4dda4827e78\") "
Sep 19 05:54:15.239446 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:15.239381    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/465e9b26-ea7c-4957-baae-d4dda4827e78-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "465e9b26-ea7c-4957-baae-d4dda4827e78" (UID: "465e9b26-ea7c-4957-baae-d4dda4827e78"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 05:54:15.239663 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:15.239495    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/465e9b26-ea7c-4957-baae-d4dda4827e78-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "465e9b26-ea7c-4957-baae-d4dda4827e78" (UID: "465e9b26-ea7c-4957-baae-d4dda4827e78"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 05:54:15.239663 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:15.239636    2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/465e9b26-ea7c-4957-baae-d4dda4827e78-istiod-ca-cert\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 05:54:15.239663 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:15.239653    2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/465e9b26-ea7c-4957-baae-d4dda4827e78-workload-socket\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 05:54:15.239827 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:15.239666    2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/465e9b26-ea7c-4957-baae-d4dda4827e78-workload-certs\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 05:54:15.239827 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:15.239670    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/465e9b26-ea7c-4957-baae-d4dda4827e78-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "465e9b26-ea7c-4957-baae-d4dda4827e78" (UID: "465e9b26-ea7c-4957-baae-d4dda4827e78"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 05:54:15.239827 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:15.239679    2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/465e9b26-ea7c-4957-baae-d4dda4827e78-istio-data\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 05:54:15.241592 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:15.241569    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/465e9b26-ea7c-4957-baae-d4dda4827e78-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "465e9b26-ea7c-4957-baae-d4dda4827e78" (UID: "465e9b26-ea7c-4957-baae-d4dda4827e78"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 19 05:54:15.241722 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:15.241593    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/465e9b26-ea7c-4957-baae-d4dda4827e78-kube-api-access-n4j8k" (OuterVolumeSpecName: "kube-api-access-n4j8k") pod "465e9b26-ea7c-4957-baae-d4dda4827e78" (UID: "465e9b26-ea7c-4957-baae-d4dda4827e78"). InnerVolumeSpecName "kube-api-access-n4j8k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 05:54:15.241722 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:15.241690    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/465e9b26-ea7c-4957-baae-d4dda4827e78-istio-token" (OuterVolumeSpecName: "istio-token") pod "465e9b26-ea7c-4957-baae-d4dda4827e78" (UID: "465e9b26-ea7c-4957-baae-d4dda4827e78"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 05:54:15.241816 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:15.241720    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/465e9b26-ea7c-4957-baae-d4dda4827e78-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "465e9b26-ea7c-4957-baae-d4dda4827e78" (UID: "465e9b26-ea7c-4957-baae-d4dda4827e78"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 05:54:15.340949 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:15.340915    2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/465e9b26-ea7c-4957-baae-d4dda4827e78-istio-envoy\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 05:54:15.340949 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:15.340944    2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/465e9b26-ea7c-4957-baae-d4dda4827e78-credential-socket\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 05:54:15.340949 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:15.340955    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n4j8k\" (UniqueName: \"kubernetes.io/projected/465e9b26-ea7c-4957-baae-d4dda4827e78-kube-api-access-n4j8k\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 05:54:15.341191 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:15.340963    2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/465e9b26-ea7c-4957-baae-d4dda4827e78-istio-token\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 05:54:15.341191 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:15.340972    2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/465e9b26-ea7c-4957-baae-d4dda4827e78-istio-podinfo\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 05:54:15.847696 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:15.847655    2583 generic.go:356] "Generic (PLEG): container finished" podID="465e9b26-ea7c-4957-baae-d4dda4827e78" containerID="300e85c5cd323e1ea731a6c523d328e7545cd0158ec702163298b876426e85c5" exitCode=0
Sep 19 05:54:15.848213 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:15.847740    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-bd4c1118-openshift-default-565f6989b4-xjv76" event={"ID":"465e9b26-ea7c-4957-baae-d4dda4827e78","Type":"ContainerDied","Data":"300e85c5cd323e1ea731a6c523d328e7545cd0158ec702163298b876426e85c5"}
Sep 19 05:54:15.848213 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:15.847757    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-bd4c1118-openshift-default-565f6989b4-xjv76"
Sep 19 05:54:15.848213 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:15.847827    2583 scope.go:117] "RemoveContainer" containerID="300e85c5cd323e1ea731a6c523d328e7545cd0158ec702163298b876426e85c5"
Sep 19 05:54:15.848213 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:15.847811    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-bd4c1118-openshift-default-565f6989b4-xjv76" event={"ID":"465e9b26-ea7c-4957-baae-d4dda4827e78","Type":"ContainerDied","Data":"819fd4476eec913a2f8f53fad0af256bc16db9d410e38b96bfee1c21bcad9c2e"}
Sep 19 05:54:15.851107 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:15.851076    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-c85316-openshift-default-57bfb64984-xgk69" event={"ID":"3b2632bb-bcca-400d-835f-197701450eeb","Type":"ContainerStarted","Data":"a3868f8b128e9a8dbfdd2f586b6243eae4a956c506d129230a1ac273abac5235"}
Sep 19 05:54:15.858696 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:15.858676    2583 scope.go:117] "RemoveContainer" containerID="adf10e4a3bf211dce9ea3ab105f04464d8c734130f9c74ca3f005ad9c863894b"
Sep 19 05:54:15.866888 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:15.866853    2583 scope.go:117] "RemoveContainer" containerID="300e85c5cd323e1ea731a6c523d328e7545cd0158ec702163298b876426e85c5"
Sep 19 05:54:15.867174 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:54:15.867151    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"300e85c5cd323e1ea731a6c523d328e7545cd0158ec702163298b876426e85c5\": container with ID starting with 300e85c5cd323e1ea731a6c523d328e7545cd0158ec702163298b876426e85c5 not found: ID does not exist" containerID="300e85c5cd323e1ea731a6c523d328e7545cd0158ec702163298b876426e85c5"
Sep 19 05:54:15.867242 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:15.867186    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"300e85c5cd323e1ea731a6c523d328e7545cd0158ec702163298b876426e85c5"} err="failed to get container status \"300e85c5cd323e1ea731a6c523d328e7545cd0158ec702163298b876426e85c5\": rpc error: code = NotFound desc = could not find container \"300e85c5cd323e1ea731a6c523d328e7545cd0158ec702163298b876426e85c5\": container with ID starting with 300e85c5cd323e1ea731a6c523d328e7545cd0158ec702163298b876426e85c5 not found: ID does not exist"
Sep 19 05:54:15.867242 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:15.867202    2583 scope.go:117] "RemoveContainer" containerID="adf10e4a3bf211dce9ea3ab105f04464d8c734130f9c74ca3f005ad9c863894b"
Sep 19 05:54:15.867450 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:54:15.867432    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"adf10e4a3bf211dce9ea3ab105f04464d8c734130f9c74ca3f005ad9c863894b\": container with ID starting with adf10e4a3bf211dce9ea3ab105f04464d8c734130f9c74ca3f005ad9c863894b not found: ID does not exist" containerID="adf10e4a3bf211dce9ea3ab105f04464d8c734130f9c74ca3f005ad9c863894b"
Sep 19 05:54:15.867496 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:15.867458    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"adf10e4a3bf211dce9ea3ab105f04464d8c734130f9c74ca3f005ad9c863894b"} err="failed to get container status \"adf10e4a3bf211dce9ea3ab105f04464d8c734130f9c74ca3f005ad9c863894b\": rpc error: code = NotFound desc = could not find container \"adf10e4a3bf211dce9ea3ab105f04464d8c734130f9c74ca3f005ad9c863894b\": container with ID starting with adf10e4a3bf211dce9ea3ab105f04464d8c734130f9c74ca3f005ad9c863894b not found: ID does not exist"
Sep 19 05:54:15.887399 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:15.887374    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-bd4c1118-openshift-default-565f6989b4-xjv76"]
Sep 19 05:54:15.889748 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:15.889725    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-bd4c1118-openshift-default-565f6989b4-xjv76"]
Sep 19 05:54:16.200185 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:16.200080    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-c85316-openshift-default-57bfb64984-xgk69"
Sep 19 05:54:16.201336 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:16.201309    2583 patch_prober.go:28] interesting pod/e2e-inf-b-c85316-openshift-default-57bfb64984-xgk69 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body=
Sep 19 05:54:16.201462 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:16.201375    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-c85316-openshift-default-57bfb64984-xgk69" podUID="3b2632bb-bcca-400d-835f-197701450eeb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused"
Sep 19 05:54:16.598331 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:16.598293    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-c85316-openshift-default-57bfb64984-xgk69"
Sep 19 05:54:17.200437 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:17.200401    2583 patch_prober.go:28] interesting pod/e2e-inf-b-c85316-openshift-default-57bfb64984-xgk69 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body=
Sep 19 05:54:17.200906 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:17.200471    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-c85316-openshift-default-57bfb64984-xgk69" podUID="3b2632bb-bcca-400d-835f-197701450eeb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused"
Sep 19 05:54:17.498254 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:17.498169    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="465e9b26-ea7c-4957-baae-d4dda4827e78" path="/var/lib/kubelet/pods/465e9b26-ea7c-4957-baae-d4dda4827e78/volumes"
Sep 19 05:54:18.200182 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:18.200124    2583 patch_prober.go:28] interesting pod/e2e-inf-b-c85316-openshift-default-57bfb64984-xgk69 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body=
Sep 19 05:54:18.200393 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:18.200223    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-c85316-openshift-default-57bfb64984-xgk69" podUID="3b2632bb-bcca-400d-835f-197701450eeb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused"
Sep 19 05:54:19.200239 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:19.200203    2583 patch_prober.go:28] interesting pod/e2e-inf-b-c85316-openshift-default-57bfb64984-xgk69 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body=
Sep 19 05:54:19.200660 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:19.200272    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-c85316-openshift-default-57bfb64984-xgk69" podUID="3b2632bb-bcca-400d-835f-197701450eeb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused"
Sep 19 05:54:20.199657 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:20.199614    2583 patch_prober.go:28] interesting pod/e2e-inf-b-c85316-openshift-default-57bfb64984-xgk69 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body=
Sep 19 05:54:20.199837 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:20.199677    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-c85316-openshift-default-57bfb64984-xgk69" podUID="3b2632bb-bcca-400d-835f-197701450eeb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused"
Sep 19 05:54:21.203463 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:21.203432    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-c85316-openshift-default-57bfb64984-xgk69"
Sep 19 05:54:21.204391 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:21.204370    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-c85316-openshift-default-57bfb64984-xgk69"
Sep 19 05:54:55.448644 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:55.448608    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-32a87fd8-openshift-default-7f58d48f6b-p76dc"]
Sep 19 05:54:55.449299 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:55.449277    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="465e9b26-ea7c-4957-baae-d4dda4827e78" containerName="istio-proxy"
Sep 19 05:54:55.449406 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:55.449302    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="465e9b26-ea7c-4957-baae-d4dda4827e78" containerName="istio-proxy"
Sep 19 05:54:55.449406 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:55.449330    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="465e9b26-ea7c-4957-baae-d4dda4827e78" containerName="istio-proxy"
Sep 19 05:54:55.449406 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:55.449338    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="465e9b26-ea7c-4957-baae-d4dda4827e78" containerName="istio-proxy"
Sep 19 05:54:55.449406 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:55.449368    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1ddb803f-d353-4008-b572-6ff3d22c9156" containerName="maas-api"
Sep 19 05:54:55.449406 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:55.449376    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ddb803f-d353-4008-b572-6ff3d22c9156" containerName="maas-api"
Sep 19 05:54:55.449655 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:55.449479    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="465e9b26-ea7c-4957-baae-d4dda4827e78" containerName="istio-proxy"
Sep 19 05:54:55.449655 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:55.449495    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="1ddb803f-d353-4008-b572-6ff3d22c9156" containerName="maas-api"
Sep 19 05:54:55.449655 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:55.449505    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="465e9b26-ea7c-4957-baae-d4dda4827e78" containerName="istio-proxy"
Sep 19 05:54:55.452480 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:55.452454    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-32a87fd8-openshift-default-7f58d48f6b-p76dc"
Sep 19 05:54:55.455262 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:55.455239    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-32a87fd8-openshift-default-dockercfg-4c4k8\""
Sep 19 05:54:55.471055 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:55.471029    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-32a87fd8-openshift-default-7f58d48f6b-p76dc"]
Sep 19 05:54:55.625566 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:55.625525    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cea2723e-34c7-44b0-a256-9651af2c6638-istio-podinfo\") pod \"e2e-ait-32a87fd8-openshift-default-7f58d48f6b-p76dc\" (UID: \"cea2723e-34c7-44b0-a256-9651af2c6638\") " pod="openshift-ingress/e2e-ait-32a87fd8-openshift-default-7f58d48f6b-p76dc"
Sep 19 05:54:55.625566 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:55.625574    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cea2723e-34c7-44b0-a256-9651af2c6638-istiod-ca-cert\") pod \"e2e-ait-32a87fd8-openshift-default-7f58d48f6b-p76dc\" (UID: \"cea2723e-34c7-44b0-a256-9651af2c6638\") " pod="openshift-ingress/e2e-ait-32a87fd8-openshift-default-7f58d48f6b-p76dc"
Sep 19 05:54:55.625843 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:55.625711    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cea2723e-34c7-44b0-a256-9651af2c6638-istio-envoy\") pod \"e2e-ait-32a87fd8-openshift-default-7f58d48f6b-p76dc\" (UID: \"cea2723e-34c7-44b0-a256-9651af2c6638\") " pod="openshift-ingress/e2e-ait-32a87fd8-openshift-default-7f58d48f6b-p76dc"
Sep 19 05:54:55.625843 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:55.625747    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cea2723e-34c7-44b0-a256-9651af2c6638-istio-data\") pod \"e2e-ait-32a87fd8-openshift-default-7f58d48f6b-p76dc\" (UID: \"cea2723e-34c7-44b0-a256-9651af2c6638\") " pod="openshift-ingress/e2e-ait-32a87fd8-openshift-default-7f58d48f6b-p76dc"
Sep 19 05:54:55.625843 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:55.625825    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cea2723e-34c7-44b0-a256-9651af2c6638-workload-socket\") pod \"e2e-ait-32a87fd8-openshift-default-7f58d48f6b-p76dc\" (UID: \"cea2723e-34c7-44b0-a256-9651af2c6638\") " pod="openshift-ingress/e2e-ait-32a87fd8-openshift-default-7f58d48f6b-p76dc"
Sep 19 05:54:55.625961 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:55.625880    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h7ftx\" (UniqueName: \"kubernetes.io/projected/cea2723e-34c7-44b0-a256-9651af2c6638-kube-api-access-h7ftx\") pod \"e2e-ait-32a87fd8-openshift-default-7f58d48f6b-p76dc\" (UID: \"cea2723e-34c7-44b0-a256-9651af2c6638\") " pod="openshift-ingress/e2e-ait-32a87fd8-openshift-default-7f58d48f6b-p76dc"
Sep 19 05:54:55.625961 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:55.625917    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cea2723e-34c7-44b0-a256-9651af2c6638-istio-token\") pod \"e2e-ait-32a87fd8-openshift-default-7f58d48f6b-p76dc\" (UID: \"cea2723e-34c7-44b0-a256-9651af2c6638\") " pod="openshift-ingress/e2e-ait-32a87fd8-openshift-default-7f58d48f6b-p76dc"
Sep 19 05:54:55.626041 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:55.625974    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cea2723e-34c7-44b0-a256-9651af2c6638-credential-socket\") pod \"e2e-ait-32a87fd8-openshift-default-7f58d48f6b-p76dc\" (UID: \"cea2723e-34c7-44b0-a256-9651af2c6638\") " pod="openshift-ingress/e2e-ait-32a87fd8-openshift-default-7f58d48f6b-p76dc"
Sep 19 05:54:55.626041 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:55.626010    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cea2723e-34c7-44b0-a256-9651af2c6638-workload-certs\") pod \"e2e-ait-32a87fd8-openshift-default-7f58d48f6b-p76dc\" (UID: \"cea2723e-34c7-44b0-a256-9651af2c6638\") " pod="openshift-ingress/e2e-ait-32a87fd8-openshift-default-7f58d48f6b-p76dc"
Sep 19 05:54:55.727541 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:55.727455    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cea2723e-34c7-44b0-a256-9651af2c6638-credential-socket\") pod \"e2e-ait-32a87fd8-openshift-default-7f58d48f6b-p76dc\" (UID: \"cea2723e-34c7-44b0-a256-9651af2c6638\") " pod="openshift-ingress/e2e-ait-32a87fd8-openshift-default-7f58d48f6b-p76dc"
Sep 19 05:54:55.727541 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:55.727496    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cea2723e-34c7-44b0-a256-9651af2c6638-workload-certs\") pod \"e2e-ait-32a87fd8-openshift-default-7f58d48f6b-p76dc\" (UID: \"cea2723e-34c7-44b0-a256-9651af2c6638\") " pod="openshift-ingress/e2e-ait-32a87fd8-openshift-default-7f58d48f6b-p76dc"
Sep 19 05:54:55.727541 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:55.727525    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cea2723e-34c7-44b0-a256-9651af2c6638-istio-podinfo\") pod \"e2e-ait-32a87fd8-openshift-default-7f58d48f6b-p76dc\" (UID: \"cea2723e-34c7-44b0-a256-9651af2c6638\") " pod="openshift-ingress/e2e-ait-32a87fd8-openshift-default-7f58d48f6b-p76dc"
Sep 19 05:54:55.727810 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:55.727646    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cea2723e-34c7-44b0-a256-9651af2c6638-istiod-ca-cert\") pod \"e2e-ait-32a87fd8-openshift-default-7f58d48f6b-p76dc\" (UID: \"cea2723e-34c7-44b0-a256-9651af2c6638\") " pod="openshift-ingress/e2e-ait-32a87fd8-openshift-default-7f58d48f6b-p76dc"
Sep 19 05:54:55.727810 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:55.727774    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cea2723e-34c7-44b0-a256-9651af2c6638-istio-envoy\") pod \"e2e-ait-32a87fd8-openshift-default-7f58d48f6b-p76dc\" (UID: \"cea2723e-34c7-44b0-a256-9651af2c6638\") " pod="openshift-ingress/e2e-ait-32a87fd8-openshift-default-7f58d48f6b-p76dc"
Sep 19 05:54:55.727810 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:55.727806    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cea2723e-34c7-44b0-a256-9651af2c6638-istio-data\") pod \"e2e-ait-32a87fd8-openshift-default-7f58d48f6b-p76dc\" (UID: \"cea2723e-34c7-44b0-a256-9651af2c6638\") " pod="openshift-ingress/e2e-ait-32a87fd8-openshift-default-7f58d48f6b-p76dc"
Sep 19 05:54:55.727980 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:55.727861    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cea2723e-34c7-44b0-a256-9651af2c6638-workload-socket\") pod \"e2e-ait-32a87fd8-openshift-default-7f58d48f6b-p76dc\" (UID: \"cea2723e-34c7-44b0-a256-9651af2c6638\") " pod="openshift-ingress/e2e-ait-32a87fd8-openshift-default-7f58d48f6b-p76dc"
Sep 19 05:54:55.727980 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:55.727907    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cea2723e-34c7-44b0-a256-9651af2c6638-credential-socket\") pod \"e2e-ait-32a87fd8-openshift-default-7f58d48f6b-p76dc\" (UID: \"cea2723e-34c7-44b0-a256-9651af2c6638\") " pod="openshift-ingress/e2e-ait-32a87fd8-openshift-default-7f58d48f6b-p76dc"
Sep 19 05:54:55.727980 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:55.727923    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h7ftx\" (UniqueName: \"kubernetes.io/projected/cea2723e-34c7-44b0-a256-9651af2c6638-kube-api-access-h7ftx\") pod \"e2e-ait-32a87fd8-openshift-default-7f58d48f6b-p76dc\" (UID: \"cea2723e-34c7-44b0-a256-9651af2c6638\") " pod="openshift-ingress/e2e-ait-32a87fd8-openshift-default-7f58d48f6b-p76dc"
Sep 19 05:54:55.728160 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:55.728002    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cea2723e-34c7-44b0-a256-9651af2c6638-istio-token\") pod \"e2e-ait-32a87fd8-openshift-default-7f58d48f6b-p76dc\" (UID: \"cea2723e-34c7-44b0-a256-9651af2c6638\") " pod="openshift-ingress/e2e-ait-32a87fd8-openshift-default-7f58d48f6b-p76dc"
Sep 19 05:54:55.728322 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:55.728296    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cea2723e-34c7-44b0-a256-9651af2c6638-workload-socket\") pod \"e2e-ait-32a87fd8-openshift-default-7f58d48f6b-p76dc\" (UID: \"cea2723e-34c7-44b0-a256-9651af2c6638\") " pod="openshift-ingress/e2e-ait-32a87fd8-openshift-default-7f58d48f6b-p76dc"
Sep 19 05:54:55.728404 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:55.728356    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cea2723e-34c7-44b0-a256-9651af2c6638-workload-certs\") pod \"e2e-ait-32a87fd8-openshift-default-7f58d48f6b-p76dc\" (UID: \"cea2723e-34c7-44b0-a256-9651af2c6638\") " pod="openshift-ingress/e2e-ait-32a87fd8-openshift-default-7f58d48f6b-p76dc"
Sep 19 05:54:55.728463 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:55.728415    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cea2723e-34c7-44b0-a256-9651af2c6638-istiod-ca-cert\") pod \"e2e-ait-32a87fd8-openshift-default-7f58d48f6b-p76dc\" (UID: \"cea2723e-34c7-44b0-a256-9651af2c6638\") " pod="openshift-ingress/e2e-ait-32a87fd8-openshift-default-7f58d48f6b-p76dc"
Sep 19 05:54:55.728518 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:55.728501    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cea2723e-34c7-44b0-a256-9651af2c6638-istio-data\") pod \"e2e-ait-32a87fd8-openshift-default-7f58d48f6b-p76dc\" (UID: \"cea2723e-34c7-44b0-a256-9651af2c6638\") " pod="openshift-ingress/e2e-ait-32a87fd8-openshift-default-7f58d48f6b-p76dc"
Sep 19 05:54:55.730477 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:55.730445    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cea2723e-34c7-44b0-a256-9651af2c6638-istio-envoy\") pod \"e2e-ait-32a87fd8-openshift-default-7f58d48f6b-p76dc\" (UID: \"cea2723e-34c7-44b0-a256-9651af2c6638\") " pod="openshift-ingress/e2e-ait-32a87fd8-openshift-default-7f58d48f6b-p76dc"
Sep 19 05:54:55.730772 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:55.730752    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cea2723e-34c7-44b0-a256-9651af2c6638-istio-podinfo\") pod \"e2e-ait-32a87fd8-openshift-default-7f58d48f6b-p76dc\" (UID: \"cea2723e-34c7-44b0-a256-9651af2c6638\") " pod="openshift-ingress/e2e-ait-32a87fd8-openshift-default-7f58d48f6b-p76dc"
Sep 19 05:54:55.735778 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:55.735750    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cea2723e-34c7-44b0-a256-9651af2c6638-istio-token\") pod \"e2e-ait-32a87fd8-openshift-default-7f58d48f6b-p76dc\" (UID: \"cea2723e-34c7-44b0-a256-9651af2c6638\") " pod="openshift-ingress/e2e-ait-32a87fd8-openshift-default-7f58d48f6b-p76dc"
Sep 19 05:54:55.736003 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:55.735979    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h7ftx\" (UniqueName: \"kubernetes.io/projected/cea2723e-34c7-44b0-a256-9651af2c6638-kube-api-access-h7ftx\") pod \"e2e-ait-32a87fd8-openshift-default-7f58d48f6b-p76dc\" (UID: \"cea2723e-34c7-44b0-a256-9651af2c6638\") " pod="openshift-ingress/e2e-ait-32a87fd8-openshift-default-7f58d48f6b-p76dc"
Sep 19 05:54:55.764793 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:55.764764    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-32a87fd8-openshift-default-7f58d48f6b-p76dc"
Sep 19 05:54:56.043282 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:56.043256    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-32a87fd8-openshift-default-7f58d48f6b-p76dc"]
Sep 19 05:54:56.045143 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:54:56.045103    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcea2723e_34c7_44b0_a256_9651af2c6638.slice/crio-eaa744b2510c39f622bf695ce0c97a0a930d4ca6ee853ca5c43f6dc176c8078f WatchSource:0}: Error finding container eaa744b2510c39f622bf695ce0c97a0a930d4ca6ee853ca5c43f6dc176c8078f: Status 404 returned error can't find the container with id eaa744b2510c39f622bf695ce0c97a0a930d4ca6ee853ca5c43f6dc176c8078f
Sep 19 05:54:56.047467 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:56.047433    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 19 05:54:56.047591 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:56.047510    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 19 05:54:56.047591 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:56.047554    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 19 05:54:56.074173 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:56.074142    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-32a87fd8-openshift-default-7f58d48f6b-p76dc" event={"ID":"cea2723e-34c7-44b0-a256-9651af2c6638","Type":"ContainerStarted","Data":"eaa744b2510c39f622bf695ce0c97a0a930d4ca6ee853ca5c43f6dc176c8078f"}
Sep 19 05:54:57.082211 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:57.082178    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-32a87fd8-openshift-default-7f58d48f6b-p76dc" event={"ID":"cea2723e-34c7-44b0-a256-9651af2c6638","Type":"ContainerStarted","Data":"e46cab024da67593eb04481bc273b6ac0cfec8e1dfde3aa48a70f83df6c984f3"}
Sep 19 05:54:57.107726 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:57.107670    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-32a87fd8-openshift-default-7f58d48f6b-p76dc" podStartSLOduration=2.107655563 podStartE2EDuration="2.107655563s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:54:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:54:57.104890435 +0000 UTC m=+855.763344997" watchObservedRunningTime="2026-09-19 05:54:57.107655563 +0000 UTC m=+855.766110122"
Sep 19 05:54:57.765855 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:57.765820    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-32a87fd8-openshift-default-7f58d48f6b-p76dc"
Sep 19 05:54:57.771099 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:57.771075    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-32a87fd8-openshift-default-7f58d48f6b-p76dc"
Sep 19 05:54:58.087866 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:58.087830    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-32a87fd8-openshift-default-7f58d48f6b-p76dc"
Sep 19 05:54:58.088695 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:54:58.088673    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-32a87fd8-openshift-default-7f58d48f6b-p76dc"
Sep 19 05:55:22.219206 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:55:22.219174    2583 generic.go:356] "Generic (PLEG): container finished" podID="cea2723e-34c7-44b0-a256-9651af2c6638" containerID="e46cab024da67593eb04481bc273b6ac0cfec8e1dfde3aa48a70f83df6c984f3" exitCode=0
Sep 19 05:55:22.219636 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:55:22.219249    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-32a87fd8-openshift-default-7f58d48f6b-p76dc" event={"ID":"cea2723e-34c7-44b0-a256-9651af2c6638","Type":"ContainerDied","Data":"e46cab024da67593eb04481bc273b6ac0cfec8e1dfde3aa48a70f83df6c984f3"}
Sep 19 05:55:22.219636 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:55:22.219623    2583 scope.go:117] "RemoveContainer" containerID="e46cab024da67593eb04481bc273b6ac0cfec8e1dfde3aa48a70f83df6c984f3"
Sep 19 05:55:22.220734 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:55:22.220710    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 19 05:55:22.220790 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:55:22.220774    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 19 05:55:22.220829 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:55:22.220805    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 19 05:55:23.231083 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:55:23.231037    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-32a87fd8-openshift-default-7f58d48f6b-p76dc" event={"ID":"cea2723e-34c7-44b0-a256-9651af2c6638","Type":"ContainerStarted","Data":"d168044e1f67363a6833e7da77f8965894e9faf9b6112f79ebf32c58cd7352c3"}
Sep 19 05:55:23.765587 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:55:23.765555    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-32a87fd8-openshift-default-7f58d48f6b-p76dc"
Sep 19 05:55:23.766617 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:55:23.766590    2583 patch_prober.go:28] interesting pod/e2e-ait-32a87fd8-openshift-default-7f58d48f6b-p76dc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body=
Sep 19 05:55:23.766761 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:55:23.766644    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-32a87fd8-openshift-default-7f58d48f6b-p76dc" podUID="cea2723e-34c7-44b0-a256-9651af2c6638" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused"
Sep 19 05:55:24.765758 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:55:24.765724    2583 patch_prober.go:28] interesting pod/e2e-ait-32a87fd8-openshift-default-7f58d48f6b-p76dc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body=
Sep 19 05:55:24.766276 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:55:24.765784    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-32a87fd8-openshift-default-7f58d48f6b-p76dc" podUID="cea2723e-34c7-44b0-a256-9651af2c6638" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused"
Sep 19 05:55:25.765336 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:55:25.765254    2583 patch_prober.go:28] interesting pod/e2e-ait-32a87fd8-openshift-default-7f58d48f6b-p76dc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body=
Sep 19 05:55:25.765336 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:55:25.765308    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-32a87fd8-openshift-default-7f58d48f6b-p76dc" podUID="cea2723e-34c7-44b0-a256-9651af2c6638" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused"
Sep 19 05:55:26.765486 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:55:26.765449    2583 patch_prober.go:28] interesting pod/e2e-ait-32a87fd8-openshift-default-7f58d48f6b-p76dc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body=
Sep 19 05:55:26.765955 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:55:26.765510    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-32a87fd8-openshift-default-7f58d48f6b-p76dc" podUID="cea2723e-34c7-44b0-a256-9651af2c6638" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused"
Sep 19 05:55:27.765942 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:55:27.765900    2583 patch_prober.go:28] interesting pod/e2e-ait-32a87fd8-openshift-default-7f58d48f6b-p76dc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body=
Sep 19 05:55:27.766348 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:55:27.765964    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-32a87fd8-openshift-default-7f58d48f6b-p76dc" podUID="cea2723e-34c7-44b0-a256-9651af2c6638" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused"
Sep 19 05:55:28.088372 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:55:28.088332    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-32a87fd8-openshift-default-7f58d48f6b-p76dc"
Sep 19 05:55:28.770019 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:55:28.769991    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-32a87fd8-openshift-default-7f58d48f6b-p76dc"
Sep 19 05:55:28.771165 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:55:28.771117    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-32a87fd8-openshift-default-7f58d48f6b-p76dc"
Sep 19 05:55:42.191302 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:55:42.191268    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-32a87fd8-openshift-default-7f58d48f6b-p76dc"]
Sep 19 05:55:42.191732 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:55:42.191548    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-32a87fd8-openshift-default-7f58d48f6b-p76dc" podUID="cea2723e-34c7-44b0-a256-9651af2c6638" containerName="istio-proxy" containerID="cri-o://d168044e1f67363a6833e7da77f8965894e9faf9b6112f79ebf32c58cd7352c3" gracePeriod=30
Sep 19 05:55:43.772461 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:55:43.772419    2583 patch_prober.go:28] interesting pod/e2e-ait-32a87fd8-openshift-default-7f58d48f6b-p76dc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 19 05:55:43.772855 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:55:43.772489    2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-32a87fd8-openshift-default-7f58d48f6b-p76dc" podUID="cea2723e-34c7-44b0-a256-9651af2c6638" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 19 05:55:47.380443 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:55:47.380400    2583 generic.go:356] "Generic (PLEG): container finished" podID="cea2723e-34c7-44b0-a256-9651af2c6638" containerID="d168044e1f67363a6833e7da77f8965894e9faf9b6112f79ebf32c58cd7352c3" exitCode=0
Sep 19 05:55:47.380870 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:55:47.380525    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-32a87fd8-openshift-default-7f58d48f6b-p76dc" event={"ID":"cea2723e-34c7-44b0-a256-9651af2c6638","Type":"ContainerDied","Data":"d168044e1f67363a6833e7da77f8965894e9faf9b6112f79ebf32c58cd7352c3"}
Sep 19 05:55:47.380870 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:55:47.380597    2583 scope.go:117] "RemoveContainer" containerID="e46cab024da67593eb04481bc273b6ac0cfec8e1dfde3aa48a70f83df6c984f3"
Sep 19 05:55:47.471167 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:55:47.471078    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-32a87fd8-openshift-default-7f58d48f6b-p76dc"
Sep 19 05:55:47.528494 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:55:47.528468    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cea2723e-34c7-44b0-a256-9651af2c6638-istio-data\") pod \"cea2723e-34c7-44b0-a256-9651af2c6638\" (UID: \"cea2723e-34c7-44b0-a256-9651af2c6638\") "
Sep 19 05:55:47.528661 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:55:47.528513    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cea2723e-34c7-44b0-a256-9651af2c6638-istiod-ca-cert\") pod \"cea2723e-34c7-44b0-a256-9651af2c6638\" (UID: \"cea2723e-34c7-44b0-a256-9651af2c6638\") "
Sep 19 05:55:47.528661 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:55:47.528537    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cea2723e-34c7-44b0-a256-9651af2c6638-istio-podinfo\") pod \"cea2723e-34c7-44b0-a256-9651af2c6638\" (UID: \"cea2723e-34c7-44b0-a256-9651af2c6638\") "
Sep 19 05:55:47.528661 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:55:47.528575    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cea2723e-34c7-44b0-a256-9651af2c6638-workload-certs\") pod \"cea2723e-34c7-44b0-a256-9651af2c6638\" (UID: \"cea2723e-34c7-44b0-a256-9651af2c6638\") "
Sep 19 05:55:47.528661 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:55:47.528638    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cea2723e-34c7-44b0-a256-9651af2c6638-istio-token\") pod \"cea2723e-34c7-44b0-a256-9651af2c6638\" (UID: \"cea2723e-34c7-44b0-a256-9651af2c6638\") "
Sep 19 05:55:47.528871 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:55:47.528674    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cea2723e-34c7-44b0-a256-9651af2c6638-workload-socket\") pod \"cea2723e-34c7-44b0-a256-9651af2c6638\" (UID: \"cea2723e-34c7-44b0-a256-9651af2c6638\") "
Sep 19 05:55:47.528871 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:55:47.528711    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cea2723e-34c7-44b0-a256-9651af2c6638-credential-socket\") pod \"cea2723e-34c7-44b0-a256-9651af2c6638\" (UID: \"cea2723e-34c7-44b0-a256-9651af2c6638\") "
Sep 19 05:55:47.528871 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:55:47.528731    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cea2723e-34c7-44b0-a256-9651af2c6638-istio-data" (OuterVolumeSpecName: "istio-data") pod "cea2723e-34c7-44b0-a256-9651af2c6638" (UID: "cea2723e-34c7-44b0-a256-9651af2c6638"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 05:55:47.528871 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:55:47.528836    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cea2723e-34c7-44b0-a256-9651af2c6638-istio-envoy\") pod \"cea2723e-34c7-44b0-a256-9651af2c6638\" (UID: \"cea2723e-34c7-44b0-a256-9651af2c6638\") "
Sep 19 05:55:47.529091 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:55:47.528873    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h7ftx\" (UniqueName: \"kubernetes.io/projected/cea2723e-34c7-44b0-a256-9651af2c6638-kube-api-access-h7ftx\") pod \"cea2723e-34c7-44b0-a256-9651af2c6638\" (UID: \"cea2723e-34c7-44b0-a256-9651af2c6638\") "
Sep 19 05:55:47.529091 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:55:47.528934    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cea2723e-34c7-44b0-a256-9651af2c6638-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "cea2723e-34c7-44b0-a256-9651af2c6638" (UID: "cea2723e-34c7-44b0-a256-9651af2c6638"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 05:55:47.529091 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:55:47.528937    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cea2723e-34c7-44b0-a256-9651af2c6638-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "cea2723e-34c7-44b0-a256-9651af2c6638" (UID: "cea2723e-34c7-44b0-a256-9651af2c6638"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 05:55:47.529091 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:55:47.528955    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cea2723e-34c7-44b0-a256-9651af2c6638-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "cea2723e-34c7-44b0-a256-9651af2c6638" (UID: "cea2723e-34c7-44b0-a256-9651af2c6638"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 05:55:47.529338 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:55:47.529294    2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cea2723e-34c7-44b0-a256-9651af2c6638-istio-data\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 05:55:47.529338 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:55:47.529315    2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cea2723e-34c7-44b0-a256-9651af2c6638-istiod-ca-cert\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 05:55:47.529338 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:55:47.529328    2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cea2723e-34c7-44b0-a256-9651af2c6638-workload-certs\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 05:55:47.529466 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:55:47.529340    2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cea2723e-34c7-44b0-a256-9651af2c6638-workload-socket\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 05:55:47.529466 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:55:47.529335    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cea2723e-34c7-44b0-a256-9651af2c6638-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "cea2723e-34c7-44b0-a256-9651af2c6638" (UID: "cea2723e-34c7-44b0-a256-9651af2c6638"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 05:55:47.531182 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:55:47.531150    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cea2723e-34c7-44b0-a256-9651af2c6638-kube-api-access-h7ftx" (OuterVolumeSpecName: "kube-api-access-h7ftx") pod "cea2723e-34c7-44b0-a256-9651af2c6638" (UID: "cea2723e-34c7-44b0-a256-9651af2c6638"). InnerVolumeSpecName "kube-api-access-h7ftx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 05:55:47.531300 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:55:47.531198    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cea2723e-34c7-44b0-a256-9651af2c6638-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "cea2723e-34c7-44b0-a256-9651af2c6638" (UID: "cea2723e-34c7-44b0-a256-9651af2c6638"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 05:55:47.531300 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:55:47.531283    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/cea2723e-34c7-44b0-a256-9651af2c6638-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "cea2723e-34c7-44b0-a256-9651af2c6638" (UID: "cea2723e-34c7-44b0-a256-9651af2c6638"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 19 05:55:47.531428 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:55:47.531286    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cea2723e-34c7-44b0-a256-9651af2c6638-istio-token" (OuterVolumeSpecName: "istio-token") pod "cea2723e-34c7-44b0-a256-9651af2c6638" (UID: "cea2723e-34c7-44b0-a256-9651af2c6638"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 05:55:47.630224 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:55:47.630192    2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cea2723e-34c7-44b0-a256-9651af2c6638-istio-token\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 05:55:47.630224 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:55:47.630218    2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cea2723e-34c7-44b0-a256-9651af2c6638-credential-socket\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 05:55:47.630224 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:55:47.630228    2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cea2723e-34c7-44b0-a256-9651af2c6638-istio-envoy\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 05:55:47.630449 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:55:47.630236    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h7ftx\" (UniqueName: \"kubernetes.io/projected/cea2723e-34c7-44b0-a256-9651af2c6638-kube-api-access-h7ftx\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 05:55:47.630449 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:55:47.630245    2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cea2723e-34c7-44b0-a256-9651af2c6638-istio-podinfo\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 05:55:48.387806 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:55:48.387779    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-32a87fd8-openshift-default-7f58d48f6b-p76dc"
Sep 19 05:55:48.388245 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:55:48.387827    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-32a87fd8-openshift-default-7f58d48f6b-p76dc" event={"ID":"cea2723e-34c7-44b0-a256-9651af2c6638","Type":"ContainerDied","Data":"eaa744b2510c39f622bf695ce0c97a0a930d4ca6ee853ca5c43f6dc176c8078f"}
Sep 19 05:55:48.388245 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:55:48.387887    2583 scope.go:117] "RemoveContainer" containerID="d168044e1f67363a6833e7da77f8965894e9faf9b6112f79ebf32c58cd7352c3"
Sep 19 05:55:48.416469 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:55:48.416438    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-32a87fd8-openshift-default-7f58d48f6b-p76dc"]
Sep 19 05:55:48.419503 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:55:48.419478    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-32a87fd8-openshift-default-7f58d48f6b-p76dc"]
Sep 19 05:55:48.471665 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:55:48.471630    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-39e34e62-6f4c68898f-ggjhm"]
Sep 19 05:55:48.472224 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:55:48.472205    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cea2723e-34c7-44b0-a256-9651af2c6638" containerName="istio-proxy"
Sep 19 05:55:48.472224 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:55:48.472224    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="cea2723e-34c7-44b0-a256-9651af2c6638" containerName="istio-proxy"
Sep 19 05:55:48.472400 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:55:48.472255    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cea2723e-34c7-44b0-a256-9651af2c6638" containerName="istio-proxy"
Sep 19 05:55:48.472400 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:55:48.472264    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="cea2723e-34c7-44b0-a256-9651af2c6638" containerName="istio-proxy"
Sep 19 05:55:48.472400 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:55:48.472357    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="cea2723e-34c7-44b0-a256-9651af2c6638" containerName="istio-proxy"
Sep 19 05:55:48.475595 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:55:48.475572    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-39e34e62-6f4c68898f-ggjhm"
Sep 19 05:55:48.478011 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:55:48.477992    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-39e34e62-serving-cert\""
Sep 19 05:55:48.487763 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:55:48.487739    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-39e34e62-6f4c68898f-ggjhm"]
Sep 19 05:55:48.539021 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:55:48.538994    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sv5xq\" (UniqueName: \"kubernetes.io/projected/14aa0449-9173-4239-b23a-621bc251c10f-kube-api-access-sv5xq\") pod \"maas-api-e2e-derive-39e34e62-6f4c68898f-ggjhm\" (UID: \"14aa0449-9173-4239-b23a-621bc251c10f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-39e34e62-6f4c68898f-ggjhm"
Sep 19 05:55:48.539212 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:55:48.539029    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/14aa0449-9173-4239-b23a-621bc251c10f-maas-api-tls\") pod \"maas-api-e2e-derive-39e34e62-6f4c68898f-ggjhm\" (UID: \"14aa0449-9173-4239-b23a-621bc251c10f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-39e34e62-6f4c68898f-ggjhm"
Sep 19 05:55:48.539212 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:55:48.539047    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/14aa0449-9173-4239-b23a-621bc251c10f-metrics-certs\") pod \"maas-api-e2e-derive-39e34e62-6f4c68898f-ggjhm\" (UID: \"14aa0449-9173-4239-b23a-621bc251c10f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-39e34e62-6f4c68898f-ggjhm"
Sep 19 05:55:48.539212 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:55:48.539205    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/14aa0449-9173-4239-b23a-621bc251c10f-openshift-service-ca\") pod \"maas-api-e2e-derive-39e34e62-6f4c68898f-ggjhm\" (UID: \"14aa0449-9173-4239-b23a-621bc251c10f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-39e34e62-6f4c68898f-ggjhm"
Sep 19 05:55:48.640670 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:55:48.640584    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/14aa0449-9173-4239-b23a-621bc251c10f-openshift-service-ca\") pod \"maas-api-e2e-derive-39e34e62-6f4c68898f-ggjhm\" (UID: \"14aa0449-9173-4239-b23a-621bc251c10f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-39e34e62-6f4c68898f-ggjhm"
Sep 19 05:55:48.640670 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:55:48.640639    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sv5xq\" (UniqueName: \"kubernetes.io/projected/14aa0449-9173-4239-b23a-621bc251c10f-kube-api-access-sv5xq\") pod \"maas-api-e2e-derive-39e34e62-6f4c68898f-ggjhm\" (UID: \"14aa0449-9173-4239-b23a-621bc251c10f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-39e34e62-6f4c68898f-ggjhm"
Sep 19 05:55:48.640670 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:55:48.640659    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/14aa0449-9173-4239-b23a-621bc251c10f-maas-api-tls\") pod \"maas-api-e2e-derive-39e34e62-6f4c68898f-ggjhm\" (UID: \"14aa0449-9173-4239-b23a-621bc251c10f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-39e34e62-6f4c68898f-ggjhm"
Sep 19 05:55:48.640971 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:55:48.640675    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/14aa0449-9173-4239-b23a-621bc251c10f-metrics-certs\") pod \"maas-api-e2e-derive-39e34e62-6f4c68898f-ggjhm\" (UID: \"14aa0449-9173-4239-b23a-621bc251c10f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-39e34e62-6f4c68898f-ggjhm"
Sep 19 05:55:48.641333 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:55:48.641306    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/14aa0449-9173-4239-b23a-621bc251c10f-openshift-service-ca\") pod \"maas-api-e2e-derive-39e34e62-6f4c68898f-ggjhm\" (UID: \"14aa0449-9173-4239-b23a-621bc251c10f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-39e34e62-6f4c68898f-ggjhm"
Sep 19 05:55:48.643070 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:55:48.643048    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/14aa0449-9173-4239-b23a-621bc251c10f-metrics-certs\") pod \"maas-api-e2e-derive-39e34e62-6f4c68898f-ggjhm\" (UID: \"14aa0449-9173-4239-b23a-621bc251c10f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-39e34e62-6f4c68898f-ggjhm"
Sep 19 05:55:48.643184 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:55:48.643100    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/14aa0449-9173-4239-b23a-621bc251c10f-maas-api-tls\") pod \"maas-api-e2e-derive-39e34e62-6f4c68898f-ggjhm\" (UID: \"14aa0449-9173-4239-b23a-621bc251c10f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-39e34e62-6f4c68898f-ggjhm"
Sep 19 05:55:48.649582 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:55:48.649560    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sv5xq\" (UniqueName: \"kubernetes.io/projected/14aa0449-9173-4239-b23a-621bc251c10f-kube-api-access-sv5xq\") pod \"maas-api-e2e-derive-39e34e62-6f4c68898f-ggjhm\" (UID: \"14aa0449-9173-4239-b23a-621bc251c10f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-39e34e62-6f4c68898f-ggjhm"
Sep 19 05:55:48.789928 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:55:48.789896    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-39e34e62-6f4c68898f-ggjhm"
Sep 19 05:55:49.063501 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:55:49.063472    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-39e34e62-6f4c68898f-ggjhm"]
Sep 19 05:55:49.064798 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:55:49.064774    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14aa0449_9173_4239_b23a_621bc251c10f.slice/crio-bb531bdaecae162c3824c71de69846ba5aeb42a87899b8a2f6de445fcc8baa37 WatchSource:0}: Error finding container bb531bdaecae162c3824c71de69846ba5aeb42a87899b8a2f6de445fcc8baa37: Status 404 returned error can't find the container with id bb531bdaecae162c3824c71de69846ba5aeb42a87899b8a2f6de445fcc8baa37
Sep 19 05:55:49.396231 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:55:49.396108    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-39e34e62-6f4c68898f-ggjhm" event={"ID":"14aa0449-9173-4239-b23a-621bc251c10f","Type":"ContainerStarted","Data":"bb531bdaecae162c3824c71de69846ba5aeb42a87899b8a2f6de445fcc8baa37"}
Sep 19 05:55:49.499108 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:55:49.499072    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cea2723e-34c7-44b0-a256-9651af2c6638" path="/var/lib/kubelet/pods/cea2723e-34c7-44b0-a256-9651af2c6638/volumes"
Sep 19 05:55:50.402747 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:55:50.402714    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-39e34e62-6f4c68898f-ggjhm" event={"ID":"14aa0449-9173-4239-b23a-621bc251c10f","Type":"ContainerStarted","Data":"37e8676eb9fc9335a8e3b023de638c304624c153b1ae14f55ede909e5d030a36"}
Sep 19 05:55:50.403175 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:55:50.402778    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-39e34e62-6f4c68898f-ggjhm"
Sep 19 05:55:50.422413 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:55:50.422363    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-39e34e62-6f4c68898f-ggjhm" podStartSLOduration=1.8655140989999999 podStartE2EDuration="2.422348466s" totalImagesPullingTime="556.834367ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:55:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:55:50.419496148 +0000 UTC m=+909.077950718" watchObservedRunningTime="2026-09-19 05:55:50.422348466 +0000 UTC m=+909.080803024"
Sep 19 05:55:56.413710 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:55:56.413657    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-39e34e62-6f4c68898f-ggjhm"
Sep 19 05:56:14.302192 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:56:14.302157    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-39e34e62-6f4c68898f-ggjhm"]
Sep 19 05:56:14.302715 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:56:14.302363    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-39e34e62-6f4c68898f-ggjhm" podUID="14aa0449-9173-4239-b23a-621bc251c10f" containerName="maas-api" containerID="cri-o://37e8676eb9fc9335a8e3b023de638c304624c153b1ae14f55ede909e5d030a36" gracePeriod=30
Sep 19 05:56:14.541163 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:56:14.541119    2583 generic.go:356] "Generic (PLEG): container finished" podID="14aa0449-9173-4239-b23a-621bc251c10f" containerID="37e8676eb9fc9335a8e3b023de638c304624c153b1ae14f55ede909e5d030a36" exitCode=0
Sep 19 05:56:14.541312 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:56:14.541195    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-39e34e62-6f4c68898f-ggjhm" event={"ID":"14aa0449-9173-4239-b23a-621bc251c10f","Type":"ContainerDied","Data":"37e8676eb9fc9335a8e3b023de638c304624c153b1ae14f55ede909e5d030a36"}
Sep 19 05:56:14.541312 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:56:14.541253    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-39e34e62-6f4c68898f-ggjhm" event={"ID":"14aa0449-9173-4239-b23a-621bc251c10f","Type":"ContainerDied","Data":"bb531bdaecae162c3824c71de69846ba5aeb42a87899b8a2f6de445fcc8baa37"}
Sep 19 05:56:14.541312 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:56:14.541269    2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bb531bdaecae162c3824c71de69846ba5aeb42a87899b8a2f6de445fcc8baa37"
Sep 19 05:56:14.551487 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:56:14.551467    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-39e34e62-6f4c68898f-ggjhm"
Sep 19 05:56:14.692669 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:56:14.692591    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/14aa0449-9173-4239-b23a-621bc251c10f-maas-api-tls\") pod \"14aa0449-9173-4239-b23a-621bc251c10f\" (UID: \"14aa0449-9173-4239-b23a-621bc251c10f\") "
Sep 19 05:56:14.692828 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:56:14.692712    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sv5xq\" (UniqueName: \"kubernetes.io/projected/14aa0449-9173-4239-b23a-621bc251c10f-kube-api-access-sv5xq\") pod \"14aa0449-9173-4239-b23a-621bc251c10f\" (UID: \"14aa0449-9173-4239-b23a-621bc251c10f\") "
Sep 19 05:56:14.692828 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:56:14.692744    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/14aa0449-9173-4239-b23a-621bc251c10f-metrics-certs\") pod \"14aa0449-9173-4239-b23a-621bc251c10f\" (UID: \"14aa0449-9173-4239-b23a-621bc251c10f\") "
Sep 19 05:56:14.693234 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:56:14.693090    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/14aa0449-9173-4239-b23a-621bc251c10f-openshift-service-ca\") pod \"14aa0449-9173-4239-b23a-621bc251c10f\" (UID: \"14aa0449-9173-4239-b23a-621bc251c10f\") "
Sep 19 05:56:14.693858 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:56:14.693805    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/14aa0449-9173-4239-b23a-621bc251c10f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "14aa0449-9173-4239-b23a-621bc251c10f" (UID: "14aa0449-9173-4239-b23a-621bc251c10f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 05:56:14.696426 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:56:14.696397    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/14aa0449-9173-4239-b23a-621bc251c10f-kube-api-access-sv5xq" (OuterVolumeSpecName: "kube-api-access-sv5xq") pod "14aa0449-9173-4239-b23a-621bc251c10f" (UID: "14aa0449-9173-4239-b23a-621bc251c10f"). InnerVolumeSpecName "kube-api-access-sv5xq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 05:56:14.697967 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:56:14.697936    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/14aa0449-9173-4239-b23a-621bc251c10f-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "14aa0449-9173-4239-b23a-621bc251c10f" (UID: "14aa0449-9173-4239-b23a-621bc251c10f"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 05:56:14.698057 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:56:14.698026    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/14aa0449-9173-4239-b23a-621bc251c10f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "14aa0449-9173-4239-b23a-621bc251c10f" (UID: "14aa0449-9173-4239-b23a-621bc251c10f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 05:56:14.794633 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:56:14.794596    2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/14aa0449-9173-4239-b23a-621bc251c10f-openshift-service-ca\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 05:56:14.794633 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:56:14.794627    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/14aa0449-9173-4239-b23a-621bc251c10f-maas-api-tls\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 05:56:14.794633 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:56:14.794638    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sv5xq\" (UniqueName: \"kubernetes.io/projected/14aa0449-9173-4239-b23a-621bc251c10f-kube-api-access-sv5xq\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 05:56:14.794870 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:56:14.794647    2583 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/14aa0449-9173-4239-b23a-621bc251c10f-metrics-certs\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 05:56:15.550533 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:56:15.550486    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-39e34e62-6f4c68898f-ggjhm"
Sep 19 05:56:15.579559 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:56:15.579526    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-39e34e62-6f4c68898f-ggjhm"]
Sep 19 05:56:15.581990 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:56:15.581965    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-39e34e62-6f4c68898f-ggjhm"]
Sep 19 05:56:17.498908 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:56:17.498879    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="14aa0449-9173-4239-b23a-621bc251c10f" path="/var/lib/kubelet/pods/14aa0449-9173-4239-b23a-621bc251c10f/volumes"
Sep 19 05:56:47.530377 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:56:47.530341    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-cc0e26/premium-w0-cc0e26-kserve-64f54b56f5-chxqb"]
Sep 19 05:56:47.530855 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:56:47.530738    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="14aa0449-9173-4239-b23a-621bc251c10f" containerName="maas-api"
Sep 19 05:56:47.530855 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:56:47.530751    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="14aa0449-9173-4239-b23a-621bc251c10f" containerName="maas-api"
Sep 19 05:56:47.530855 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:56:47.530816    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="cea2723e-34c7-44b0-a256-9651af2c6638" containerName="istio-proxy"
Sep 19 05:56:47.530855 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:56:47.530826    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="14aa0449-9173-4239-b23a-621bc251c10f" containerName="maas-api"
Sep 19 05:56:47.533589 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:56:47.533572    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-cc0e26/premium-w0-cc0e26-kserve-64f54b56f5-chxqb"
Sep 19 05:56:47.537377 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:56:47.537340    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-cc0e26\"/\"openshift-service-ca.crt\""
Sep 19 05:56:47.537377 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:56:47.537353    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-cc0e26\"/\"premium-w0-cc0e26-kserve-self-signed-certs\""
Sep 19 05:56:47.537606 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:56:47.537593    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-cc0e26\"/\"kube-root-ca.crt\""
Sep 19 05:56:47.537685 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:56:47.537597    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-cc0e26\"/\"default-dockercfg-mwx7m\""
Sep 19 05:56:47.544395 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:56:47.544374    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-cc0e26/premium-w0-cc0e26-kserve-64f54b56f5-chxqb"]
Sep 19 05:56:47.606049 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:56:47.606019    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/96e17cef-e0e1-4422-aa6e-7eba33cae0c6-home\") pod \"premium-w0-cc0e26-kserve-64f54b56f5-chxqb\" (UID: \"96e17cef-e0e1-4422-aa6e-7eba33cae0c6\") " pod="e2e-models-e2e-worker-w0-cc0e26/premium-w0-cc0e26-kserve-64f54b56f5-chxqb"
Sep 19 05:56:47.606049 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:56:47.606051    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/96e17cef-e0e1-4422-aa6e-7eba33cae0c6-tls-certs\") pod \"premium-w0-cc0e26-kserve-64f54b56f5-chxqb\" (UID: \"96e17cef-e0e1-4422-aa6e-7eba33cae0c6\") " pod="e2e-models-e2e-worker-w0-cc0e26/premium-w0-cc0e26-kserve-64f54b56f5-chxqb"
Sep 19 05:56:47.606269 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:56:47.606073    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/96e17cef-e0e1-4422-aa6e-7eba33cae0c6-dshm\") pod \"premium-w0-cc0e26-kserve-64f54b56f5-chxqb\" (UID: \"96e17cef-e0e1-4422-aa6e-7eba33cae0c6\") " pod="e2e-models-e2e-worker-w0-cc0e26/premium-w0-cc0e26-kserve-64f54b56f5-chxqb"
Sep 19 05:56:47.606269 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:56:47.606146    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/96e17cef-e0e1-4422-aa6e-7eba33cae0c6-tmp-dir\") pod \"premium-w0-cc0e26-kserve-64f54b56f5-chxqb\" (UID: \"96e17cef-e0e1-4422-aa6e-7eba33cae0c6\") " pod="e2e-models-e2e-worker-w0-cc0e26/premium-w0-cc0e26-kserve-64f54b56f5-chxqb"
Sep 19 05:56:47.606269 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:56:47.606181    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/96e17cef-e0e1-4422-aa6e-7eba33cae0c6-model-cache\") pod \"premium-w0-cc0e26-kserve-64f54b56f5-chxqb\" (UID: \"96e17cef-e0e1-4422-aa6e-7eba33cae0c6\") " pod="e2e-models-e2e-worker-w0-cc0e26/premium-w0-cc0e26-kserve-64f54b56f5-chxqb"
Sep 19 05:56:47.606269 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:56:47.606202    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dfrtx\" (UniqueName: \"kubernetes.io/projected/96e17cef-e0e1-4422-aa6e-7eba33cae0c6-kube-api-access-dfrtx\") pod \"premium-w0-cc0e26-kserve-64f54b56f5-chxqb\" (UID: \"96e17cef-e0e1-4422-aa6e-7eba33cae0c6\") " pod="e2e-models-e2e-worker-w0-cc0e26/premium-w0-cc0e26-kserve-64f54b56f5-chxqb"
Sep 19 05:56:47.707815 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:56:47.707778    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/96e17cef-e0e1-4422-aa6e-7eba33cae0c6-home\") pod \"premium-w0-cc0e26-kserve-64f54b56f5-chxqb\" (UID: \"96e17cef-e0e1-4422-aa6e-7eba33cae0c6\") " pod="e2e-models-e2e-worker-w0-cc0e26/premium-w0-cc0e26-kserve-64f54b56f5-chxqb"
Sep 19 05:56:47.707815 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:56:47.707814    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/96e17cef-e0e1-4422-aa6e-7eba33cae0c6-tls-certs\") pod \"premium-w0-cc0e26-kserve-64f54b56f5-chxqb\" (UID: \"96e17cef-e0e1-4422-aa6e-7eba33cae0c6\") " pod="e2e-models-e2e-worker-w0-cc0e26/premium-w0-cc0e26-kserve-64f54b56f5-chxqb"
Sep 19 05:56:47.708048 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:56:47.707832    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/96e17cef-e0e1-4422-aa6e-7eba33cae0c6-dshm\") pod \"premium-w0-cc0e26-kserve-64f54b56f5-chxqb\" (UID: \"96e17cef-e0e1-4422-aa6e-7eba33cae0c6\") " pod="e2e-models-e2e-worker-w0-cc0e26/premium-w0-cc0e26-kserve-64f54b56f5-chxqb"
Sep 19 05:56:47.708048 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:56:47.707896    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/96e17cef-e0e1-4422-aa6e-7eba33cae0c6-tmp-dir\") pod \"premium-w0-cc0e26-kserve-64f54b56f5-chxqb\" (UID: \"96e17cef-e0e1-4422-aa6e-7eba33cae0c6\") " pod="e2e-models-e2e-worker-w0-cc0e26/premium-w0-cc0e26-kserve-64f54b56f5-chxqb"
Sep 19 05:56:47.708048 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:56:47.707930    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/96e17cef-e0e1-4422-aa6e-7eba33cae0c6-model-cache\") pod \"premium-w0-cc0e26-kserve-64f54b56f5-chxqb\" (UID: \"96e17cef-e0e1-4422-aa6e-7eba33cae0c6\") " pod="e2e-models-e2e-worker-w0-cc0e26/premium-w0-cc0e26-kserve-64f54b56f5-chxqb"
Sep 19 05:56:47.708048 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:56:47.707951    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dfrtx\" (UniqueName: \"kubernetes.io/projected/96e17cef-e0e1-4422-aa6e-7eba33cae0c6-kube-api-access-dfrtx\") pod \"premium-w0-cc0e26-kserve-64f54b56f5-chxqb\" (UID: \"96e17cef-e0e1-4422-aa6e-7eba33cae0c6\") " pod="e2e-models-e2e-worker-w0-cc0e26/premium-w0-cc0e26-kserve-64f54b56f5-chxqb"
Sep 19 05:56:47.708368 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:56:47.708349    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/96e17cef-e0e1-4422-aa6e-7eba33cae0c6-tmp-dir\") pod \"premium-w0-cc0e26-kserve-64f54b56f5-chxqb\" (UID: \"96e17cef-e0e1-4422-aa6e-7eba33cae0c6\") " pod="e2e-models-e2e-worker-w0-cc0e26/premium-w0-cc0e26-kserve-64f54b56f5-chxqb"
Sep 19 05:56:47.708437 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:56:47.708416    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/96e17cef-e0e1-4422-aa6e-7eba33cae0c6-home\") pod \"premium-w0-cc0e26-kserve-64f54b56f5-chxqb\" (UID: \"96e17cef-e0e1-4422-aa6e-7eba33cae0c6\") " pod="e2e-models-e2e-worker-w0-cc0e26/premium-w0-cc0e26-kserve-64f54b56f5-chxqb"
Sep 19 05:56:47.708545 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:56:47.708523    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/96e17cef-e0e1-4422-aa6e-7eba33cae0c6-model-cache\") pod \"premium-w0-cc0e26-kserve-64f54b56f5-chxqb\" (UID: \"96e17cef-e0e1-4422-aa6e-7eba33cae0c6\") " pod="e2e-models-e2e-worker-w0-cc0e26/premium-w0-cc0e26-kserve-64f54b56f5-chxqb"
Sep 19 05:56:47.710041 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:56:47.710011    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/96e17cef-e0e1-4422-aa6e-7eba33cae0c6-dshm\") pod \"premium-w0-cc0e26-kserve-64f54b56f5-chxqb\" (UID: \"96e17cef-e0e1-4422-aa6e-7eba33cae0c6\") " pod="e2e-models-e2e-worker-w0-cc0e26/premium-w0-cc0e26-kserve-64f54b56f5-chxqb"
Sep 19 05:56:47.710550 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:56:47.710529    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/96e17cef-e0e1-4422-aa6e-7eba33cae0c6-tls-certs\") pod \"premium-w0-cc0e26-kserve-64f54b56f5-chxqb\" (UID: \"96e17cef-e0e1-4422-aa6e-7eba33cae0c6\") " pod="e2e-models-e2e-worker-w0-cc0e26/premium-w0-cc0e26-kserve-64f54b56f5-chxqb"
Sep 19 05:56:47.716324 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:56:47.716301    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dfrtx\" (UniqueName: \"kubernetes.io/projected/96e17cef-e0e1-4422-aa6e-7eba33cae0c6-kube-api-access-dfrtx\") pod \"premium-w0-cc0e26-kserve-64f54b56f5-chxqb\" (UID: \"96e17cef-e0e1-4422-aa6e-7eba33cae0c6\") " pod="e2e-models-e2e-worker-w0-cc0e26/premium-w0-cc0e26-kserve-64f54b56f5-chxqb"
Sep 19 05:56:47.842934 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:56:47.842853    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-cc0e26/premium-w0-cc0e26-kserve-64f54b56f5-chxqb"
Sep 19 05:56:48.113841 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:56:48.113814    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-cc0e26/premium-w0-cc0e26-kserve-64f54b56f5-chxqb"]
Sep 19 05:56:48.114787 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:56:48.114766    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod96e17cef_e0e1_4422_aa6e_7eba33cae0c6.slice/crio-8aa8692b173ea22318014c4228915bb299aa4907e3015c134eaa4dd91dc3b755 WatchSource:0}: Error finding container 8aa8692b173ea22318014c4228915bb299aa4907e3015c134eaa4dd91dc3b755: Status 404 returned error can't find the container with id 8aa8692b173ea22318014c4228915bb299aa4907e3015c134eaa4dd91dc3b755
Sep 19 05:56:48.755865 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:56:48.755828    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-cc0e26/premium-w0-cc0e26-kserve-64f54b56f5-chxqb" event={"ID":"96e17cef-e0e1-4422-aa6e-7eba33cae0c6","Type":"ContainerStarted","Data":"3d4cf28f7e05f95cbba26b2859c31f3bd2924e1a247d94d1a3963a9f136c565c"}
Sep 19 05:56:48.756328 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:56:48.755874    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-cc0e26/premium-w0-cc0e26-kserve-64f54b56f5-chxqb" event={"ID":"96e17cef-e0e1-4422-aa6e-7eba33cae0c6","Type":"ContainerStarted","Data":"8aa8692b173ea22318014c4228915bb299aa4907e3015c134eaa4dd91dc3b755"}
Sep 19 05:56:48.775211 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:56:48.775162    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-cc0e26/premium-w0-cc0e26-kserve-64f54b56f5-chxqb" podStartSLOduration=1.7751214640000001 podStartE2EDuration="1.775121464s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:56:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:56:48.773451598 +0000 UTC m=+967.431906158" watchObservedRunningTime="2026-09-19 05:56:48.775121464 +0000 UTC m=+967.433576022"
Sep 19 05:56:57.843930 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:56:57.843847    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-cc0e26/premium-w0-cc0e26-kserve-64f54b56f5-chxqb"
Sep 19 05:56:57.843930 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:56:57.843910    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-cc0e26/premium-w0-cc0e26-kserve-64f54b56f5-chxqb"
Sep 19 05:56:57.856935 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:56:57.856911    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-cc0e26/premium-w0-cc0e26-kserve-64f54b56f5-chxqb"
Sep 19 05:56:58.823412 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:56:58.823382    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-cc0e26/premium-w0-cc0e26-kserve-64f54b56f5-chxqb"
Sep 19 05:57:25.304728 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:57:25.304691    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-explicit-875220/explicit-model-875220-kserve-698fd9bfd5-txppj"]
Sep 19 05:57:25.307173 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:57:25.307155    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-875220/explicit-model-875220-kserve-698fd9bfd5-txppj"
Sep 19 05:57:25.312760 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:57:25.312730    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-explicit-875220\"/\"kube-root-ca.crt\""
Sep 19 05:57:25.313915 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:57:25.313892    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-explicit-875220\"/\"default-dockercfg-c9p9g\""
Sep 19 05:57:25.314028 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:57:25.313952    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-explicit-875220\"/\"openshift-service-ca.crt\""
Sep 19 05:57:25.314028 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:57:25.313952    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-explicit-875220\"/\"explicit-model-875220-kserve-self-signed-certs\""
Sep 19 05:57:25.326326 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:57:25.326304    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-explicit-875220/explicit-model-875220-kserve-698fd9bfd5-txppj"]
Sep 19 05:57:25.468526 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:57:25.468483    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/637b66a5-e2e0-4883-9e17-64125856c5d4-tmp-dir\") pod \"explicit-model-875220-kserve-698fd9bfd5-txppj\" (UID: \"637b66a5-e2e0-4883-9e17-64125856c5d4\") " pod="ai-tenant-e2e-explicit-875220/explicit-model-875220-kserve-698fd9bfd5-txppj"
Sep 19 05:57:25.468526 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:57:25.468526    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-twlxq\" (UniqueName: \"kubernetes.io/projected/637b66a5-e2e0-4883-9e17-64125856c5d4-kube-api-access-twlxq\") pod \"explicit-model-875220-kserve-698fd9bfd5-txppj\" (UID: \"637b66a5-e2e0-4883-9e17-64125856c5d4\") " pod="ai-tenant-e2e-explicit-875220/explicit-model-875220-kserve-698fd9bfd5-txppj"
Sep 19 05:57:25.468793 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:57:25.468548    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/637b66a5-e2e0-4883-9e17-64125856c5d4-model-cache\") pod \"explicit-model-875220-kserve-698fd9bfd5-txppj\" (UID: \"637b66a5-e2e0-4883-9e17-64125856c5d4\") " pod="ai-tenant-e2e-explicit-875220/explicit-model-875220-kserve-698fd9bfd5-txppj"
Sep 19 05:57:25.468793 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:57:25.468668    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/637b66a5-e2e0-4883-9e17-64125856c5d4-dshm\") pod \"explicit-model-875220-kserve-698fd9bfd5-txppj\" (UID: \"637b66a5-e2e0-4883-9e17-64125856c5d4\") " pod="ai-tenant-e2e-explicit-875220/explicit-model-875220-kserve-698fd9bfd5-txppj"
Sep 19 05:57:25.468793 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:57:25.468711    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/637b66a5-e2e0-4883-9e17-64125856c5d4-tls-certs\") pod \"explicit-model-875220-kserve-698fd9bfd5-txppj\" (UID: \"637b66a5-e2e0-4883-9e17-64125856c5d4\") " pod="ai-tenant-e2e-explicit-875220/explicit-model-875220-kserve-698fd9bfd5-txppj"
Sep 19 05:57:25.468793 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:57:25.468762    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/637b66a5-e2e0-4883-9e17-64125856c5d4-home\") pod \"explicit-model-875220-kserve-698fd9bfd5-txppj\" (UID: \"637b66a5-e2e0-4883-9e17-64125856c5d4\") " pod="ai-tenant-e2e-explicit-875220/explicit-model-875220-kserve-698fd9bfd5-txppj"
Sep 19 05:57:25.569584 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:57:25.569503    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/637b66a5-e2e0-4883-9e17-64125856c5d4-tmp-dir\") pod \"explicit-model-875220-kserve-698fd9bfd5-txppj\" (UID: \"637b66a5-e2e0-4883-9e17-64125856c5d4\") " pod="ai-tenant-e2e-explicit-875220/explicit-model-875220-kserve-698fd9bfd5-txppj"
Sep 19 05:57:25.569584 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:57:25.569539    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-twlxq\" (UniqueName: \"kubernetes.io/projected/637b66a5-e2e0-4883-9e17-64125856c5d4-kube-api-access-twlxq\") pod \"explicit-model-875220-kserve-698fd9bfd5-txppj\" (UID: \"637b66a5-e2e0-4883-9e17-64125856c5d4\") " pod="ai-tenant-e2e-explicit-875220/explicit-model-875220-kserve-698fd9bfd5-txppj"
Sep 19 05:57:25.569584 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:57:25.569561    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/637b66a5-e2e0-4883-9e17-64125856c5d4-model-cache\") pod \"explicit-model-875220-kserve-698fd9bfd5-txppj\" (UID: \"637b66a5-e2e0-4883-9e17-64125856c5d4\") " pod="ai-tenant-e2e-explicit-875220/explicit-model-875220-kserve-698fd9bfd5-txppj"
Sep 19 05:57:25.569806 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:57:25.569690    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/637b66a5-e2e0-4883-9e17-64125856c5d4-dshm\") pod \"explicit-model-875220-kserve-698fd9bfd5-txppj\" (UID: \"637b66a5-e2e0-4883-9e17-64125856c5d4\") " pod="ai-tenant-e2e-explicit-875220/explicit-model-875220-kserve-698fd9bfd5-txppj"
Sep 19 05:57:25.569806 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:57:25.569733    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/637b66a5-e2e0-4883-9e17-64125856c5d4-tls-certs\") pod \"explicit-model-875220-kserve-698fd9bfd5-txppj\" (UID: \"637b66a5-e2e0-4883-9e17-64125856c5d4\") " pod="ai-tenant-e2e-explicit-875220/explicit-model-875220-kserve-698fd9bfd5-txppj"
Sep 19 05:57:25.569806 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:57:25.569768    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/637b66a5-e2e0-4883-9e17-64125856c5d4-home\") pod \"explicit-model-875220-kserve-698fd9bfd5-txppj\" (UID: \"637b66a5-e2e0-4883-9e17-64125856c5d4\") " pod="ai-tenant-e2e-explicit-875220/explicit-model-875220-kserve-698fd9bfd5-txppj"
Sep 19 05:57:25.569971 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:57:25.569938    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/637b66a5-e2e0-4883-9e17-64125856c5d4-tmp-dir\") pod \"explicit-model-875220-kserve-698fd9bfd5-txppj\" (UID: \"637b66a5-e2e0-4883-9e17-64125856c5d4\") " pod="ai-tenant-e2e-explicit-875220/explicit-model-875220-kserve-698fd9bfd5-txppj"
Sep 19 05:57:25.569971 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:57:25.569950    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/637b66a5-e2e0-4883-9e17-64125856c5d4-model-cache\") pod \"explicit-model-875220-kserve-698fd9bfd5-txppj\" (UID: \"637b66a5-e2e0-4883-9e17-64125856c5d4\") " pod="ai-tenant-e2e-explicit-875220/explicit-model-875220-kserve-698fd9bfd5-txppj"
Sep 19 05:57:25.570112 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:57:25.570092    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/637b66a5-e2e0-4883-9e17-64125856c5d4-home\") pod \"explicit-model-875220-kserve-698fd9bfd5-txppj\" (UID: \"637b66a5-e2e0-4883-9e17-64125856c5d4\") " pod="ai-tenant-e2e-explicit-875220/explicit-model-875220-kserve-698fd9bfd5-txppj"
Sep 19 05:57:25.571852 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:57:25.571831    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/637b66a5-e2e0-4883-9e17-64125856c5d4-dshm\") pod \"explicit-model-875220-kserve-698fd9bfd5-txppj\" (UID: \"637b66a5-e2e0-4883-9e17-64125856c5d4\") " pod="ai-tenant-e2e-explicit-875220/explicit-model-875220-kserve-698fd9bfd5-txppj"
Sep 19 05:57:25.572282 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:57:25.572263    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/637b66a5-e2e0-4883-9e17-64125856c5d4-tls-certs\") pod \"explicit-model-875220-kserve-698fd9bfd5-txppj\" (UID: \"637b66a5-e2e0-4883-9e17-64125856c5d4\") " pod="ai-tenant-e2e-explicit-875220/explicit-model-875220-kserve-698fd9bfd5-txppj"
Sep 19 05:57:25.577955 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:57:25.577932    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-twlxq\" (UniqueName: \"kubernetes.io/projected/637b66a5-e2e0-4883-9e17-64125856c5d4-kube-api-access-twlxq\") pod \"explicit-model-875220-kserve-698fd9bfd5-txppj\" (UID: \"637b66a5-e2e0-4883-9e17-64125856c5d4\") " pod="ai-tenant-e2e-explicit-875220/explicit-model-875220-kserve-698fd9bfd5-txppj"
Sep 19 05:57:25.617013 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:57:25.616989    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-875220/explicit-model-875220-kserve-698fd9bfd5-txppj"
Sep 19 05:57:25.908894 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:57:25.908793    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-explicit-875220/explicit-model-875220-kserve-698fd9bfd5-txppj"]
Sep 19 05:57:25.913973 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:57:25.913943    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod637b66a5_e2e0_4883_9e17_64125856c5d4.slice/crio-bafa61fbc643ab9d1fc949a2070b8c7ba754623b1c24bcfca88d4e8c91f6ee34 WatchSource:0}: Error finding container bafa61fbc643ab9d1fc949a2070b8c7ba754623b1c24bcfca88d4e8c91f6ee34: Status 404 returned error can't find the container with id bafa61fbc643ab9d1fc949a2070b8c7ba754623b1c24bcfca88d4e8c91f6ee34
Sep 19 05:57:25.961187 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:57:25.961155    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-875220/explicit-model-875220-kserve-698fd9bfd5-txppj" event={"ID":"637b66a5-e2e0-4883-9e17-64125856c5d4","Type":"ContainerStarted","Data":"bafa61fbc643ab9d1fc949a2070b8c7ba754623b1c24bcfca88d4e8c91f6ee34"}
Sep 19 05:57:26.968259 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:57:26.968215    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-875220/explicit-model-875220-kserve-698fd9bfd5-txppj" event={"ID":"637b66a5-e2e0-4883-9e17-64125856c5d4","Type":"ContainerStarted","Data":"0f80059f9a94f4df985988b4452c77adb2efcff3f4f797d1f4bc8603564cc509"}
Sep 19 05:57:26.985749 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:57:26.985699    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-explicit-875220/explicit-model-875220-kserve-698fd9bfd5-txppj" podStartSLOduration=1.9856831929999998 podStartE2EDuration="1.985683193s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:57:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:57:26.984957522 +0000 UTC m=+1005.643412094" watchObservedRunningTime="2026-09-19 05:57:26.985683193 +0000 UTC m=+1005.644137751"
Sep 19 05:57:35.617566 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:57:35.617513    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-explicit-875220/explicit-model-875220-kserve-698fd9bfd5-txppj"
Sep 19 05:57:35.617566 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:57:35.617565    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-explicit-875220/explicit-model-875220-kserve-698fd9bfd5-txppj"
Sep 19 05:57:35.638297 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:57:35.638257    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-explicit-875220/explicit-model-875220-kserve-698fd9bfd5-txppj"
Sep 19 05:57:36.029557 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:57:36.029527    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-explicit-875220/explicit-model-875220-kserve-698fd9bfd5-txppj"
Sep 19 05:57:43.608253 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:57:43.608220    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-explicit-875220/explicit-model-875220-kserve-698fd9bfd5-txppj"]
Sep 19 05:57:58.627541 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:57:58.627494    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-explicit-875220/explicit-model-875220-kserve-698fd9bfd5-txppj" podUID="637b66a5-e2e0-4883-9e17-64125856c5d4" containerName="main" containerID="cri-o://0f80059f9a94f4df985988b4452c77adb2efcff3f4f797d1f4bc8603564cc509" gracePeriod=45
Sep 19 05:57:58.878399 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:57:58.878332    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-875220/explicit-model-875220-kserve-698fd9bfd5-txppj"
Sep 19 05:57:58.991184 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:57:58.991150    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/637b66a5-e2e0-4883-9e17-64125856c5d4-dshm\") pod \"637b66a5-e2e0-4883-9e17-64125856c5d4\" (UID: \"637b66a5-e2e0-4883-9e17-64125856c5d4\") "
Sep 19 05:57:58.991350 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:57:58.991293    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-twlxq\" (UniqueName: \"kubernetes.io/projected/637b66a5-e2e0-4883-9e17-64125856c5d4-kube-api-access-twlxq\") pod \"637b66a5-e2e0-4883-9e17-64125856c5d4\" (UID: \"637b66a5-e2e0-4883-9e17-64125856c5d4\") "
Sep 19 05:57:58.991404 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:57:58.991360    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/637b66a5-e2e0-4883-9e17-64125856c5d4-tmp-dir\") pod \"637b66a5-e2e0-4883-9e17-64125856c5d4\" (UID: \"637b66a5-e2e0-4883-9e17-64125856c5d4\") "
Sep 19 05:57:58.991538 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:57:58.991515    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/637b66a5-e2e0-4883-9e17-64125856c5d4-model-cache\") pod \"637b66a5-e2e0-4883-9e17-64125856c5d4\" (UID: \"637b66a5-e2e0-4883-9e17-64125856c5d4\") "
Sep 19 05:57:58.991603 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:57:58.991574    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/637b66a5-e2e0-4883-9e17-64125856c5d4-tls-certs\") pod \"637b66a5-e2e0-4883-9e17-64125856c5d4\" (UID: \"637b66a5-e2e0-4883-9e17-64125856c5d4\") "
Sep 19 05:57:58.991603 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:57:58.991578    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/637b66a5-e2e0-4883-9e17-64125856c5d4-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "637b66a5-e2e0-4883-9e17-64125856c5d4" (UID: "637b66a5-e2e0-4883-9e17-64125856c5d4"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 05:57:58.991688 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:57:58.991607    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/637b66a5-e2e0-4883-9e17-64125856c5d4-home\") pod \"637b66a5-e2e0-4883-9e17-64125856c5d4\" (UID: \"637b66a5-e2e0-4883-9e17-64125856c5d4\") "
Sep 19 05:57:58.991853 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:57:58.991806    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/637b66a5-e2e0-4883-9e17-64125856c5d4-model-cache" (OuterVolumeSpecName: "model-cache") pod "637b66a5-e2e0-4883-9e17-64125856c5d4" (UID: "637b66a5-e2e0-4883-9e17-64125856c5d4"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 05:57:58.991917 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:57:58.991877    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/637b66a5-e2e0-4883-9e17-64125856c5d4-home" (OuterVolumeSpecName: "home") pod "637b66a5-e2e0-4883-9e17-64125856c5d4" (UID: "637b66a5-e2e0-4883-9e17-64125856c5d4"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 05:57:58.992049 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:57:58.992030    2583 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/637b66a5-e2e0-4883-9e17-64125856c5d4-tmp-dir\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 05:57:58.992096 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:57:58.992054    2583 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/637b66a5-e2e0-4883-9e17-64125856c5d4-model-cache\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 05:57:58.992096 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:57:58.992068    2583 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/637b66a5-e2e0-4883-9e17-64125856c5d4-home\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 05:57:58.993604 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:57:58.993579    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/637b66a5-e2e0-4883-9e17-64125856c5d4-dshm" (OuterVolumeSpecName: "dshm") pod "637b66a5-e2e0-4883-9e17-64125856c5d4" (UID: "637b66a5-e2e0-4883-9e17-64125856c5d4"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 05:57:58.993709 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:57:58.993615    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/637b66a5-e2e0-4883-9e17-64125856c5d4-kube-api-access-twlxq" (OuterVolumeSpecName: "kube-api-access-twlxq") pod "637b66a5-e2e0-4883-9e17-64125856c5d4" (UID: "637b66a5-e2e0-4883-9e17-64125856c5d4"). InnerVolumeSpecName "kube-api-access-twlxq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 05:57:58.993709 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:57:58.993638    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/637b66a5-e2e0-4883-9e17-64125856c5d4-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "637b66a5-e2e0-4883-9e17-64125856c5d4" (UID: "637b66a5-e2e0-4883-9e17-64125856c5d4"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 05:57:59.092673 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:57:59.092635    2583 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/637b66a5-e2e0-4883-9e17-64125856c5d4-tls-certs\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 05:57:59.092673 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:57:59.092667    2583 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/637b66a5-e2e0-4883-9e17-64125856c5d4-dshm\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 05:57:59.092673 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:57:59.092676    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-twlxq\" (UniqueName: \"kubernetes.io/projected/637b66a5-e2e0-4883-9e17-64125856c5d4-kube-api-access-twlxq\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 05:57:59.161620 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:57:59.161512    2583 generic.go:356] "Generic (PLEG): container finished" podID="637b66a5-e2e0-4883-9e17-64125856c5d4" containerID="0f80059f9a94f4df985988b4452c77adb2efcff3f4f797d1f4bc8603564cc509" exitCode=0
Sep 19 05:57:59.161620 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:57:59.161577    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-875220/explicit-model-875220-kserve-698fd9bfd5-txppj"
Sep 19 05:57:59.161620 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:57:59.161583    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-875220/explicit-model-875220-kserve-698fd9bfd5-txppj" event={"ID":"637b66a5-e2e0-4883-9e17-64125856c5d4","Type":"ContainerDied","Data":"0f80059f9a94f4df985988b4452c77adb2efcff3f4f797d1f4bc8603564cc509"}
Sep 19 05:57:59.161620 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:57:59.161618    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-875220/explicit-model-875220-kserve-698fd9bfd5-txppj" event={"ID":"637b66a5-e2e0-4883-9e17-64125856c5d4","Type":"ContainerDied","Data":"bafa61fbc643ab9d1fc949a2070b8c7ba754623b1c24bcfca88d4e8c91f6ee34"}
Sep 19 05:57:59.161936 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:57:59.161645    2583 scope.go:117] "RemoveContainer" containerID="0f80059f9a94f4df985988b4452c77adb2efcff3f4f797d1f4bc8603564cc509"
Sep 19 05:57:59.171988 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:57:59.171963    2583 scope.go:117] "RemoveContainer" containerID="0f80059f9a94f4df985988b4452c77adb2efcff3f4f797d1f4bc8603564cc509"
Sep 19 05:57:59.172252 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:57:59.172233    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0f80059f9a94f4df985988b4452c77adb2efcff3f4f797d1f4bc8603564cc509\": container with ID starting with 0f80059f9a94f4df985988b4452c77adb2efcff3f4f797d1f4bc8603564cc509 not found: ID does not exist" containerID="0f80059f9a94f4df985988b4452c77adb2efcff3f4f797d1f4bc8603564cc509"
Sep 19 05:57:59.172310 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:57:59.172259    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0f80059f9a94f4df985988b4452c77adb2efcff3f4f797d1f4bc8603564cc509"} err="failed to get container status \"0f80059f9a94f4df985988b4452c77adb2efcff3f4f797d1f4bc8603564cc509\": rpc error: code = NotFound desc = could not find container \"0f80059f9a94f4df985988b4452c77adb2efcff3f4f797d1f4bc8603564cc509\": container with ID starting with 0f80059f9a94f4df985988b4452c77adb2efcff3f4f797d1f4bc8603564cc509 not found: ID does not exist"
Sep 19 05:57:59.189282 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:57:59.189258    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-explicit-875220/explicit-model-875220-kserve-698fd9bfd5-txppj"]
Sep 19 05:57:59.192526 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:57:59.192502    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-explicit-875220/explicit-model-875220-kserve-698fd9bfd5-txppj"]
Sep 19 05:57:59.498845 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:57:59.498768    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="637b66a5-e2e0-4883-9e17-64125856c5d4" path="/var/lib/kubelet/pods/637b66a5-e2e0-4883-9e17-64125856c5d4/volumes"
Sep 19 05:58:01.988879 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:58:01.988847    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-aec2317b-5cf4ffcc9b-75b2d"]
Sep 19 05:58:01.989387 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:58:01.989252    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="637b66a5-e2e0-4883-9e17-64125856c5d4" containerName="main"
Sep 19 05:58:01.989387 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:58:01.989265    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="637b66a5-e2e0-4883-9e17-64125856c5d4" containerName="main"
Sep 19 05:58:01.989387 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:58:01.989343    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="637b66a5-e2e0-4883-9e17-64125856c5d4" containerName="main"
Sep 19 05:58:01.991479 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:58:01.991459    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-aec2317b-5cf4ffcc9b-75b2d"
Sep 19 05:58:01.994569 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:58:01.994546    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-aec2317b-serving-cert\""
Sep 19 05:58:02.003165 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:58:02.003123    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-aec2317b-5cf4ffcc9b-75b2d"]
Sep 19 05:58:02.122184 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:58:02.122117    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/46db5bf6-fc21-4183-ad3a-f30dbb6e91d4-maas-api-tls\") pod \"maas-api-e2e-mt-aec2317b-5cf4ffcc9b-75b2d\" (UID: \"46db5bf6-fc21-4183-ad3a-f30dbb6e91d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-aec2317b-5cf4ffcc9b-75b2d"
Sep 19 05:58:02.122378 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:58:02.122266    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z9fwm\" (UniqueName: \"kubernetes.io/projected/46db5bf6-fc21-4183-ad3a-f30dbb6e91d4-kube-api-access-z9fwm\") pod \"maas-api-e2e-mt-aec2317b-5cf4ffcc9b-75b2d\" (UID: \"46db5bf6-fc21-4183-ad3a-f30dbb6e91d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-aec2317b-5cf4ffcc9b-75b2d"
Sep 19 05:58:02.122450 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:58:02.122418    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/46db5bf6-fc21-4183-ad3a-f30dbb6e91d4-metrics-certs\") pod \"maas-api-e2e-mt-aec2317b-5cf4ffcc9b-75b2d\" (UID: \"46db5bf6-fc21-4183-ad3a-f30dbb6e91d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-aec2317b-5cf4ffcc9b-75b2d"
Sep 19 05:58:02.122540 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:58:02.122523    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/46db5bf6-fc21-4183-ad3a-f30dbb6e91d4-openshift-service-ca\") pod \"maas-api-e2e-mt-aec2317b-5cf4ffcc9b-75b2d\" (UID: \"46db5bf6-fc21-4183-ad3a-f30dbb6e91d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-aec2317b-5cf4ffcc9b-75b2d"
Sep 19 05:58:02.223017 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:58:02.222980    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/46db5bf6-fc21-4183-ad3a-f30dbb6e91d4-openshift-service-ca\") pod \"maas-api-e2e-mt-aec2317b-5cf4ffcc9b-75b2d\" (UID: \"46db5bf6-fc21-4183-ad3a-f30dbb6e91d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-aec2317b-5cf4ffcc9b-75b2d"
Sep 19 05:58:02.223178 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:58:02.223022    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/46db5bf6-fc21-4183-ad3a-f30dbb6e91d4-maas-api-tls\") pod \"maas-api-e2e-mt-aec2317b-5cf4ffcc9b-75b2d\" (UID: \"46db5bf6-fc21-4183-ad3a-f30dbb6e91d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-aec2317b-5cf4ffcc9b-75b2d"
Sep 19 05:58:02.223178 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:58:02.223065    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z9fwm\" (UniqueName: \"kubernetes.io/projected/46db5bf6-fc21-4183-ad3a-f30dbb6e91d4-kube-api-access-z9fwm\") pod \"maas-api-e2e-mt-aec2317b-5cf4ffcc9b-75b2d\" (UID: \"46db5bf6-fc21-4183-ad3a-f30dbb6e91d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-aec2317b-5cf4ffcc9b-75b2d"
Sep 19 05:58:02.223178 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:58:02.223172    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/46db5bf6-fc21-4183-ad3a-f30dbb6e91d4-metrics-certs\") pod \"maas-api-e2e-mt-aec2317b-5cf4ffcc9b-75b2d\" (UID: \"46db5bf6-fc21-4183-ad3a-f30dbb6e91d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-aec2317b-5cf4ffcc9b-75b2d"
Sep 19 05:58:02.223811 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:58:02.223785    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/46db5bf6-fc21-4183-ad3a-f30dbb6e91d4-openshift-service-ca\") pod \"maas-api-e2e-mt-aec2317b-5cf4ffcc9b-75b2d\" (UID: \"46db5bf6-fc21-4183-ad3a-f30dbb6e91d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-aec2317b-5cf4ffcc9b-75b2d"
Sep 19 05:58:02.225742 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:58:02.225714    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/46db5bf6-fc21-4183-ad3a-f30dbb6e91d4-metrics-certs\") pod \"maas-api-e2e-mt-aec2317b-5cf4ffcc9b-75b2d\" (UID: \"46db5bf6-fc21-4183-ad3a-f30dbb6e91d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-aec2317b-5cf4ffcc9b-75b2d"
Sep 19 05:58:02.225864 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:58:02.225745    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/46db5bf6-fc21-4183-ad3a-f30dbb6e91d4-maas-api-tls\") pod \"maas-api-e2e-mt-aec2317b-5cf4ffcc9b-75b2d\" (UID: \"46db5bf6-fc21-4183-ad3a-f30dbb6e91d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-aec2317b-5cf4ffcc9b-75b2d"
Sep 19 05:58:02.235025 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:58:02.235000    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z9fwm\" (UniqueName: \"kubernetes.io/projected/46db5bf6-fc21-4183-ad3a-f30dbb6e91d4-kube-api-access-z9fwm\") pod \"maas-api-e2e-mt-aec2317b-5cf4ffcc9b-75b2d\" (UID: \"46db5bf6-fc21-4183-ad3a-f30dbb6e91d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-aec2317b-5cf4ffcc9b-75b2d"
Sep 19 05:58:02.304988 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:58:02.304953    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-aec2317b-5cf4ffcc9b-75b2d"
Sep 19 05:58:02.589622 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:58:02.589596    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-aec2317b-5cf4ffcc9b-75b2d"]
Sep 19 05:58:02.591865 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:58:02.591831    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46db5bf6_fc21_4183_ad3a_f30dbb6e91d4.slice/crio-1dcb97bdb4eddaf5a8b433208118d3142d86138b6779a0f532f2f989895bced3 WatchSource:0}: Error finding container 1dcb97bdb4eddaf5a8b433208118d3142d86138b6779a0f532f2f989895bced3: Status 404 returned error can't find the container with id 1dcb97bdb4eddaf5a8b433208118d3142d86138b6779a0f532f2f989895bced3
Sep 19 05:58:03.186261 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:58:03.186229    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-aec2317b-5cf4ffcc9b-75b2d" event={"ID":"46db5bf6-fc21-4183-ad3a-f30dbb6e91d4","Type":"ContainerStarted","Data":"da047b53279a345f0fb8a2dec16096c8a9013ac2717099745bb67c29be6072c4"}
Sep 19 05:58:03.186658 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:58:03.186279    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-aec2317b-5cf4ffcc9b-75b2d" event={"ID":"46db5bf6-fc21-4183-ad3a-f30dbb6e91d4","Type":"ContainerStarted","Data":"1dcb97bdb4eddaf5a8b433208118d3142d86138b6779a0f532f2f989895bced3"}
Sep 19 05:58:03.186658 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:58:03.186353    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-aec2317b-5cf4ffcc9b-75b2d"
Sep 19 05:58:03.213920 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:58:03.213865    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-aec2317b-5cf4ffcc9b-75b2d" podStartSLOduration=1.75144291 podStartE2EDuration="2.21384846s" totalImagesPullingTime="462.40555ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:58:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:58:03.212724486 +0000 UTC m=+1041.871179056" watchObservedRunningTime="2026-09-19 05:58:03.21384846 +0000 UTC m=+1041.872303018"
Sep 19 05:58:09.198260 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:58:09.198229    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-aec2317b-5cf4ffcc9b-75b2d"
Sep 19 05:58:46.160762 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:58:46.160724    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-aec2317b-5cf4ffcc9b-75b2d"]
Sep 19 05:58:46.161266 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:58:46.160945    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-aec2317b-5cf4ffcc9b-75b2d" podUID="46db5bf6-fc21-4183-ad3a-f30dbb6e91d4" containerName="maas-api" containerID="cri-o://da047b53279a345f0fb8a2dec16096c8a9013ac2717099745bb67c29be6072c4" gracePeriod=30
Sep 19 05:58:46.407036 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:58:46.407010    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-aec2317b-5cf4ffcc9b-75b2d"
Sep 19 05:58:46.433737 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:58:46.433657    2583 generic.go:356] "Generic (PLEG): container finished" podID="46db5bf6-fc21-4183-ad3a-f30dbb6e91d4" containerID="da047b53279a345f0fb8a2dec16096c8a9013ac2717099745bb67c29be6072c4" exitCode=0
Sep 19 05:58:46.433737 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:58:46.433711    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-aec2317b-5cf4ffcc9b-75b2d"
Sep 19 05:58:46.433737 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:58:46.433723    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-aec2317b-5cf4ffcc9b-75b2d" event={"ID":"46db5bf6-fc21-4183-ad3a-f30dbb6e91d4","Type":"ContainerDied","Data":"da047b53279a345f0fb8a2dec16096c8a9013ac2717099745bb67c29be6072c4"}
Sep 19 05:58:46.434011 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:58:46.433776    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-aec2317b-5cf4ffcc9b-75b2d" event={"ID":"46db5bf6-fc21-4183-ad3a-f30dbb6e91d4","Type":"ContainerDied","Data":"1dcb97bdb4eddaf5a8b433208118d3142d86138b6779a0f532f2f989895bced3"}
Sep 19 05:58:46.434011 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:58:46.433809    2583 scope.go:117] "RemoveContainer" containerID="da047b53279a345f0fb8a2dec16096c8a9013ac2717099745bb67c29be6072c4"
Sep 19 05:58:46.443660 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:58:46.443634    2583 scope.go:117] "RemoveContainer" containerID="da047b53279a345f0fb8a2dec16096c8a9013ac2717099745bb67c29be6072c4"
Sep 19 05:58:46.443992 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:58:46.443972    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"da047b53279a345f0fb8a2dec16096c8a9013ac2717099745bb67c29be6072c4\": container with ID starting with da047b53279a345f0fb8a2dec16096c8a9013ac2717099745bb67c29be6072c4 not found: ID does not exist" containerID="da047b53279a345f0fb8a2dec16096c8a9013ac2717099745bb67c29be6072c4"
Sep 19 05:58:46.444079 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:58:46.444001    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da047b53279a345f0fb8a2dec16096c8a9013ac2717099745bb67c29be6072c4"} err="failed to get container status \"da047b53279a345f0fb8a2dec16096c8a9013ac2717099745bb67c29be6072c4\": rpc error: code = NotFound desc = could not find container \"da047b53279a345f0fb8a2dec16096c8a9013ac2717099745bb67c29be6072c4\": container with ID starting with da047b53279a345f0fb8a2dec16096c8a9013ac2717099745bb67c29be6072c4 not found: ID does not exist"
Sep 19 05:58:46.553854 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:58:46.553805    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/46db5bf6-fc21-4183-ad3a-f30dbb6e91d4-metrics-certs\") pod \"46db5bf6-fc21-4183-ad3a-f30dbb6e91d4\" (UID: \"46db5bf6-fc21-4183-ad3a-f30dbb6e91d4\") "
Sep 19 05:58:46.554047 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:58:46.553900    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/46db5bf6-fc21-4183-ad3a-f30dbb6e91d4-openshift-service-ca\") pod \"46db5bf6-fc21-4183-ad3a-f30dbb6e91d4\" (UID: \"46db5bf6-fc21-4183-ad3a-f30dbb6e91d4\") "
Sep 19 05:58:46.554630 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:58:46.554589    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/46db5bf6-fc21-4183-ad3a-f30dbb6e91d4-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "46db5bf6-fc21-4183-ad3a-f30dbb6e91d4" (UID: "46db5bf6-fc21-4183-ad3a-f30dbb6e91d4"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 05:58:46.554754 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:58:46.554080    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/46db5bf6-fc21-4183-ad3a-f30dbb6e91d4-maas-api-tls\") pod \"46db5bf6-fc21-4183-ad3a-f30dbb6e91d4\" (UID: \"46db5bf6-fc21-4183-ad3a-f30dbb6e91d4\") "
Sep 19 05:58:46.554754 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:58:46.554740    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z9fwm\" (UniqueName: \"kubernetes.io/projected/46db5bf6-fc21-4183-ad3a-f30dbb6e91d4-kube-api-access-z9fwm\") pod \"46db5bf6-fc21-4183-ad3a-f30dbb6e91d4\" (UID: \"46db5bf6-fc21-4183-ad3a-f30dbb6e91d4\") "
Sep 19 05:58:46.555486 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:58:46.555463    2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/46db5bf6-fc21-4183-ad3a-f30dbb6e91d4-openshift-service-ca\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 05:58:46.560148 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:58:46.560085    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/46db5bf6-fc21-4183-ad3a-f30dbb6e91d4-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "46db5bf6-fc21-4183-ad3a-f30dbb6e91d4" (UID: "46db5bf6-fc21-4183-ad3a-f30dbb6e91d4"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 05:58:46.560519 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:58:46.560485    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/46db5bf6-fc21-4183-ad3a-f30dbb6e91d4-kube-api-access-z9fwm" (OuterVolumeSpecName: "kube-api-access-z9fwm") pod "46db5bf6-fc21-4183-ad3a-f30dbb6e91d4" (UID: "46db5bf6-fc21-4183-ad3a-f30dbb6e91d4"). InnerVolumeSpecName "kube-api-access-z9fwm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 05:58:46.561166 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:58:46.560850    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/46db5bf6-fc21-4183-ad3a-f30dbb6e91d4-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "46db5bf6-fc21-4183-ad3a-f30dbb6e91d4" (UID: "46db5bf6-fc21-4183-ad3a-f30dbb6e91d4"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 05:58:46.656173 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:58:46.656120    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z9fwm\" (UniqueName: \"kubernetes.io/projected/46db5bf6-fc21-4183-ad3a-f30dbb6e91d4-kube-api-access-z9fwm\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 05:58:46.656173 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:58:46.656168    2583 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/46db5bf6-fc21-4183-ad3a-f30dbb6e91d4-metrics-certs\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 05:58:46.656173 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:58:46.656180    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/46db5bf6-fc21-4183-ad3a-f30dbb6e91d4-maas-api-tls\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 05:58:46.758993 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:58:46.758954    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-aec2317b-5cf4ffcc9b-75b2d"]
Sep 19 05:58:46.765151 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:58:46.765111    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-aec2317b-5cf4ffcc9b-75b2d"]
Sep 19 05:58:47.499075 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:58:47.499041    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="46db5bf6-fc21-4183-ad3a-f30dbb6e91d4" path="/var/lib/kubelet/pods/46db5bf6-fc21-4183-ad3a-f30dbb6e91d4/volumes"
Sep 19 05:59:13.678662 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:59:13.678618    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-2b3fb3-54449d8cc4-6n8n8"]
Sep 19 05:59:13.679574 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:59:13.679550    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="46db5bf6-fc21-4183-ad3a-f30dbb6e91d4" containerName="maas-api"
Sep 19 05:59:13.679574 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:59:13.679575    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="46db5bf6-fc21-4183-ad3a-f30dbb6e91d4" containerName="maas-api"
Sep 19 05:59:13.679749 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:59:13.679671    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="46db5bf6-fc21-4183-ad3a-f30dbb6e91d4" containerName="maas-api"
Sep 19 05:59:13.682904 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:59:13.682884    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-2b3fb3-54449d8cc4-6n8n8"
Sep 19 05:59:13.685517 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:59:13.685493    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w2-2b3fb3-serving-cert\""
Sep 19 05:59:13.694005 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:59:13.693976    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-2b3fb3-54449d8cc4-6n8n8"]
Sep 19 05:59:13.819438 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:59:13.819393    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/635f4638-2b98-4ff8-b247-cf98593ef0ca-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-2b3fb3-54449d8cc4-6n8n8\" (UID: \"635f4638-2b98-4ff8-b247-cf98593ef0ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-2b3fb3-54449d8cc4-6n8n8"
Sep 19 05:59:13.819638 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:59:13.819511    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/635f4638-2b98-4ff8-b247-cf98593ef0ca-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-2b3fb3-54449d8cc4-6n8n8\" (UID: \"635f4638-2b98-4ff8-b247-cf98593ef0ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-2b3fb3-54449d8cc4-6n8n8"
Sep 19 05:59:13.819638 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:59:13.819542    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/635f4638-2b98-4ff8-b247-cf98593ef0ca-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-2b3fb3-54449d8cc4-6n8n8\" (UID: \"635f4638-2b98-4ff8-b247-cf98593ef0ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-2b3fb3-54449d8cc4-6n8n8"
Sep 19 05:59:13.819748 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:59:13.819642    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c7l8w\" (UniqueName: \"kubernetes.io/projected/635f4638-2b98-4ff8-b247-cf98593ef0ca-kube-api-access-c7l8w\") pod \"maas-api-e2e-shared-a-w2-2b3fb3-54449d8cc4-6n8n8\" (UID: \"635f4638-2b98-4ff8-b247-cf98593ef0ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-2b3fb3-54449d8cc4-6n8n8"
Sep 19 05:59:13.920657 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:59:13.920618    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c7l8w\" (UniqueName: \"kubernetes.io/projected/635f4638-2b98-4ff8-b247-cf98593ef0ca-kube-api-access-c7l8w\") pod \"maas-api-e2e-shared-a-w2-2b3fb3-54449d8cc4-6n8n8\" (UID: \"635f4638-2b98-4ff8-b247-cf98593ef0ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-2b3fb3-54449d8cc4-6n8n8"
Sep 19 05:59:13.920818 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:59:13.920663    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/635f4638-2b98-4ff8-b247-cf98593ef0ca-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-2b3fb3-54449d8cc4-6n8n8\" (UID: \"635f4638-2b98-4ff8-b247-cf98593ef0ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-2b3fb3-54449d8cc4-6n8n8"
Sep 19 05:59:13.920818 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:59:13.920723    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/635f4638-2b98-4ff8-b247-cf98593ef0ca-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-2b3fb3-54449d8cc4-6n8n8\" (UID: \"635f4638-2b98-4ff8-b247-cf98593ef0ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-2b3fb3-54449d8cc4-6n8n8"
Sep 19 05:59:13.920818 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:59:13.920738    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/635f4638-2b98-4ff8-b247-cf98593ef0ca-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-2b3fb3-54449d8cc4-6n8n8\" (UID: \"635f4638-2b98-4ff8-b247-cf98593ef0ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-2b3fb3-54449d8cc4-6n8n8"
Sep 19 05:59:13.921017 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:59:13.920863    2583 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-2b3fb3-serving-cert: secret "maas-api-e2e-shared-a-w2-2b3fb3-serving-cert" not found
Sep 19 05:59:13.921017 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:59:13.920939    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/635f4638-2b98-4ff8-b247-cf98593ef0ca-maas-api-tls podName:635f4638-2b98-4ff8-b247-cf98593ef0ca nodeName:}" failed. No retries permitted until 2026-09-19 05:59:14.420918738 +0000 UTC m=+1113.079373289 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/635f4638-2b98-4ff8-b247-cf98593ef0ca-maas-api-tls") pod "maas-api-e2e-shared-a-w2-2b3fb3-54449d8cc4-6n8n8" (UID: "635f4638-2b98-4ff8-b247-cf98593ef0ca") : secret "maas-api-e2e-shared-a-w2-2b3fb3-serving-cert" not found
Sep 19 05:59:13.921604 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:59:13.921576    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/635f4638-2b98-4ff8-b247-cf98593ef0ca-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-2b3fb3-54449d8cc4-6n8n8\" (UID: \"635f4638-2b98-4ff8-b247-cf98593ef0ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-2b3fb3-54449d8cc4-6n8n8"
Sep 19 05:59:13.923345 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:59:13.923328    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/635f4638-2b98-4ff8-b247-cf98593ef0ca-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-2b3fb3-54449d8cc4-6n8n8\" (UID: \"635f4638-2b98-4ff8-b247-cf98593ef0ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-2b3fb3-54449d8cc4-6n8n8"
Sep 19 05:59:13.933990 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:59:13.933909    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c7l8w\" (UniqueName: \"kubernetes.io/projected/635f4638-2b98-4ff8-b247-cf98593ef0ca-kube-api-access-c7l8w\") pod \"maas-api-e2e-shared-a-w2-2b3fb3-54449d8cc4-6n8n8\" (UID: \"635f4638-2b98-4ff8-b247-cf98593ef0ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-2b3fb3-54449d8cc4-6n8n8"
Sep 19 05:59:14.424987 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:59:14.424951    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/635f4638-2b98-4ff8-b247-cf98593ef0ca-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-2b3fb3-54449d8cc4-6n8n8\" (UID: \"635f4638-2b98-4ff8-b247-cf98593ef0ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-2b3fb3-54449d8cc4-6n8n8"
Sep 19 05:59:14.427596 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:59:14.427571    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/635f4638-2b98-4ff8-b247-cf98593ef0ca-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-2b3fb3-54449d8cc4-6n8n8\" (UID: \"635f4638-2b98-4ff8-b247-cf98593ef0ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-2b3fb3-54449d8cc4-6n8n8"
Sep 19 05:59:14.594508 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:59:14.594478    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-2b3fb3-54449d8cc4-6n8n8"
Sep 19 05:59:14.876348 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:59:14.876316    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-2b3fb3-54449d8cc4-6n8n8"]
Sep 19 05:59:14.877453 ip-10-0-139-144 kubenswrapper[2583]: W0919 05:59:14.877424    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod635f4638_2b98_4ff8_b247_cf98593ef0ca.slice/crio-23d9c1ed4daf9019b3b62fb4d80be7d9c87b04456f139c44c0fa373232df308d WatchSource:0}: Error finding container 23d9c1ed4daf9019b3b62fb4d80be7d9c87b04456f139c44c0fa373232df308d: Status 404 returned error can't find the container with id 23d9c1ed4daf9019b3b62fb4d80be7d9c87b04456f139c44c0fa373232df308d
Sep 19 05:59:14.878812 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:59:14.878794    2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 19 05:59:15.602911 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:59:15.602874    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-2b3fb3-54449d8cc4-6n8n8" event={"ID":"635f4638-2b98-4ff8-b247-cf98593ef0ca","Type":"ContainerStarted","Data":"a2616a6d9b36ac6905de31bce89ae877b79782960b4c506f74eafb0121d4dbd2"}
Sep 19 05:59:15.603098 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:59:15.602939    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-2b3fb3-54449d8cc4-6n8n8" event={"ID":"635f4638-2b98-4ff8-b247-cf98593ef0ca","Type":"ContainerStarted","Data":"23d9c1ed4daf9019b3b62fb4d80be7d9c87b04456f139c44c0fa373232df308d"}
Sep 19 05:59:15.603098 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:59:15.603085    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-2b3fb3-54449d8cc4-6n8n8"
Sep 19 05:59:15.622948 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:59:15.622897    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-2b3fb3-54449d8cc4-6n8n8" podStartSLOduration=2.236781966 podStartE2EDuration="2.622881179s" totalImagesPullingTime="386.099213ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:59:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:59:15.620277124 +0000 UTC m=+1114.278731682" watchObservedRunningTime="2026-09-19 05:59:15.622881179 +0000 UTC m=+1114.281335736"
Sep 19 05:59:21.613624 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:59:21.613592    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-2b3fb3-54449d8cc4-6n8n8"
Sep 19 05:59:50.380566 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:59:50.380536    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-4b78e2-6bf5c5f97b-75ldh"]
Sep 19 05:59:50.381166 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:59:50.380800    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4b78e2-6bf5c5f97b-75ldh" podUID="9b70e086-50fc-498a-9e22-ff7fa70d4738" containerName="maas-api" containerID="cri-o://5579c2f7804809dda888ea44e6bd483413620b630a38d1250091650e279f3827" gracePeriod=30
Sep 19 05:59:50.647181 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:59:50.647109    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4b78e2-6bf5c5f97b-75ldh"
Sep 19 05:59:50.766544 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:59:50.766509    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9b70e086-50fc-498a-9e22-ff7fa70d4738-maas-api-tls\") pod \"9b70e086-50fc-498a-9e22-ff7fa70d4738\" (UID: \"9b70e086-50fc-498a-9e22-ff7fa70d4738\") "
Sep 19 05:59:50.766710 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:59:50.766609    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b70e086-50fc-498a-9e22-ff7fa70d4738-openshift-service-ca\") pod \"9b70e086-50fc-498a-9e22-ff7fa70d4738\" (UID: \"9b70e086-50fc-498a-9e22-ff7fa70d4738\") "
Sep 19 05:59:50.766710 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:59:50.766688    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nnccl\" (UniqueName: \"kubernetes.io/projected/9b70e086-50fc-498a-9e22-ff7fa70d4738-kube-api-access-nnccl\") pod \"9b70e086-50fc-498a-9e22-ff7fa70d4738\" (UID: \"9b70e086-50fc-498a-9e22-ff7fa70d4738\") "
Sep 19 05:59:50.766819 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:59:50.766728    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9b70e086-50fc-498a-9e22-ff7fa70d4738-metrics-certs\") pod \"9b70e086-50fc-498a-9e22-ff7fa70d4738\" (UID: \"9b70e086-50fc-498a-9e22-ff7fa70d4738\") "
Sep 19 05:59:50.766998 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:59:50.766967    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9b70e086-50fc-498a-9e22-ff7fa70d4738-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "9b70e086-50fc-498a-9e22-ff7fa70d4738" (UID: "9b70e086-50fc-498a-9e22-ff7fa70d4738"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 05:59:50.767178 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:59:50.767116    2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b70e086-50fc-498a-9e22-ff7fa70d4738-openshift-service-ca\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 05:59:50.768779 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:59:50.768746    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9b70e086-50fc-498a-9e22-ff7fa70d4738-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "9b70e086-50fc-498a-9e22-ff7fa70d4738" (UID: "9b70e086-50fc-498a-9e22-ff7fa70d4738"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 05:59:50.768925 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:59:50.768900    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9b70e086-50fc-498a-9e22-ff7fa70d4738-kube-api-access-nnccl" (OuterVolumeSpecName: "kube-api-access-nnccl") pod "9b70e086-50fc-498a-9e22-ff7fa70d4738" (UID: "9b70e086-50fc-498a-9e22-ff7fa70d4738"). InnerVolumeSpecName "kube-api-access-nnccl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 05:59:50.768990 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:59:50.768903    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9b70e086-50fc-498a-9e22-ff7fa70d4738-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "9b70e086-50fc-498a-9e22-ff7fa70d4738" (UID: "9b70e086-50fc-498a-9e22-ff7fa70d4738"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 05:59:50.804000 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:59:50.803967    2583 generic.go:356] "Generic (PLEG): container finished" podID="9b70e086-50fc-498a-9e22-ff7fa70d4738" containerID="5579c2f7804809dda888ea44e6bd483413620b630a38d1250091650e279f3827" exitCode=0
Sep 19 05:59:50.804180 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:59:50.804003    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4b78e2-6bf5c5f97b-75ldh"
Sep 19 05:59:50.804180 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:59:50.804045    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4b78e2-6bf5c5f97b-75ldh" event={"ID":"9b70e086-50fc-498a-9e22-ff7fa70d4738","Type":"ContainerDied","Data":"5579c2f7804809dda888ea44e6bd483413620b630a38d1250091650e279f3827"}
Sep 19 05:59:50.804180 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:59:50.804141    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4b78e2-6bf5c5f97b-75ldh" event={"ID":"9b70e086-50fc-498a-9e22-ff7fa70d4738","Type":"ContainerDied","Data":"ffb5966f2d2a1d14797f69cc590881ec25bfe9d5baf885f2f8564b0815318a6b"}
Sep 19 05:59:50.804180 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:59:50.804156    2583 scope.go:117] "RemoveContainer" containerID="5579c2f7804809dda888ea44e6bd483413620b630a38d1250091650e279f3827"
Sep 19 05:59:50.816910 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:59:50.816886    2583 scope.go:117] "RemoveContainer" containerID="5579c2f7804809dda888ea44e6bd483413620b630a38d1250091650e279f3827"
Sep 19 05:59:50.817709 ip-10-0-139-144 kubenswrapper[2583]: E0919 05:59:50.817675    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5579c2f7804809dda888ea44e6bd483413620b630a38d1250091650e279f3827\": container with ID starting with 5579c2f7804809dda888ea44e6bd483413620b630a38d1250091650e279f3827 not found: ID does not exist" containerID="5579c2f7804809dda888ea44e6bd483413620b630a38d1250091650e279f3827"
Sep 19 05:59:50.817798 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:59:50.817720    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5579c2f7804809dda888ea44e6bd483413620b630a38d1250091650e279f3827"} err="failed to get container status \"5579c2f7804809dda888ea44e6bd483413620b630a38d1250091650e279f3827\": rpc error: code = NotFound desc = could not find container \"5579c2f7804809dda888ea44e6bd483413620b630a38d1250091650e279f3827\": container with ID starting with 5579c2f7804809dda888ea44e6bd483413620b630a38d1250091650e279f3827 not found: ID does not exist"
Sep 19 05:59:50.835755 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:59:50.835729    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-4b78e2-6bf5c5f97b-75ldh"]
Sep 19 05:59:50.838202 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:59:50.838180    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-4b78e2-6bf5c5f97b-75ldh"]
Sep 19 05:59:50.868567 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:59:50.868546    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nnccl\" (UniqueName: \"kubernetes.io/projected/9b70e086-50fc-498a-9e22-ff7fa70d4738-kube-api-access-nnccl\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 05:59:50.868567 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:59:50.868568    2583 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9b70e086-50fc-498a-9e22-ff7fa70d4738-metrics-certs\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 05:59:50.868709 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:59:50.868580    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9b70e086-50fc-498a-9e22-ff7fa70d4738-maas-api-tls\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 05:59:51.498474 ip-10-0-139-144 kubenswrapper[2583]: I0919 05:59:51.498441    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9b70e086-50fc-498a-9e22-ff7fa70d4738" path="/var/lib/kubelet/pods/9b70e086-50fc-498a-9e22-ff7fa70d4738/volumes"
Sep 19 06:00:32.571152 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:00:32.571069    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-957d1190-67ccbbbcfb-shmzk"]
Sep 19 06:00:32.573716 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:00:32.571666    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b70e086-50fc-498a-9e22-ff7fa70d4738" containerName="maas-api"
Sep 19 06:00:32.573716 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:00:32.571683    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b70e086-50fc-498a-9e22-ff7fa70d4738" containerName="maas-api"
Sep 19 06:00:32.573716 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:00:32.571784    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b70e086-50fc-498a-9e22-ff7fa70d4738" containerName="maas-api"
Sep 19 06:00:32.575002 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:00:32.574981    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-957d1190-67ccbbbcfb-shmzk"
Sep 19 06:00:32.577988 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:00:32.577966    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-957d1190-serving-cert\""
Sep 19 06:00:32.587624 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:00:32.587598    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-957d1190-67ccbbbcfb-shmzk"]
Sep 19 06:00:32.661364 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:00:32.661335    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0f0e722a-eafd-4d32-bebd-7627639a45a7-metrics-certs\") pod \"maas-api-e2e-mt-957d1190-67ccbbbcfb-shmzk\" (UID: \"0f0e722a-eafd-4d32-bebd-7627639a45a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-957d1190-67ccbbbcfb-shmzk"
Sep 19 06:00:32.661517 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:00:32.661412    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0f0e722a-eafd-4d32-bebd-7627639a45a7-openshift-service-ca\") pod \"maas-api-e2e-mt-957d1190-67ccbbbcfb-shmzk\" (UID: \"0f0e722a-eafd-4d32-bebd-7627639a45a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-957d1190-67ccbbbcfb-shmzk"
Sep 19 06:00:32.661517 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:00:32.661456    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0f0e722a-eafd-4d32-bebd-7627639a45a7-maas-api-tls\") pod \"maas-api-e2e-mt-957d1190-67ccbbbcfb-shmzk\" (UID: \"0f0e722a-eafd-4d32-bebd-7627639a45a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-957d1190-67ccbbbcfb-shmzk"
Sep 19 06:00:32.661517 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:00:32.661474    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m5z78\" (UniqueName: \"kubernetes.io/projected/0f0e722a-eafd-4d32-bebd-7627639a45a7-kube-api-access-m5z78\") pod \"maas-api-e2e-mt-957d1190-67ccbbbcfb-shmzk\" (UID: \"0f0e722a-eafd-4d32-bebd-7627639a45a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-957d1190-67ccbbbcfb-shmzk"
Sep 19 06:00:32.762402 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:00:32.762366    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0f0e722a-eafd-4d32-bebd-7627639a45a7-metrics-certs\") pod \"maas-api-e2e-mt-957d1190-67ccbbbcfb-shmzk\" (UID: \"0f0e722a-eafd-4d32-bebd-7627639a45a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-957d1190-67ccbbbcfb-shmzk"
Sep 19 06:00:32.762569 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:00:32.762532    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0f0e722a-eafd-4d32-bebd-7627639a45a7-openshift-service-ca\") pod \"maas-api-e2e-mt-957d1190-67ccbbbcfb-shmzk\" (UID: \"0f0e722a-eafd-4d32-bebd-7627639a45a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-957d1190-67ccbbbcfb-shmzk"
Sep 19 06:00:32.762619 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:00:32.762591    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0f0e722a-eafd-4d32-bebd-7627639a45a7-maas-api-tls\") pod \"maas-api-e2e-mt-957d1190-67ccbbbcfb-shmzk\" (UID: \"0f0e722a-eafd-4d32-bebd-7627639a45a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-957d1190-67ccbbbcfb-shmzk"
Sep 19 06:00:32.762655 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:00:32.762619    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m5z78\" (UniqueName: \"kubernetes.io/projected/0f0e722a-eafd-4d32-bebd-7627639a45a7-kube-api-access-m5z78\") pod \"maas-api-e2e-mt-957d1190-67ccbbbcfb-shmzk\" (UID: \"0f0e722a-eafd-4d32-bebd-7627639a45a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-957d1190-67ccbbbcfb-shmzk"
Sep 19 06:00:32.762768 ip-10-0-139-144 kubenswrapper[2583]: E0919 06:00:32.762746    2583 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-957d1190-serving-cert: secret "maas-api-e2e-mt-957d1190-serving-cert" not found
Sep 19 06:00:32.762852 ip-10-0-139-144 kubenswrapper[2583]: E0919 06:00:32.762834    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0f0e722a-eafd-4d32-bebd-7627639a45a7-maas-api-tls podName:0f0e722a-eafd-4d32-bebd-7627639a45a7 nodeName:}" failed. No retries permitted until 2026-09-19 06:00:33.26281377 +0000 UTC m=+1191.921268327 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/0f0e722a-eafd-4d32-bebd-7627639a45a7-maas-api-tls") pod "maas-api-e2e-mt-957d1190-67ccbbbcfb-shmzk" (UID: "0f0e722a-eafd-4d32-bebd-7627639a45a7") : secret "maas-api-e2e-mt-957d1190-serving-cert" not found
Sep 19 06:00:32.763231 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:00:32.763210    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0f0e722a-eafd-4d32-bebd-7627639a45a7-openshift-service-ca\") pod \"maas-api-e2e-mt-957d1190-67ccbbbcfb-shmzk\" (UID: \"0f0e722a-eafd-4d32-bebd-7627639a45a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-957d1190-67ccbbbcfb-shmzk"
Sep 19 06:00:32.765027 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:00:32.765004    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0f0e722a-eafd-4d32-bebd-7627639a45a7-metrics-certs\") pod \"maas-api-e2e-mt-957d1190-67ccbbbcfb-shmzk\" (UID: \"0f0e722a-eafd-4d32-bebd-7627639a45a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-957d1190-67ccbbbcfb-shmzk"
Sep 19 06:00:32.771303 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:00:32.771283    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m5z78\" (UniqueName: \"kubernetes.io/projected/0f0e722a-eafd-4d32-bebd-7627639a45a7-kube-api-access-m5z78\") pod \"maas-api-e2e-mt-957d1190-67ccbbbcfb-shmzk\" (UID: \"0f0e722a-eafd-4d32-bebd-7627639a45a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-957d1190-67ccbbbcfb-shmzk"
Sep 19 06:00:33.268064 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:00:33.268028    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0f0e722a-eafd-4d32-bebd-7627639a45a7-maas-api-tls\") pod \"maas-api-e2e-mt-957d1190-67ccbbbcfb-shmzk\" (UID: \"0f0e722a-eafd-4d32-bebd-7627639a45a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-957d1190-67ccbbbcfb-shmzk"
Sep 19 06:00:33.270379 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:00:33.270351    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0f0e722a-eafd-4d32-bebd-7627639a45a7-maas-api-tls\") pod \"maas-api-e2e-mt-957d1190-67ccbbbcfb-shmzk\" (UID: \"0f0e722a-eafd-4d32-bebd-7627639a45a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-957d1190-67ccbbbcfb-shmzk"
Sep 19 06:00:33.487269 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:00:33.487232    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-957d1190-67ccbbbcfb-shmzk"
Sep 19 06:00:33.765514 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:00:33.765489    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-957d1190-67ccbbbcfb-shmzk"]
Sep 19 06:00:33.767106 ip-10-0-139-144 kubenswrapper[2583]: W0919 06:00:33.767078    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f0e722a_eafd_4d32_bebd_7627639a45a7.slice/crio-23c95f3b2c18ae8d34cff20aaa3f6b4c5ec97ec6fba3cb7cdac5fd66328f8fdd WatchSource:0}: Error finding container 23c95f3b2c18ae8d34cff20aaa3f6b4c5ec97ec6fba3cb7cdac5fd66328f8fdd: Status 404 returned error can't find the container with id 23c95f3b2c18ae8d34cff20aaa3f6b4c5ec97ec6fba3cb7cdac5fd66328f8fdd
Sep 19 06:00:34.050544 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:00:34.050504    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-957d1190-67ccbbbcfb-shmzk" event={"ID":"0f0e722a-eafd-4d32-bebd-7627639a45a7","Type":"ContainerStarted","Data":"23c95f3b2c18ae8d34cff20aaa3f6b4c5ec97ec6fba3cb7cdac5fd66328f8fdd"}
Sep 19 06:00:35.063879 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:00:35.063834    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-957d1190-67ccbbbcfb-shmzk" event={"ID":"0f0e722a-eafd-4d32-bebd-7627639a45a7","Type":"ContainerStarted","Data":"4ab4f5e5202d7117d0f84751d7c52ac94aa93db0eb31a0de271e9fa8644b81e9"}
Sep 19 06:00:35.064299 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:00:35.063958    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-957d1190-67ccbbbcfb-shmzk"
Sep 19 06:00:35.082310 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:00:35.082263    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-957d1190-67ccbbbcfb-shmzk" podStartSLOduration=2.672352391 podStartE2EDuration="3.08225055s" totalImagesPullingTime="409.898159ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 06:00:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 06:00:35.080951646 +0000 UTC m=+1193.739406204" watchObservedRunningTime="2026-09-19 06:00:35.08225055 +0000 UTC m=+1193.740705148"
Sep 19 06:00:41.075435 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:00:41.075406    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-957d1190-67ccbbbcfb-shmzk"
Sep 19 06:00:47.974394 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:00:47.974356    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-cc0e26/distinct-w0-cc0e26-kserve-6b7bdf546b-dqvpj"]
Sep 19 06:00:47.979291 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:00:47.979275    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-cc0e26/distinct-w0-cc0e26-kserve-6b7bdf546b-dqvpj"
Sep 19 06:00:47.985946 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:00:47.985679    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-cc0e26\"/\"distinct-w0-cc0e26-kserve-self-signed-certs\""
Sep 19 06:00:48.001057 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:00:48.001031    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-cc0e26/distinct-w0-cc0e26-kserve-6b7bdf546b-dqvpj"]
Sep 19 06:00:48.112673 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:00:48.112634    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q8p96\" (UniqueName: \"kubernetes.io/projected/9a6c8743-44fc-465e-a9b0-2353781c2aae-kube-api-access-q8p96\") pod \"distinct-w0-cc0e26-kserve-6b7bdf546b-dqvpj\" (UID: \"9a6c8743-44fc-465e-a9b0-2353781c2aae\") " pod="e2e-models-e2e-worker-w0-cc0e26/distinct-w0-cc0e26-kserve-6b7bdf546b-dqvpj"
Sep 19 06:00:48.112828 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:00:48.112723    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9a6c8743-44fc-465e-a9b0-2353781c2aae-tls-certs\") pod \"distinct-w0-cc0e26-kserve-6b7bdf546b-dqvpj\" (UID: \"9a6c8743-44fc-465e-a9b0-2353781c2aae\") " pod="e2e-models-e2e-worker-w0-cc0e26/distinct-w0-cc0e26-kserve-6b7bdf546b-dqvpj"
Sep 19 06:00:48.112828 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:00:48.112761    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9a6c8743-44fc-465e-a9b0-2353781c2aae-dshm\") pod \"distinct-w0-cc0e26-kserve-6b7bdf546b-dqvpj\" (UID: \"9a6c8743-44fc-465e-a9b0-2353781c2aae\") " pod="e2e-models-e2e-worker-w0-cc0e26/distinct-w0-cc0e26-kserve-6b7bdf546b-dqvpj"
Sep 19 06:00:48.112902 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:00:48.112824    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9a6c8743-44fc-465e-a9b0-2353781c2aae-home\") pod \"distinct-w0-cc0e26-kserve-6b7bdf546b-dqvpj\" (UID: \"9a6c8743-44fc-465e-a9b0-2353781c2aae\") " pod="e2e-models-e2e-worker-w0-cc0e26/distinct-w0-cc0e26-kserve-6b7bdf546b-dqvpj"
Sep 19 06:00:48.112938 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:00:48.112899    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9a6c8743-44fc-465e-a9b0-2353781c2aae-model-cache\") pod \"distinct-w0-cc0e26-kserve-6b7bdf546b-dqvpj\" (UID: \"9a6c8743-44fc-465e-a9b0-2353781c2aae\") " pod="e2e-models-e2e-worker-w0-cc0e26/distinct-w0-cc0e26-kserve-6b7bdf546b-dqvpj"
Sep 19 06:00:48.112972 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:00:48.112950    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9a6c8743-44fc-465e-a9b0-2353781c2aae-tmp-dir\") pod \"distinct-w0-cc0e26-kserve-6b7bdf546b-dqvpj\" (UID: \"9a6c8743-44fc-465e-a9b0-2353781c2aae\") " pod="e2e-models-e2e-worker-w0-cc0e26/distinct-w0-cc0e26-kserve-6b7bdf546b-dqvpj"
Sep 19 06:00:48.215834 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:00:48.215792    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9a6c8743-44fc-465e-a9b0-2353781c2aae-home\") pod \"distinct-w0-cc0e26-kserve-6b7bdf546b-dqvpj\" (UID: \"9a6c8743-44fc-465e-a9b0-2353781c2aae\") " pod="e2e-models-e2e-worker-w0-cc0e26/distinct-w0-cc0e26-kserve-6b7bdf546b-dqvpj"
Sep 19 06:00:48.216167 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:00:48.216105    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9a6c8743-44fc-465e-a9b0-2353781c2aae-model-cache\") pod \"distinct-w0-cc0e26-kserve-6b7bdf546b-dqvpj\" (UID: \"9a6c8743-44fc-465e-a9b0-2353781c2aae\") " pod="e2e-models-e2e-worker-w0-cc0e26/distinct-w0-cc0e26-kserve-6b7bdf546b-dqvpj"
Sep 19 06:00:48.216304 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:00:48.216200    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9a6c8743-44fc-465e-a9b0-2353781c2aae-tmp-dir\") pod \"distinct-w0-cc0e26-kserve-6b7bdf546b-dqvpj\" (UID: \"9a6c8743-44fc-465e-a9b0-2353781c2aae\") " pod="e2e-models-e2e-worker-w0-cc0e26/distinct-w0-cc0e26-kserve-6b7bdf546b-dqvpj"
Sep 19 06:00:48.216304 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:00:48.216292    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q8p96\" (UniqueName: \"kubernetes.io/projected/9a6c8743-44fc-465e-a9b0-2353781c2aae-kube-api-access-q8p96\") pod \"distinct-w0-cc0e26-kserve-6b7bdf546b-dqvpj\" (UID: \"9a6c8743-44fc-465e-a9b0-2353781c2aae\") " pod="e2e-models-e2e-worker-w0-cc0e26/distinct-w0-cc0e26-kserve-6b7bdf546b-dqvpj"
Sep 19 06:00:48.216490 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:00:48.216368    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9a6c8743-44fc-465e-a9b0-2353781c2aae-tls-certs\") pod \"distinct-w0-cc0e26-kserve-6b7bdf546b-dqvpj\" (UID: \"9a6c8743-44fc-465e-a9b0-2353781c2aae\") " pod="e2e-models-e2e-worker-w0-cc0e26/distinct-w0-cc0e26-kserve-6b7bdf546b-dqvpj"
Sep 19 06:00:48.216490 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:00:48.216407    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9a6c8743-44fc-465e-a9b0-2353781c2aae-dshm\") pod \"distinct-w0-cc0e26-kserve-6b7bdf546b-dqvpj\" (UID: \"9a6c8743-44fc-465e-a9b0-2353781c2aae\") " pod="e2e-models-e2e-worker-w0-cc0e26/distinct-w0-cc0e26-kserve-6b7bdf546b-dqvpj"
Sep 19 06:00:48.216490 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:00:48.216427    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9a6c8743-44fc-465e-a9b0-2353781c2aae-home\") pod \"distinct-w0-cc0e26-kserve-6b7bdf546b-dqvpj\" (UID: \"9a6c8743-44fc-465e-a9b0-2353781c2aae\") " pod="e2e-models-e2e-worker-w0-cc0e26/distinct-w0-cc0e26-kserve-6b7bdf546b-dqvpj"
Sep 19 06:00:48.216865 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:00:48.216846    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9a6c8743-44fc-465e-a9b0-2353781c2aae-tmp-dir\") pod \"distinct-w0-cc0e26-kserve-6b7bdf546b-dqvpj\" (UID: \"9a6c8743-44fc-465e-a9b0-2353781c2aae\") " pod="e2e-models-e2e-worker-w0-cc0e26/distinct-w0-cc0e26-kserve-6b7bdf546b-dqvpj"
Sep 19 06:00:48.217154 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:00:48.217101    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9a6c8743-44fc-465e-a9b0-2353781c2aae-model-cache\") pod \"distinct-w0-cc0e26-kserve-6b7bdf546b-dqvpj\" (UID: \"9a6c8743-44fc-465e-a9b0-2353781c2aae\") " pod="e2e-models-e2e-worker-w0-cc0e26/distinct-w0-cc0e26-kserve-6b7bdf546b-dqvpj"
Sep 19 06:00:48.219774 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:00:48.219729    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9a6c8743-44fc-465e-a9b0-2353781c2aae-dshm\") pod \"distinct-w0-cc0e26-kserve-6b7bdf546b-dqvpj\" (UID: \"9a6c8743-44fc-465e-a9b0-2353781c2aae\") " pod="e2e-models-e2e-worker-w0-cc0e26/distinct-w0-cc0e26-kserve-6b7bdf546b-dqvpj"
Sep 19 06:00:48.221480 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:00:48.221028    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9a6c8743-44fc-465e-a9b0-2353781c2aae-tls-certs\") pod \"distinct-w0-cc0e26-kserve-6b7bdf546b-dqvpj\" (UID: \"9a6c8743-44fc-465e-a9b0-2353781c2aae\") " pod="e2e-models-e2e-worker-w0-cc0e26/distinct-w0-cc0e26-kserve-6b7bdf546b-dqvpj"
Sep 19 06:00:48.227163 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:00:48.227041    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q8p96\" (UniqueName: \"kubernetes.io/projected/9a6c8743-44fc-465e-a9b0-2353781c2aae-kube-api-access-q8p96\") pod \"distinct-w0-cc0e26-kserve-6b7bdf546b-dqvpj\" (UID: \"9a6c8743-44fc-465e-a9b0-2353781c2aae\") " pod="e2e-models-e2e-worker-w0-cc0e26/distinct-w0-cc0e26-kserve-6b7bdf546b-dqvpj"
Sep 19 06:00:48.288614 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:00:48.288581    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-cc0e26/distinct-w0-cc0e26-kserve-6b7bdf546b-dqvpj"
Sep 19 06:00:48.563172 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:00:48.563146    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-cc0e26/distinct-w0-cc0e26-kserve-6b7bdf546b-dqvpj"]
Sep 19 06:00:48.564533 ip-10-0-139-144 kubenswrapper[2583]: W0919 06:00:48.564502    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9a6c8743_44fc_465e_a9b0_2353781c2aae.slice/crio-9667e88e1f9f08bccdc3c49e839f842dc1d8f5db330f43dface893db44deff4f WatchSource:0}: Error finding container 9667e88e1f9f08bccdc3c49e839f842dc1d8f5db330f43dface893db44deff4f: Status 404 returned error can't find the container with id 9667e88e1f9f08bccdc3c49e839f842dc1d8f5db330f43dface893db44deff4f
Sep 19 06:00:49.142295 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:00:49.142262    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-cc0e26/distinct-w0-cc0e26-kserve-6b7bdf546b-dqvpj" event={"ID":"9a6c8743-44fc-465e-a9b0-2353781c2aae","Type":"ContainerStarted","Data":"f27f0cd1d58dc08b7de1e1d04feac9df7b8bd875aba5424fde91193fa1563ab9"}
Sep 19 06:00:49.142698 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:00:49.142304    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-cc0e26/distinct-w0-cc0e26-kserve-6b7bdf546b-dqvpj" event={"ID":"9a6c8743-44fc-465e-a9b0-2353781c2aae","Type":"ContainerStarted","Data":"9667e88e1f9f08bccdc3c49e839f842dc1d8f5db330f43dface893db44deff4f"}
Sep 19 06:00:49.162995 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:00:49.162945    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-cc0e26/distinct-w0-cc0e26-kserve-6b7bdf546b-dqvpj" podStartSLOduration=2.162928967 podStartE2EDuration="2.162928967s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 06:00:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 06:00:49.16029319 +0000 UTC m=+1207.818747749" watchObservedRunningTime="2026-09-19 06:00:49.162928967 +0000 UTC m=+1207.821383524"
Sep 19 06:00:57.756482 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:00:57.756448    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-4b78e2-openshift-default-7b7664c598-rz5lx"]
Sep 19 06:00:57.756893 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:00:57.756701    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-4b78e2-openshift-default-7b7664c598-rz5lx" podUID="57450d37-8096-4ff3-a23f-5ce98d6cbc22" containerName="istio-proxy" containerID="cri-o://c33be8bfb63320a231e5d408ec225e19cd71ebf3c73acbf7d68ce98b29804fcf" gracePeriod=30
Sep 19 06:00:58.288927 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:00:58.288892    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-cc0e26/distinct-w0-cc0e26-kserve-6b7bdf546b-dqvpj"
Sep 19 06:00:58.288927 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:00:58.288932    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-cc0e26/distinct-w0-cc0e26-kserve-6b7bdf546b-dqvpj"
Sep 19 06:00:58.301719 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:00:58.301688    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-cc0e26/distinct-w0-cc0e26-kserve-6b7bdf546b-dqvpj"
Sep 19 06:00:59.210681 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:00:59.210649    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-cc0e26/distinct-w0-cc0e26-kserve-6b7bdf546b-dqvpj"
Sep 19 06:01:03.007574 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:03.007547    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-4b78e2-openshift-default-7b7664c598-rz5lx"
Sep 19 06:01:03.166419 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:03.166374    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/57450d37-8096-4ff3-a23f-5ce98d6cbc22-workload-certs\") pod \"57450d37-8096-4ff3-a23f-5ce98d6cbc22\" (UID: \"57450d37-8096-4ff3-a23f-5ce98d6cbc22\") "
Sep 19 06:01:03.166419 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:03.166418    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/57450d37-8096-4ff3-a23f-5ce98d6cbc22-istio-data\") pod \"57450d37-8096-4ff3-a23f-5ce98d6cbc22\" (UID: \"57450d37-8096-4ff3-a23f-5ce98d6cbc22\") "
Sep 19 06:01:03.166655 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:03.166463    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/57450d37-8096-4ff3-a23f-5ce98d6cbc22-credential-socket\") pod \"57450d37-8096-4ff3-a23f-5ce98d6cbc22\" (UID: \"57450d37-8096-4ff3-a23f-5ce98d6cbc22\") "
Sep 19 06:01:03.166655 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:03.166484    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/57450d37-8096-4ff3-a23f-5ce98d6cbc22-workload-socket\") pod \"57450d37-8096-4ff3-a23f-5ce98d6cbc22\" (UID: \"57450d37-8096-4ff3-a23f-5ce98d6cbc22\") "
Sep 19 06:01:03.166655 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:03.166535    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x57rl\" (UniqueName: \"kubernetes.io/projected/57450d37-8096-4ff3-a23f-5ce98d6cbc22-kube-api-access-x57rl\") pod \"57450d37-8096-4ff3-a23f-5ce98d6cbc22\" (UID: \"57450d37-8096-4ff3-a23f-5ce98d6cbc22\") "
Sep 19 06:01:03.166655 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:03.166574    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/57450d37-8096-4ff3-a23f-5ce98d6cbc22-istiod-ca-cert\") pod \"57450d37-8096-4ff3-a23f-5ce98d6cbc22\" (UID: \"57450d37-8096-4ff3-a23f-5ce98d6cbc22\") "
Sep 19 06:01:03.166655 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:03.166608    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/57450d37-8096-4ff3-a23f-5ce98d6cbc22-istio-podinfo\") pod \"57450d37-8096-4ff3-a23f-5ce98d6cbc22\" (UID: \"57450d37-8096-4ff3-a23f-5ce98d6cbc22\") "
Sep 19 06:01:03.166655 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:03.166644    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/57450d37-8096-4ff3-a23f-5ce98d6cbc22-istio-token\") pod \"57450d37-8096-4ff3-a23f-5ce98d6cbc22\" (UID: \"57450d37-8096-4ff3-a23f-5ce98d6cbc22\") "
Sep 19 06:01:03.166973 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:03.166678    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/57450d37-8096-4ff3-a23f-5ce98d6cbc22-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "57450d37-8096-4ff3-a23f-5ce98d6cbc22" (UID: "57450d37-8096-4ff3-a23f-5ce98d6cbc22"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 06:01:03.166973 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:03.166689    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/57450d37-8096-4ff3-a23f-5ce98d6cbc22-istio-envoy\") pod \"57450d37-8096-4ff3-a23f-5ce98d6cbc22\" (UID: \"57450d37-8096-4ff3-a23f-5ce98d6cbc22\") "
Sep 19 06:01:03.166973 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:03.166731    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/57450d37-8096-4ff3-a23f-5ce98d6cbc22-istio-data" (OuterVolumeSpecName: "istio-data") pod "57450d37-8096-4ff3-a23f-5ce98d6cbc22" (UID: "57450d37-8096-4ff3-a23f-5ce98d6cbc22"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 06:01:03.166973 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:03.166814    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/57450d37-8096-4ff3-a23f-5ce98d6cbc22-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "57450d37-8096-4ff3-a23f-5ce98d6cbc22" (UID: "57450d37-8096-4ff3-a23f-5ce98d6cbc22"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 06:01:03.166973 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:03.166910    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/57450d37-8096-4ff3-a23f-5ce98d6cbc22-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "57450d37-8096-4ff3-a23f-5ce98d6cbc22" (UID: "57450d37-8096-4ff3-a23f-5ce98d6cbc22"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 06:01:03.167230 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:03.167080    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/57450d37-8096-4ff3-a23f-5ce98d6cbc22-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "57450d37-8096-4ff3-a23f-5ce98d6cbc22" (UID: "57450d37-8096-4ff3-a23f-5ce98d6cbc22"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 06:01:03.167230 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:03.167199    2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/57450d37-8096-4ff3-a23f-5ce98d6cbc22-workload-certs\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 06:01:03.167230 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:03.167216    2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/57450d37-8096-4ff3-a23f-5ce98d6cbc22-istio-data\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 06:01:03.167350 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:03.167237    2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/57450d37-8096-4ff3-a23f-5ce98d6cbc22-credential-socket\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 06:01:03.167350 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:03.167250    2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/57450d37-8096-4ff3-a23f-5ce98d6cbc22-workload-socket\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 06:01:03.167350 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:03.167265    2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/57450d37-8096-4ff3-a23f-5ce98d6cbc22-istiod-ca-cert\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 06:01:03.168935 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:03.168897    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/57450d37-8096-4ff3-a23f-5ce98d6cbc22-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "57450d37-8096-4ff3-a23f-5ce98d6cbc22" (UID: "57450d37-8096-4ff3-a23f-5ce98d6cbc22"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 19 06:01:03.169045 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:03.168974    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/57450d37-8096-4ff3-a23f-5ce98d6cbc22-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "57450d37-8096-4ff3-a23f-5ce98d6cbc22" (UID: "57450d37-8096-4ff3-a23f-5ce98d6cbc22"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 06:01:03.169045 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:03.169003    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/57450d37-8096-4ff3-a23f-5ce98d6cbc22-kube-api-access-x57rl" (OuterVolumeSpecName: "kube-api-access-x57rl") pod "57450d37-8096-4ff3-a23f-5ce98d6cbc22" (UID: "57450d37-8096-4ff3-a23f-5ce98d6cbc22"). InnerVolumeSpecName "kube-api-access-x57rl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 06:01:03.169045 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:03.169027    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/57450d37-8096-4ff3-a23f-5ce98d6cbc22-istio-token" (OuterVolumeSpecName: "istio-token") pod "57450d37-8096-4ff3-a23f-5ce98d6cbc22" (UID: "57450d37-8096-4ff3-a23f-5ce98d6cbc22"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 06:01:03.228555 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:03.228515    2583 generic.go:356] "Generic (PLEG): container finished" podID="57450d37-8096-4ff3-a23f-5ce98d6cbc22" containerID="c33be8bfb63320a231e5d408ec225e19cd71ebf3c73acbf7d68ce98b29804fcf" exitCode=0
Sep 19 06:01:03.228713 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:03.228559    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-4b78e2-openshift-default-7b7664c598-rz5lx"
Sep 19 06:01:03.228713 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:03.228568    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-4b78e2-openshift-default-7b7664c598-rz5lx" event={"ID":"57450d37-8096-4ff3-a23f-5ce98d6cbc22","Type":"ContainerDied","Data":"c33be8bfb63320a231e5d408ec225e19cd71ebf3c73acbf7d68ce98b29804fcf"}
Sep 19 06:01:03.228713 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:03.228618    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-4b78e2-openshift-default-7b7664c598-rz5lx" event={"ID":"57450d37-8096-4ff3-a23f-5ce98d6cbc22","Type":"ContainerDied","Data":"5b874ef4ee9dab004826779b4707e7f00d1e4328fe7f331c8b46c8db04b60a17"}
Sep 19 06:01:03.228713 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:03.228643    2583 scope.go:117] "RemoveContainer" containerID="c33be8bfb63320a231e5d408ec225e19cd71ebf3c73acbf7d68ce98b29804fcf"
Sep 19 06:01:03.238411 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:03.238394    2583 scope.go:117] "RemoveContainer" containerID="e0ed9229f52bc20a9a8c8b116f2575ff5abaeb42445d4ef500940a278d7a0bf8"
Sep 19 06:01:03.246875 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:03.246857    2583 scope.go:117] "RemoveContainer" containerID="c33be8bfb63320a231e5d408ec225e19cd71ebf3c73acbf7d68ce98b29804fcf"
Sep 19 06:01:03.247118 ip-10-0-139-144 kubenswrapper[2583]: E0919 06:01:03.247090    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c33be8bfb63320a231e5d408ec225e19cd71ebf3c73acbf7d68ce98b29804fcf\": container with ID starting with c33be8bfb63320a231e5d408ec225e19cd71ebf3c73acbf7d68ce98b29804fcf not found: ID does not exist" containerID="c33be8bfb63320a231e5d408ec225e19cd71ebf3c73acbf7d68ce98b29804fcf"
Sep 19 06:01:03.247226 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:03.247116    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c33be8bfb63320a231e5d408ec225e19cd71ebf3c73acbf7d68ce98b29804fcf"} err="failed to get container status \"c33be8bfb63320a231e5d408ec225e19cd71ebf3c73acbf7d68ce98b29804fcf\": rpc error: code = NotFound desc = could not find container \"c33be8bfb63320a231e5d408ec225e19cd71ebf3c73acbf7d68ce98b29804fcf\": container with ID starting with c33be8bfb63320a231e5d408ec225e19cd71ebf3c73acbf7d68ce98b29804fcf not found: ID does not exist"
Sep 19 06:01:03.247226 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:03.247158    2583 scope.go:117] "RemoveContainer" containerID="e0ed9229f52bc20a9a8c8b116f2575ff5abaeb42445d4ef500940a278d7a0bf8"
Sep 19 06:01:03.247403 ip-10-0-139-144 kubenswrapper[2583]: E0919 06:01:03.247388    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e0ed9229f52bc20a9a8c8b116f2575ff5abaeb42445d4ef500940a278d7a0bf8\": container with ID starting with e0ed9229f52bc20a9a8c8b116f2575ff5abaeb42445d4ef500940a278d7a0bf8 not found: ID does not exist" containerID="e0ed9229f52bc20a9a8c8b116f2575ff5abaeb42445d4ef500940a278d7a0bf8"
Sep 19 06:01:03.247442 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:03.247407    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e0ed9229f52bc20a9a8c8b116f2575ff5abaeb42445d4ef500940a278d7a0bf8"} err="failed to get container status \"e0ed9229f52bc20a9a8c8b116f2575ff5abaeb42445d4ef500940a278d7a0bf8\": rpc error: code = NotFound desc = could not find container \"e0ed9229f52bc20a9a8c8b116f2575ff5abaeb42445d4ef500940a278d7a0bf8\": container with ID starting with e0ed9229f52bc20a9a8c8b116f2575ff5abaeb42445d4ef500940a278d7a0bf8 not found: ID does not exist"
Sep 19 06:01:03.253644 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:03.253621    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-4b78e2-openshift-default-7b7664c598-rz5lx"]
Sep 19 06:01:03.259714 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:03.259695    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-4b78e2-openshift-default-7b7664c598-rz5lx"]
Sep 19 06:01:03.268451 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:03.268431    2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/57450d37-8096-4ff3-a23f-5ce98d6cbc22-istio-podinfo\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 06:01:03.268451 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:03.268451    2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/57450d37-8096-4ff3-a23f-5ce98d6cbc22-istio-token\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 06:01:03.268585 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:03.268461    2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/57450d37-8096-4ff3-a23f-5ce98d6cbc22-istio-envoy\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 06:01:03.268585 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:03.268469    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x57rl\" (UniqueName: \"kubernetes.io/projected/57450d37-8096-4ff3-a23f-5ce98d6cbc22-kube-api-access-x57rl\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 06:01:03.499886 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:03.499799    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="57450d37-8096-4ff3-a23f-5ce98d6cbc22" path="/var/lib/kubelet/pods/57450d37-8096-4ff3-a23f-5ce98d6cbc22/volumes"
Sep 19 06:01:08.875790 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:08.875756    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-c85316-7f7d79c99-2tz2f"]
Sep 19 06:01:08.876293 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:08.876169    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c85316-7f7d79c99-2tz2f" podUID="d1e19012-831d-430e-b711-c2b0a4665df8" containerName="maas-api" containerID="cri-o://00f3f973469ff192cd8cb1d83568e64aba1e1425b25bd40badaebff372daa9da" gracePeriod=30
Sep 19 06:01:09.137742 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:09.137678    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c85316-7f7d79c99-2tz2f"
Sep 19 06:01:09.224197 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:09.224162    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d1e19012-831d-430e-b711-c2b0a4665df8-openshift-service-ca\") pod \"d1e19012-831d-430e-b711-c2b0a4665df8\" (UID: \"d1e19012-831d-430e-b711-c2b0a4665df8\") "
Sep 19 06:01:09.224348 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:09.224289    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d1e19012-831d-430e-b711-c2b0a4665df8-metrics-certs\") pod \"d1e19012-831d-430e-b711-c2b0a4665df8\" (UID: \"d1e19012-831d-430e-b711-c2b0a4665df8\") "
Sep 19 06:01:09.224414 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:09.224345    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d1e19012-831d-430e-b711-c2b0a4665df8-maas-api-tls\") pod \"d1e19012-831d-430e-b711-c2b0a4665df8\" (UID: \"d1e19012-831d-430e-b711-c2b0a4665df8\") "
Sep 19 06:01:09.224414 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:09.224385    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7ftjc\" (UniqueName: \"kubernetes.io/projected/d1e19012-831d-430e-b711-c2b0a4665df8-kube-api-access-7ftjc\") pod \"d1e19012-831d-430e-b711-c2b0a4665df8\" (UID: \"d1e19012-831d-430e-b711-c2b0a4665df8\") "
Sep 19 06:01:09.224543 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:09.224518    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d1e19012-831d-430e-b711-c2b0a4665df8-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d1e19012-831d-430e-b711-c2b0a4665df8" (UID: "d1e19012-831d-430e-b711-c2b0a4665df8"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 06:01:09.224764 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:09.224746    2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d1e19012-831d-430e-b711-c2b0a4665df8-openshift-service-ca\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 06:01:09.226494 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:09.226462    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d1e19012-831d-430e-b711-c2b0a4665df8-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "d1e19012-831d-430e-b711-c2b0a4665df8" (UID: "d1e19012-831d-430e-b711-c2b0a4665df8"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 06:01:09.226494 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:09.226488    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d1e19012-831d-430e-b711-c2b0a4665df8-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d1e19012-831d-430e-b711-c2b0a4665df8" (UID: "d1e19012-831d-430e-b711-c2b0a4665df8"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 06:01:09.226849 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:09.226834    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d1e19012-831d-430e-b711-c2b0a4665df8-kube-api-access-7ftjc" (OuterVolumeSpecName: "kube-api-access-7ftjc") pod "d1e19012-831d-430e-b711-c2b0a4665df8" (UID: "d1e19012-831d-430e-b711-c2b0a4665df8"). InnerVolumeSpecName "kube-api-access-7ftjc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 06:01:09.265731 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:09.265688    2583 generic.go:356] "Generic (PLEG): container finished" podID="d1e19012-831d-430e-b711-c2b0a4665df8" containerID="00f3f973469ff192cd8cb1d83568e64aba1e1425b25bd40badaebff372daa9da" exitCode=0
Sep 19 06:01:09.271572 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:09.265750    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c85316-7f7d79c99-2tz2f"
Sep 19 06:01:09.271572 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:09.265757    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c85316-7f7d79c99-2tz2f" event={"ID":"d1e19012-831d-430e-b711-c2b0a4665df8","Type":"ContainerDied","Data":"00f3f973469ff192cd8cb1d83568e64aba1e1425b25bd40badaebff372daa9da"}
Sep 19 06:01:09.271572 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:09.265805    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c85316-7f7d79c99-2tz2f" event={"ID":"d1e19012-831d-430e-b711-c2b0a4665df8","Type":"ContainerDied","Data":"502118f2c0fba9edac334652dd77c05d6fcad3efe0b489807f2548d5dcb030a6"}
Sep 19 06:01:09.271572 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:09.265833    2583 scope.go:117] "RemoveContainer" containerID="00f3f973469ff192cd8cb1d83568e64aba1e1425b25bd40badaebff372daa9da"
Sep 19 06:01:09.285744 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:09.285683    2583 scope.go:117] "RemoveContainer" containerID="00f3f973469ff192cd8cb1d83568e64aba1e1425b25bd40badaebff372daa9da"
Sep 19 06:01:09.286107 ip-10-0-139-144 kubenswrapper[2583]: E0919 06:01:09.286018    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"00f3f973469ff192cd8cb1d83568e64aba1e1425b25bd40badaebff372daa9da\": container with ID starting with 00f3f973469ff192cd8cb1d83568e64aba1e1425b25bd40badaebff372daa9da not found: ID does not exist" containerID="00f3f973469ff192cd8cb1d83568e64aba1e1425b25bd40badaebff372daa9da"
Sep 19 06:01:09.286107 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:09.286057    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"00f3f973469ff192cd8cb1d83568e64aba1e1425b25bd40badaebff372daa9da"} err="failed to get container status \"00f3f973469ff192cd8cb1d83568e64aba1e1425b25bd40badaebff372daa9da\": rpc error: code = NotFound desc = could not find container \"00f3f973469ff192cd8cb1d83568e64aba1e1425b25bd40badaebff372daa9da\": container with ID starting with 00f3f973469ff192cd8cb1d83568e64aba1e1425b25bd40badaebff372daa9da not found: ID does not exist"
Sep 19 06:01:09.303684 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:09.303624    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-c85316-7f7d79c99-2tz2f"]
Sep 19 06:01:09.308205 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:09.308151    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-c85316-7f7d79c99-2tz2f"]
Sep 19 06:01:09.326299 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:09.326272    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7ftjc\" (UniqueName: \"kubernetes.io/projected/d1e19012-831d-430e-b711-c2b0a4665df8-kube-api-access-7ftjc\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 06:01:09.326299 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:09.326295    2583 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d1e19012-831d-430e-b711-c2b0a4665df8-metrics-certs\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 06:01:09.326484 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:09.326306    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d1e19012-831d-430e-b711-c2b0a4665df8-maas-api-tls\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 06:01:09.499247 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:09.499162    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d1e19012-831d-430e-b711-c2b0a4665df8" path="/var/lib/kubelet/pods/d1e19012-831d-430e-b711-c2b0a4665df8/volumes"
Sep 19 06:01:14.143511 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:14.143472    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-3182b5/e2e-conflict-b-95222d07-kserve-5fdbcc596d-hd8vs"]
Sep 19 06:01:14.144112 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:14.144092    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d1e19012-831d-430e-b711-c2b0a4665df8" containerName="maas-api"
Sep 19 06:01:14.144241 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:14.144116    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="d1e19012-831d-430e-b711-c2b0a4665df8" containerName="maas-api"
Sep 19 06:01:14.144241 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:14.144164    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="57450d37-8096-4ff3-a23f-5ce98d6cbc22" containerName="istio-proxy"
Sep 19 06:01:14.144241 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:14.144173    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="57450d37-8096-4ff3-a23f-5ce98d6cbc22" containerName="istio-proxy"
Sep 19 06:01:14.144241 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:14.144195    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="57450d37-8096-4ff3-a23f-5ce98d6cbc22" containerName="istio-proxy"
Sep 19 06:01:14.144241 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:14.144204    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="57450d37-8096-4ff3-a23f-5ce98d6cbc22" containerName="istio-proxy"
Sep 19 06:01:14.144488 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:14.144318    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="57450d37-8096-4ff3-a23f-5ce98d6cbc22" containerName="istio-proxy"
Sep 19 06:01:14.144488 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:14.144331    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="57450d37-8096-4ff3-a23f-5ce98d6cbc22" containerName="istio-proxy"
Sep 19 06:01:14.144488 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:14.144345    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="d1e19012-831d-430e-b711-c2b0a4665df8" containerName="maas-api"
Sep 19 06:01:14.148767 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:14.148742    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-3182b5/e2e-conflict-b-95222d07-kserve-5fdbcc596d-hd8vs"
Sep 19 06:01:14.151615 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:14.151595    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-3182b5\"/\"openshift-service-ca.crt\""
Sep 19 06:01:14.152519 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:14.152498    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-3182b5\"/\"default-dockercfg-2h5kn\""
Sep 19 06:01:14.152630 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:14.152498    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-3182b5\"/\"e2e-conflict-b-95222d07-kserve-self-signed-certs\""
Sep 19 06:01:14.152630 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:14.152497    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-3182b5\"/\"kube-root-ca.crt\""
Sep 19 06:01:14.160300 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:14.160277    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-3182b5/e2e-conflict-b-95222d07-kserve-5fdbcc596d-hd8vs"]
Sep 19 06:01:14.272958 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:14.272929    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ca7338c6-7d7d-4f91-97a3-4f12d048d2dc-model-cache\") pod \"e2e-conflict-b-95222d07-kserve-5fdbcc596d-hd8vs\" (UID: \"ca7338c6-7d7d-4f91-97a3-4f12d048d2dc\") " pod="e2e-models-e2e-worker-w4-3182b5/e2e-conflict-b-95222d07-kserve-5fdbcc596d-hd8vs"
Sep 19 06:01:14.273125 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:14.272974    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ca7338c6-7d7d-4f91-97a3-4f12d048d2dc-dshm\") pod \"e2e-conflict-b-95222d07-kserve-5fdbcc596d-hd8vs\" (UID: \"ca7338c6-7d7d-4f91-97a3-4f12d048d2dc\") " pod="e2e-models-e2e-worker-w4-3182b5/e2e-conflict-b-95222d07-kserve-5fdbcc596d-hd8vs"
Sep 19 06:01:14.273125 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:14.273052    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ca7338c6-7d7d-4f91-97a3-4f12d048d2dc-tmp-dir\") pod \"e2e-conflict-b-95222d07-kserve-5fdbcc596d-hd8vs\" (UID: \"ca7338c6-7d7d-4f91-97a3-4f12d048d2dc\") " pod="e2e-models-e2e-worker-w4-3182b5/e2e-conflict-b-95222d07-kserve-5fdbcc596d-hd8vs"
Sep 19 06:01:14.273125 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:14.273074    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ca7338c6-7d7d-4f91-97a3-4f12d048d2dc-home\") pod \"e2e-conflict-b-95222d07-kserve-5fdbcc596d-hd8vs\" (UID: \"ca7338c6-7d7d-4f91-97a3-4f12d048d2dc\") " pod="e2e-models-e2e-worker-w4-3182b5/e2e-conflict-b-95222d07-kserve-5fdbcc596d-hd8vs"
Sep 19 06:01:14.273274 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:14.273184    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d8wlk\" (UniqueName: \"kubernetes.io/projected/ca7338c6-7d7d-4f91-97a3-4f12d048d2dc-kube-api-access-d8wlk\") pod \"e2e-conflict-b-95222d07-kserve-5fdbcc596d-hd8vs\" (UID: \"ca7338c6-7d7d-4f91-97a3-4f12d048d2dc\") " pod="e2e-models-e2e-worker-w4-3182b5/e2e-conflict-b-95222d07-kserve-5fdbcc596d-hd8vs"
Sep 19 06:01:14.273274 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:14.273241    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ca7338c6-7d7d-4f91-97a3-4f12d048d2dc-tls-certs\") pod \"e2e-conflict-b-95222d07-kserve-5fdbcc596d-hd8vs\" (UID: \"ca7338c6-7d7d-4f91-97a3-4f12d048d2dc\") " pod="e2e-models-e2e-worker-w4-3182b5/e2e-conflict-b-95222d07-kserve-5fdbcc596d-hd8vs"
Sep 19 06:01:14.374414 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:14.374380    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d8wlk\" (UniqueName: \"kubernetes.io/projected/ca7338c6-7d7d-4f91-97a3-4f12d048d2dc-kube-api-access-d8wlk\") pod \"e2e-conflict-b-95222d07-kserve-5fdbcc596d-hd8vs\" (UID: \"ca7338c6-7d7d-4f91-97a3-4f12d048d2dc\") " pod="e2e-models-e2e-worker-w4-3182b5/e2e-conflict-b-95222d07-kserve-5fdbcc596d-hd8vs"
Sep 19 06:01:14.374604 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:14.374427    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ca7338c6-7d7d-4f91-97a3-4f12d048d2dc-tls-certs\") pod \"e2e-conflict-b-95222d07-kserve-5fdbcc596d-hd8vs\" (UID: \"ca7338c6-7d7d-4f91-97a3-4f12d048d2dc\") " pod="e2e-models-e2e-worker-w4-3182b5/e2e-conflict-b-95222d07-kserve-5fdbcc596d-hd8vs"
Sep 19 06:01:14.374604 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:14.374459    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ca7338c6-7d7d-4f91-97a3-4f12d048d2dc-model-cache\") pod \"e2e-conflict-b-95222d07-kserve-5fdbcc596d-hd8vs\" (UID: \"ca7338c6-7d7d-4f91-97a3-4f12d048d2dc\") " pod="e2e-models-e2e-worker-w4-3182b5/e2e-conflict-b-95222d07-kserve-5fdbcc596d-hd8vs"
Sep 19 06:01:14.374604 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:14.374490    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ca7338c6-7d7d-4f91-97a3-4f12d048d2dc-dshm\") pod \"e2e-conflict-b-95222d07-kserve-5fdbcc596d-hd8vs\" (UID: \"ca7338c6-7d7d-4f91-97a3-4f12d048d2dc\") " pod="e2e-models-e2e-worker-w4-3182b5/e2e-conflict-b-95222d07-kserve-5fdbcc596d-hd8vs"
Sep 19 06:01:14.374604 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:14.374518    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ca7338c6-7d7d-4f91-97a3-4f12d048d2dc-tmp-dir\") pod \"e2e-conflict-b-95222d07-kserve-5fdbcc596d-hd8vs\" (UID: \"ca7338c6-7d7d-4f91-97a3-4f12d048d2dc\") " pod="e2e-models-e2e-worker-w4-3182b5/e2e-conflict-b-95222d07-kserve-5fdbcc596d-hd8vs"
Sep 19 06:01:14.374604 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:14.374535    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ca7338c6-7d7d-4f91-97a3-4f12d048d2dc-home\") pod \"e2e-conflict-b-95222d07-kserve-5fdbcc596d-hd8vs\" (UID: \"ca7338c6-7d7d-4f91-97a3-4f12d048d2dc\") " pod="e2e-models-e2e-worker-w4-3182b5/e2e-conflict-b-95222d07-kserve-5fdbcc596d-hd8vs"
Sep 19 06:01:14.374936 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:14.374917    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ca7338c6-7d7d-4f91-97a3-4f12d048d2dc-home\") pod \"e2e-conflict-b-95222d07-kserve-5fdbcc596d-hd8vs\" (UID: \"ca7338c6-7d7d-4f91-97a3-4f12d048d2dc\") " pod="e2e-models-e2e-worker-w4-3182b5/e2e-conflict-b-95222d07-kserve-5fdbcc596d-hd8vs"
Sep 19 06:01:14.375005 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:14.374944    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ca7338c6-7d7d-4f91-97a3-4f12d048d2dc-model-cache\") pod \"e2e-conflict-b-95222d07-kserve-5fdbcc596d-hd8vs\" (UID: \"ca7338c6-7d7d-4f91-97a3-4f12d048d2dc\") " pod="e2e-models-e2e-worker-w4-3182b5/e2e-conflict-b-95222d07-kserve-5fdbcc596d-hd8vs"
Sep 19 06:01:14.375005 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:14.374989    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ca7338c6-7d7d-4f91-97a3-4f12d048d2dc-tmp-dir\") pod \"e2e-conflict-b-95222d07-kserve-5fdbcc596d-hd8vs\" (UID: \"ca7338c6-7d7d-4f91-97a3-4f12d048d2dc\") " pod="e2e-models-e2e-worker-w4-3182b5/e2e-conflict-b-95222d07-kserve-5fdbcc596d-hd8vs"
Sep 19 06:01:14.376849 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:14.376824    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ca7338c6-7d7d-4f91-97a3-4f12d048d2dc-dshm\") pod \"e2e-conflict-b-95222d07-kserve-5fdbcc596d-hd8vs\" (UID: \"ca7338c6-7d7d-4f91-97a3-4f12d048d2dc\") " pod="e2e-models-e2e-worker-w4-3182b5/e2e-conflict-b-95222d07-kserve-5fdbcc596d-hd8vs"
Sep 19 06:01:14.377076 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:14.377058    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ca7338c6-7d7d-4f91-97a3-4f12d048d2dc-tls-certs\") pod \"e2e-conflict-b-95222d07-kserve-5fdbcc596d-hd8vs\" (UID: \"ca7338c6-7d7d-4f91-97a3-4f12d048d2dc\") " pod="e2e-models-e2e-worker-w4-3182b5/e2e-conflict-b-95222d07-kserve-5fdbcc596d-hd8vs"
Sep 19 06:01:14.382703 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:14.382676    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d8wlk\" (UniqueName: \"kubernetes.io/projected/ca7338c6-7d7d-4f91-97a3-4f12d048d2dc-kube-api-access-d8wlk\") pod \"e2e-conflict-b-95222d07-kserve-5fdbcc596d-hd8vs\" (UID: \"ca7338c6-7d7d-4f91-97a3-4f12d048d2dc\") " pod="e2e-models-e2e-worker-w4-3182b5/e2e-conflict-b-95222d07-kserve-5fdbcc596d-hd8vs"
Sep 19 06:01:14.460495 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:14.460409    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-3182b5/e2e-conflict-b-95222d07-kserve-5fdbcc596d-hd8vs"
Sep 19 06:01:14.735607 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:14.735571    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-3182b5/e2e-conflict-b-95222d07-kserve-5fdbcc596d-hd8vs"]
Sep 19 06:01:14.737188 ip-10-0-139-144 kubenswrapper[2583]: W0919 06:01:14.737164    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podca7338c6_7d7d_4f91_97a3_4f12d048d2dc.slice/crio-19043a7b3021ff6025783b61abde19c98b0e50948798b25ce03e8ecbd8591700 WatchSource:0}: Error finding container 19043a7b3021ff6025783b61abde19c98b0e50948798b25ce03e8ecbd8591700: Status 404 returned error can't find the container with id 19043a7b3021ff6025783b61abde19c98b0e50948798b25ce03e8ecbd8591700
Sep 19 06:01:15.310278 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:15.310245    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-3182b5/e2e-conflict-b-95222d07-kserve-5fdbcc596d-hd8vs" event={"ID":"ca7338c6-7d7d-4f91-97a3-4f12d048d2dc","Type":"ContainerStarted","Data":"df4e50b1e2b59ce5fb09590e6486fda09bc481b2d3fac2d7b712c45cc6c614f4"}
Sep 19 06:01:15.310677 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:15.310287    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-3182b5/e2e-conflict-b-95222d07-kserve-5fdbcc596d-hd8vs" event={"ID":"ca7338c6-7d7d-4f91-97a3-4f12d048d2dc","Type":"ContainerStarted","Data":"19043a7b3021ff6025783b61abde19c98b0e50948798b25ce03e8ecbd8591700"}
Sep 19 06:01:15.351689 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:15.351638    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-3182b5/e2e-conflict-b-95222d07-kserve-5fdbcc596d-hd8vs" podStartSLOduration=1.351623703 podStartE2EDuration="1.351623703s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 06:01:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 06:01:15.349288999 +0000 UTC m=+1234.007743566" watchObservedRunningTime="2026-09-19 06:01:15.351623703 +0000 UTC m=+1234.010078260"
Sep 19 06:01:15.797165 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:15.797113    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-c85316-openshift-default-57bfb64984-xgk69"]
Sep 19 06:01:15.797464 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:15.797417    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-c85316-openshift-default-57bfb64984-xgk69" podUID="3b2632bb-bcca-400d-835f-197701450eeb" containerName="istio-proxy" containerID="cri-o://a3868f8b128e9a8dbfdd2f586b6243eae4a956c506d129230a1ac273abac5235" gracePeriod=30
Sep 19 06:01:16.709305 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:16.709269    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w3-888f75-openshift-default-599bd5457b-8mhn2"]
Sep 19 06:01:16.714232 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:16.714210    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-888f75-openshift-default-599bd5457b-8mhn2"
Sep 19 06:01:16.716818 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:16.716798    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w3-888f75-openshift-default-dockercfg-xxspr\""
Sep 19 06:01:16.725323 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:16.725295    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-888f75-openshift-default-599bd5457b-8mhn2"]
Sep 19 06:01:16.804904 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:16.804860    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/21ad9ff3-d47f-477a-ac4a-176a0258ec36-istio-token\") pod \"e2e-shared-a-w3-888f75-openshift-default-599bd5457b-8mhn2\" (UID: \"21ad9ff3-d47f-477a-ac4a-176a0258ec36\") " pod="openshift-ingress/e2e-shared-a-w3-888f75-openshift-default-599bd5457b-8mhn2"
Sep 19 06:01:16.805095 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:16.804930    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/21ad9ff3-d47f-477a-ac4a-176a0258ec36-credential-socket\") pod \"e2e-shared-a-w3-888f75-openshift-default-599bd5457b-8mhn2\" (UID: \"21ad9ff3-d47f-477a-ac4a-176a0258ec36\") " pod="openshift-ingress/e2e-shared-a-w3-888f75-openshift-default-599bd5457b-8mhn2"
Sep 19 06:01:16.805095 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:16.804964    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/21ad9ff3-d47f-477a-ac4a-176a0258ec36-istiod-ca-cert\") pod \"e2e-shared-a-w3-888f75-openshift-default-599bd5457b-8mhn2\" (UID: \"21ad9ff3-d47f-477a-ac4a-176a0258ec36\") " pod="openshift-ingress/e2e-shared-a-w3-888f75-openshift-default-599bd5457b-8mhn2"
Sep 19 06:01:16.805095 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:16.804997    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/21ad9ff3-d47f-477a-ac4a-176a0258ec36-istio-podinfo\") pod \"e2e-shared-a-w3-888f75-openshift-default-599bd5457b-8mhn2\" (UID: \"21ad9ff3-d47f-477a-ac4a-176a0258ec36\") " pod="openshift-ingress/e2e-shared-a-w3-888f75-openshift-default-599bd5457b-8mhn2"
Sep 19 06:01:16.805095 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:16.805038    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/21ad9ff3-d47f-477a-ac4a-176a0258ec36-istio-envoy\") pod \"e2e-shared-a-w3-888f75-openshift-default-599bd5457b-8mhn2\" (UID: \"21ad9ff3-d47f-477a-ac4a-176a0258ec36\") " pod="openshift-ingress/e2e-shared-a-w3-888f75-openshift-default-599bd5457b-8mhn2"
Sep 19 06:01:16.805346 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:16.805108    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/21ad9ff3-d47f-477a-ac4a-176a0258ec36-istio-data\") pod \"e2e-shared-a-w3-888f75-openshift-default-599bd5457b-8mhn2\" (UID: \"21ad9ff3-d47f-477a-ac4a-176a0258ec36\") " pod="openshift-ingress/e2e-shared-a-w3-888f75-openshift-default-599bd5457b-8mhn2"
Sep 19 06:01:16.805346 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:16.805168    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cxk9p\" (UniqueName: \"kubernetes.io/projected/21ad9ff3-d47f-477a-ac4a-176a0258ec36-kube-api-access-cxk9p\") pod \"e2e-shared-a-w3-888f75-openshift-default-599bd5457b-8mhn2\" (UID: \"21ad9ff3-d47f-477a-ac4a-176a0258ec36\") " pod="openshift-ingress/e2e-shared-a-w3-888f75-openshift-default-599bd5457b-8mhn2"
Sep 19 06:01:16.805346 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:16.805233    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/21ad9ff3-d47f-477a-ac4a-176a0258ec36-workload-socket\") pod \"e2e-shared-a-w3-888f75-openshift-default-599bd5457b-8mhn2\" (UID: \"21ad9ff3-d47f-477a-ac4a-176a0258ec36\") " pod="openshift-ingress/e2e-shared-a-w3-888f75-openshift-default-599bd5457b-8mhn2"
Sep 19 06:01:16.805346 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:16.805290    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/21ad9ff3-d47f-477a-ac4a-176a0258ec36-workload-certs\") pod \"e2e-shared-a-w3-888f75-openshift-default-599bd5457b-8mhn2\" (UID: \"21ad9ff3-d47f-477a-ac4a-176a0258ec36\") " pod="openshift-ingress/e2e-shared-a-w3-888f75-openshift-default-599bd5457b-8mhn2"
Sep 19 06:01:16.906300 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:16.906262    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/21ad9ff3-d47f-477a-ac4a-176a0258ec36-istio-data\") pod \"e2e-shared-a-w3-888f75-openshift-default-599bd5457b-8mhn2\" (UID: \"21ad9ff3-d47f-477a-ac4a-176a0258ec36\") " pod="openshift-ingress/e2e-shared-a-w3-888f75-openshift-default-599bd5457b-8mhn2"
Sep 19 06:01:16.906462 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:16.906308    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cxk9p\" (UniqueName: \"kubernetes.io/projected/21ad9ff3-d47f-477a-ac4a-176a0258ec36-kube-api-access-cxk9p\") pod \"e2e-shared-a-w3-888f75-openshift-default-599bd5457b-8mhn2\" (UID: \"21ad9ff3-d47f-477a-ac4a-176a0258ec36\") " pod="openshift-ingress/e2e-shared-a-w3-888f75-openshift-default-599bd5457b-8mhn2"
Sep 19 06:01:16.906462 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:16.906353    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/21ad9ff3-d47f-477a-ac4a-176a0258ec36-workload-socket\") pod \"e2e-shared-a-w3-888f75-openshift-default-599bd5457b-8mhn2\" (UID: \"21ad9ff3-d47f-477a-ac4a-176a0258ec36\") " pod="openshift-ingress/e2e-shared-a-w3-888f75-openshift-default-599bd5457b-8mhn2"
Sep 19 06:01:16.906462 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:16.906385    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/21ad9ff3-d47f-477a-ac4a-176a0258ec36-workload-certs\") pod \"e2e-shared-a-w3-888f75-openshift-default-599bd5457b-8mhn2\" (UID: \"21ad9ff3-d47f-477a-ac4a-176a0258ec36\") " pod="openshift-ingress/e2e-shared-a-w3-888f75-openshift-default-599bd5457b-8mhn2"
Sep 19 06:01:16.906462 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:16.906428    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/21ad9ff3-d47f-477a-ac4a-176a0258ec36-istio-token\") pod \"e2e-shared-a-w3-888f75-openshift-default-599bd5457b-8mhn2\" (UID: \"21ad9ff3-d47f-477a-ac4a-176a0258ec36\") " pod="openshift-ingress/e2e-shared-a-w3-888f75-openshift-default-599bd5457b-8mhn2"
Sep 19 06:01:16.906627 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:16.906461    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/21ad9ff3-d47f-477a-ac4a-176a0258ec36-credential-socket\") pod \"e2e-shared-a-w3-888f75-openshift-default-599bd5457b-8mhn2\" (UID: \"21ad9ff3-d47f-477a-ac4a-176a0258ec36\") " pod="openshift-ingress/e2e-shared-a-w3-888f75-openshift-default-599bd5457b-8mhn2"
Sep 19 06:01:16.906627 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:16.906559    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/21ad9ff3-d47f-477a-ac4a-176a0258ec36-istiod-ca-cert\") pod \"e2e-shared-a-w3-888f75-openshift-default-599bd5457b-8mhn2\" (UID: \"21ad9ff3-d47f-477a-ac4a-176a0258ec36\") " pod="openshift-ingress/e2e-shared-a-w3-888f75-openshift-default-599bd5457b-8mhn2"
Sep 19 06:01:16.906627 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:16.906594    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/21ad9ff3-d47f-477a-ac4a-176a0258ec36-istio-podinfo\") pod \"e2e-shared-a-w3-888f75-openshift-default-599bd5457b-8mhn2\" (UID: \"21ad9ff3-d47f-477a-ac4a-176a0258ec36\") " pod="openshift-ingress/e2e-shared-a-w3-888f75-openshift-default-599bd5457b-8mhn2"
Sep 19 06:01:16.906782 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:16.906635    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/21ad9ff3-d47f-477a-ac4a-176a0258ec36-istio-envoy\") pod \"e2e-shared-a-w3-888f75-openshift-default-599bd5457b-8mhn2\" (UID: \"21ad9ff3-d47f-477a-ac4a-176a0258ec36\") " pod="openshift-ingress/e2e-shared-a-w3-888f75-openshift-default-599bd5457b-8mhn2"
Sep 19 06:01:16.906782 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:16.906739    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/21ad9ff3-d47f-477a-ac4a-176a0258ec36-istio-data\") pod \"e2e-shared-a-w3-888f75-openshift-default-599bd5457b-8mhn2\" (UID: \"21ad9ff3-d47f-477a-ac4a-176a0258ec36\") " pod="openshift-ingress/e2e-shared-a-w3-888f75-openshift-default-599bd5457b-8mhn2"
Sep 19 06:01:16.906892 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:16.906787    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/21ad9ff3-d47f-477a-ac4a-176a0258ec36-workload-socket\") pod \"e2e-shared-a-w3-888f75-openshift-default-599bd5457b-8mhn2\" (UID: \"21ad9ff3-d47f-477a-ac4a-176a0258ec36\") " pod="openshift-ingress/e2e-shared-a-w3-888f75-openshift-default-599bd5457b-8mhn2"
Sep 19 06:01:16.906892 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:16.906839    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/21ad9ff3-d47f-477a-ac4a-176a0258ec36-workload-certs\") pod \"e2e-shared-a-w3-888f75-openshift-default-599bd5457b-8mhn2\" (UID: \"21ad9ff3-d47f-477a-ac4a-176a0258ec36\") " pod="openshift-ingress/e2e-shared-a-w3-888f75-openshift-default-599bd5457b-8mhn2"
Sep 19 06:01:16.907306 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:16.907281    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/21ad9ff3-d47f-477a-ac4a-176a0258ec36-credential-socket\") pod \"e2e-shared-a-w3-888f75-openshift-default-599bd5457b-8mhn2\" (UID: \"21ad9ff3-d47f-477a-ac4a-176a0258ec36\") " pod="openshift-ingress/e2e-shared-a-w3-888f75-openshift-default-599bd5457b-8mhn2"
Sep 19 06:01:16.907516 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:16.907470    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/21ad9ff3-d47f-477a-ac4a-176a0258ec36-istiod-ca-cert\") pod \"e2e-shared-a-w3-888f75-openshift-default-599bd5457b-8mhn2\" (UID: \"21ad9ff3-d47f-477a-ac4a-176a0258ec36\") " pod="openshift-ingress/e2e-shared-a-w3-888f75-openshift-default-599bd5457b-8mhn2"
Sep 19 06:01:16.909063 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:16.909045    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/21ad9ff3-d47f-477a-ac4a-176a0258ec36-istio-envoy\") pod \"e2e-shared-a-w3-888f75-openshift-default-599bd5457b-8mhn2\" (UID: \"21ad9ff3-d47f-477a-ac4a-176a0258ec36\") " pod="openshift-ingress/e2e-shared-a-w3-888f75-openshift-default-599bd5457b-8mhn2"
Sep 19 06:01:16.909180 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:16.909160    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/21ad9ff3-d47f-477a-ac4a-176a0258ec36-istio-podinfo\") pod \"e2e-shared-a-w3-888f75-openshift-default-599bd5457b-8mhn2\" (UID: \"21ad9ff3-d47f-477a-ac4a-176a0258ec36\") " pod="openshift-ingress/e2e-shared-a-w3-888f75-openshift-default-599bd5457b-8mhn2"
Sep 19 06:01:16.916590 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:16.916562    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/21ad9ff3-d47f-477a-ac4a-176a0258ec36-istio-token\") pod \"e2e-shared-a-w3-888f75-openshift-default-599bd5457b-8mhn2\" (UID: \"21ad9ff3-d47f-477a-ac4a-176a0258ec36\") " pod="openshift-ingress/e2e-shared-a-w3-888f75-openshift-default-599bd5457b-8mhn2"
Sep 19 06:01:16.916924 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:16.916903    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cxk9p\" (UniqueName: \"kubernetes.io/projected/21ad9ff3-d47f-477a-ac4a-176a0258ec36-kube-api-access-cxk9p\") pod \"e2e-shared-a-w3-888f75-openshift-default-599bd5457b-8mhn2\" (UID: \"21ad9ff3-d47f-477a-ac4a-176a0258ec36\") " pod="openshift-ingress/e2e-shared-a-w3-888f75-openshift-default-599bd5457b-8mhn2"
Sep 19 06:01:17.027543 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:17.027515    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-888f75-openshift-default-599bd5457b-8mhn2"
Sep 19 06:01:17.305014 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:17.304986    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-888f75-openshift-default-599bd5457b-8mhn2"]
Sep 19 06:01:17.306835 ip-10-0-139-144 kubenswrapper[2583]: W0919 06:01:17.306808    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21ad9ff3_d47f_477a_ac4a_176a0258ec36.slice/crio-33b06c89ec68e40a922033855fc8f6fe41db2aca2577397e9297255978dbace3 WatchSource:0}: Error finding container 33b06c89ec68e40a922033855fc8f6fe41db2aca2577397e9297255978dbace3: Status 404 returned error can't find the container with id 33b06c89ec68e40a922033855fc8f6fe41db2aca2577397e9297255978dbace3
Sep 19 06:01:17.309152 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:17.309110    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 19 06:01:17.309216 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:17.309188    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 19 06:01:17.309266 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:17.309220    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 19 06:01:17.322808 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:17.322779    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-888f75-openshift-default-599bd5457b-8mhn2" event={"ID":"21ad9ff3-d47f-477a-ac4a-176a0258ec36","Type":"ContainerStarted","Data":"33b06c89ec68e40a922033855fc8f6fe41db2aca2577397e9297255978dbace3"}
Sep 19 06:01:18.330776 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:18.330739    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-888f75-openshift-default-599bd5457b-8mhn2" event={"ID":"21ad9ff3-d47f-477a-ac4a-176a0258ec36","Type":"ContainerStarted","Data":"05f733480345e86473ec333947be6bfa9ec4c4d6beeaf41ab4e4952b7e5e298b"}
Sep 19 06:01:18.354521 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:18.354471    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w3-888f75-openshift-default-599bd5457b-8mhn2" podStartSLOduration=2.354456319 podStartE2EDuration="2.354456319s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 06:01:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 06:01:18.351902518 +0000 UTC m=+1237.010357077" watchObservedRunningTime="2026-09-19 06:01:18.354456319 +0000 UTC m=+1237.012910926"
Sep 19 06:01:19.028061 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:19.028023    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-888f75-openshift-default-599bd5457b-8mhn2"
Sep 19 06:01:19.032681 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:19.032655    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-888f75-openshift-default-599bd5457b-8mhn2"
Sep 19 06:01:19.336332 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:19.336240    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-888f75-openshift-default-599bd5457b-8mhn2"
Sep 19 06:01:19.337057 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:19.337037    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-888f75-openshift-default-599bd5457b-8mhn2"
Sep 19 06:01:21.046857 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:21.046819    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-c85316-openshift-default-57bfb64984-xgk69"
Sep 19 06:01:21.148881 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:21.148849    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bzckb\" (UniqueName: \"kubernetes.io/projected/3b2632bb-bcca-400d-835f-197701450eeb-kube-api-access-bzckb\") pod \"3b2632bb-bcca-400d-835f-197701450eeb\" (UID: \"3b2632bb-bcca-400d-835f-197701450eeb\") "
Sep 19 06:01:21.148881 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:21.148887    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3b2632bb-bcca-400d-835f-197701450eeb-istio-podinfo\") pod \"3b2632bb-bcca-400d-835f-197701450eeb\" (UID: \"3b2632bb-bcca-400d-835f-197701450eeb\") "
Sep 19 06:01:21.149156 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:21.148915    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3b2632bb-bcca-400d-835f-197701450eeb-istio-token\") pod \"3b2632bb-bcca-400d-835f-197701450eeb\" (UID: \"3b2632bb-bcca-400d-835f-197701450eeb\") "
Sep 19 06:01:21.149156 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:21.148942    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3b2632bb-bcca-400d-835f-197701450eeb-istio-envoy\") pod \"3b2632bb-bcca-400d-835f-197701450eeb\" (UID: \"3b2632bb-bcca-400d-835f-197701450eeb\") "
Sep 19 06:01:21.149156 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:21.148998    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3b2632bb-bcca-400d-835f-197701450eeb-istio-data\") pod \"3b2632bb-bcca-400d-835f-197701450eeb\" (UID: \"3b2632bb-bcca-400d-835f-197701450eeb\") "
Sep 19 06:01:21.149156 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:21.149035    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b2632bb-bcca-400d-835f-197701450eeb-credential-socket\") pod \"3b2632bb-bcca-400d-835f-197701450eeb\" (UID: \"3b2632bb-bcca-400d-835f-197701450eeb\") "
Sep 19 06:01:21.149406 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:21.149166    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3b2632bb-bcca-400d-835f-197701450eeb-istiod-ca-cert\") pod \"3b2632bb-bcca-400d-835f-197701450eeb\" (UID: \"3b2632bb-bcca-400d-835f-197701450eeb\") "
Sep 19 06:01:21.149406 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:21.149192    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b2632bb-bcca-400d-835f-197701450eeb-workload-socket\") pod \"3b2632bb-bcca-400d-835f-197701450eeb\" (UID: \"3b2632bb-bcca-400d-835f-197701450eeb\") "
Sep 19 06:01:21.149406 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:21.149231    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3b2632bb-bcca-400d-835f-197701450eeb-workload-certs\") pod \"3b2632bb-bcca-400d-835f-197701450eeb\" (UID: \"3b2632bb-bcca-400d-835f-197701450eeb\") "
Sep 19 06:01:21.149406 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:21.149369    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b2632bb-bcca-400d-835f-197701450eeb-istio-data" (OuterVolumeSpecName: "istio-data") pod "3b2632bb-bcca-400d-835f-197701450eeb" (UID: "3b2632bb-bcca-400d-835f-197701450eeb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 06:01:21.149602 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:21.149420    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b2632bb-bcca-400d-835f-197701450eeb-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3b2632bb-bcca-400d-835f-197701450eeb" (UID: "3b2632bb-bcca-400d-835f-197701450eeb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 06:01:21.149696 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:21.149642    2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3b2632bb-bcca-400d-835f-197701450eeb-istio-data\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 06:01:21.149696 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:21.149664    2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b2632bb-bcca-400d-835f-197701450eeb-credential-socket\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 06:01:21.149696 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:21.149677    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b2632bb-bcca-400d-835f-197701450eeb-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3b2632bb-bcca-400d-835f-197701450eeb" (UID: "3b2632bb-bcca-400d-835f-197701450eeb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 06:01:21.149856 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:21.149704    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b2632bb-bcca-400d-835f-197701450eeb-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3b2632bb-bcca-400d-835f-197701450eeb" (UID: "3b2632bb-bcca-400d-835f-197701450eeb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 06:01:21.149856 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:21.149723    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3b2632bb-bcca-400d-835f-197701450eeb-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3b2632bb-bcca-400d-835f-197701450eeb" (UID: "3b2632bb-bcca-400d-835f-197701450eeb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 06:01:21.151181 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:21.151154    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b2632bb-bcca-400d-835f-197701450eeb-istio-token" (OuterVolumeSpecName: "istio-token") pod "3b2632bb-bcca-400d-835f-197701450eeb" (UID: "3b2632bb-bcca-400d-835f-197701450eeb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 06:01:21.151308 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:21.151241    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3b2632bb-bcca-400d-835f-197701450eeb-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3b2632bb-bcca-400d-835f-197701450eeb" (UID: "3b2632bb-bcca-400d-835f-197701450eeb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 19 06:01:21.151353 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:21.151317    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b2632bb-bcca-400d-835f-197701450eeb-kube-api-access-bzckb" (OuterVolumeSpecName: "kube-api-access-bzckb") pod "3b2632bb-bcca-400d-835f-197701450eeb" (UID: "3b2632bb-bcca-400d-835f-197701450eeb"). InnerVolumeSpecName "kube-api-access-bzckb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 06:01:21.151404 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:21.151388    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b2632bb-bcca-400d-835f-197701450eeb-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3b2632bb-bcca-400d-835f-197701450eeb" (UID: "3b2632bb-bcca-400d-835f-197701450eeb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 06:01:21.250831 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:21.250758    2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3b2632bb-bcca-400d-835f-197701450eeb-istiod-ca-cert\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 06:01:21.250831 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:21.250784    2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b2632bb-bcca-400d-835f-197701450eeb-workload-socket\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 06:01:21.250831 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:21.250793    2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3b2632bb-bcca-400d-835f-197701450eeb-workload-certs\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 06:01:21.250831 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:21.250802    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bzckb\" (UniqueName: \"kubernetes.io/projected/3b2632bb-bcca-400d-835f-197701450eeb-kube-api-access-bzckb\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 06:01:21.250831 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:21.250812    2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3b2632bb-bcca-400d-835f-197701450eeb-istio-podinfo\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 06:01:21.250831 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:21.250820    2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3b2632bb-bcca-400d-835f-197701450eeb-istio-token\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 06:01:21.250831 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:21.250828    2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3b2632bb-bcca-400d-835f-197701450eeb-istio-envoy\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 06:01:21.349225 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:21.349185    2583 generic.go:356] "Generic (PLEG): container finished" podID="3b2632bb-bcca-400d-835f-197701450eeb" containerID="a3868f8b128e9a8dbfdd2f586b6243eae4a956c506d129230a1ac273abac5235" exitCode=0
Sep 19 06:01:21.349392 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:21.349247    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-c85316-openshift-default-57bfb64984-xgk69"
Sep 19 06:01:21.349392 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:21.349269    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-c85316-openshift-default-57bfb64984-xgk69" event={"ID":"3b2632bb-bcca-400d-835f-197701450eeb","Type":"ContainerDied","Data":"a3868f8b128e9a8dbfdd2f586b6243eae4a956c506d129230a1ac273abac5235"}
Sep 19 06:01:21.349392 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:21.349321    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-c85316-openshift-default-57bfb64984-xgk69" event={"ID":"3b2632bb-bcca-400d-835f-197701450eeb","Type":"ContainerDied","Data":"56f646bbecad89ac3d30f994991c68c60f6072456632735bc1091abc3b16846a"}
Sep 19 06:01:21.349392 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:21.349343    2583 scope.go:117] "RemoveContainer" containerID="a3868f8b128e9a8dbfdd2f586b6243eae4a956c506d129230a1ac273abac5235"
Sep 19 06:01:21.359337 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:21.359319    2583 scope.go:117] "RemoveContainer" containerID="589b669105a8f4fc5c1ef39d7c000ffc9fcae2006395787366059c24a0d1c43b"
Sep 19 06:01:21.368272 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:21.368254    2583 scope.go:117] "RemoveContainer" containerID="a3868f8b128e9a8dbfdd2f586b6243eae4a956c506d129230a1ac273abac5235"
Sep 19 06:01:21.368544 ip-10-0-139-144 kubenswrapper[2583]: E0919 06:01:21.368526    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a3868f8b128e9a8dbfdd2f586b6243eae4a956c506d129230a1ac273abac5235\": container with ID starting with a3868f8b128e9a8dbfdd2f586b6243eae4a956c506d129230a1ac273abac5235 not found: ID does not exist" containerID="a3868f8b128e9a8dbfdd2f586b6243eae4a956c506d129230a1ac273abac5235"
Sep 19 06:01:21.368595 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:21.368554    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a3868f8b128e9a8dbfdd2f586b6243eae4a956c506d129230a1ac273abac5235"} err="failed to get container status \"a3868f8b128e9a8dbfdd2f586b6243eae4a956c506d129230a1ac273abac5235\": rpc error: code = NotFound desc = could not find container \"a3868f8b128e9a8dbfdd2f586b6243eae4a956c506d129230a1ac273abac5235\": container with ID starting with a3868f8b128e9a8dbfdd2f586b6243eae4a956c506d129230a1ac273abac5235 not found: ID does not exist"
Sep 19 06:01:21.368595 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:21.368570    2583 scope.go:117] "RemoveContainer" containerID="589b669105a8f4fc5c1ef39d7c000ffc9fcae2006395787366059c24a0d1c43b"
Sep 19 06:01:21.368805 ip-10-0-139-144 kubenswrapper[2583]: E0919 06:01:21.368784    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"589b669105a8f4fc5c1ef39d7c000ffc9fcae2006395787366059c24a0d1c43b\": container with ID starting with 589b669105a8f4fc5c1ef39d7c000ffc9fcae2006395787366059c24a0d1c43b not found: ID does not exist" containerID="589b669105a8f4fc5c1ef39d7c000ffc9fcae2006395787366059c24a0d1c43b"
Sep 19 06:01:21.368861 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:21.368815    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"589b669105a8f4fc5c1ef39d7c000ffc9fcae2006395787366059c24a0d1c43b"} err="failed to get container status \"589b669105a8f4fc5c1ef39d7c000ffc9fcae2006395787366059c24a0d1c43b\": rpc error: code = NotFound desc = could not find container \"589b669105a8f4fc5c1ef39d7c000ffc9fcae2006395787366059c24a0d1c43b\": container with ID starting with 589b669105a8f4fc5c1ef39d7c000ffc9fcae2006395787366059c24a0d1c43b not found: ID does not exist"
Sep 19 06:01:21.374585 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:21.374562    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-c85316-openshift-default-57bfb64984-xgk69"]
Sep 19 06:01:21.378372 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:21.378351    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-c85316-openshift-default-57bfb64984-xgk69"]
Sep 19 06:01:21.498738 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:21.498705    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3b2632bb-bcca-400d-835f-197701450eeb" path="/var/lib/kubelet/pods/3b2632bb-bcca-400d-835f-197701450eeb/volumes"
Sep 19 06:01:24.461365 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:24.461334    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-3182b5/e2e-conflict-b-95222d07-kserve-5fdbcc596d-hd8vs"
Sep 19 06:01:24.461775 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:24.461412    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-3182b5/e2e-conflict-b-95222d07-kserve-5fdbcc596d-hd8vs"
Sep 19 06:01:24.473970 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:24.473943    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-3182b5/e2e-conflict-b-95222d07-kserve-5fdbcc596d-hd8vs"
Sep 19 06:01:24.578604 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:24.578330    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-888f75-559c558bcc-nczl4"]
Sep 19 06:01:24.584463 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:24.584423    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b2632bb-bcca-400d-835f-197701450eeb" containerName="istio-proxy"
Sep 19 06:01:24.584463 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:24.584452    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b2632bb-bcca-400d-835f-197701450eeb" containerName="istio-proxy"
Sep 19 06:01:24.584463 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:24.584470    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b2632bb-bcca-400d-835f-197701450eeb" containerName="istio-proxy"
Sep 19 06:01:24.584730 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:24.584479    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b2632bb-bcca-400d-835f-197701450eeb" containerName="istio-proxy"
Sep 19 06:01:24.584730 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:24.584609    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="3b2632bb-bcca-400d-835f-197701450eeb" containerName="istio-proxy"
Sep 19 06:01:24.589586 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:24.589563    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-888f75-559c558bcc-nczl4"]
Sep 19 06:01:24.589714 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:24.589644    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-888f75-559c558bcc-nczl4"
Sep 19 06:01:24.592034 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:24.592009    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w3-888f75-serving-cert\""
Sep 19 06:01:24.685578 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:24.685541    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-66bbx\" (UniqueName: \"kubernetes.io/projected/98d89776-8cfe-4f8f-8441-16fd67eedd1f-kube-api-access-66bbx\") pod \"maas-api-e2e-shared-a-w3-888f75-559c558bcc-nczl4\" (UID: \"98d89776-8cfe-4f8f-8441-16fd67eedd1f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-888f75-559c558bcc-nczl4"
Sep 19 06:01:24.685786 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:24.685592    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/98d89776-8cfe-4f8f-8441-16fd67eedd1f-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-888f75-559c558bcc-nczl4\" (UID: \"98d89776-8cfe-4f8f-8441-16fd67eedd1f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-888f75-559c558bcc-nczl4"
Sep 19 06:01:24.685786 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:24.685622    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/98d89776-8cfe-4f8f-8441-16fd67eedd1f-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-888f75-559c558bcc-nczl4\" (UID: \"98d89776-8cfe-4f8f-8441-16fd67eedd1f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-888f75-559c558bcc-nczl4"
Sep 19 06:01:24.685786 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:24.685722    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/98d89776-8cfe-4f8f-8441-16fd67eedd1f-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-888f75-559c558bcc-nczl4\" (UID: \"98d89776-8cfe-4f8f-8441-16fd67eedd1f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-888f75-559c558bcc-nczl4"
Sep 19 06:01:24.786849 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:24.786818    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-66bbx\" (UniqueName: \"kubernetes.io/projected/98d89776-8cfe-4f8f-8441-16fd67eedd1f-kube-api-access-66bbx\") pod \"maas-api-e2e-shared-a-w3-888f75-559c558bcc-nczl4\" (UID: \"98d89776-8cfe-4f8f-8441-16fd67eedd1f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-888f75-559c558bcc-nczl4"
Sep 19 06:01:24.787047 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:24.786870    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/98d89776-8cfe-4f8f-8441-16fd67eedd1f-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-888f75-559c558bcc-nczl4\" (UID: \"98d89776-8cfe-4f8f-8441-16fd67eedd1f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-888f75-559c558bcc-nczl4"
Sep 19 06:01:24.787047 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:24.786900    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/98d89776-8cfe-4f8f-8441-16fd67eedd1f-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-888f75-559c558bcc-nczl4\" (UID: \"98d89776-8cfe-4f8f-8441-16fd67eedd1f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-888f75-559c558bcc-nczl4"
Sep 19 06:01:24.787047 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:24.786956    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/98d89776-8cfe-4f8f-8441-16fd67eedd1f-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-888f75-559c558bcc-nczl4\" (UID: \"98d89776-8cfe-4f8f-8441-16fd67eedd1f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-888f75-559c558bcc-nczl4"
Sep 19 06:01:24.787635 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:24.787604    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/98d89776-8cfe-4f8f-8441-16fd67eedd1f-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-888f75-559c558bcc-nczl4\" (UID: \"98d89776-8cfe-4f8f-8441-16fd67eedd1f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-888f75-559c558bcc-nczl4"
Sep 19 06:01:24.789376 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:24.789351    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/98d89776-8cfe-4f8f-8441-16fd67eedd1f-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-888f75-559c558bcc-nczl4\" (UID: \"98d89776-8cfe-4f8f-8441-16fd67eedd1f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-888f75-559c558bcc-nczl4"
Sep 19 06:01:24.789471 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:24.789397    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/98d89776-8cfe-4f8f-8441-16fd67eedd1f-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-888f75-559c558bcc-nczl4\" (UID: \"98d89776-8cfe-4f8f-8441-16fd67eedd1f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-888f75-559c558bcc-nczl4"
Sep 19 06:01:24.795217 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:24.795191    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-66bbx\" (UniqueName: \"kubernetes.io/projected/98d89776-8cfe-4f8f-8441-16fd67eedd1f-kube-api-access-66bbx\") pod \"maas-api-e2e-shared-a-w3-888f75-559c558bcc-nczl4\" (UID: \"98d89776-8cfe-4f8f-8441-16fd67eedd1f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-888f75-559c558bcc-nczl4"
Sep 19 06:01:24.901588 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:24.901541    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-888f75-559c558bcc-nczl4"
Sep 19 06:01:25.187654 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:25.187623    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-888f75-559c558bcc-nczl4"]
Sep 19 06:01:25.188998 ip-10-0-139-144 kubenswrapper[2583]: W0919 06:01:25.188971    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98d89776_8cfe_4f8f_8441_16fd67eedd1f.slice/crio-ff7eca5dbfc7f68bc407527e34b5dc3f202c7999cf87a3f1fdfaa8deb69e70af WatchSource:0}: Error finding container ff7eca5dbfc7f68bc407527e34b5dc3f202c7999cf87a3f1fdfaa8deb69e70af: Status 404 returned error can't find the container with id ff7eca5dbfc7f68bc407527e34b5dc3f202c7999cf87a3f1fdfaa8deb69e70af
Sep 19 06:01:25.374020 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:25.373940    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-888f75-559c558bcc-nczl4" event={"ID":"98d89776-8cfe-4f8f-8441-16fd67eedd1f","Type":"ContainerStarted","Data":"ff7eca5dbfc7f68bc407527e34b5dc3f202c7999cf87a3f1fdfaa8deb69e70af"}
Sep 19 06:01:25.385401 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:25.385376    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-3182b5/e2e-conflict-b-95222d07-kserve-5fdbcc596d-hd8vs"
Sep 19 06:01:26.381322 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:26.381281    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-888f75-559c558bcc-nczl4" event={"ID":"98d89776-8cfe-4f8f-8441-16fd67eedd1f","Type":"ContainerStarted","Data":"2014f118f5411ae5bd5763b06c39e714e4316d7b085157b4d31ecfc57d776be9"}
Sep 19 06:01:26.400649 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:26.400598    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-888f75-559c558bcc-nczl4" podStartSLOduration=1.8288939659999999 podStartE2EDuration="2.400582679s" totalImagesPullingTime="571.688713ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 06:01:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 06:01:26.39919058 +0000 UTC m=+1245.057645137" watchObservedRunningTime="2026-09-19 06:01:26.400582679 +0000 UTC m=+1245.059037237"
Sep 19 06:01:27.387032 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:27.386995    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-888f75-559c558bcc-nczl4"
Sep 19 06:01:33.398410 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:33.398379    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-888f75-559c558bcc-nczl4"
Sep 19 06:01:36.574164 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:36.574108    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-cc0e26/distinct-w0-cc0e26-kserve-6b7bdf546b-dqvpj"]
Sep 19 06:01:36.579583 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:36.579554    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-cc0e26/premium-w0-cc0e26-kserve-64f54b56f5-chxqb"]
Sep 19 06:01:40.861622 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:40.861589    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w3-f1aad1-openshift-default-c6fd9dd46-l98nd"]
Sep 19 06:01:40.862269 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:40.862253    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="3b2632bb-bcca-400d-835f-197701450eeb" containerName="istio-proxy"
Sep 19 06:01:40.866221 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:40.866197    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-f1aad1-openshift-default-c6fd9dd46-l98nd"
Sep 19 06:01:40.869376 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:40.869348    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w3-f1aad1-openshift-default-dockercfg-22887\""
Sep 19 06:01:40.877198 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:40.877173    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-f1aad1-openshift-default-c6fd9dd46-l98nd"]
Sep 19 06:01:40.945740 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:40.945704    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/726c78fe-d902-4d20-a882-1e0df84d795c-credential-socket\") pod \"e2e-shared-b-w3-f1aad1-openshift-default-c6fd9dd46-l98nd\" (UID: \"726c78fe-d902-4d20-a882-1e0df84d795c\") " pod="openshift-ingress/e2e-shared-b-w3-f1aad1-openshift-default-c6fd9dd46-l98nd"
Sep 19 06:01:40.945926 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:40.945755    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/726c78fe-d902-4d20-a882-1e0df84d795c-workload-certs\") pod \"e2e-shared-b-w3-f1aad1-openshift-default-c6fd9dd46-l98nd\" (UID: \"726c78fe-d902-4d20-a882-1e0df84d795c\") " pod="openshift-ingress/e2e-shared-b-w3-f1aad1-openshift-default-c6fd9dd46-l98nd"
Sep 19 06:01:40.945926 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:40.945809    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4bmkn\" (UniqueName: \"kubernetes.io/projected/726c78fe-d902-4d20-a882-1e0df84d795c-kube-api-access-4bmkn\") pod \"e2e-shared-b-w3-f1aad1-openshift-default-c6fd9dd46-l98nd\" (UID: \"726c78fe-d902-4d20-a882-1e0df84d795c\") " pod="openshift-ingress/e2e-shared-b-w3-f1aad1-openshift-default-c6fd9dd46-l98nd"
Sep 19 06:01:40.945926 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:40.945893    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/726c78fe-d902-4d20-a882-1e0df84d795c-istio-envoy\") pod \"e2e-shared-b-w3-f1aad1-openshift-default-c6fd9dd46-l98nd\" (UID: \"726c78fe-d902-4d20-a882-1e0df84d795c\") " pod="openshift-ingress/e2e-shared-b-w3-f1aad1-openshift-default-c6fd9dd46-l98nd"
Sep 19 06:01:40.946072 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:40.945947    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/726c78fe-d902-4d20-a882-1e0df84d795c-workload-socket\") pod \"e2e-shared-b-w3-f1aad1-openshift-default-c6fd9dd46-l98nd\" (UID: \"726c78fe-d902-4d20-a882-1e0df84d795c\") " pod="openshift-ingress/e2e-shared-b-w3-f1aad1-openshift-default-c6fd9dd46-l98nd"
Sep 19 06:01:40.946072 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:40.945973    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/726c78fe-d902-4d20-a882-1e0df84d795c-istio-podinfo\") pod \"e2e-shared-b-w3-f1aad1-openshift-default-c6fd9dd46-l98nd\" (UID: \"726c78fe-d902-4d20-a882-1e0df84d795c\") " pod="openshift-ingress/e2e-shared-b-w3-f1aad1-openshift-default-c6fd9dd46-l98nd"
Sep 19 06:01:40.946072 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:40.946020    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/726c78fe-d902-4d20-a882-1e0df84d795c-istio-token\") pod \"e2e-shared-b-w3-f1aad1-openshift-default-c6fd9dd46-l98nd\" (UID: \"726c78fe-d902-4d20-a882-1e0df84d795c\") " pod="openshift-ingress/e2e-shared-b-w3-f1aad1-openshift-default-c6fd9dd46-l98nd"
Sep 19 06:01:40.946262 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:40.946111    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/726c78fe-d902-4d20-a882-1e0df84d795c-istio-data\") pod \"e2e-shared-b-w3-f1aad1-openshift-default-c6fd9dd46-l98nd\" (UID: \"726c78fe-d902-4d20-a882-1e0df84d795c\") " pod="openshift-ingress/e2e-shared-b-w3-f1aad1-openshift-default-c6fd9dd46-l98nd"
Sep 19 06:01:40.946262 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:40.946180    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/726c78fe-d902-4d20-a882-1e0df84d795c-istiod-ca-cert\") pod \"e2e-shared-b-w3-f1aad1-openshift-default-c6fd9dd46-l98nd\" (UID: \"726c78fe-d902-4d20-a882-1e0df84d795c\") " pod="openshift-ingress/e2e-shared-b-w3-f1aad1-openshift-default-c6fd9dd46-l98nd"
Sep 19 06:01:41.047503 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:41.047462    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/726c78fe-d902-4d20-a882-1e0df84d795c-istio-data\") pod \"e2e-shared-b-w3-f1aad1-openshift-default-c6fd9dd46-l98nd\" (UID: \"726c78fe-d902-4d20-a882-1e0df84d795c\") " pod="openshift-ingress/e2e-shared-b-w3-f1aad1-openshift-default-c6fd9dd46-l98nd"
Sep 19 06:01:41.047503 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:41.047507    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/726c78fe-d902-4d20-a882-1e0df84d795c-istiod-ca-cert\") pod \"e2e-shared-b-w3-f1aad1-openshift-default-c6fd9dd46-l98nd\" (UID: \"726c78fe-d902-4d20-a882-1e0df84d795c\") " pod="openshift-ingress/e2e-shared-b-w3-f1aad1-openshift-default-c6fd9dd46-l98nd"
Sep 19 06:01:41.047774 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:41.047536    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/726c78fe-d902-4d20-a882-1e0df84d795c-credential-socket\") pod \"e2e-shared-b-w3-f1aad1-openshift-default-c6fd9dd46-l98nd\" (UID: \"726c78fe-d902-4d20-a882-1e0df84d795c\") " pod="openshift-ingress/e2e-shared-b-w3-f1aad1-openshift-default-c6fd9dd46-l98nd"
Sep 19 06:01:41.047774 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:41.047557    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/726c78fe-d902-4d20-a882-1e0df84d795c-workload-certs\") pod \"e2e-shared-b-w3-f1aad1-openshift-default-c6fd9dd46-l98nd\" (UID: \"726c78fe-d902-4d20-a882-1e0df84d795c\") " pod="openshift-ingress/e2e-shared-b-w3-f1aad1-openshift-default-c6fd9dd46-l98nd"
Sep 19 06:01:41.047774 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:41.047577    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4bmkn\" (UniqueName: \"kubernetes.io/projected/726c78fe-d902-4d20-a882-1e0df84d795c-kube-api-access-4bmkn\") pod \"e2e-shared-b-w3-f1aad1-openshift-default-c6fd9dd46-l98nd\" (UID: \"726c78fe-d902-4d20-a882-1e0df84d795c\") " pod="openshift-ingress/e2e-shared-b-w3-f1aad1-openshift-default-c6fd9dd46-l98nd"
Sep 19 06:01:41.047774 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:41.047616    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/726c78fe-d902-4d20-a882-1e0df84d795c-istio-envoy\") pod \"e2e-shared-b-w3-f1aad1-openshift-default-c6fd9dd46-l98nd\" (UID: \"726c78fe-d902-4d20-a882-1e0df84d795c\") " pod="openshift-ingress/e2e-shared-b-w3-f1aad1-openshift-default-c6fd9dd46-l98nd"
Sep 19 06:01:41.047774 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:41.047642    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/726c78fe-d902-4d20-a882-1e0df84d795c-workload-socket\") pod \"e2e-shared-b-w3-f1aad1-openshift-default-c6fd9dd46-l98nd\" (UID: \"726c78fe-d902-4d20-a882-1e0df84d795c\") " pod="openshift-ingress/e2e-shared-b-w3-f1aad1-openshift-default-c6fd9dd46-l98nd"
Sep 19 06:01:41.047774 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:41.047658    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/726c78fe-d902-4d20-a882-1e0df84d795c-istio-podinfo\") pod \"e2e-shared-b-w3-f1aad1-openshift-default-c6fd9dd46-l98nd\" (UID: \"726c78fe-d902-4d20-a882-1e0df84d795c\") " pod="openshift-ingress/e2e-shared-b-w3-f1aad1-openshift-default-c6fd9dd46-l98nd"
Sep 19 06:01:41.047774 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:41.047690    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/726c78fe-d902-4d20-a882-1e0df84d795c-istio-token\") pod \"e2e-shared-b-w3-f1aad1-openshift-default-c6fd9dd46-l98nd\" (UID: \"726c78fe-d902-4d20-a882-1e0df84d795c\") " pod="openshift-ingress/e2e-shared-b-w3-f1aad1-openshift-default-c6fd9dd46-l98nd"
Sep 19 06:01:41.048158 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:41.047937    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/726c78fe-d902-4d20-a882-1e0df84d795c-istio-data\") pod \"e2e-shared-b-w3-f1aad1-openshift-default-c6fd9dd46-l98nd\" (UID: \"726c78fe-d902-4d20-a882-1e0df84d795c\") " pod="openshift-ingress/e2e-shared-b-w3-f1aad1-openshift-default-c6fd9dd46-l98nd"
Sep 19 06:01:41.048518 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:41.048494    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/726c78fe-d902-4d20-a882-1e0df84d795c-istiod-ca-cert\") pod \"e2e-shared-b-w3-f1aad1-openshift-default-c6fd9dd46-l98nd\" (UID: \"726c78fe-d902-4d20-a882-1e0df84d795c\") " pod="openshift-ingress/e2e-shared-b-w3-f1aad1-openshift-default-c6fd9dd46-l98nd"
Sep 19 06:01:41.048637 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:41.048525    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/726c78fe-d902-4d20-a882-1e0df84d795c-credential-socket\") pod \"e2e-shared-b-w3-f1aad1-openshift-default-c6fd9dd46-l98nd\" (UID: \"726c78fe-d902-4d20-a882-1e0df84d795c\") " pod="openshift-ingress/e2e-shared-b-w3-f1aad1-openshift-default-c6fd9dd46-l98nd"
Sep 19 06:01:41.048700 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:41.048689    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/726c78fe-d902-4d20-a882-1e0df84d795c-workload-socket\") pod \"e2e-shared-b-w3-f1aad1-openshift-default-c6fd9dd46-l98nd\" (UID: \"726c78fe-d902-4d20-a882-1e0df84d795c\") " pod="openshift-ingress/e2e-shared-b-w3-f1aad1-openshift-default-c6fd9dd46-l98nd"
Sep 19 06:01:41.048989 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:41.048971    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/726c78fe-d902-4d20-a882-1e0df84d795c-workload-certs\") pod \"e2e-shared-b-w3-f1aad1-openshift-default-c6fd9dd46-l98nd\" (UID: \"726c78fe-d902-4d20-a882-1e0df84d795c\") " pod="openshift-ingress/e2e-shared-b-w3-f1aad1-openshift-default-c6fd9dd46-l98nd"
Sep 19 06:01:41.050756 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:41.050737    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/726c78fe-d902-4d20-a882-1e0df84d795c-istio-envoy\") pod \"e2e-shared-b-w3-f1aad1-openshift-default-c6fd9dd46-l98nd\" (UID: \"726c78fe-d902-4d20-a882-1e0df84d795c\") " pod="openshift-ingress/e2e-shared-b-w3-f1aad1-openshift-default-c6fd9dd46-l98nd"
Sep 19 06:01:41.051094 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:41.051079    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/726c78fe-d902-4d20-a882-1e0df84d795c-istio-podinfo\") pod \"e2e-shared-b-w3-f1aad1-openshift-default-c6fd9dd46-l98nd\" (UID: \"726c78fe-d902-4d20-a882-1e0df84d795c\") " pod="openshift-ingress/e2e-shared-b-w3-f1aad1-openshift-default-c6fd9dd46-l98nd"
Sep 19 06:01:41.056542 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:41.056519    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4bmkn\" (UniqueName: \"kubernetes.io/projected/726c78fe-d902-4d20-a882-1e0df84d795c-kube-api-access-4bmkn\") pod \"e2e-shared-b-w3-f1aad1-openshift-default-c6fd9dd46-l98nd\" (UID: \"726c78fe-d902-4d20-a882-1e0df84d795c\") " pod="openshift-ingress/e2e-shared-b-w3-f1aad1-openshift-default-c6fd9dd46-l98nd"
Sep 19 06:01:41.058215 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:41.058191    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/726c78fe-d902-4d20-a882-1e0df84d795c-istio-token\") pod \"e2e-shared-b-w3-f1aad1-openshift-default-c6fd9dd46-l98nd\" (UID: \"726c78fe-d902-4d20-a882-1e0df84d795c\") " pod="openshift-ingress/e2e-shared-b-w3-f1aad1-openshift-default-c6fd9dd46-l98nd"
Sep 19 06:01:41.180177 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:41.180079    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-f1aad1-openshift-default-c6fd9dd46-l98nd"
Sep 19 06:01:41.494943 ip-10-0-139-144 kubenswrapper[2583]: W0919 06:01:41.494915    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod726c78fe_d902_4d20_a882_1e0df84d795c.slice/crio-7f2b8edd233b81d1dccd10227fc200550ef47e329d1e97158be88f6426e19a67 WatchSource:0}: Error finding container 7f2b8edd233b81d1dccd10227fc200550ef47e329d1e97158be88f6426e19a67: Status 404 returned error can't find the container with id 7f2b8edd233b81d1dccd10227fc200550ef47e329d1e97158be88f6426e19a67
Sep 19 06:01:41.497406 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:41.497359    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 19 06:01:41.497514 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:41.497457    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 19 06:01:41.497514 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:41.497501    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 19 06:01:41.502523 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:41.502478    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-f1aad1-openshift-default-c6fd9dd46-l98nd"]
Sep 19 06:01:42.479883 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:42.479842    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-f1aad1-openshift-default-c6fd9dd46-l98nd" event={"ID":"726c78fe-d902-4d20-a882-1e0df84d795c","Type":"ContainerStarted","Data":"4a613c439eae9f8cf1cdfb3d18bc3ff2c84434b736e10afc07ff72f366776189"}
Sep 19 06:01:42.480370 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:42.479908    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-f1aad1-openshift-default-c6fd9dd46-l98nd" event={"ID":"726c78fe-d902-4d20-a882-1e0df84d795c","Type":"ContainerStarted","Data":"7f2b8edd233b81d1dccd10227fc200550ef47e329d1e97158be88f6426e19a67"}
Sep 19 06:01:42.500829 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:42.500767    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w3-f1aad1-openshift-default-c6fd9dd46-l98nd" podStartSLOduration=2.500749217 podStartE2EDuration="2.500749217s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 06:01:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 06:01:42.500449341 +0000 UTC m=+1261.158903903" watchObservedRunningTime="2026-09-19 06:01:42.500749217 +0000 UTC m=+1261.159203783"
Sep 19 06:01:43.180877 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:43.180834    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-f1aad1-openshift-default-c6fd9dd46-l98nd"
Sep 19 06:01:43.186509 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:43.186481    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-f1aad1-openshift-default-c6fd9dd46-l98nd"
Sep 19 06:01:43.486534 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:43.486446    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-f1aad1-openshift-default-c6fd9dd46-l98nd"
Sep 19 06:01:43.487329 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:43.487304    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-f1aad1-openshift-default-c6fd9dd46-l98nd"
Sep 19 06:01:45.500092 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:45.500061    2583 generic.go:356] "Generic (PLEG): container finished" podID="21ad9ff3-d47f-477a-ac4a-176a0258ec36" containerID="05f733480345e86473ec333947be6bfa9ec4c4d6beeaf41ab4e4952b7e5e298b" exitCode=0
Sep 19 06:01:45.500634 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:45.500141    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-888f75-openshift-default-599bd5457b-8mhn2" event={"ID":"21ad9ff3-d47f-477a-ac4a-176a0258ec36","Type":"ContainerDied","Data":"05f733480345e86473ec333947be6bfa9ec4c4d6beeaf41ab4e4952b7e5e298b"}
Sep 19 06:01:45.500634 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:45.500557    2583 scope.go:117] "RemoveContainer" containerID="05f733480345e86473ec333947be6bfa9ec4c4d6beeaf41ab4e4952b7e5e298b"
Sep 19 06:01:45.501814 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:45.501790    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 19 06:01:45.501872 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:45.501854    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 19 06:01:45.501909 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:45.501887    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 19 06:01:46.508115 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:46.508080    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-888f75-openshift-default-599bd5457b-8mhn2" event={"ID":"21ad9ff3-d47f-477a-ac4a-176a0258ec36","Type":"ContainerStarted","Data":"e7cbd576b7f0b5d941a661fac184636eee6975fcb375ba2dca4a838ce2ff3b26"}
Sep 19 06:01:47.028302 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:47.028254    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-888f75-openshift-default-599bd5457b-8mhn2"
Sep 19 06:01:47.029875 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:47.029835    2583 patch_prober.go:28] interesting pod/e2e-shared-a-w3-888f75-openshift-default-599bd5457b-8mhn2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body=
Sep 19 06:01:47.030030 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:47.029914    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-888f75-openshift-default-599bd5457b-8mhn2" podUID="21ad9ff3-d47f-477a-ac4a-176a0258ec36" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused"
Sep 19 06:01:48.028747 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:48.028705    2583 patch_prober.go:28] interesting pod/e2e-shared-a-w3-888f75-openshift-default-599bd5457b-8mhn2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body=
Sep 19 06:01:48.029391 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:48.029357    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-888f75-openshift-default-599bd5457b-8mhn2" podUID="21ad9ff3-d47f-477a-ac4a-176a0258ec36" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused"
Sep 19 06:01:49.032154 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:49.029518    2583 patch_prober.go:28] interesting pod/e2e-shared-a-w3-888f75-openshift-default-599bd5457b-8mhn2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body=
Sep 19 06:01:49.032154 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:49.029581    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-888f75-openshift-default-599bd5457b-8mhn2" podUID="21ad9ff3-d47f-477a-ac4a-176a0258ec36" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused"
Sep 19 06:01:49.337381 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:49.337285    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-888f75-openshift-default-599bd5457b-8mhn2"
Sep 19 06:01:50.028817 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:50.028776    2583 patch_prober.go:28] interesting pod/e2e-shared-a-w3-888f75-openshift-default-599bd5457b-8mhn2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body=
Sep 19 06:01:50.029096 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:50.029064    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-888f75-openshift-default-599bd5457b-8mhn2" podUID="21ad9ff3-d47f-477a-ac4a-176a0258ec36" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused"
Sep 19 06:01:51.028656 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:51.028614    2583 patch_prober.go:28] interesting pod/e2e-shared-a-w3-888f75-openshift-default-599bd5457b-8mhn2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body=
Sep 19 06:01:51.029695 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:51.029284    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-888f75-openshift-default-599bd5457b-8mhn2" podUID="21ad9ff3-d47f-477a-ac4a-176a0258ec36" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused"
Sep 19 06:01:51.594931 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:51.594879    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-cc0e26/distinct-w0-cc0e26-kserve-6b7bdf546b-dqvpj" podUID="9a6c8743-44fc-465e-a9b0-2353781c2aae" containerName="main" containerID="cri-o://f27f0cd1d58dc08b7de1e1d04feac9df7b8bd875aba5424fde91193fa1563ab9" gracePeriod=45
Sep 19 06:01:51.598335 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:51.598296    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-cc0e26/premium-w0-cc0e26-kserve-64f54b56f5-chxqb" podUID="96e17cef-e0e1-4422-aa6e-7eba33cae0c6" containerName="main" containerID="cri-o://3d4cf28f7e05f95cbba26b2859c31f3bd2924e1a247d94d1a3963a9f136c565c" gracePeriod=45
Sep 19 06:01:51.886889 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:51.886859    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-cc0e26/distinct-w0-cc0e26-kserve-6b7bdf546b-dqvpj"
Sep 19 06:01:51.890315 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:51.890281    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-cc0e26/premium-w0-cc0e26-kserve-64f54b56f5-chxqb"
Sep 19 06:01:51.971582 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:51.971551    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9a6c8743-44fc-465e-a9b0-2353781c2aae-model-cache\") pod \"9a6c8743-44fc-465e-a9b0-2353781c2aae\" (UID: \"9a6c8743-44fc-465e-a9b0-2353781c2aae\") "
Sep 19 06:01:51.971779 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:51.971607    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9a6c8743-44fc-465e-a9b0-2353781c2aae-tmp-dir\") pod \"9a6c8743-44fc-465e-a9b0-2353781c2aae\" (UID: \"9a6c8743-44fc-465e-a9b0-2353781c2aae\") "
Sep 19 06:01:51.971779 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:51.971639    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/96e17cef-e0e1-4422-aa6e-7eba33cae0c6-tmp-dir\") pod \"96e17cef-e0e1-4422-aa6e-7eba33cae0c6\" (UID: \"96e17cef-e0e1-4422-aa6e-7eba33cae0c6\") "
Sep 19 06:01:51.971779 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:51.971681    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/96e17cef-e0e1-4422-aa6e-7eba33cae0c6-dshm\") pod \"96e17cef-e0e1-4422-aa6e-7eba33cae0c6\" (UID: \"96e17cef-e0e1-4422-aa6e-7eba33cae0c6\") "
Sep 19 06:01:51.971779 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:51.971706    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/96e17cef-e0e1-4422-aa6e-7eba33cae0c6-model-cache\") pod \"96e17cef-e0e1-4422-aa6e-7eba33cae0c6\" (UID: \"96e17cef-e0e1-4422-aa6e-7eba33cae0c6\") "
Sep 19 06:01:51.971779 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:51.971742    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9a6c8743-44fc-465e-a9b0-2353781c2aae-dshm\") pod \"9a6c8743-44fc-465e-a9b0-2353781c2aae\" (UID: \"9a6c8743-44fc-465e-a9b0-2353781c2aae\") "
Sep 19 06:01:51.971779 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:51.971770    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/96e17cef-e0e1-4422-aa6e-7eba33cae0c6-home\") pod \"96e17cef-e0e1-4422-aa6e-7eba33cae0c6\" (UID: \"96e17cef-e0e1-4422-aa6e-7eba33cae0c6\") "
Sep 19 06:01:51.972104 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:51.971868    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a6c8743-44fc-465e-a9b0-2353781c2aae-model-cache" (OuterVolumeSpecName: "model-cache") pod "9a6c8743-44fc-465e-a9b0-2353781c2aae" (UID: "9a6c8743-44fc-465e-a9b0-2353781c2aae"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 06:01:51.972104 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:51.971898    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96e17cef-e0e1-4422-aa6e-7eba33cae0c6-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "96e17cef-e0e1-4422-aa6e-7eba33cae0c6" (UID: "96e17cef-e0e1-4422-aa6e-7eba33cae0c6"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 06:01:51.972104 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:51.971923    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a6c8743-44fc-465e-a9b0-2353781c2aae-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "9a6c8743-44fc-465e-a9b0-2353781c2aae" (UID: "9a6c8743-44fc-465e-a9b0-2353781c2aae"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 06:01:51.972104 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:51.972056    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96e17cef-e0e1-4422-aa6e-7eba33cae0c6-model-cache" (OuterVolumeSpecName: "model-cache") pod "96e17cef-e0e1-4422-aa6e-7eba33cae0c6" (UID: "96e17cef-e0e1-4422-aa6e-7eba33cae0c6"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 06:01:51.972373 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:51.972163    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96e17cef-e0e1-4422-aa6e-7eba33cae0c6-home" (OuterVolumeSpecName: "home") pod "96e17cef-e0e1-4422-aa6e-7eba33cae0c6" (UID: "96e17cef-e0e1-4422-aa6e-7eba33cae0c6"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 06:01:51.972373 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:51.972283    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q8p96\" (UniqueName: \"kubernetes.io/projected/9a6c8743-44fc-465e-a9b0-2353781c2aae-kube-api-access-q8p96\") pod \"9a6c8743-44fc-465e-a9b0-2353781c2aae\" (UID: \"9a6c8743-44fc-465e-a9b0-2353781c2aae\") "
Sep 19 06:01:51.972373 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:51.972318    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dfrtx\" (UniqueName: \"kubernetes.io/projected/96e17cef-e0e1-4422-aa6e-7eba33cae0c6-kube-api-access-dfrtx\") pod \"96e17cef-e0e1-4422-aa6e-7eba33cae0c6\" (UID: \"96e17cef-e0e1-4422-aa6e-7eba33cae0c6\") "
Sep 19 06:01:51.972373 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:51.972369    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9a6c8743-44fc-465e-a9b0-2353781c2aae-home\") pod \"9a6c8743-44fc-465e-a9b0-2353781c2aae\" (UID: \"9a6c8743-44fc-465e-a9b0-2353781c2aae\") "
Sep 19 06:01:51.972578 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:51.972400    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/96e17cef-e0e1-4422-aa6e-7eba33cae0c6-tls-certs\") pod \"96e17cef-e0e1-4422-aa6e-7eba33cae0c6\" (UID: \"96e17cef-e0e1-4422-aa6e-7eba33cae0c6\") "
Sep 19 06:01:51.972578 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:51.972482    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9a6c8743-44fc-465e-a9b0-2353781c2aae-tls-certs\") pod \"9a6c8743-44fc-465e-a9b0-2353781c2aae\" (UID: \"9a6c8743-44fc-465e-a9b0-2353781c2aae\") "
Sep 19 06:01:51.972688 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:51.972640    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a6c8743-44fc-465e-a9b0-2353781c2aae-home" (OuterVolumeSpecName: "home") pod "9a6c8743-44fc-465e-a9b0-2353781c2aae" (UID: "9a6c8743-44fc-465e-a9b0-2353781c2aae"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 06:01:51.972954 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:51.972928    2583 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/96e17cef-e0e1-4422-aa6e-7eba33cae0c6-tmp-dir\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 06:01:51.973043 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:51.972958    2583 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/96e17cef-e0e1-4422-aa6e-7eba33cae0c6-model-cache\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 06:01:51.973043 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:51.972971    2583 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/96e17cef-e0e1-4422-aa6e-7eba33cae0c6-home\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 06:01:51.973043 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:51.972981    2583 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9a6c8743-44fc-465e-a9b0-2353781c2aae-home\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 06:01:51.973043 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:51.972993    2583 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9a6c8743-44fc-465e-a9b0-2353781c2aae-model-cache\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 06:01:51.973043 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:51.973006    2583 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9a6c8743-44fc-465e-a9b0-2353781c2aae-tmp-dir\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 06:01:51.974684 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:51.974659    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96e17cef-e0e1-4422-aa6e-7eba33cae0c6-dshm" (OuterVolumeSpecName: "dshm") pod "96e17cef-e0e1-4422-aa6e-7eba33cae0c6" (UID: "96e17cef-e0e1-4422-aa6e-7eba33cae0c6"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 06:01:51.974802 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:51.974752    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9a6c8743-44fc-465e-a9b0-2353781c2aae-kube-api-access-q8p96" (OuterVolumeSpecName: "kube-api-access-q8p96") pod "9a6c8743-44fc-465e-a9b0-2353781c2aae" (UID: "9a6c8743-44fc-465e-a9b0-2353781c2aae"). InnerVolumeSpecName "kube-api-access-q8p96". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 06:01:51.975041 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:51.975016    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a6c8743-44fc-465e-a9b0-2353781c2aae-dshm" (OuterVolumeSpecName: "dshm") pod "9a6c8743-44fc-465e-a9b0-2353781c2aae" (UID: "9a6c8743-44fc-465e-a9b0-2353781c2aae"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 06:01:51.975118 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:51.975100    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/96e17cef-e0e1-4422-aa6e-7eba33cae0c6-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "96e17cef-e0e1-4422-aa6e-7eba33cae0c6" (UID: "96e17cef-e0e1-4422-aa6e-7eba33cae0c6"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 06:01:51.975196 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:51.975122    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9a6c8743-44fc-465e-a9b0-2353781c2aae-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "9a6c8743-44fc-465e-a9b0-2353781c2aae" (UID: "9a6c8743-44fc-465e-a9b0-2353781c2aae"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 06:01:51.975196 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:51.975154    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/96e17cef-e0e1-4422-aa6e-7eba33cae0c6-kube-api-access-dfrtx" (OuterVolumeSpecName: "kube-api-access-dfrtx") pod "96e17cef-e0e1-4422-aa6e-7eba33cae0c6" (UID: "96e17cef-e0e1-4422-aa6e-7eba33cae0c6"). InnerVolumeSpecName "kube-api-access-dfrtx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 06:01:52.032815 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:52.032783    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-888f75-openshift-default-599bd5457b-8mhn2"
Sep 19 06:01:52.034577 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:52.034548    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-888f75-openshift-default-599bd5457b-8mhn2"
Sep 19 06:01:52.073686 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:52.073647    2583 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/96e17cef-e0e1-4422-aa6e-7eba33cae0c6-dshm\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 06:01:52.073686 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:52.073677    2583 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9a6c8743-44fc-465e-a9b0-2353781c2aae-dshm\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 06:01:52.073686 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:52.073687    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q8p96\" (UniqueName: \"kubernetes.io/projected/9a6c8743-44fc-465e-a9b0-2353781c2aae-kube-api-access-q8p96\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 06:01:52.073923 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:52.073696    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dfrtx\" (UniqueName: \"kubernetes.io/projected/96e17cef-e0e1-4422-aa6e-7eba33cae0c6-kube-api-access-dfrtx\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 06:01:52.073923 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:52.073712    2583 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/96e17cef-e0e1-4422-aa6e-7eba33cae0c6-tls-certs\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 06:01:52.073923 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:52.073724    2583 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9a6c8743-44fc-465e-a9b0-2353781c2aae-tls-certs\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 06:01:52.554015 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:52.553983    2583 generic.go:356] "Generic (PLEG): container finished" podID="9a6c8743-44fc-465e-a9b0-2353781c2aae" containerID="f27f0cd1d58dc08b7de1e1d04feac9df7b8bd875aba5424fde91193fa1563ab9" exitCode=0
Sep 19 06:01:52.554239 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:52.554036    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-cc0e26/distinct-w0-cc0e26-kserve-6b7bdf546b-dqvpj"
Sep 19 06:01:52.554239 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:52.554071    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-cc0e26/distinct-w0-cc0e26-kserve-6b7bdf546b-dqvpj" event={"ID":"9a6c8743-44fc-465e-a9b0-2353781c2aae","Type":"ContainerDied","Data":"f27f0cd1d58dc08b7de1e1d04feac9df7b8bd875aba5424fde91193fa1563ab9"}
Sep 19 06:01:52.554239 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:52.554124    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-cc0e26/distinct-w0-cc0e26-kserve-6b7bdf546b-dqvpj" event={"ID":"9a6c8743-44fc-465e-a9b0-2353781c2aae","Type":"ContainerDied","Data":"9667e88e1f9f08bccdc3c49e839f842dc1d8f5db330f43dface893db44deff4f"}
Sep 19 06:01:52.554239 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:52.554173    2583 scope.go:117] "RemoveContainer" containerID="f27f0cd1d58dc08b7de1e1d04feac9df7b8bd875aba5424fde91193fa1563ab9"
Sep 19 06:01:52.555802 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:52.555775    2583 generic.go:356] "Generic (PLEG): container finished" podID="96e17cef-e0e1-4422-aa6e-7eba33cae0c6" containerID="3d4cf28f7e05f95cbba26b2859c31f3bd2924e1a247d94d1a3963a9f136c565c" exitCode=0
Sep 19 06:01:52.555941 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:52.555807    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-cc0e26/premium-w0-cc0e26-kserve-64f54b56f5-chxqb" event={"ID":"96e17cef-e0e1-4422-aa6e-7eba33cae0c6","Type":"ContainerDied","Data":"3d4cf28f7e05f95cbba26b2859c31f3bd2924e1a247d94d1a3963a9f136c565c"}
Sep 19 06:01:52.555941 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:52.555846    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-cc0e26/premium-w0-cc0e26-kserve-64f54b56f5-chxqb"
Sep 19 06:01:52.555941 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:52.555858    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-cc0e26/premium-w0-cc0e26-kserve-64f54b56f5-chxqb" event={"ID":"96e17cef-e0e1-4422-aa6e-7eba33cae0c6","Type":"ContainerDied","Data":"8aa8692b173ea22318014c4228915bb299aa4907e3015c134eaa4dd91dc3b755"}
Sep 19 06:01:52.565479 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:52.565327    2583 scope.go:117] "RemoveContainer" containerID="f27f0cd1d58dc08b7de1e1d04feac9df7b8bd875aba5424fde91193fa1563ab9"
Sep 19 06:01:52.565718 ip-10-0-139-144 kubenswrapper[2583]: E0919 06:01:52.565685    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f27f0cd1d58dc08b7de1e1d04feac9df7b8bd875aba5424fde91193fa1563ab9\": container with ID starting with f27f0cd1d58dc08b7de1e1d04feac9df7b8bd875aba5424fde91193fa1563ab9 not found: ID does not exist" containerID="f27f0cd1d58dc08b7de1e1d04feac9df7b8bd875aba5424fde91193fa1563ab9"
Sep 19 06:01:52.565814 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:52.565728    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f27f0cd1d58dc08b7de1e1d04feac9df7b8bd875aba5424fde91193fa1563ab9"} err="failed to get container status \"f27f0cd1d58dc08b7de1e1d04feac9df7b8bd875aba5424fde91193fa1563ab9\": rpc error: code = NotFound desc = could not find container \"f27f0cd1d58dc08b7de1e1d04feac9df7b8bd875aba5424fde91193fa1563ab9\": container with ID starting with f27f0cd1d58dc08b7de1e1d04feac9df7b8bd875aba5424fde91193fa1563ab9 not found: ID does not exist"
Sep 19 06:01:52.565814 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:52.565744    2583 scope.go:117] "RemoveContainer" containerID="3d4cf28f7e05f95cbba26b2859c31f3bd2924e1a247d94d1a3963a9f136c565c"
Sep 19 06:01:52.573974 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:52.573957    2583 scope.go:117] "RemoveContainer" containerID="3d4cf28f7e05f95cbba26b2859c31f3bd2924e1a247d94d1a3963a9f136c565c"
Sep 19 06:01:52.574242 ip-10-0-139-144 kubenswrapper[2583]: E0919 06:01:52.574224    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3d4cf28f7e05f95cbba26b2859c31f3bd2924e1a247d94d1a3963a9f136c565c\": container with ID starting with 3d4cf28f7e05f95cbba26b2859c31f3bd2924e1a247d94d1a3963a9f136c565c not found: ID does not exist" containerID="3d4cf28f7e05f95cbba26b2859c31f3bd2924e1a247d94d1a3963a9f136c565c"
Sep 19 06:01:52.574300 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:52.574247    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d4cf28f7e05f95cbba26b2859c31f3bd2924e1a247d94d1a3963a9f136c565c"} err="failed to get container status \"3d4cf28f7e05f95cbba26b2859c31f3bd2924e1a247d94d1a3963a9f136c565c\": rpc error: code = NotFound desc = could not find container \"3d4cf28f7e05f95cbba26b2859c31f3bd2924e1a247d94d1a3963a9f136c565c\": container with ID starting with 3d4cf28f7e05f95cbba26b2859c31f3bd2924e1a247d94d1a3963a9f136c565c not found: ID does not exist"
Sep 19 06:01:52.585973 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:52.585950    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-cc0e26/premium-w0-cc0e26-kserve-64f54b56f5-chxqb"]
Sep 19 06:01:52.590630 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:52.590605    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-cc0e26/premium-w0-cc0e26-kserve-64f54b56f5-chxqb"]
Sep 19 06:01:52.607829 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:52.607803    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-cc0e26/distinct-w0-cc0e26-kserve-6b7bdf546b-dqvpj"]
Sep 19 06:01:52.610410 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:52.610389    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-cc0e26/distinct-w0-cc0e26-kserve-6b7bdf546b-dqvpj"]
Sep 19 06:01:53.502289 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:53.502254    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="96e17cef-e0e1-4422-aa6e-7eba33cae0c6" path="/var/lib/kubelet/pods/96e17cef-e0e1-4422-aa6e-7eba33cae0c6/volumes"
Sep 19 06:01:53.502692 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:01:53.502677    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9a6c8743-44fc-465e-a9b0-2353781c2aae" path="/var/lib/kubelet/pods/9a6c8743-44fc-465e-a9b0-2353781c2aae/volumes"
Sep 19 06:02:14.488316 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:02:14.488270    2583 patch_prober.go:28] interesting pod/e2e-shared-b-w3-f1aad1-openshift-default-c6fd9dd46-l98nd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 19 06:02:14.488753 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:02:14.488354    2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w3-f1aad1-openshift-default-c6fd9dd46-l98nd" podUID="726c78fe-d902-4d20-a882-1e0df84d795c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 19 06:02:15.694336 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:02:15.694305    2583 generic.go:356] "Generic (PLEG): container finished" podID="726c78fe-d902-4d20-a882-1e0df84d795c" containerID="4a613c439eae9f8cf1cdfb3d18bc3ff2c84434b736e10afc07ff72f366776189" exitCode=0
Sep 19 06:02:15.694730 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:02:15.694380    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-f1aad1-openshift-default-c6fd9dd46-l98nd" event={"ID":"726c78fe-d902-4d20-a882-1e0df84d795c","Type":"ContainerDied","Data":"4a613c439eae9f8cf1cdfb3d18bc3ff2c84434b736e10afc07ff72f366776189"}
Sep 19 06:02:15.694808 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:02:15.694793    2583 scope.go:117] "RemoveContainer" containerID="4a613c439eae9f8cf1cdfb3d18bc3ff2c84434b736e10afc07ff72f366776189"
Sep 19 06:02:15.695950 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:02:15.695918    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 19 06:02:15.696041 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:02:15.695989    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 19 06:02:15.696041 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:02:15.696021    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 19 06:02:16.701660 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:02:16.701628    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-f1aad1-openshift-default-c6fd9dd46-l98nd" event={"ID":"726c78fe-d902-4d20-a882-1e0df84d795c","Type":"ContainerStarted","Data":"799c004db5135b82b56ef8d1d32bac6c520f6ba1aa437248c729f2c02fda599b"}
Sep 19 06:02:17.180426 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:02:17.180398    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-f1aad1-openshift-default-c6fd9dd46-l98nd"
Sep 19 06:02:17.180599 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:02:17.180568    2583 patch_prober.go:28] interesting pod/e2e-shared-b-w3-f1aad1-openshift-default-c6fd9dd46-l98nd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Sep 19 06:02:17.180642 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:02:17.180613    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-f1aad1-openshift-default-c6fd9dd46-l98nd" podUID="726c78fe-d902-4d20-a882-1e0df84d795c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Sep 19 06:02:18.180785 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:02:18.180745    2583 patch_prober.go:28] interesting pod/e2e-shared-b-w3-f1aad1-openshift-default-c6fd9dd46-l98nd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Sep 19 06:02:18.181225 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:02:18.180801    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-f1aad1-openshift-default-c6fd9dd46-l98nd" podUID="726c78fe-d902-4d20-a882-1e0df84d795c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Sep 19 06:02:19.181388 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:02:19.181334    2583 patch_prober.go:28] interesting pod/e2e-shared-b-w3-f1aad1-openshift-default-c6fd9dd46-l98nd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Sep 19 06:02:19.181904 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:02:19.181430    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-f1aad1-openshift-default-c6fd9dd46-l98nd" podUID="726c78fe-d902-4d20-a882-1e0df84d795c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Sep 19 06:02:20.180956 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:02:20.180920    2583 patch_prober.go:28] interesting pod/e2e-shared-b-w3-f1aad1-openshift-default-c6fd9dd46-l98nd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Sep 19 06:02:20.181160 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:02:20.180985    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-f1aad1-openshift-default-c6fd9dd46-l98nd" podUID="726c78fe-d902-4d20-a882-1e0df84d795c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Sep 19 06:02:21.180861 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:02:21.180825    2583 patch_prober.go:28] interesting pod/e2e-shared-b-w3-f1aad1-openshift-default-c6fd9dd46-l98nd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Sep 19 06:02:21.181272 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:02:21.180880    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-f1aad1-openshift-default-c6fd9dd46-l98nd" podUID="726c78fe-d902-4d20-a882-1e0df84d795c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Sep 19 06:02:22.184910 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:02:22.184876    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-f1aad1-openshift-default-c6fd9dd46-l98nd"
Sep 19 06:02:22.185365 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:02:22.185180    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-f1aad1-openshift-default-c6fd9dd46-l98nd"
Sep 19 06:02:22.186030 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:02:22.186008    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-f1aad1-openshift-default-c6fd9dd46-l98nd"
Sep 19 06:02:27.174778 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:02:27.174731    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-3182b5/e2e-conflict-b-95222d07-kserve-5fdbcc596d-hd8vs"]
Sep 19 06:02:42.194941 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:02:42.194889    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-3182b5/e2e-conflict-b-95222d07-kserve-5fdbcc596d-hd8vs" podUID="ca7338c6-7d7d-4f91-97a3-4f12d048d2dc" containerName="main" containerID="cri-o://df4e50b1e2b59ce5fb09590e6486fda09bc481b2d3fac2d7b712c45cc6c614f4" gracePeriod=45
Sep 19 06:02:42.448318 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:02:42.448252    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-3182b5/e2e-conflict-b-95222d07-kserve-5fdbcc596d-hd8vs"
Sep 19 06:02:42.564865 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:02:42.564833    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ca7338c6-7d7d-4f91-97a3-4f12d048d2dc-tmp-dir\") pod \"ca7338c6-7d7d-4f91-97a3-4f12d048d2dc\" (UID: \"ca7338c6-7d7d-4f91-97a3-4f12d048d2dc\") "
Sep 19 06:02:42.565026 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:02:42.564934    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d8wlk\" (UniqueName: \"kubernetes.io/projected/ca7338c6-7d7d-4f91-97a3-4f12d048d2dc-kube-api-access-d8wlk\") pod \"ca7338c6-7d7d-4f91-97a3-4f12d048d2dc\" (UID: \"ca7338c6-7d7d-4f91-97a3-4f12d048d2dc\") "
Sep 19 06:02:42.565026 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:02:42.564978    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ca7338c6-7d7d-4f91-97a3-4f12d048d2dc-model-cache\") pod \"ca7338c6-7d7d-4f91-97a3-4f12d048d2dc\" (UID: \"ca7338c6-7d7d-4f91-97a3-4f12d048d2dc\") "
Sep 19 06:02:42.565112 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:02:42.565026    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ca7338c6-7d7d-4f91-97a3-4f12d048d2dc-dshm\") pod \"ca7338c6-7d7d-4f91-97a3-4f12d048d2dc\" (UID: \"ca7338c6-7d7d-4f91-97a3-4f12d048d2dc\") "
Sep 19 06:02:42.565112 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:02:42.565062    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ca7338c6-7d7d-4f91-97a3-4f12d048d2dc-home\") pod \"ca7338c6-7d7d-4f91-97a3-4f12d048d2dc\" (UID: \"ca7338c6-7d7d-4f91-97a3-4f12d048d2dc\") "
Sep 19 06:02:42.565112 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:02:42.565079    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ca7338c6-7d7d-4f91-97a3-4f12d048d2dc-tls-certs\") pod \"ca7338c6-7d7d-4f91-97a3-4f12d048d2dc\" (UID: \"ca7338c6-7d7d-4f91-97a3-4f12d048d2dc\") "
Sep 19 06:02:42.565112 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:02:42.565074    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca7338c6-7d7d-4f91-97a3-4f12d048d2dc-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ca7338c6-7d7d-4f91-97a3-4f12d048d2dc" (UID: "ca7338c6-7d7d-4f91-97a3-4f12d048d2dc"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 06:02:42.565354 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:02:42.565271    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca7338c6-7d7d-4f91-97a3-4f12d048d2dc-model-cache" (OuterVolumeSpecName: "model-cache") pod "ca7338c6-7d7d-4f91-97a3-4f12d048d2dc" (UID: "ca7338c6-7d7d-4f91-97a3-4f12d048d2dc"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 06:02:42.565354 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:02:42.565285    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca7338c6-7d7d-4f91-97a3-4f12d048d2dc-home" (OuterVolumeSpecName: "home") pod "ca7338c6-7d7d-4f91-97a3-4f12d048d2dc" (UID: "ca7338c6-7d7d-4f91-97a3-4f12d048d2dc"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 06:02:42.565513 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:02:42.565487    2583 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ca7338c6-7d7d-4f91-97a3-4f12d048d2dc-home\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 06:02:42.565513 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:02:42.565514    2583 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ca7338c6-7d7d-4f91-97a3-4f12d048d2dc-tmp-dir\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 06:02:42.565712 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:02:42.565527    2583 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ca7338c6-7d7d-4f91-97a3-4f12d048d2dc-model-cache\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 06:02:42.567369 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:02:42.567347    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca7338c6-7d7d-4f91-97a3-4f12d048d2dc-kube-api-access-d8wlk" (OuterVolumeSpecName: "kube-api-access-d8wlk") pod "ca7338c6-7d7d-4f91-97a3-4f12d048d2dc" (UID: "ca7338c6-7d7d-4f91-97a3-4f12d048d2dc"). InnerVolumeSpecName "kube-api-access-d8wlk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 06:02:42.567465 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:02:42.567366    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca7338c6-7d7d-4f91-97a3-4f12d048d2dc-dshm" (OuterVolumeSpecName: "dshm") pod "ca7338c6-7d7d-4f91-97a3-4f12d048d2dc" (UID: "ca7338c6-7d7d-4f91-97a3-4f12d048d2dc"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 06:02:42.567465 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:02:42.567411    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ca7338c6-7d7d-4f91-97a3-4f12d048d2dc-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ca7338c6-7d7d-4f91-97a3-4f12d048d2dc" (UID: "ca7338c6-7d7d-4f91-97a3-4f12d048d2dc"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 06:02:42.667099 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:02:42.667012    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d8wlk\" (UniqueName: \"kubernetes.io/projected/ca7338c6-7d7d-4f91-97a3-4f12d048d2dc-kube-api-access-d8wlk\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 06:02:42.667099 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:02:42.667054    2583 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ca7338c6-7d7d-4f91-97a3-4f12d048d2dc-dshm\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 06:02:42.667099 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:02:42.667071    2583 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ca7338c6-7d7d-4f91-97a3-4f12d048d2dc-tls-certs\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 06:02:42.864556 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:02:42.864523    2583 generic.go:356] "Generic (PLEG): container finished" podID="ca7338c6-7d7d-4f91-97a3-4f12d048d2dc" containerID="df4e50b1e2b59ce5fb09590e6486fda09bc481b2d3fac2d7b712c45cc6c614f4" exitCode=0
Sep 19 06:02:42.864752 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:02:42.864571    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-3182b5/e2e-conflict-b-95222d07-kserve-5fdbcc596d-hd8vs"
Sep 19 06:02:42.864752 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:02:42.864601    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-3182b5/e2e-conflict-b-95222d07-kserve-5fdbcc596d-hd8vs" event={"ID":"ca7338c6-7d7d-4f91-97a3-4f12d048d2dc","Type":"ContainerDied","Data":"df4e50b1e2b59ce5fb09590e6486fda09bc481b2d3fac2d7b712c45cc6c614f4"}
Sep 19 06:02:42.864752 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:02:42.864646    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-3182b5/e2e-conflict-b-95222d07-kserve-5fdbcc596d-hd8vs" event={"ID":"ca7338c6-7d7d-4f91-97a3-4f12d048d2dc","Type":"ContainerDied","Data":"19043a7b3021ff6025783b61abde19c98b0e50948798b25ce03e8ecbd8591700"}
Sep 19 06:02:42.864752 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:02:42.864664    2583 scope.go:117] "RemoveContainer" containerID="df4e50b1e2b59ce5fb09590e6486fda09bc481b2d3fac2d7b712c45cc6c614f4"
Sep 19 06:02:42.874440 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:02:42.874420    2583 scope.go:117] "RemoveContainer" containerID="df4e50b1e2b59ce5fb09590e6486fda09bc481b2d3fac2d7b712c45cc6c614f4"
Sep 19 06:02:42.874719 ip-10-0-139-144 kubenswrapper[2583]: E0919 06:02:42.874695    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"df4e50b1e2b59ce5fb09590e6486fda09bc481b2d3fac2d7b712c45cc6c614f4\": container with ID starting with df4e50b1e2b59ce5fb09590e6486fda09bc481b2d3fac2d7b712c45cc6c614f4 not found: ID does not exist" containerID="df4e50b1e2b59ce5fb09590e6486fda09bc481b2d3fac2d7b712c45cc6c614f4"
Sep 19 06:02:42.874790 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:02:42.874730    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"df4e50b1e2b59ce5fb09590e6486fda09bc481b2d3fac2d7b712c45cc6c614f4"} err="failed to get container status \"df4e50b1e2b59ce5fb09590e6486fda09bc481b2d3fac2d7b712c45cc6c614f4\": rpc error: code = NotFound desc = could not find container \"df4e50b1e2b59ce5fb09590e6486fda09bc481b2d3fac2d7b712c45cc6c614f4\": container with ID starting with df4e50b1e2b59ce5fb09590e6486fda09bc481b2d3fac2d7b712c45cc6c614f4 not found: ID does not exist"
Sep 19 06:02:42.886496 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:02:42.886472    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-3182b5/e2e-conflict-b-95222d07-kserve-5fdbcc596d-hd8vs"]
Sep 19 06:02:42.891541 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:02:42.891521    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-3182b5/e2e-conflict-b-95222d07-kserve-5fdbcc596d-hd8vs"]
Sep 19 06:02:43.498688 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:02:43.498656    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ca7338c6-7d7d-4f91-97a3-4f12d048d2dc" path="/var/lib/kubelet/pods/ca7338c6-7d7d-4f91-97a3-4f12d048d2dc/volumes"
Sep 19 06:02:43.612787 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:02:43.612758    2583 scope.go:117] "RemoveContainer" containerID="37e8676eb9fc9335a8e3b023de638c304624c153b1ae14f55ede909e5d030a36"
Sep 19 06:02:56.232922 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:02:56.232840    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-957d1190-67ccbbbcfb-shmzk"]
Sep 19 06:02:56.233527 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:02:56.233095    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-957d1190-67ccbbbcfb-shmzk" podUID="0f0e722a-eafd-4d32-bebd-7627639a45a7" containerName="maas-api" containerID="cri-o://4ab4f5e5202d7117d0f84751d7c52ac94aa93db0eb31a0de271e9fa8644b81e9" gracePeriod=30
Sep 19 06:02:56.480157 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:02:56.480116    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-957d1190-67ccbbbcfb-shmzk"
Sep 19 06:02:56.596971 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:02:56.596936    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0f0e722a-eafd-4d32-bebd-7627639a45a7-openshift-service-ca\") pod \"0f0e722a-eafd-4d32-bebd-7627639a45a7\" (UID: \"0f0e722a-eafd-4d32-bebd-7627639a45a7\") "
Sep 19 06:02:56.597169 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:02:56.597000    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0f0e722a-eafd-4d32-bebd-7627639a45a7-metrics-certs\") pod \"0f0e722a-eafd-4d32-bebd-7627639a45a7\" (UID: \"0f0e722a-eafd-4d32-bebd-7627639a45a7\") "
Sep 19 06:02:56.597169 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:02:56.597051    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0f0e722a-eafd-4d32-bebd-7627639a45a7-maas-api-tls\") pod \"0f0e722a-eafd-4d32-bebd-7627639a45a7\" (UID: \"0f0e722a-eafd-4d32-bebd-7627639a45a7\") "
Sep 19 06:02:56.597169 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:02:56.597078    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m5z78\" (UniqueName: \"kubernetes.io/projected/0f0e722a-eafd-4d32-bebd-7627639a45a7-kube-api-access-m5z78\") pod \"0f0e722a-eafd-4d32-bebd-7627639a45a7\" (UID: \"0f0e722a-eafd-4d32-bebd-7627639a45a7\") "
Sep 19 06:02:56.597431 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:02:56.597401    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0f0e722a-eafd-4d32-bebd-7627639a45a7-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "0f0e722a-eafd-4d32-bebd-7627639a45a7" (UID: "0f0e722a-eafd-4d32-bebd-7627639a45a7"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 06:02:56.599272 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:02:56.599239    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0f0e722a-eafd-4d32-bebd-7627639a45a7-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "0f0e722a-eafd-4d32-bebd-7627639a45a7" (UID: "0f0e722a-eafd-4d32-bebd-7627639a45a7"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 06:02:56.599377 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:02:56.599280    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0f0e722a-eafd-4d32-bebd-7627639a45a7-kube-api-access-m5z78" (OuterVolumeSpecName: "kube-api-access-m5z78") pod "0f0e722a-eafd-4d32-bebd-7627639a45a7" (UID: "0f0e722a-eafd-4d32-bebd-7627639a45a7"). InnerVolumeSpecName "kube-api-access-m5z78". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 06:02:56.599377 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:02:56.599321    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0f0e722a-eafd-4d32-bebd-7627639a45a7-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "0f0e722a-eafd-4d32-bebd-7627639a45a7" (UID: "0f0e722a-eafd-4d32-bebd-7627639a45a7"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 06:02:56.698221 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:02:56.698186    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m5z78\" (UniqueName: \"kubernetes.io/projected/0f0e722a-eafd-4d32-bebd-7627639a45a7-kube-api-access-m5z78\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 06:02:56.698221 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:02:56.698215    2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0f0e722a-eafd-4d32-bebd-7627639a45a7-openshift-service-ca\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 06:02:56.698221 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:02:56.698227    2583 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0f0e722a-eafd-4d32-bebd-7627639a45a7-metrics-certs\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 06:02:56.698454 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:02:56.698236    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0f0e722a-eafd-4d32-bebd-7627639a45a7-maas-api-tls\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 06:02:56.943762 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:02:56.943671    2583 generic.go:356] "Generic (PLEG): container finished" podID="0f0e722a-eafd-4d32-bebd-7627639a45a7" containerID="4ab4f5e5202d7117d0f84751d7c52ac94aa93db0eb31a0de271e9fa8644b81e9" exitCode=0
Sep 19 06:02:56.943762 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:02:56.943718    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-957d1190-67ccbbbcfb-shmzk"
Sep 19 06:02:56.943947 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:02:56.943753    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-957d1190-67ccbbbcfb-shmzk" event={"ID":"0f0e722a-eafd-4d32-bebd-7627639a45a7","Type":"ContainerDied","Data":"4ab4f5e5202d7117d0f84751d7c52ac94aa93db0eb31a0de271e9fa8644b81e9"}
Sep 19 06:02:56.943947 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:02:56.943812    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-957d1190-67ccbbbcfb-shmzk" event={"ID":"0f0e722a-eafd-4d32-bebd-7627639a45a7","Type":"ContainerDied","Data":"23c95f3b2c18ae8d34cff20aaa3f6b4c5ec97ec6fba3cb7cdac5fd66328f8fdd"}
Sep 19 06:02:56.943947 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:02:56.943839    2583 scope.go:117] "RemoveContainer" containerID="4ab4f5e5202d7117d0f84751d7c52ac94aa93db0eb31a0de271e9fa8644b81e9"
Sep 19 06:02:56.959852 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:02:56.959816    2583 scope.go:117] "RemoveContainer" containerID="4ab4f5e5202d7117d0f84751d7c52ac94aa93db0eb31a0de271e9fa8644b81e9"
Sep 19 06:02:56.960332 ip-10-0-139-144 kubenswrapper[2583]: E0919 06:02:56.960292    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4ab4f5e5202d7117d0f84751d7c52ac94aa93db0eb31a0de271e9fa8644b81e9\": container with ID starting with 4ab4f5e5202d7117d0f84751d7c52ac94aa93db0eb31a0de271e9fa8644b81e9 not found: ID does not exist" containerID="4ab4f5e5202d7117d0f84751d7c52ac94aa93db0eb31a0de271e9fa8644b81e9"
Sep 19 06:02:56.960463 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:02:56.960347    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ab4f5e5202d7117d0f84751d7c52ac94aa93db0eb31a0de271e9fa8644b81e9"} err="failed to get container status \"4ab4f5e5202d7117d0f84751d7c52ac94aa93db0eb31a0de271e9fa8644b81e9\": rpc error: code = NotFound desc = could not find container \"4ab4f5e5202d7117d0f84751d7c52ac94aa93db0eb31a0de271e9fa8644b81e9\": container with ID starting with 4ab4f5e5202d7117d0f84751d7c52ac94aa93db0eb31a0de271e9fa8644b81e9 not found: ID does not exist"
Sep 19 06:02:56.974058 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:02:56.974026    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-957d1190-67ccbbbcfb-shmzk"]
Sep 19 06:02:56.976366 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:02:56.976348    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-957d1190-67ccbbbcfb-shmzk"]
Sep 19 06:02:57.499291 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:02:57.499263    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0f0e722a-eafd-4d32-bebd-7627639a45a7" path="/var/lib/kubelet/pods/0f0e722a-eafd-4d32-bebd-7627639a45a7/volumes"
Sep 19 06:03:51.250066 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:03:51.250028    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-f1aad1/auth-test-model-f1aad1-kserve-57476cc487-jqqzm"]
Sep 19 06:03:51.250680 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:03:51.250656    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="96e17cef-e0e1-4422-aa6e-7eba33cae0c6" containerName="main"
Sep 19 06:03:51.250791 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:03:51.250684    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="96e17cef-e0e1-4422-aa6e-7eba33cae0c6" containerName="main"
Sep 19 06:03:51.250791 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:03:51.250708    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ca7338c6-7d7d-4f91-97a3-4f12d048d2dc" containerName="main"
Sep 19 06:03:51.250791 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:03:51.250718    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="ca7338c6-7d7d-4f91-97a3-4f12d048d2dc" containerName="main"
Sep 19 06:03:51.250791 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:03:51.250747    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a6c8743-44fc-465e-a9b0-2353781c2aae" containerName="main"
Sep 19 06:03:51.250791 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:03:51.250757    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a6c8743-44fc-465e-a9b0-2353781c2aae" containerName="main"
Sep 19 06:03:51.250791 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:03:51.250770    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0f0e722a-eafd-4d32-bebd-7627639a45a7" containerName="maas-api"
Sep 19 06:03:51.250791 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:03:51.250779    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="0f0e722a-eafd-4d32-bebd-7627639a45a7" containerName="maas-api"
Sep 19 06:03:51.251188 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:03:51.250896    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="96e17cef-e0e1-4422-aa6e-7eba33cae0c6" containerName="main"
Sep 19 06:03:51.251188 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:03:51.250910    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="ca7338c6-7d7d-4f91-97a3-4f12d048d2dc" containerName="main"
Sep 19 06:03:51.251188 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:03:51.250922    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="0f0e722a-eafd-4d32-bebd-7627639a45a7" containerName="maas-api"
Sep 19 06:03:51.251188 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:03:51.250932    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="9a6c8743-44fc-465e-a9b0-2353781c2aae" containerName="main"
Sep 19 06:03:51.254170 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:03:51.254149    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-f1aad1/auth-test-model-f1aad1-kserve-57476cc487-jqqzm"
Sep 19 06:03:51.257423 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:03:51.257390    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-f1aad1\"/\"kube-root-ca.crt\""
Sep 19 06:03:51.257423 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:03:51.257410    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-f1aad1\"/\"auth-test-model-f1aad1-kserve-self-signed-certs\""
Sep 19 06:03:51.258243 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:03:51.257708    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-f1aad1\"/\"default-dockercfg-kpxlv\""
Sep 19 06:03:51.258243 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:03:51.257900    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-f1aad1\"/\"openshift-service-ca.crt\""
Sep 19 06:03:51.260694 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:03:51.260665    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-f1aad1/auth-test-model-f1aad1-kserve-57476cc487-jqqzm"]
Sep 19 06:03:51.307713 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:03:51.307681    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/38f09d4e-1855-4f7b-9165-12a2f05a4fa8-model-cache\") pod \"auth-test-model-f1aad1-kserve-57476cc487-jqqzm\" (UID: \"38f09d4e-1855-4f7b-9165-12a2f05a4fa8\") " pod="ai-tenant-e2e-shared-b-w3-f1aad1/auth-test-model-f1aad1-kserve-57476cc487-jqqzm"
Sep 19 06:03:51.307889 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:03:51.307718    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/38f09d4e-1855-4f7b-9165-12a2f05a4fa8-home\") pod \"auth-test-model-f1aad1-kserve-57476cc487-jqqzm\" (UID: \"38f09d4e-1855-4f7b-9165-12a2f05a4fa8\") " pod="ai-tenant-e2e-shared-b-w3-f1aad1/auth-test-model-f1aad1-kserve-57476cc487-jqqzm"
Sep 19 06:03:51.307889 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:03:51.307786    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f229c\" (UniqueName: \"kubernetes.io/projected/38f09d4e-1855-4f7b-9165-12a2f05a4fa8-kube-api-access-f229c\") pod \"auth-test-model-f1aad1-kserve-57476cc487-jqqzm\" (UID: \"38f09d4e-1855-4f7b-9165-12a2f05a4fa8\") " pod="ai-tenant-e2e-shared-b-w3-f1aad1/auth-test-model-f1aad1-kserve-57476cc487-jqqzm"
Sep 19 06:03:51.308017 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:03:51.307927    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/38f09d4e-1855-4f7b-9165-12a2f05a4fa8-dshm\") pod \"auth-test-model-f1aad1-kserve-57476cc487-jqqzm\" (UID: \"38f09d4e-1855-4f7b-9165-12a2f05a4fa8\") " pod="ai-tenant-e2e-shared-b-w3-f1aad1/auth-test-model-f1aad1-kserve-57476cc487-jqqzm"
Sep 19 06:03:51.308017 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:03:51.307972    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/38f09d4e-1855-4f7b-9165-12a2f05a4fa8-tmp-dir\") pod \"auth-test-model-f1aad1-kserve-57476cc487-jqqzm\" (UID: \"38f09d4e-1855-4f7b-9165-12a2f05a4fa8\") " pod="ai-tenant-e2e-shared-b-w3-f1aad1/auth-test-model-f1aad1-kserve-57476cc487-jqqzm"
Sep 19 06:03:51.308087 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:03:51.308026    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/38f09d4e-1855-4f7b-9165-12a2f05a4fa8-tls-certs\") pod \"auth-test-model-f1aad1-kserve-57476cc487-jqqzm\" (UID: \"38f09d4e-1855-4f7b-9165-12a2f05a4fa8\") " pod="ai-tenant-e2e-shared-b-w3-f1aad1/auth-test-model-f1aad1-kserve-57476cc487-jqqzm"
Sep 19 06:03:51.409481 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:03:51.409443    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/38f09d4e-1855-4f7b-9165-12a2f05a4fa8-dshm\") pod \"auth-test-model-f1aad1-kserve-57476cc487-jqqzm\" (UID: \"38f09d4e-1855-4f7b-9165-12a2f05a4fa8\") " pod="ai-tenant-e2e-shared-b-w3-f1aad1/auth-test-model-f1aad1-kserve-57476cc487-jqqzm"
Sep 19 06:03:51.409688 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:03:51.409487    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/38f09d4e-1855-4f7b-9165-12a2f05a4fa8-tmp-dir\") pod \"auth-test-model-f1aad1-kserve-57476cc487-jqqzm\" (UID: \"38f09d4e-1855-4f7b-9165-12a2f05a4fa8\") " pod="ai-tenant-e2e-shared-b-w3-f1aad1/auth-test-model-f1aad1-kserve-57476cc487-jqqzm"
Sep 19 06:03:51.409688 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:03:51.409527    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/38f09d4e-1855-4f7b-9165-12a2f05a4fa8-tls-certs\") pod \"auth-test-model-f1aad1-kserve-57476cc487-jqqzm\" (UID: \"38f09d4e-1855-4f7b-9165-12a2f05a4fa8\") " pod="ai-tenant-e2e-shared-b-w3-f1aad1/auth-test-model-f1aad1-kserve-57476cc487-jqqzm"
Sep 19 06:03:51.409688 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:03:51.409565    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/38f09d4e-1855-4f7b-9165-12a2f05a4fa8-model-cache\") pod \"auth-test-model-f1aad1-kserve-57476cc487-jqqzm\" (UID: \"38f09d4e-1855-4f7b-9165-12a2f05a4fa8\") " pod="ai-tenant-e2e-shared-b-w3-f1aad1/auth-test-model-f1aad1-kserve-57476cc487-jqqzm"
Sep 19 06:03:51.409863 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:03:51.409709    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/38f09d4e-1855-4f7b-9165-12a2f05a4fa8-home\") pod \"auth-test-model-f1aad1-kserve-57476cc487-jqqzm\" (UID: \"38f09d4e-1855-4f7b-9165-12a2f05a4fa8\") " pod="ai-tenant-e2e-shared-b-w3-f1aad1/auth-test-model-f1aad1-kserve-57476cc487-jqqzm"
Sep 19 06:03:51.409863 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:03:51.409784    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f229c\" (UniqueName: \"kubernetes.io/projected/38f09d4e-1855-4f7b-9165-12a2f05a4fa8-kube-api-access-f229c\") pod \"auth-test-model-f1aad1-kserve-57476cc487-jqqzm\" (UID: \"38f09d4e-1855-4f7b-9165-12a2f05a4fa8\") " pod="ai-tenant-e2e-shared-b-w3-f1aad1/auth-test-model-f1aad1-kserve-57476cc487-jqqzm"
Sep 19 06:03:51.409974 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:03:51.409954    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/38f09d4e-1855-4f7b-9165-12a2f05a4fa8-model-cache\") pod \"auth-test-model-f1aad1-kserve-57476cc487-jqqzm\" (UID: \"38f09d4e-1855-4f7b-9165-12a2f05a4fa8\") " pod="ai-tenant-e2e-shared-b-w3-f1aad1/auth-test-model-f1aad1-kserve-57476cc487-jqqzm"
Sep 19 06:03:51.410047 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:03:51.410030    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/38f09d4e-1855-4f7b-9165-12a2f05a4fa8-home\") pod \"auth-test-model-f1aad1-kserve-57476cc487-jqqzm\" (UID: \"38f09d4e-1855-4f7b-9165-12a2f05a4fa8\") " pod="ai-tenant-e2e-shared-b-w3-f1aad1/auth-test-model-f1aad1-kserve-57476cc487-jqqzm"
Sep 19 06:03:51.410113 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:03:51.410093    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/38f09d4e-1855-4f7b-9165-12a2f05a4fa8-tmp-dir\") pod \"auth-test-model-f1aad1-kserve-57476cc487-jqqzm\" (UID: \"38f09d4e-1855-4f7b-9165-12a2f05a4fa8\") " pod="ai-tenant-e2e-shared-b-w3-f1aad1/auth-test-model-f1aad1-kserve-57476cc487-jqqzm"
Sep 19 06:03:51.411925 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:03:51.411902    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/38f09d4e-1855-4f7b-9165-12a2f05a4fa8-dshm\") pod \"auth-test-model-f1aad1-kserve-57476cc487-jqqzm\" (UID: \"38f09d4e-1855-4f7b-9165-12a2f05a4fa8\") " pod="ai-tenant-e2e-shared-b-w3-f1aad1/auth-test-model-f1aad1-kserve-57476cc487-jqqzm"
Sep 19 06:03:51.412244 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:03:51.412224    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/38f09d4e-1855-4f7b-9165-12a2f05a4fa8-tls-certs\") pod \"auth-test-model-f1aad1-kserve-57476cc487-jqqzm\" (UID: \"38f09d4e-1855-4f7b-9165-12a2f05a4fa8\") " pod="ai-tenant-e2e-shared-b-w3-f1aad1/auth-test-model-f1aad1-kserve-57476cc487-jqqzm"
Sep 19 06:03:51.417140 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:03:51.417108    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f229c\" (UniqueName: \"kubernetes.io/projected/38f09d4e-1855-4f7b-9165-12a2f05a4fa8-kube-api-access-f229c\") pod \"auth-test-model-f1aad1-kserve-57476cc487-jqqzm\" (UID: \"38f09d4e-1855-4f7b-9165-12a2f05a4fa8\") " pod="ai-tenant-e2e-shared-b-w3-f1aad1/auth-test-model-f1aad1-kserve-57476cc487-jqqzm"
Sep 19 06:03:51.567121 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:03:51.567088    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-f1aad1/auth-test-model-f1aad1-kserve-57476cc487-jqqzm"
Sep 19 06:03:51.839079 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:03:51.839050    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-f1aad1/auth-test-model-f1aad1-kserve-57476cc487-jqqzm"]
Sep 19 06:03:51.840523 ip-10-0-139-144 kubenswrapper[2583]: W0919 06:03:51.840495    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod38f09d4e_1855_4f7b_9165_12a2f05a4fa8.slice/crio-90a00dd3869c69c6be63ccbf08601055f672199e216ac98c8ba23d541b0c96b0 WatchSource:0}: Error finding container 90a00dd3869c69c6be63ccbf08601055f672199e216ac98c8ba23d541b0c96b0: Status 404 returned error can't find the container with id 90a00dd3869c69c6be63ccbf08601055f672199e216ac98c8ba23d541b0c96b0
Sep 19 06:03:52.255665 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:03:52.255578    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-f1aad1/auth-test-model-f1aad1-kserve-57476cc487-jqqzm" event={"ID":"38f09d4e-1855-4f7b-9165-12a2f05a4fa8","Type":"ContainerStarted","Data":"1c2ef0e820f7d6bac619eaf73d4c3f8652780bfe7d10a743432195b83b114de0"}
Sep 19 06:03:52.255665 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:03:52.255622    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-f1aad1/auth-test-model-f1aad1-kserve-57476cc487-jqqzm" event={"ID":"38f09d4e-1855-4f7b-9165-12a2f05a4fa8","Type":"ContainerStarted","Data":"90a00dd3869c69c6be63ccbf08601055f672199e216ac98c8ba23d541b0c96b0"}
Sep 19 06:03:52.272997 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:03:52.272951    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-f1aad1/auth-test-model-f1aad1-kserve-57476cc487-jqqzm" podStartSLOduration=1.272936917 podStartE2EDuration="1.272936917s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 06:03:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 06:03:52.270433942 +0000 UTC m=+1390.928888501" watchObservedRunningTime="2026-09-19 06:03:52.272936917 +0000 UTC m=+1390.931391474"
Sep 19 06:04:01.567838 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:04:01.567798    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-f1aad1/auth-test-model-f1aad1-kserve-57476cc487-jqqzm"
Sep 19 06:04:01.568311 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:04:01.567866    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-f1aad1/auth-test-model-f1aad1-kserve-57476cc487-jqqzm"
Sep 19 06:04:01.580629 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:04:01.580598    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-f1aad1/auth-test-model-f1aad1-kserve-57476cc487-jqqzm"
Sep 19 06:04:02.319887 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:04:02.319858    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-f1aad1/auth-test-model-f1aad1-kserve-57476cc487-jqqzm"
Sep 19 06:05:11.286103 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:05:11.286063    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-311e2d-6b6d79b59c-2phcd"]
Sep 19 06:05:11.289098 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:05:11.289072    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-311e2d-6b6d79b59c-2phcd"
Sep 19 06:05:11.291429 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:05:11.291405    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-311e2d-serving-cert\""
Sep 19 06:05:11.300440 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:05:11.300398    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-311e2d-6b6d79b59c-2phcd"]
Sep 19 06:05:11.359032 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:05:11.358997    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d637e261-4039-40e0-859f-2a105352401a-metrics-certs\") pod \"maas-api-e2e-api-a-311e2d-6b6d79b59c-2phcd\" (UID: \"d637e261-4039-40e0-859f-2a105352401a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-311e2d-6b6d79b59c-2phcd"
Sep 19 06:05:11.359199 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:05:11.359124    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d637e261-4039-40e0-859f-2a105352401a-openshift-service-ca\") pod \"maas-api-e2e-api-a-311e2d-6b6d79b59c-2phcd\" (UID: \"d637e261-4039-40e0-859f-2a105352401a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-311e2d-6b6d79b59c-2phcd"
Sep 19 06:05:11.359253 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:05:11.359237    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d637e261-4039-40e0-859f-2a105352401a-maas-api-tls\") pod \"maas-api-e2e-api-a-311e2d-6b6d79b59c-2phcd\" (UID: \"d637e261-4039-40e0-859f-2a105352401a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-311e2d-6b6d79b59c-2phcd"
Sep 19 06:05:11.359290 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:05:11.359262    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tkppv\" (UniqueName: \"kubernetes.io/projected/d637e261-4039-40e0-859f-2a105352401a-kube-api-access-tkppv\") pod \"maas-api-e2e-api-a-311e2d-6b6d79b59c-2phcd\" (UID: \"d637e261-4039-40e0-859f-2a105352401a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-311e2d-6b6d79b59c-2phcd"
Sep 19 06:05:11.459740 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:05:11.459696    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d637e261-4039-40e0-859f-2a105352401a-openshift-service-ca\") pod \"maas-api-e2e-api-a-311e2d-6b6d79b59c-2phcd\" (UID: \"d637e261-4039-40e0-859f-2a105352401a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-311e2d-6b6d79b59c-2phcd"
Sep 19 06:05:11.459923 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:05:11.459797    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d637e261-4039-40e0-859f-2a105352401a-maas-api-tls\") pod \"maas-api-e2e-api-a-311e2d-6b6d79b59c-2phcd\" (UID: \"d637e261-4039-40e0-859f-2a105352401a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-311e2d-6b6d79b59c-2phcd"
Sep 19 06:05:11.459923 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:05:11.459831    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tkppv\" (UniqueName: \"kubernetes.io/projected/d637e261-4039-40e0-859f-2a105352401a-kube-api-access-tkppv\") pod \"maas-api-e2e-api-a-311e2d-6b6d79b59c-2phcd\" (UID: \"d637e261-4039-40e0-859f-2a105352401a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-311e2d-6b6d79b59c-2phcd"
Sep 19 06:05:11.459923 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:05:11.459868    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d637e261-4039-40e0-859f-2a105352401a-metrics-certs\") pod \"maas-api-e2e-api-a-311e2d-6b6d79b59c-2phcd\" (UID: \"d637e261-4039-40e0-859f-2a105352401a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-311e2d-6b6d79b59c-2phcd"
Sep 19 06:05:11.460419 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:05:11.460395    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d637e261-4039-40e0-859f-2a105352401a-openshift-service-ca\") pod \"maas-api-e2e-api-a-311e2d-6b6d79b59c-2phcd\" (UID: \"d637e261-4039-40e0-859f-2a105352401a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-311e2d-6b6d79b59c-2phcd"
Sep 19 06:05:11.465860 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:05:11.465821    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d637e261-4039-40e0-859f-2a105352401a-metrics-certs\") pod \"maas-api-e2e-api-a-311e2d-6b6d79b59c-2phcd\" (UID: \"d637e261-4039-40e0-859f-2a105352401a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-311e2d-6b6d79b59c-2phcd"
Sep 19 06:05:11.468684 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:05:11.468651    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d637e261-4039-40e0-859f-2a105352401a-maas-api-tls\") pod \"maas-api-e2e-api-a-311e2d-6b6d79b59c-2phcd\" (UID: \"d637e261-4039-40e0-859f-2a105352401a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-311e2d-6b6d79b59c-2phcd"
Sep 19 06:05:11.469976 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:05:11.469949    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tkppv\" (UniqueName: \"kubernetes.io/projected/d637e261-4039-40e0-859f-2a105352401a-kube-api-access-tkppv\") pod \"maas-api-e2e-api-a-311e2d-6b6d79b59c-2phcd\" (UID: \"d637e261-4039-40e0-859f-2a105352401a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-311e2d-6b6d79b59c-2phcd"
Sep 19 06:05:11.604035 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:05:11.603940    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-311e2d-6b6d79b59c-2phcd"
Sep 19 06:05:11.886884 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:05:11.886859    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-311e2d-6b6d79b59c-2phcd"]
Sep 19 06:05:11.889223 ip-10-0-139-144 kubenswrapper[2583]: W0919 06:05:11.889193    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd637e261_4039_40e0_859f_2a105352401a.slice/crio-b08f33db58c779ee94e68858f1010dcc7d4ccffcab3ea72732638e9d826644d0 WatchSource:0}: Error finding container b08f33db58c779ee94e68858f1010dcc7d4ccffcab3ea72732638e9d826644d0: Status 404 returned error can't find the container with id b08f33db58c779ee94e68858f1010dcc7d4ccffcab3ea72732638e9d826644d0
Sep 19 06:05:11.890825 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:05:11.890803    2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 19 06:05:12.724911 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:05:12.724874    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-311e2d-6b6d79b59c-2phcd" event={"ID":"d637e261-4039-40e0-859f-2a105352401a","Type":"ContainerStarted","Data":"8f72894924c2acec4fde6d793e84f915b842f6eb40ede8ec006314bc8f73153c"}
Sep 19 06:05:12.725413 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:05:12.724957    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-311e2d-6b6d79b59c-2phcd" event={"ID":"d637e261-4039-40e0-859f-2a105352401a","Type":"ContainerStarted","Data":"b08f33db58c779ee94e68858f1010dcc7d4ccffcab3ea72732638e9d826644d0"}
Sep 19 06:05:12.725413 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:05:12.724979    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-311e2d-6b6d79b59c-2phcd"
Sep 19 06:05:12.744191 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:05:12.744075    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-311e2d-6b6d79b59c-2phcd" podStartSLOduration=1.14787305 podStartE2EDuration="1.744056618s" totalImagesPullingTime="596.183568ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 06:05:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 06:05:12.742927972 +0000 UTC m=+1471.401382531" watchObservedRunningTime="2026-09-19 06:05:12.744056618 +0000 UTC m=+1471.402511178"
Sep 19 06:05:18.737475 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:05:18.737446    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-311e2d-6b6d79b59c-2phcd"
Sep 19 06:05:38.342324 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:05:38.342287    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-aea238-openshift-default-7579fbf658-nmd8x"]
Sep 19 06:05:38.345408 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:05:38.345384    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-aea238-openshift-default-7579fbf658-nmd8x"
Sep 19 06:05:38.348608 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:05:38.348581    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-aea238-openshift-default-dockercfg-2hggz\""
Sep 19 06:05:38.358270 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:05:38.358247    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-aea238-openshift-default-7579fbf658-nmd8x"]
Sep 19 06:05:38.437563 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:05:38.437521    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l9cn8\" (UniqueName: \"kubernetes.io/projected/484a06e7-0464-448b-86aa-1f166f59dfca-kube-api-access-l9cn8\") pod \"e2e-api-b-aea238-openshift-default-7579fbf658-nmd8x\" (UID: \"484a06e7-0464-448b-86aa-1f166f59dfca\") " pod="openshift-ingress/e2e-api-b-aea238-openshift-default-7579fbf658-nmd8x"
Sep 19 06:05:38.437883 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:05:38.437859    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/484a06e7-0464-448b-86aa-1f166f59dfca-workload-socket\") pod \"e2e-api-b-aea238-openshift-default-7579fbf658-nmd8x\" (UID: \"484a06e7-0464-448b-86aa-1f166f59dfca\") " pod="openshift-ingress/e2e-api-b-aea238-openshift-default-7579fbf658-nmd8x"
Sep 19 06:05:38.438061 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:05:38.438044    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/484a06e7-0464-448b-86aa-1f166f59dfca-istio-data\") pod \"e2e-api-b-aea238-openshift-default-7579fbf658-nmd8x\" (UID: \"484a06e7-0464-448b-86aa-1f166f59dfca\") " pod="openshift-ingress/e2e-api-b-aea238-openshift-default-7579fbf658-nmd8x"
Sep 19 06:05:38.438240 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:05:38.438224    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/484a06e7-0464-448b-86aa-1f166f59dfca-istio-token\") pod \"e2e-api-b-aea238-openshift-default-7579fbf658-nmd8x\" (UID: \"484a06e7-0464-448b-86aa-1f166f59dfca\") " pod="openshift-ingress/e2e-api-b-aea238-openshift-default-7579fbf658-nmd8x"
Sep 19 06:05:38.438403 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:05:38.438388    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/484a06e7-0464-448b-86aa-1f166f59dfca-workload-certs\") pod \"e2e-api-b-aea238-openshift-default-7579fbf658-nmd8x\" (UID: \"484a06e7-0464-448b-86aa-1f166f59dfca\") " pod="openshift-ingress/e2e-api-b-aea238-openshift-default-7579fbf658-nmd8x"
Sep 19 06:05:38.438539 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:05:38.438525    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/484a06e7-0464-448b-86aa-1f166f59dfca-istiod-ca-cert\") pod \"e2e-api-b-aea238-openshift-default-7579fbf658-nmd8x\" (UID: \"484a06e7-0464-448b-86aa-1f166f59dfca\") " pod="openshift-ingress/e2e-api-b-aea238-openshift-default-7579fbf658-nmd8x"
Sep 19 06:05:38.438745 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:05:38.438728    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/484a06e7-0464-448b-86aa-1f166f59dfca-istio-podinfo\") pod \"e2e-api-b-aea238-openshift-default-7579fbf658-nmd8x\" (UID: \"484a06e7-0464-448b-86aa-1f166f59dfca\") " pod="openshift-ingress/e2e-api-b-aea238-openshift-default-7579fbf658-nmd8x"
Sep 19 06:05:38.438937 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:05:38.438923    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/484a06e7-0464-448b-86aa-1f166f59dfca-istio-envoy\") pod \"e2e-api-b-aea238-openshift-default-7579fbf658-nmd8x\" (UID: \"484a06e7-0464-448b-86aa-1f166f59dfca\") " pod="openshift-ingress/e2e-api-b-aea238-openshift-default-7579fbf658-nmd8x"
Sep 19 06:05:38.439073 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:05:38.439058    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/484a06e7-0464-448b-86aa-1f166f59dfca-credential-socket\") pod \"e2e-api-b-aea238-openshift-default-7579fbf658-nmd8x\" (UID: \"484a06e7-0464-448b-86aa-1f166f59dfca\") " pod="openshift-ingress/e2e-api-b-aea238-openshift-default-7579fbf658-nmd8x"
Sep 19 06:05:38.539733 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:05:38.539700    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/484a06e7-0464-448b-86aa-1f166f59dfca-workload-certs\") pod \"e2e-api-b-aea238-openshift-default-7579fbf658-nmd8x\" (UID: \"484a06e7-0464-448b-86aa-1f166f59dfca\") " pod="openshift-ingress/e2e-api-b-aea238-openshift-default-7579fbf658-nmd8x"
Sep 19 06:05:38.539733 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:05:38.539739    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/484a06e7-0464-448b-86aa-1f166f59dfca-istiod-ca-cert\") pod \"e2e-api-b-aea238-openshift-default-7579fbf658-nmd8x\" (UID: \"484a06e7-0464-448b-86aa-1f166f59dfca\") " pod="openshift-ingress/e2e-api-b-aea238-openshift-default-7579fbf658-nmd8x"
Sep 19 06:05:38.539983 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:05:38.539797    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/484a06e7-0464-448b-86aa-1f166f59dfca-istio-podinfo\") pod \"e2e-api-b-aea238-openshift-default-7579fbf658-nmd8x\" (UID: \"484a06e7-0464-448b-86aa-1f166f59dfca\") " pod="openshift-ingress/e2e-api-b-aea238-openshift-default-7579fbf658-nmd8x"
Sep 19 06:05:38.539983 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:05:38.539849    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/484a06e7-0464-448b-86aa-1f166f59dfca-istio-envoy\") pod \"e2e-api-b-aea238-openshift-default-7579fbf658-nmd8x\" (UID: \"484a06e7-0464-448b-86aa-1f166f59dfca\") " pod="openshift-ingress/e2e-api-b-aea238-openshift-default-7579fbf658-nmd8x"
Sep 19 06:05:38.539983 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:05:38.539866    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/484a06e7-0464-448b-86aa-1f166f59dfca-credential-socket\") pod \"e2e-api-b-aea238-openshift-default-7579fbf658-nmd8x\" (UID: \"484a06e7-0464-448b-86aa-1f166f59dfca\") " pod="openshift-ingress/e2e-api-b-aea238-openshift-default-7579fbf658-nmd8x"
Sep 19 06:05:38.539983 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:05:38.539893    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l9cn8\" (UniqueName: \"kubernetes.io/projected/484a06e7-0464-448b-86aa-1f166f59dfca-kube-api-access-l9cn8\") pod \"e2e-api-b-aea238-openshift-default-7579fbf658-nmd8x\" (UID: \"484a06e7-0464-448b-86aa-1f166f59dfca\") " pod="openshift-ingress/e2e-api-b-aea238-openshift-default-7579fbf658-nmd8x"
Sep 19 06:05:38.540226 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:05:38.540165    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/484a06e7-0464-448b-86aa-1f166f59dfca-workload-certs\") pod \"e2e-api-b-aea238-openshift-default-7579fbf658-nmd8x\" (UID: \"484a06e7-0464-448b-86aa-1f166f59dfca\") " pod="openshift-ingress/e2e-api-b-aea238-openshift-default-7579fbf658-nmd8x"
Sep 19 06:05:38.540320 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:05:38.540295    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/484a06e7-0464-448b-86aa-1f166f59dfca-credential-socket\") pod \"e2e-api-b-aea238-openshift-default-7579fbf658-nmd8x\" (UID: \"484a06e7-0464-448b-86aa-1f166f59dfca\") " pod="openshift-ingress/e2e-api-b-aea238-openshift-default-7579fbf658-nmd8x"
Sep 19 06:05:38.540371 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:05:38.540312    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/484a06e7-0464-448b-86aa-1f166f59dfca-workload-socket\") pod \"e2e-api-b-aea238-openshift-default-7579fbf658-nmd8x\" (UID: \"484a06e7-0464-448b-86aa-1f166f59dfca\") " pod="openshift-ingress/e2e-api-b-aea238-openshift-default-7579fbf658-nmd8x"
Sep 19 06:05:38.540486 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:05:38.540379    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/484a06e7-0464-448b-86aa-1f166f59dfca-istio-data\") pod \"e2e-api-b-aea238-openshift-default-7579fbf658-nmd8x\" (UID: \"484a06e7-0464-448b-86aa-1f166f59dfca\") " pod="openshift-ingress/e2e-api-b-aea238-openshift-default-7579fbf658-nmd8x"
Sep 19 06:05:38.540486 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:05:38.540408    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/484a06e7-0464-448b-86aa-1f166f59dfca-istio-token\") pod \"e2e-api-b-aea238-openshift-default-7579fbf658-nmd8x\" (UID: \"484a06e7-0464-448b-86aa-1f166f59dfca\") " pod="openshift-ingress/e2e-api-b-aea238-openshift-default-7579fbf658-nmd8x"
Sep 19 06:05:38.540599 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:05:38.540582    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/484a06e7-0464-448b-86aa-1f166f59dfca-workload-socket\") pod \"e2e-api-b-aea238-openshift-default-7579fbf658-nmd8x\" (UID: \"484a06e7-0464-448b-86aa-1f166f59dfca\") " pod="openshift-ingress/e2e-api-b-aea238-openshift-default-7579fbf658-nmd8x"
Sep 19 06:05:38.540657 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:05:38.540610    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/484a06e7-0464-448b-86aa-1f166f59dfca-istiod-ca-cert\") pod \"e2e-api-b-aea238-openshift-default-7579fbf658-nmd8x\" (UID: \"484a06e7-0464-448b-86aa-1f166f59dfca\") " pod="openshift-ingress/e2e-api-b-aea238-openshift-default-7579fbf658-nmd8x"
Sep 19 06:05:38.540843 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:05:38.540818    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/484a06e7-0464-448b-86aa-1f166f59dfca-istio-data\") pod \"e2e-api-b-aea238-openshift-default-7579fbf658-nmd8x\" (UID: \"484a06e7-0464-448b-86aa-1f166f59dfca\") " pod="openshift-ingress/e2e-api-b-aea238-openshift-default-7579fbf658-nmd8x"
Sep 19 06:05:38.542325 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:05:38.542305    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/484a06e7-0464-448b-86aa-1f166f59dfca-istio-envoy\") pod \"e2e-api-b-aea238-openshift-default-7579fbf658-nmd8x\" (UID: \"484a06e7-0464-448b-86aa-1f166f59dfca\") " pod="openshift-ingress/e2e-api-b-aea238-openshift-default-7579fbf658-nmd8x"
Sep 19 06:05:38.543045 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:05:38.543023    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/484a06e7-0464-448b-86aa-1f166f59dfca-istio-podinfo\") pod \"e2e-api-b-aea238-openshift-default-7579fbf658-nmd8x\" (UID: \"484a06e7-0464-448b-86aa-1f166f59dfca\") " pod="openshift-ingress/e2e-api-b-aea238-openshift-default-7579fbf658-nmd8x"
Sep 19 06:05:38.548292 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:05:38.548269    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/484a06e7-0464-448b-86aa-1f166f59dfca-istio-token\") pod \"e2e-api-b-aea238-openshift-default-7579fbf658-nmd8x\" (UID: \"484a06e7-0464-448b-86aa-1f166f59dfca\") " pod="openshift-ingress/e2e-api-b-aea238-openshift-default-7579fbf658-nmd8x"
Sep 19 06:05:38.548841 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:05:38.548820    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l9cn8\" (UniqueName: \"kubernetes.io/projected/484a06e7-0464-448b-86aa-1f166f59dfca-kube-api-access-l9cn8\") pod \"e2e-api-b-aea238-openshift-default-7579fbf658-nmd8x\" (UID: \"484a06e7-0464-448b-86aa-1f166f59dfca\") " pod="openshift-ingress/e2e-api-b-aea238-openshift-default-7579fbf658-nmd8x"
Sep 19 06:05:38.657899 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:05:38.657820    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-aea238-openshift-default-7579fbf658-nmd8x"
Sep 19 06:05:38.940108 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:05:38.940079    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-aea238-openshift-default-7579fbf658-nmd8x"]
Sep 19 06:05:38.940470 ip-10-0-139-144 kubenswrapper[2583]: W0919 06:05:38.940446    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod484a06e7_0464_448b_86aa_1f166f59dfca.slice/crio-728a6a6135f887363b5baee225ec7f8f87f08bb2508bb104bcf959b61278c33e WatchSource:0}: Error finding container 728a6a6135f887363b5baee225ec7f8f87f08bb2508bb104bcf959b61278c33e: Status 404 returned error can't find the container with id 728a6a6135f887363b5baee225ec7f8f87f08bb2508bb104bcf959b61278c33e
Sep 19 06:05:38.943036 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:05:38.942994    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 19 06:05:38.943160 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:05:38.943080    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 19 06:05:38.943160 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:05:38.943144    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 19 06:05:39.893143 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:05:39.893089    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-aea238-openshift-default-7579fbf658-nmd8x" event={"ID":"484a06e7-0464-448b-86aa-1f166f59dfca","Type":"ContainerStarted","Data":"f6624bb13b86bb691f1e70fc28b04636cf3be78d83a3fac105a114f8e85968ee"}
Sep 19 06:05:39.893628 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:05:39.893166    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-aea238-openshift-default-7579fbf658-nmd8x" event={"ID":"484a06e7-0464-448b-86aa-1f166f59dfca","Type":"ContainerStarted","Data":"728a6a6135f887363b5baee225ec7f8f87f08bb2508bb104bcf959b61278c33e"}
Sep 19 06:05:39.915621 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:05:39.915565    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-aea238-openshift-default-7579fbf658-nmd8x" podStartSLOduration=1.915550713 podStartE2EDuration="1.915550713s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 06:05:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 06:05:39.912185435 +0000 UTC m=+1498.570639992" watchObservedRunningTime="2026-09-19 06:05:39.915550713 +0000 UTC m=+1498.574005271"
Sep 19 06:05:40.658377 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:05:40.658321    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-aea238-openshift-default-7579fbf658-nmd8x"
Sep 19 06:05:40.663652 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:05:40.663626    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-aea238-openshift-default-7579fbf658-nmd8x"
Sep 19 06:05:40.899007 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:05:40.898974    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-aea238-openshift-default-7579fbf658-nmd8x"
Sep 19 06:05:40.899982 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:05:40.899961    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-aea238-openshift-default-7579fbf658-nmd8x"
Sep 19 06:05:47.251391 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:05:47.251350    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-aea238-58d9db7fc8-htghb"]
Sep 19 06:05:47.256486 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:05:47.256466    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-aea238-58d9db7fc8-htghb"
Sep 19 06:05:47.259068 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:05:47.259039    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-aea238-serving-cert\""
Sep 19 06:05:47.268182 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:05:47.268119    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-aea238-58d9db7fc8-htghb"]
Sep 19 06:05:47.326781 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:05:47.326748    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/912e198c-c64b-487a-b063-c8eb2b2386a4-metrics-certs\") pod \"maas-api-e2e-api-b-aea238-58d9db7fc8-htghb\" (UID: \"912e198c-c64b-487a-b063-c8eb2b2386a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-aea238-58d9db7fc8-htghb"
Sep 19 06:05:47.326951 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:05:47.326787    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c5pcm\" (UniqueName: \"kubernetes.io/projected/912e198c-c64b-487a-b063-c8eb2b2386a4-kube-api-access-c5pcm\") pod \"maas-api-e2e-api-b-aea238-58d9db7fc8-htghb\" (UID: \"912e198c-c64b-487a-b063-c8eb2b2386a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-aea238-58d9db7fc8-htghb"
Sep 19 06:05:47.326951 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:05:47.326805    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/912e198c-c64b-487a-b063-c8eb2b2386a4-openshift-service-ca\") pod \"maas-api-e2e-api-b-aea238-58d9db7fc8-htghb\" (UID: \"912e198c-c64b-487a-b063-c8eb2b2386a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-aea238-58d9db7fc8-htghb"
Sep 19 06:05:47.326951 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:05:47.326933    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/912e198c-c64b-487a-b063-c8eb2b2386a4-maas-api-tls\") pod \"maas-api-e2e-api-b-aea238-58d9db7fc8-htghb\" (UID: \"912e198c-c64b-487a-b063-c8eb2b2386a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-aea238-58d9db7fc8-htghb"
Sep 19 06:05:47.428540 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:05:47.428505    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/912e198c-c64b-487a-b063-c8eb2b2386a4-metrics-certs\") pod \"maas-api-e2e-api-b-aea238-58d9db7fc8-htghb\" (UID: \"912e198c-c64b-487a-b063-c8eb2b2386a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-aea238-58d9db7fc8-htghb"
Sep 19 06:05:47.428540 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:05:47.428551    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c5pcm\" (UniqueName: \"kubernetes.io/projected/912e198c-c64b-487a-b063-c8eb2b2386a4-kube-api-access-c5pcm\") pod \"maas-api-e2e-api-b-aea238-58d9db7fc8-htghb\" (UID: \"912e198c-c64b-487a-b063-c8eb2b2386a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-aea238-58d9db7fc8-htghb"
Sep 19 06:05:47.428828 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:05:47.428572    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/912e198c-c64b-487a-b063-c8eb2b2386a4-openshift-service-ca\") pod \"maas-api-e2e-api-b-aea238-58d9db7fc8-htghb\" (UID: \"912e198c-c64b-487a-b063-c8eb2b2386a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-aea238-58d9db7fc8-htghb"
Sep 19 06:05:47.428828 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:05:47.428605    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/912e198c-c64b-487a-b063-c8eb2b2386a4-maas-api-tls\") pod \"maas-api-e2e-api-b-aea238-58d9db7fc8-htghb\" (UID: \"912e198c-c64b-487a-b063-c8eb2b2386a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-aea238-58d9db7fc8-htghb"
Sep 19 06:05:47.428828 ip-10-0-139-144 kubenswrapper[2583]: E0919 06:05:47.428720    2583 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-aea238-serving-cert: secret "maas-api-e2e-api-b-aea238-serving-cert" not found
Sep 19 06:05:47.428828 ip-10-0-139-144 kubenswrapper[2583]: E0919 06:05:47.428778    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/912e198c-c64b-487a-b063-c8eb2b2386a4-maas-api-tls podName:912e198c-c64b-487a-b063-c8eb2b2386a4 nodeName:}" failed. No retries permitted until 2026-09-19 06:05:47.928761741 +0000 UTC m=+1506.587216277 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/912e198c-c64b-487a-b063-c8eb2b2386a4-maas-api-tls") pod "maas-api-e2e-api-b-aea238-58d9db7fc8-htghb" (UID: "912e198c-c64b-487a-b063-c8eb2b2386a4") : secret "maas-api-e2e-api-b-aea238-serving-cert" not found
Sep 19 06:05:47.429371 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:05:47.429347    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/912e198c-c64b-487a-b063-c8eb2b2386a4-openshift-service-ca\") pod \"maas-api-e2e-api-b-aea238-58d9db7fc8-htghb\" (UID: \"912e198c-c64b-487a-b063-c8eb2b2386a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-aea238-58d9db7fc8-htghb"
Sep 19 06:05:47.431352 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:05:47.431326    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/912e198c-c64b-487a-b063-c8eb2b2386a4-metrics-certs\") pod \"maas-api-e2e-api-b-aea238-58d9db7fc8-htghb\" (UID: \"912e198c-c64b-487a-b063-c8eb2b2386a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-aea238-58d9db7fc8-htghb"
Sep 19 06:05:47.436258 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:05:47.436235    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c5pcm\" (UniqueName: \"kubernetes.io/projected/912e198c-c64b-487a-b063-c8eb2b2386a4-kube-api-access-c5pcm\") pod \"maas-api-e2e-api-b-aea238-58d9db7fc8-htghb\" (UID: \"912e198c-c64b-487a-b063-c8eb2b2386a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-aea238-58d9db7fc8-htghb"
Sep 19 06:05:47.933199 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:05:47.933168    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/912e198c-c64b-487a-b063-c8eb2b2386a4-maas-api-tls\") pod \"maas-api-e2e-api-b-aea238-58d9db7fc8-htghb\" (UID: \"912e198c-c64b-487a-b063-c8eb2b2386a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-aea238-58d9db7fc8-htghb"
Sep 19 06:05:47.933389 ip-10-0-139-144 kubenswrapper[2583]: E0919 06:05:47.933344    2583 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-aea238-serving-cert: secret "maas-api-e2e-api-b-aea238-serving-cert" not found
Sep 19 06:05:47.933451 ip-10-0-139-144 kubenswrapper[2583]: E0919 06:05:47.933422    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/912e198c-c64b-487a-b063-c8eb2b2386a4-maas-api-tls podName:912e198c-c64b-487a-b063-c8eb2b2386a4 nodeName:}" failed. No retries permitted until 2026-09-19 06:05:48.933402615 +0000 UTC m=+1507.591857154 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/912e198c-c64b-487a-b063-c8eb2b2386a4-maas-api-tls") pod "maas-api-e2e-api-b-aea238-58d9db7fc8-htghb" (UID: "912e198c-c64b-487a-b063-c8eb2b2386a4") : secret "maas-api-e2e-api-b-aea238-serving-cert" not found
Sep 19 06:05:48.944139 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:05:48.944106    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/912e198c-c64b-487a-b063-c8eb2b2386a4-maas-api-tls\") pod \"maas-api-e2e-api-b-aea238-58d9db7fc8-htghb\" (UID: \"912e198c-c64b-487a-b063-c8eb2b2386a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-aea238-58d9db7fc8-htghb"
Sep 19 06:05:48.946503 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:05:48.946483    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/912e198c-c64b-487a-b063-c8eb2b2386a4-maas-api-tls\") pod \"maas-api-e2e-api-b-aea238-58d9db7fc8-htghb\" (UID: \"912e198c-c64b-487a-b063-c8eb2b2386a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-aea238-58d9db7fc8-htghb"
Sep 19 06:05:49.072290 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:05:49.072238    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-aea238-58d9db7fc8-htghb"
Sep 19 06:05:49.346757 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:05:49.346726    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-aea238-58d9db7fc8-htghb"]
Sep 19 06:05:49.347807 ip-10-0-139-144 kubenswrapper[2583]: W0919 06:05:49.347773    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod912e198c_c64b_487a_b063_c8eb2b2386a4.slice/crio-75c069811eb8c644bb100ee986e69defdd657b23d891b7949cf1644bc7630686 WatchSource:0}: Error finding container 75c069811eb8c644bb100ee986e69defdd657b23d891b7949cf1644bc7630686: Status 404 returned error can't find the container with id 75c069811eb8c644bb100ee986e69defdd657b23d891b7949cf1644bc7630686
Sep 19 06:05:49.951997 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:05:49.951907    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-aea238-58d9db7fc8-htghb" event={"ID":"912e198c-c64b-487a-b063-c8eb2b2386a4","Type":"ContainerStarted","Data":"8c0f9069f6ca2db1724af52858b88939db7300d537469e5d4bd2b6bb307f606a"}
Sep 19 06:05:49.952388 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:05:49.951999    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-aea238-58d9db7fc8-htghb" event={"ID":"912e198c-c64b-487a-b063-c8eb2b2386a4","Type":"ContainerStarted","Data":"75c069811eb8c644bb100ee986e69defdd657b23d891b7949cf1644bc7630686"}
Sep 19 06:05:49.952388 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:05:49.952157    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-aea238-58d9db7fc8-htghb"
Sep 19 06:05:49.969002 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:05:49.968960    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-aea238-58d9db7fc8-htghb" podStartSLOduration=2.628675384 podStartE2EDuration="2.968945083s" totalImagesPullingTime="340.269699ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 06:05:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 06:05:49.967681683 +0000 UTC m=+1508.626136242" watchObservedRunningTime="2026-09-19 06:05:49.968945083 +0000 UTC m=+1508.627399640"
Sep 19 06:05:55.963528 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:05:55.963417    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-aea238-58d9db7fc8-htghb"
Sep 19 06:06:09.097885 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:06:09.097847    2583 generic.go:356] "Generic (PLEG): container finished" podID="484a06e7-0464-448b-86aa-1f166f59dfca" containerID="f6624bb13b86bb691f1e70fc28b04636cf3be78d83a3fac105a114f8e85968ee" exitCode=0
Sep 19 06:06:09.098322 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:06:09.097924    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-aea238-openshift-default-7579fbf658-nmd8x" event={"ID":"484a06e7-0464-448b-86aa-1f166f59dfca","Type":"ContainerDied","Data":"f6624bb13b86bb691f1e70fc28b04636cf3be78d83a3fac105a114f8e85968ee"}
Sep 19 06:06:09.098426 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:06:09.098398    2583 scope.go:117] "RemoveContainer" containerID="f6624bb13b86bb691f1e70fc28b04636cf3be78d83a3fac105a114f8e85968ee"
Sep 19 06:06:09.099680 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:06:09.099653    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 19 06:06:09.099735 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:06:09.099717    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 19 06:06:09.099775 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:06:09.099750    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 19 06:06:10.105974 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:06:10.105936    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-aea238-openshift-default-7579fbf658-nmd8x" event={"ID":"484a06e7-0464-448b-86aa-1f166f59dfca","Type":"ContainerStarted","Data":"74fc93b5b02f27b9c7e16b01cfbd8d7b3b0342b651bc4e2ac96672d372653b8b"}
Sep 19 06:06:10.658835 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:06:10.658800    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-aea238-openshift-default-7579fbf658-nmd8x"
Sep 19 06:06:10.659858 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:06:10.659817    2583 patch_prober.go:28] interesting pod/e2e-api-b-aea238-openshift-default-7579fbf658-nmd8x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body=
Sep 19 06:06:10.660005 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:06:10.659874    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-aea238-openshift-default-7579fbf658-nmd8x" podUID="484a06e7-0464-448b-86aa-1f166f59dfca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused"
Sep 19 06:06:10.899594 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:06:10.899564    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-aea238-openshift-default-7579fbf658-nmd8x"
Sep 19 06:06:11.658536 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:06:11.658502    2583 patch_prober.go:28] interesting pod/e2e-api-b-aea238-openshift-default-7579fbf658-nmd8x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body=
Sep 19 06:06:11.658961 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:06:11.658566    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-aea238-openshift-default-7579fbf658-nmd8x" podUID="484a06e7-0464-448b-86aa-1f166f59dfca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused"
Sep 19 06:06:12.659231 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:06:12.659192    2583 patch_prober.go:28] interesting pod/e2e-api-b-aea238-openshift-default-7579fbf658-nmd8x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body=
Sep 19 06:06:12.659651 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:06:12.659258    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-aea238-openshift-default-7579fbf658-nmd8x" podUID="484a06e7-0464-448b-86aa-1f166f59dfca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused"
Sep 19 06:06:13.659104 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:06:13.659064    2583 patch_prober.go:28] interesting pod/e2e-api-b-aea238-openshift-default-7579fbf658-nmd8x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body=
Sep 19 06:06:13.659311 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:06:13.659151    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-aea238-openshift-default-7579fbf658-nmd8x" podUID="484a06e7-0464-448b-86aa-1f166f59dfca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused"
Sep 19 06:06:14.658765 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:06:14.658720    2583 patch_prober.go:28] interesting pod/e2e-api-b-aea238-openshift-default-7579fbf658-nmd8x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body=
Sep 19 06:06:14.658976 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:06:14.658799    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-aea238-openshift-default-7579fbf658-nmd8x" podUID="484a06e7-0464-448b-86aa-1f166f59dfca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused"
Sep 19 06:06:15.663097 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:06:15.663064    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-aea238-openshift-default-7579fbf658-nmd8x"
Sep 19 06:06:15.664101 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:06:15.664083    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-aea238-openshift-default-7579fbf658-nmd8x"
Sep 19 06:06:30.513831 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:06:30.513794    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-311e2d-6b6d79b59c-2phcd"]
Sep 19 06:06:30.514355 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:06:30.514051    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-311e2d-6b6d79b59c-2phcd" podUID="d637e261-4039-40e0-859f-2a105352401a" containerName="maas-api" containerID="cri-o://8f72894924c2acec4fde6d793e84f915b842f6eb40ede8ec006314bc8f73153c" gracePeriod=30
Sep 19 06:06:30.760512 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:06:30.760485    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-311e2d-6b6d79b59c-2phcd"
Sep 19 06:06:30.859643 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:06:30.859615    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d637e261-4039-40e0-859f-2a105352401a-metrics-certs\") pod \"d637e261-4039-40e0-859f-2a105352401a\" (UID: \"d637e261-4039-40e0-859f-2a105352401a\") "
Sep 19 06:06:30.859822 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:06:30.859664    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d637e261-4039-40e0-859f-2a105352401a-maas-api-tls\") pod \"d637e261-4039-40e0-859f-2a105352401a\" (UID: \"d637e261-4039-40e0-859f-2a105352401a\") "
Sep 19 06:06:30.859822 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:06:30.859695    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tkppv\" (UniqueName: \"kubernetes.io/projected/d637e261-4039-40e0-859f-2a105352401a-kube-api-access-tkppv\") pod \"d637e261-4039-40e0-859f-2a105352401a\" (UID: \"d637e261-4039-40e0-859f-2a105352401a\") "
Sep 19 06:06:30.859822 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:06:30.859744    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d637e261-4039-40e0-859f-2a105352401a-openshift-service-ca\") pod \"d637e261-4039-40e0-859f-2a105352401a\" (UID: \"d637e261-4039-40e0-859f-2a105352401a\") "
Sep 19 06:06:30.860244 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:06:30.860216    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d637e261-4039-40e0-859f-2a105352401a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d637e261-4039-40e0-859f-2a105352401a" (UID: "d637e261-4039-40e0-859f-2a105352401a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 06:06:30.861803 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:06:30.861767    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d637e261-4039-40e0-859f-2a105352401a-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "d637e261-4039-40e0-859f-2a105352401a" (UID: "d637e261-4039-40e0-859f-2a105352401a"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 06:06:30.861803 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:06:30.861784    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d637e261-4039-40e0-859f-2a105352401a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d637e261-4039-40e0-859f-2a105352401a" (UID: "d637e261-4039-40e0-859f-2a105352401a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 06:06:30.861949 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:06:30.861846    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d637e261-4039-40e0-859f-2a105352401a-kube-api-access-tkppv" (OuterVolumeSpecName: "kube-api-access-tkppv") pod "d637e261-4039-40e0-859f-2a105352401a" (UID: "d637e261-4039-40e0-859f-2a105352401a"). InnerVolumeSpecName "kube-api-access-tkppv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 06:06:30.960806 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:06:30.960771    2583 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d637e261-4039-40e0-859f-2a105352401a-metrics-certs\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 06:06:30.960806 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:06:30.960805    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d637e261-4039-40e0-859f-2a105352401a-maas-api-tls\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 06:06:30.961028 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:06:30.960818    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tkppv\" (UniqueName: \"kubernetes.io/projected/d637e261-4039-40e0-859f-2a105352401a-kube-api-access-tkppv\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 06:06:30.961028 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:06:30.960832    2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d637e261-4039-40e0-859f-2a105352401a-openshift-service-ca\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 06:06:31.240933 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:06:31.240848    2583 generic.go:356] "Generic (PLEG): container finished" podID="d637e261-4039-40e0-859f-2a105352401a" containerID="8f72894924c2acec4fde6d793e84f915b842f6eb40ede8ec006314bc8f73153c" exitCode=0
Sep 19 06:06:31.240933 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:06:31.240896    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-311e2d-6b6d79b59c-2phcd"
Sep 19 06:06:31.241120 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:06:31.240932    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-311e2d-6b6d79b59c-2phcd" event={"ID":"d637e261-4039-40e0-859f-2a105352401a","Type":"ContainerDied","Data":"8f72894924c2acec4fde6d793e84f915b842f6eb40ede8ec006314bc8f73153c"}
Sep 19 06:06:31.241120 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:06:31.240987    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-311e2d-6b6d79b59c-2phcd" event={"ID":"d637e261-4039-40e0-859f-2a105352401a","Type":"ContainerDied","Data":"b08f33db58c779ee94e68858f1010dcc7d4ccffcab3ea72732638e9d826644d0"}
Sep 19 06:06:31.241120 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:06:31.241015    2583 scope.go:117] "RemoveContainer" containerID="8f72894924c2acec4fde6d793e84f915b842f6eb40ede8ec006314bc8f73153c"
Sep 19 06:06:31.253395 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:06:31.253371    2583 scope.go:117] "RemoveContainer" containerID="8f72894924c2acec4fde6d793e84f915b842f6eb40ede8ec006314bc8f73153c"
Sep 19 06:06:31.253754 ip-10-0-139-144 kubenswrapper[2583]: E0919 06:06:31.253727    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8f72894924c2acec4fde6d793e84f915b842f6eb40ede8ec006314bc8f73153c\": container with ID starting with 8f72894924c2acec4fde6d793e84f915b842f6eb40ede8ec006314bc8f73153c not found: ID does not exist" containerID="8f72894924c2acec4fde6d793e84f915b842f6eb40ede8ec006314bc8f73153c"
Sep 19 06:06:31.253855 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:06:31.253767    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f72894924c2acec4fde6d793e84f915b842f6eb40ede8ec006314bc8f73153c"} err="failed to get container status \"8f72894924c2acec4fde6d793e84f915b842f6eb40ede8ec006314bc8f73153c\": rpc error: code = NotFound desc = could not find container \"8f72894924c2acec4fde6d793e84f915b842f6eb40ede8ec006314bc8f73153c\": container with ID starting with 8f72894924c2acec4fde6d793e84f915b842f6eb40ede8ec006314bc8f73153c not found: ID does not exist"
Sep 19 06:06:31.270264 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:06:31.270217    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-311e2d-6b6d79b59c-2phcd"]
Sep 19 06:06:31.272882 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:06:31.272859    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-311e2d-6b6d79b59c-2phcd"]
Sep 19 06:06:31.499046 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:06:31.498963    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d637e261-4039-40e0-859f-2a105352401a" path="/var/lib/kubelet/pods/d637e261-4039-40e0-859f-2a105352401a/volumes"
Sep 19 06:06:53.101028 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:06:53.100991    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-aea238-58d9db7fc8-htghb"]
Sep 19 06:06:53.101628 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:06:53.101248    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-aea238-58d9db7fc8-htghb" podUID="912e198c-c64b-487a-b063-c8eb2b2386a4" containerName="maas-api" containerID="cri-o://8c0f9069f6ca2db1724af52858b88939db7300d537469e5d4bd2b6bb307f606a" gracePeriod=30
Sep 19 06:06:53.349245 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:06:53.349214    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-aea238-58d9db7fc8-htghb"
Sep 19 06:06:53.370938 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:06:53.370861    2583 generic.go:356] "Generic (PLEG): container finished" podID="912e198c-c64b-487a-b063-c8eb2b2386a4" containerID="8c0f9069f6ca2db1724af52858b88939db7300d537469e5d4bd2b6bb307f606a" exitCode=0
Sep 19 06:06:53.370938 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:06:53.370899    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-aea238-58d9db7fc8-htghb"
Sep 19 06:06:53.370938 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:06:53.370917    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-aea238-58d9db7fc8-htghb" event={"ID":"912e198c-c64b-487a-b063-c8eb2b2386a4","Type":"ContainerDied","Data":"8c0f9069f6ca2db1724af52858b88939db7300d537469e5d4bd2b6bb307f606a"}
Sep 19 06:06:53.371230 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:06:53.370970    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-aea238-58d9db7fc8-htghb" event={"ID":"912e198c-c64b-487a-b063-c8eb2b2386a4","Type":"ContainerDied","Data":"75c069811eb8c644bb100ee986e69defdd657b23d891b7949cf1644bc7630686"}
Sep 19 06:06:53.371230 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:06:53.371001    2583 scope.go:117] "RemoveContainer" containerID="8c0f9069f6ca2db1724af52858b88939db7300d537469e5d4bd2b6bb307f606a"
Sep 19 06:06:53.381697 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:06:53.381678    2583 scope.go:117] "RemoveContainer" containerID="8c0f9069f6ca2db1724af52858b88939db7300d537469e5d4bd2b6bb307f606a"
Sep 19 06:06:53.381964 ip-10-0-139-144 kubenswrapper[2583]: E0919 06:06:53.381941    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8c0f9069f6ca2db1724af52858b88939db7300d537469e5d4bd2b6bb307f606a\": container with ID starting with 8c0f9069f6ca2db1724af52858b88939db7300d537469e5d4bd2b6bb307f606a not found: ID does not exist" containerID="8c0f9069f6ca2db1724af52858b88939db7300d537469e5d4bd2b6bb307f606a"
Sep 19 06:06:53.382068 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:06:53.381970    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c0f9069f6ca2db1724af52858b88939db7300d537469e5d4bd2b6bb307f606a"} err="failed to get container status \"8c0f9069f6ca2db1724af52858b88939db7300d537469e5d4bd2b6bb307f606a\": rpc error: code = NotFound desc = could not find container \"8c0f9069f6ca2db1724af52858b88939db7300d537469e5d4bd2b6bb307f606a\": container with ID starting with 8c0f9069f6ca2db1724af52858b88939db7300d537469e5d4bd2b6bb307f606a not found: ID does not exist"
Sep 19 06:06:53.498344 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:06:53.497441    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/912e198c-c64b-487a-b063-c8eb2b2386a4-maas-api-tls\") pod \"912e198c-c64b-487a-b063-c8eb2b2386a4\" (UID: \"912e198c-c64b-487a-b063-c8eb2b2386a4\") "
Sep 19 06:06:53.498344 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:06:53.497530    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c5pcm\" (UniqueName: \"kubernetes.io/projected/912e198c-c64b-487a-b063-c8eb2b2386a4-kube-api-access-c5pcm\") pod \"912e198c-c64b-487a-b063-c8eb2b2386a4\" (UID: \"912e198c-c64b-487a-b063-c8eb2b2386a4\") "
Sep 19 06:06:53.498344 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:06:53.497580    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/912e198c-c64b-487a-b063-c8eb2b2386a4-openshift-service-ca\") pod \"912e198c-c64b-487a-b063-c8eb2b2386a4\" (UID: \"912e198c-c64b-487a-b063-c8eb2b2386a4\") "
Sep 19 06:06:53.498344 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:06:53.497723    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/912e198c-c64b-487a-b063-c8eb2b2386a4-metrics-certs\") pod \"912e198c-c64b-487a-b063-c8eb2b2386a4\" (UID: \"912e198c-c64b-487a-b063-c8eb2b2386a4\") "
Sep 19 06:06:53.502509 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:06:53.502475    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/912e198c-c64b-487a-b063-c8eb2b2386a4-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "912e198c-c64b-487a-b063-c8eb2b2386a4" (UID: "912e198c-c64b-487a-b063-c8eb2b2386a4"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 06:06:53.515614 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:06:53.515393    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/912e198c-c64b-487a-b063-c8eb2b2386a4-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "912e198c-c64b-487a-b063-c8eb2b2386a4" (UID: "912e198c-c64b-487a-b063-c8eb2b2386a4"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 06:06:53.516909 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:06:53.516882    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/912e198c-c64b-487a-b063-c8eb2b2386a4-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "912e198c-c64b-487a-b063-c8eb2b2386a4" (UID: "912e198c-c64b-487a-b063-c8eb2b2386a4"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 06:06:53.524355 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:06:53.524322    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/912e198c-c64b-487a-b063-c8eb2b2386a4-kube-api-access-c5pcm" (OuterVolumeSpecName: "kube-api-access-c5pcm") pod "912e198c-c64b-487a-b063-c8eb2b2386a4" (UID: "912e198c-c64b-487a-b063-c8eb2b2386a4"). InnerVolumeSpecName "kube-api-access-c5pcm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 06:06:53.598406 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:06:53.598361    2583 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/912e198c-c64b-487a-b063-c8eb2b2386a4-metrics-certs\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 06:06:53.598406 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:06:53.598392    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/912e198c-c64b-487a-b063-c8eb2b2386a4-maas-api-tls\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 06:06:53.598406 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:06:53.598401    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c5pcm\" (UniqueName: \"kubernetes.io/projected/912e198c-c64b-487a-b063-c8eb2b2386a4-kube-api-access-c5pcm\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 06:06:53.598406 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:06:53.598411    2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/912e198c-c64b-487a-b063-c8eb2b2386a4-openshift-service-ca\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 06:06:53.699811 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:06:53.699771    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-aea238-58d9db7fc8-htghb"]
Sep 19 06:06:53.705244 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:06:53.705209    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-aea238-58d9db7fc8-htghb"]
Sep 19 06:06:55.501857 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:06:55.501821    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="912e198c-c64b-487a-b063-c8eb2b2386a4" path="/var/lib/kubelet/pods/912e198c-c64b-487a-b063-c8eb2b2386a4/volumes"
Sep 19 06:06:59.625428 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:06:59.625397    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-aea238-openshift-default-7579fbf658-nmd8x"]
Sep 19 06:06:59.625858 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:06:59.625636    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-aea238-openshift-default-7579fbf658-nmd8x" podUID="484a06e7-0464-448b-86aa-1f166f59dfca" containerName="istio-proxy" containerID="cri-o://74fc93b5b02f27b9c7e16b01cfbd8d7b3b0342b651bc4e2ac96672d372653b8b" gracePeriod=30
Sep 19 06:07:00.667063 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:00.666735    2583 patch_prober.go:28] interesting pod/e2e-api-b-aea238-openshift-default-7579fbf658-nmd8x container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 19 06:07:00.667063 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:00.666815    2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-aea238-openshift-default-7579fbf658-nmd8x" podUID="484a06e7-0464-448b-86aa-1f166f59dfca" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 19 06:07:04.889448 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:04.889389    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-aea238-openshift-default-7579fbf658-nmd8x"
Sep 19 06:07:05.005373 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:05.005339    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/484a06e7-0464-448b-86aa-1f166f59dfca-istio-envoy\") pod \"484a06e7-0464-448b-86aa-1f166f59dfca\" (UID: \"484a06e7-0464-448b-86aa-1f166f59dfca\") "
Sep 19 06:07:05.005373 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:05.005377    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/484a06e7-0464-448b-86aa-1f166f59dfca-workload-socket\") pod \"484a06e7-0464-448b-86aa-1f166f59dfca\" (UID: \"484a06e7-0464-448b-86aa-1f166f59dfca\") "
Sep 19 06:07:05.005627 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:05.005412    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l9cn8\" (UniqueName: \"kubernetes.io/projected/484a06e7-0464-448b-86aa-1f166f59dfca-kube-api-access-l9cn8\") pod \"484a06e7-0464-448b-86aa-1f166f59dfca\" (UID: \"484a06e7-0464-448b-86aa-1f166f59dfca\") "
Sep 19 06:07:05.005627 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:05.005483    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/484a06e7-0464-448b-86aa-1f166f59dfca-workload-certs\") pod \"484a06e7-0464-448b-86aa-1f166f59dfca\" (UID: \"484a06e7-0464-448b-86aa-1f166f59dfca\") "
Sep 19 06:07:05.005627 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:05.005498    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/484a06e7-0464-448b-86aa-1f166f59dfca-istio-token\") pod \"484a06e7-0464-448b-86aa-1f166f59dfca\" (UID: \"484a06e7-0464-448b-86aa-1f166f59dfca\") "
Sep 19 06:07:05.005627 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:05.005528    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/484a06e7-0464-448b-86aa-1f166f59dfca-istiod-ca-cert\") pod \"484a06e7-0464-448b-86aa-1f166f59dfca\" (UID: \"484a06e7-0464-448b-86aa-1f166f59dfca\") "
Sep 19 06:07:05.005627 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:05.005562    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/484a06e7-0464-448b-86aa-1f166f59dfca-istio-podinfo\") pod \"484a06e7-0464-448b-86aa-1f166f59dfca\" (UID: \"484a06e7-0464-448b-86aa-1f166f59dfca\") "
Sep 19 06:07:05.005627 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:05.005615    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/484a06e7-0464-448b-86aa-1f166f59dfca-credential-socket\") pod \"484a06e7-0464-448b-86aa-1f166f59dfca\" (UID: \"484a06e7-0464-448b-86aa-1f166f59dfca\") "
Sep 19 06:07:05.005941 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:05.005650    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/484a06e7-0464-448b-86aa-1f166f59dfca-istio-data\") pod \"484a06e7-0464-448b-86aa-1f166f59dfca\" (UID: \"484a06e7-0464-448b-86aa-1f166f59dfca\") "
Sep 19 06:07:05.005941 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:05.005700    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/484a06e7-0464-448b-86aa-1f166f59dfca-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "484a06e7-0464-448b-86aa-1f166f59dfca" (UID: "484a06e7-0464-448b-86aa-1f166f59dfca"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 06:07:05.006045 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:05.005933    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/484a06e7-0464-448b-86aa-1f166f59dfca-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "484a06e7-0464-448b-86aa-1f166f59dfca" (UID: "484a06e7-0464-448b-86aa-1f166f59dfca"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 06:07:05.006045 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:05.005950    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/484a06e7-0464-448b-86aa-1f166f59dfca-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "484a06e7-0464-448b-86aa-1f166f59dfca" (UID: "484a06e7-0464-448b-86aa-1f166f59dfca"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 06:07:05.006045 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:05.005992    2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/484a06e7-0464-448b-86aa-1f166f59dfca-workload-socket\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 06:07:05.006045 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:05.005998    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/484a06e7-0464-448b-86aa-1f166f59dfca-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "484a06e7-0464-448b-86aa-1f166f59dfca" (UID: "484a06e7-0464-448b-86aa-1f166f59dfca"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 06:07:05.006293 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:05.006190    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/484a06e7-0464-448b-86aa-1f166f59dfca-istio-data" (OuterVolumeSpecName: "istio-data") pod "484a06e7-0464-448b-86aa-1f166f59dfca" (UID: "484a06e7-0464-448b-86aa-1f166f59dfca"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 06:07:05.007730 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:05.007696    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/484a06e7-0464-448b-86aa-1f166f59dfca-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "484a06e7-0464-448b-86aa-1f166f59dfca" (UID: "484a06e7-0464-448b-86aa-1f166f59dfca"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 19 06:07:05.007872 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:05.007772    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/484a06e7-0464-448b-86aa-1f166f59dfca-istio-token" (OuterVolumeSpecName: "istio-token") pod "484a06e7-0464-448b-86aa-1f166f59dfca" (UID: "484a06e7-0464-448b-86aa-1f166f59dfca"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 06:07:05.007872 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:05.007775    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/484a06e7-0464-448b-86aa-1f166f59dfca-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "484a06e7-0464-448b-86aa-1f166f59dfca" (UID: "484a06e7-0464-448b-86aa-1f166f59dfca"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 06:07:05.007970 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:05.007876    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/484a06e7-0464-448b-86aa-1f166f59dfca-kube-api-access-l9cn8" (OuterVolumeSpecName: "kube-api-access-l9cn8") pod "484a06e7-0464-448b-86aa-1f166f59dfca" (UID: "484a06e7-0464-448b-86aa-1f166f59dfca"). InnerVolumeSpecName "kube-api-access-l9cn8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 06:07:05.113220 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:05.113174    2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/484a06e7-0464-448b-86aa-1f166f59dfca-istio-envoy\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 06:07:05.113220 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:05.113225    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l9cn8\" (UniqueName: \"kubernetes.io/projected/484a06e7-0464-448b-86aa-1f166f59dfca-kube-api-access-l9cn8\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 06:07:05.113471 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:05.113252    2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/484a06e7-0464-448b-86aa-1f166f59dfca-workload-certs\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 06:07:05.113471 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:05.113273    2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/484a06e7-0464-448b-86aa-1f166f59dfca-istio-token\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 06:07:05.113471 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:05.113286    2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/484a06e7-0464-448b-86aa-1f166f59dfca-istiod-ca-cert\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 06:07:05.113471 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:05.113298    2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/484a06e7-0464-448b-86aa-1f166f59dfca-istio-podinfo\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 06:07:05.113471 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:05.113310    2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/484a06e7-0464-448b-86aa-1f166f59dfca-credential-socket\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 06:07:05.113471 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:05.113328    2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/484a06e7-0464-448b-86aa-1f166f59dfca-istio-data\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 06:07:05.443595 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:05.443557    2583 generic.go:356] "Generic (PLEG): container finished" podID="484a06e7-0464-448b-86aa-1f166f59dfca" containerID="74fc93b5b02f27b9c7e16b01cfbd8d7b3b0342b651bc4e2ac96672d372653b8b" exitCode=0
Sep 19 06:07:05.443796 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:05.443610    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-aea238-openshift-default-7579fbf658-nmd8x"
Sep 19 06:07:05.443796 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:05.443627    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-aea238-openshift-default-7579fbf658-nmd8x" event={"ID":"484a06e7-0464-448b-86aa-1f166f59dfca","Type":"ContainerDied","Data":"74fc93b5b02f27b9c7e16b01cfbd8d7b3b0342b651bc4e2ac96672d372653b8b"}
Sep 19 06:07:05.443796 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:05.443711    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-aea238-openshift-default-7579fbf658-nmd8x" event={"ID":"484a06e7-0464-448b-86aa-1f166f59dfca","Type":"ContainerDied","Data":"728a6a6135f887363b5baee225ec7f8f87f08bb2508bb104bcf959b61278c33e"}
Sep 19 06:07:05.443796 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:05.443744    2583 scope.go:117] "RemoveContainer" containerID="74fc93b5b02f27b9c7e16b01cfbd8d7b3b0342b651bc4e2ac96672d372653b8b"
Sep 19 06:07:05.453864 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:05.453841    2583 scope.go:117] "RemoveContainer" containerID="f6624bb13b86bb691f1e70fc28b04636cf3be78d83a3fac105a114f8e85968ee"
Sep 19 06:07:05.462084 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:05.462067    2583 scope.go:117] "RemoveContainer" containerID="74fc93b5b02f27b9c7e16b01cfbd8d7b3b0342b651bc4e2ac96672d372653b8b"
Sep 19 06:07:05.462341 ip-10-0-139-144 kubenswrapper[2583]: E0919 06:07:05.462320    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"74fc93b5b02f27b9c7e16b01cfbd8d7b3b0342b651bc4e2ac96672d372653b8b\": container with ID starting with 74fc93b5b02f27b9c7e16b01cfbd8d7b3b0342b651bc4e2ac96672d372653b8b not found: ID does not exist" containerID="74fc93b5b02f27b9c7e16b01cfbd8d7b3b0342b651bc4e2ac96672d372653b8b"
Sep 19 06:07:05.462400 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:05.462348    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"74fc93b5b02f27b9c7e16b01cfbd8d7b3b0342b651bc4e2ac96672d372653b8b"} err="failed to get container status \"74fc93b5b02f27b9c7e16b01cfbd8d7b3b0342b651bc4e2ac96672d372653b8b\": rpc error: code = NotFound desc = could not find container \"74fc93b5b02f27b9c7e16b01cfbd8d7b3b0342b651bc4e2ac96672d372653b8b\": container with ID starting with 74fc93b5b02f27b9c7e16b01cfbd8d7b3b0342b651bc4e2ac96672d372653b8b not found: ID does not exist"
Sep 19 06:07:05.462400 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:05.462365    2583 scope.go:117] "RemoveContainer" containerID="f6624bb13b86bb691f1e70fc28b04636cf3be78d83a3fac105a114f8e85968ee"
Sep 19 06:07:05.462615 ip-10-0-139-144 kubenswrapper[2583]: E0919 06:07:05.462601    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f6624bb13b86bb691f1e70fc28b04636cf3be78d83a3fac105a114f8e85968ee\": container with ID starting with f6624bb13b86bb691f1e70fc28b04636cf3be78d83a3fac105a114f8e85968ee not found: ID does not exist" containerID="f6624bb13b86bb691f1e70fc28b04636cf3be78d83a3fac105a114f8e85968ee"
Sep 19 06:07:05.462665 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:05.462618    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f6624bb13b86bb691f1e70fc28b04636cf3be78d83a3fac105a114f8e85968ee"} err="failed to get container status \"f6624bb13b86bb691f1e70fc28b04636cf3be78d83a3fac105a114f8e85968ee\": rpc error: code = NotFound desc = could not find container \"f6624bb13b86bb691f1e70fc28b04636cf3be78d83a3fac105a114f8e85968ee\": container with ID starting with f6624bb13b86bb691f1e70fc28b04636cf3be78d83a3fac105a114f8e85968ee not found: ID does not exist"
Sep 19 06:07:05.466932 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:05.466909    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-aea238-openshift-default-7579fbf658-nmd8x"]
Sep 19 06:07:05.470554 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:05.470531    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-aea238-openshift-default-7579fbf658-nmd8x"]
Sep 19 06:07:05.498423 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:05.498401    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="484a06e7-0464-448b-86aa-1f166f59dfca" path="/var/lib/kubelet/pods/484a06e7-0464-448b-86aa-1f166f59dfca/volumes"
Sep 19 06:07:10.994909 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:10.994871    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-2b3fb3-54449d8cc4-6n8n8"]
Sep 19 06:07:10.996220 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:10.996162    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-2b3fb3-54449d8cc4-6n8n8" podUID="635f4638-2b98-4ff8-b247-cf98593ef0ca" containerName="maas-api" containerID="cri-o://a2616a6d9b36ac6905de31bce89ae877b79782960b4c506f74eafb0121d4dbd2" gracePeriod=30
Sep 19 06:07:11.258082 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:11.258025    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-2b3fb3-54449d8cc4-6n8n8"
Sep 19 06:07:11.370592 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:11.370557    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c7l8w\" (UniqueName: \"kubernetes.io/projected/635f4638-2b98-4ff8-b247-cf98593ef0ca-kube-api-access-c7l8w\") pod \"635f4638-2b98-4ff8-b247-cf98593ef0ca\" (UID: \"635f4638-2b98-4ff8-b247-cf98593ef0ca\") "
Sep 19 06:07:11.370759 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:11.370639    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/635f4638-2b98-4ff8-b247-cf98593ef0ca-maas-api-tls\") pod \"635f4638-2b98-4ff8-b247-cf98593ef0ca\" (UID: \"635f4638-2b98-4ff8-b247-cf98593ef0ca\") "
Sep 19 06:07:11.370759 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:11.370685    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/635f4638-2b98-4ff8-b247-cf98593ef0ca-metrics-certs\") pod \"635f4638-2b98-4ff8-b247-cf98593ef0ca\" (UID: \"635f4638-2b98-4ff8-b247-cf98593ef0ca\") "
Sep 19 06:07:11.370759 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:11.370708    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/635f4638-2b98-4ff8-b247-cf98593ef0ca-openshift-service-ca\") pod \"635f4638-2b98-4ff8-b247-cf98593ef0ca\" (UID: \"635f4638-2b98-4ff8-b247-cf98593ef0ca\") "
Sep 19 06:07:11.371090 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:11.371057    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/635f4638-2b98-4ff8-b247-cf98593ef0ca-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "635f4638-2b98-4ff8-b247-cf98593ef0ca" (UID: "635f4638-2b98-4ff8-b247-cf98593ef0ca"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 06:07:11.372878 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:11.372848    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/635f4638-2b98-4ff8-b247-cf98593ef0ca-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "635f4638-2b98-4ff8-b247-cf98593ef0ca" (UID: "635f4638-2b98-4ff8-b247-cf98593ef0ca"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 06:07:11.372878 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:11.372866    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/635f4638-2b98-4ff8-b247-cf98593ef0ca-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "635f4638-2b98-4ff8-b247-cf98593ef0ca" (UID: "635f4638-2b98-4ff8-b247-cf98593ef0ca"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 06:07:11.373016 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:11.372870    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/635f4638-2b98-4ff8-b247-cf98593ef0ca-kube-api-access-c7l8w" (OuterVolumeSpecName: "kube-api-access-c7l8w") pod "635f4638-2b98-4ff8-b247-cf98593ef0ca" (UID: "635f4638-2b98-4ff8-b247-cf98593ef0ca"). InnerVolumeSpecName "kube-api-access-c7l8w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 06:07:11.471686 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:11.471645    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c7l8w\" (UniqueName: \"kubernetes.io/projected/635f4638-2b98-4ff8-b247-cf98593ef0ca-kube-api-access-c7l8w\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 06:07:11.471686 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:11.471680    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/635f4638-2b98-4ff8-b247-cf98593ef0ca-maas-api-tls\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 06:07:11.471686 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:11.471689    2583 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/635f4638-2b98-4ff8-b247-cf98593ef0ca-metrics-certs\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 06:07:11.471686 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:11.471697    2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/635f4638-2b98-4ff8-b247-cf98593ef0ca-openshift-service-ca\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 06:07:11.488642 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:11.488610    2583 generic.go:356] "Generic (PLEG): container finished" podID="635f4638-2b98-4ff8-b247-cf98593ef0ca" containerID="a2616a6d9b36ac6905de31bce89ae877b79782960b4c506f74eafb0121d4dbd2" exitCode=0
Sep 19 06:07:11.488802 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:11.488655    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-2b3fb3-54449d8cc4-6n8n8"
Sep 19 06:07:11.488802 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:11.488693    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-2b3fb3-54449d8cc4-6n8n8" event={"ID":"635f4638-2b98-4ff8-b247-cf98593ef0ca","Type":"ContainerDied","Data":"a2616a6d9b36ac6905de31bce89ae877b79782960b4c506f74eafb0121d4dbd2"}
Sep 19 06:07:11.488802 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:11.488748    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-2b3fb3-54449d8cc4-6n8n8" event={"ID":"635f4638-2b98-4ff8-b247-cf98593ef0ca","Type":"ContainerDied","Data":"23d9c1ed4daf9019b3b62fb4d80be7d9c87b04456f139c44c0fa373232df308d"}
Sep 19 06:07:11.488802 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:11.488774    2583 scope.go:117] "RemoveContainer" containerID="a2616a6d9b36ac6905de31bce89ae877b79782960b4c506f74eafb0121d4dbd2"
Sep 19 06:07:11.498890 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:11.498871    2583 scope.go:117] "RemoveContainer" containerID="a2616a6d9b36ac6905de31bce89ae877b79782960b4c506f74eafb0121d4dbd2"
Sep 19 06:07:11.499142 ip-10-0-139-144 kubenswrapper[2583]: E0919 06:07:11.499113    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a2616a6d9b36ac6905de31bce89ae877b79782960b4c506f74eafb0121d4dbd2\": container with ID starting with a2616a6d9b36ac6905de31bce89ae877b79782960b4c506f74eafb0121d4dbd2 not found: ID does not exist" containerID="a2616a6d9b36ac6905de31bce89ae877b79782960b4c506f74eafb0121d4dbd2"
Sep 19 06:07:11.499245 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:11.499174    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a2616a6d9b36ac6905de31bce89ae877b79782960b4c506f74eafb0121d4dbd2"} err="failed to get container status \"a2616a6d9b36ac6905de31bce89ae877b79782960b4c506f74eafb0121d4dbd2\": rpc error: code = NotFound desc = could not find container \"a2616a6d9b36ac6905de31bce89ae877b79782960b4c506f74eafb0121d4dbd2\": container with ID starting with a2616a6d9b36ac6905de31bce89ae877b79782960b4c506f74eafb0121d4dbd2 not found: ID does not exist"
Sep 19 06:07:11.510357 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:11.510300    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-2b3fb3-54449d8cc4-6n8n8"]
Sep 19 06:07:11.514264 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:11.514239    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-2b3fb3-54449d8cc4-6n8n8"]
Sep 19 06:07:13.498811 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:13.498781    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="635f4638-2b98-4ff8-b247-cf98593ef0ca" path="/var/lib/kubelet/pods/635f4638-2b98-4ff8-b247-cf98593ef0ca/volumes"
Sep 19 06:07:17.602279 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:17.602244    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-3182b5-d6c9696c6-jwzw7"]
Sep 19 06:07:17.602746 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:17.602472    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-3182b5-d6c9696c6-jwzw7" podUID="1980603d-cfad-4f61-959c-e9b7507da10f" containerName="maas-api" containerID="cri-o://5bc6d41a7bb6e4fed869ab419b59e36e25d438ff862af39e09dd81b41419d2f9" gracePeriod=30
Sep 19 06:07:18.081745 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:18.081720    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-3182b5-d6c9696c6-jwzw7"
Sep 19 06:07:18.239305 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:18.239222    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1980603d-cfad-4f61-959c-e9b7507da10f-metrics-certs\") pod \"1980603d-cfad-4f61-959c-e9b7507da10f\" (UID: \"1980603d-cfad-4f61-959c-e9b7507da10f\") "
Sep 19 06:07:18.239467 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:18.239342    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1980603d-cfad-4f61-959c-e9b7507da10f-openshift-service-ca\") pod \"1980603d-cfad-4f61-959c-e9b7507da10f\" (UID: \"1980603d-cfad-4f61-959c-e9b7507da10f\") "
Sep 19 06:07:18.239467 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:18.239391    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1980603d-cfad-4f61-959c-e9b7507da10f-maas-api-tls\") pod \"1980603d-cfad-4f61-959c-e9b7507da10f\" (UID: \"1980603d-cfad-4f61-959c-e9b7507da10f\") "
Sep 19 06:07:18.239467 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:18.239409    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-swnm2\" (UniqueName: \"kubernetes.io/projected/1980603d-cfad-4f61-959c-e9b7507da10f-kube-api-access-swnm2\") pod \"1980603d-cfad-4f61-959c-e9b7507da10f\" (UID: \"1980603d-cfad-4f61-959c-e9b7507da10f\") "
Sep 19 06:07:18.239794 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:18.239757    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1980603d-cfad-4f61-959c-e9b7507da10f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "1980603d-cfad-4f61-959c-e9b7507da10f" (UID: "1980603d-cfad-4f61-959c-e9b7507da10f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 06:07:18.241549 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:18.241519    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1980603d-cfad-4f61-959c-e9b7507da10f-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "1980603d-cfad-4f61-959c-e9b7507da10f" (UID: "1980603d-cfad-4f61-959c-e9b7507da10f"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 06:07:18.241640 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:18.241543    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1980603d-cfad-4f61-959c-e9b7507da10f-kube-api-access-swnm2" (OuterVolumeSpecName: "kube-api-access-swnm2") pod "1980603d-cfad-4f61-959c-e9b7507da10f" (UID: "1980603d-cfad-4f61-959c-e9b7507da10f"). InnerVolumeSpecName "kube-api-access-swnm2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 06:07:18.241640 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:18.241604    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1980603d-cfad-4f61-959c-e9b7507da10f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "1980603d-cfad-4f61-959c-e9b7507da10f" (UID: "1980603d-cfad-4f61-959c-e9b7507da10f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 06:07:18.340348 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:18.340316    2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1980603d-cfad-4f61-959c-e9b7507da10f-openshift-service-ca\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 06:07:18.340348 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:18.340347    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1980603d-cfad-4f61-959c-e9b7507da10f-maas-api-tls\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 06:07:18.340348 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:18.340357    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-swnm2\" (UniqueName: \"kubernetes.io/projected/1980603d-cfad-4f61-959c-e9b7507da10f-kube-api-access-swnm2\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 06:07:18.340572 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:18.340366    2583 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1980603d-cfad-4f61-959c-e9b7507da10f-metrics-certs\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 06:07:18.528123 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:18.528087    2583 generic.go:356] "Generic (PLEG): container finished" podID="1980603d-cfad-4f61-959c-e9b7507da10f" containerID="5bc6d41a7bb6e4fed869ab419b59e36e25d438ff862af39e09dd81b41419d2f9" exitCode=0
Sep 19 06:07:18.528123 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:18.528139    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-3182b5-d6c9696c6-jwzw7"
Sep 19 06:07:18.528359 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:18.528175    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-3182b5-d6c9696c6-jwzw7" event={"ID":"1980603d-cfad-4f61-959c-e9b7507da10f","Type":"ContainerDied","Data":"5bc6d41a7bb6e4fed869ab419b59e36e25d438ff862af39e09dd81b41419d2f9"}
Sep 19 06:07:18.528359 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:18.528226    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-3182b5-d6c9696c6-jwzw7" event={"ID":"1980603d-cfad-4f61-959c-e9b7507da10f","Type":"ContainerDied","Data":"ec0972eb601f5d8dec482e0537334b019e75a20c43f8dcd55032313b8147ff52"}
Sep 19 06:07:18.528359 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:18.528255    2583 scope.go:117] "RemoveContainer" containerID="5bc6d41a7bb6e4fed869ab419b59e36e25d438ff862af39e09dd81b41419d2f9"
Sep 19 06:07:18.541717 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:18.541694    2583 scope.go:117] "RemoveContainer" containerID="5bc6d41a7bb6e4fed869ab419b59e36e25d438ff862af39e09dd81b41419d2f9"
Sep 19 06:07:18.542062 ip-10-0-139-144 kubenswrapper[2583]: E0919 06:07:18.542040    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5bc6d41a7bb6e4fed869ab419b59e36e25d438ff862af39e09dd81b41419d2f9\": container with ID starting with 5bc6d41a7bb6e4fed869ab419b59e36e25d438ff862af39e09dd81b41419d2f9 not found: ID does not exist" containerID="5bc6d41a7bb6e4fed869ab419b59e36e25d438ff862af39e09dd81b41419d2f9"
Sep 19 06:07:18.542187 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:18.542072    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5bc6d41a7bb6e4fed869ab419b59e36e25d438ff862af39e09dd81b41419d2f9"} err="failed to get container status \"5bc6d41a7bb6e4fed869ab419b59e36e25d438ff862af39e09dd81b41419d2f9\": rpc error: code = NotFound desc = could not find container \"5bc6d41a7bb6e4fed869ab419b59e36e25d438ff862af39e09dd81b41419d2f9\": container with ID starting with 5bc6d41a7bb6e4fed869ab419b59e36e25d438ff862af39e09dd81b41419d2f9 not found: ID does not exist"
Sep 19 06:07:18.550043 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:18.550021    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-3182b5-d6c9696c6-jwzw7"]
Sep 19 06:07:18.553413 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:18.553385    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-3182b5-d6c9696c6-jwzw7"]
Sep 19 06:07:19.499774 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:19.499738    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1980603d-cfad-4f61-959c-e9b7507da10f" path="/var/lib/kubelet/pods/1980603d-cfad-4f61-959c-e9b7507da10f/volumes"
Sep 19 06:07:23.435001 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:23.434963    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-f1aad1/auth-test-model-f1aad1-kserve-57476cc487-jqqzm"]
Sep 19 06:07:24.173289 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:24.173251    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-3182b5-openshift-default-76dd9fb6bb-b5p7s"]
Sep 19 06:07:24.173531 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:24.173497    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-3182b5-openshift-default-76dd9fb6bb-b5p7s" podUID="b57edf34-4a41-4cb6-bc1f-f73dae8adad1" containerName="istio-proxy" containerID="cri-o://e69dfe65342b7667dfb020cee99016f94a0570281307afc79f6487d15004b2e3" gracePeriod=30
Sep 19 06:07:25.477426 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:25.477342    2583 patch_prober.go:28] interesting pod/e2e-worker-w4-3182b5-openshift-default-76dd9fb6bb-b5p7s container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 19 06:07:25.477426 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:25.477407    2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-3182b5-openshift-default-76dd9fb6bb-b5p7s" podUID="b57edf34-4a41-4cb6-bc1f-f73dae8adad1" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 19 06:07:29.447908 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:29.447853    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-3182b5-openshift-default-76dd9fb6bb-b5p7s"
Sep 19 06:07:29.550459 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:29.550426    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b57edf34-4a41-4cb6-bc1f-f73dae8adad1-istio-envoy\") pod \"b57edf34-4a41-4cb6-bc1f-f73dae8adad1\" (UID: \"b57edf34-4a41-4cb6-bc1f-f73dae8adad1\") "
Sep 19 06:07:29.550639 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:29.550472    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b57edf34-4a41-4cb6-bc1f-f73dae8adad1-credential-socket\") pod \"b57edf34-4a41-4cb6-bc1f-f73dae8adad1\" (UID: \"b57edf34-4a41-4cb6-bc1f-f73dae8adad1\") "
Sep 19 06:07:29.550639 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:29.550504    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b57edf34-4a41-4cb6-bc1f-f73dae8adad1-istiod-ca-cert\") pod \"b57edf34-4a41-4cb6-bc1f-f73dae8adad1\" (UID: \"b57edf34-4a41-4cb6-bc1f-f73dae8adad1\") "
Sep 19 06:07:29.550639 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:29.550525    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b57edf34-4a41-4cb6-bc1f-f73dae8adad1-workload-certs\") pod \"b57edf34-4a41-4cb6-bc1f-f73dae8adad1\" (UID: \"b57edf34-4a41-4cb6-bc1f-f73dae8adad1\") "
Sep 19 06:07:29.550639 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:29.550555    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b57edf34-4a41-4cb6-bc1f-f73dae8adad1-istio-data\") pod \"b57edf34-4a41-4cb6-bc1f-f73dae8adad1\" (UID: \"b57edf34-4a41-4cb6-bc1f-f73dae8adad1\") "
Sep 19 06:07:29.550639 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:29.550580    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dn57q\" (UniqueName: \"kubernetes.io/projected/b57edf34-4a41-4cb6-bc1f-f73dae8adad1-kube-api-access-dn57q\") pod \"b57edf34-4a41-4cb6-bc1f-f73dae8adad1\" (UID: \"b57edf34-4a41-4cb6-bc1f-f73dae8adad1\") "
Sep 19 06:07:29.550639 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:29.550625    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b57edf34-4a41-4cb6-bc1f-f73dae8adad1-istio-podinfo\") pod \"b57edf34-4a41-4cb6-bc1f-f73dae8adad1\" (UID: \"b57edf34-4a41-4cb6-bc1f-f73dae8adad1\") "
Sep 19 06:07:29.551015 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:29.550688    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b57edf34-4a41-4cb6-bc1f-f73dae8adad1-istio-token\") pod \"b57edf34-4a41-4cb6-bc1f-f73dae8adad1\" (UID: \"b57edf34-4a41-4cb6-bc1f-f73dae8adad1\") "
Sep 19 06:07:29.551015 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:29.550729    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b57edf34-4a41-4cb6-bc1f-f73dae8adad1-workload-socket\") pod \"b57edf34-4a41-4cb6-bc1f-f73dae8adad1\" (UID: \"b57edf34-4a41-4cb6-bc1f-f73dae8adad1\") "
Sep 19 06:07:29.551015 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:29.550845    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b57edf34-4a41-4cb6-bc1f-f73dae8adad1-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b57edf34-4a41-4cb6-bc1f-f73dae8adad1" (UID: "b57edf34-4a41-4cb6-bc1f-f73dae8adad1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 06:07:29.551015 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:29.550894    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b57edf34-4a41-4cb6-bc1f-f73dae8adad1-istio-data" (OuterVolumeSpecName: "istio-data") pod "b57edf34-4a41-4cb6-bc1f-f73dae8adad1" (UID: "b57edf34-4a41-4cb6-bc1f-f73dae8adad1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 06:07:29.551015 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:29.550911    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b57edf34-4a41-4cb6-bc1f-f73dae8adad1-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b57edf34-4a41-4cb6-bc1f-f73dae8adad1" (UID: "b57edf34-4a41-4cb6-bc1f-f73dae8adad1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 06:07:29.551308 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:29.551158    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b57edf34-4a41-4cb6-bc1f-f73dae8adad1-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b57edf34-4a41-4cb6-bc1f-f73dae8adad1" (UID: "b57edf34-4a41-4cb6-bc1f-f73dae8adad1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 06:07:29.551308 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:29.551168    2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b57edf34-4a41-4cb6-bc1f-f73dae8adad1-credential-socket\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 06:07:29.551308 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:29.551192    2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b57edf34-4a41-4cb6-bc1f-f73dae8adad1-workload-certs\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 06:07:29.551308 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:29.551206    2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b57edf34-4a41-4cb6-bc1f-f73dae8adad1-istio-data\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 06:07:29.551308 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:29.551258    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b57edf34-4a41-4cb6-bc1f-f73dae8adad1-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b57edf34-4a41-4cb6-bc1f-f73dae8adad1" (UID: "b57edf34-4a41-4cb6-bc1f-f73dae8adad1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 06:07:29.552766 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:29.552745    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b57edf34-4a41-4cb6-bc1f-f73dae8adad1-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b57edf34-4a41-4cb6-bc1f-f73dae8adad1" (UID: "b57edf34-4a41-4cb6-bc1f-f73dae8adad1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 19 06:07:29.552923 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:29.552910    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b57edf34-4a41-4cb6-bc1f-f73dae8adad1-istio-token" (OuterVolumeSpecName: "istio-token") pod "b57edf34-4a41-4cb6-bc1f-f73dae8adad1" (UID: "b57edf34-4a41-4cb6-bc1f-f73dae8adad1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 06:07:29.553010 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:29.552996    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b57edf34-4a41-4cb6-bc1f-f73dae8adad1-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b57edf34-4a41-4cb6-bc1f-f73dae8adad1" (UID: "b57edf34-4a41-4cb6-bc1f-f73dae8adad1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 06:07:29.553110 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:29.553085    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b57edf34-4a41-4cb6-bc1f-f73dae8adad1-kube-api-access-dn57q" (OuterVolumeSpecName: "kube-api-access-dn57q") pod "b57edf34-4a41-4cb6-bc1f-f73dae8adad1" (UID: "b57edf34-4a41-4cb6-bc1f-f73dae8adad1"). InnerVolumeSpecName "kube-api-access-dn57q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 06:07:29.592291 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:29.592260    2583 generic.go:356] "Generic (PLEG): container finished" podID="b57edf34-4a41-4cb6-bc1f-f73dae8adad1" containerID="e69dfe65342b7667dfb020cee99016f94a0570281307afc79f6487d15004b2e3" exitCode=0
Sep 19 06:07:29.592425 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:29.592322    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-3182b5-openshift-default-76dd9fb6bb-b5p7s"
Sep 19 06:07:29.592425 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:29.592350    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-3182b5-openshift-default-76dd9fb6bb-b5p7s" event={"ID":"b57edf34-4a41-4cb6-bc1f-f73dae8adad1","Type":"ContainerDied","Data":"e69dfe65342b7667dfb020cee99016f94a0570281307afc79f6487d15004b2e3"}
Sep 19 06:07:29.592425 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:29.592404    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-3182b5-openshift-default-76dd9fb6bb-b5p7s" event={"ID":"b57edf34-4a41-4cb6-bc1f-f73dae8adad1","Type":"ContainerDied","Data":"1cf10e3ed82925e97de55f0e82102d5b297ce34566c4560e174216ca5a389484"}
Sep 19 06:07:29.592425 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:29.592425    2583 scope.go:117] "RemoveContainer" containerID="e69dfe65342b7667dfb020cee99016f94a0570281307afc79f6487d15004b2e3"
Sep 19 06:07:29.602352 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:29.602337    2583 scope.go:117] "RemoveContainer" containerID="718372ca6c871216a911c583987ab7a96ce5d8ea38e760b37b4a66ec0d50da8e"
Sep 19 06:07:29.610611 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:29.610592    2583 scope.go:117] "RemoveContainer" containerID="e69dfe65342b7667dfb020cee99016f94a0570281307afc79f6487d15004b2e3"
Sep 19 06:07:29.610852 ip-10-0-139-144 kubenswrapper[2583]: E0919 06:07:29.610832    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e69dfe65342b7667dfb020cee99016f94a0570281307afc79f6487d15004b2e3\": container with ID starting with e69dfe65342b7667dfb020cee99016f94a0570281307afc79f6487d15004b2e3 not found: ID does not exist" containerID="e69dfe65342b7667dfb020cee99016f94a0570281307afc79f6487d15004b2e3"
Sep 19 06:07:29.610915 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:29.610860    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e69dfe65342b7667dfb020cee99016f94a0570281307afc79f6487d15004b2e3"} err="failed to get container status \"e69dfe65342b7667dfb020cee99016f94a0570281307afc79f6487d15004b2e3\": rpc error: code = NotFound desc = could not find container \"e69dfe65342b7667dfb020cee99016f94a0570281307afc79f6487d15004b2e3\": container with ID starting with e69dfe65342b7667dfb020cee99016f94a0570281307afc79f6487d15004b2e3 not found: ID does not exist"
Sep 19 06:07:29.610915 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:29.610878    2583 scope.go:117] "RemoveContainer" containerID="718372ca6c871216a911c583987ab7a96ce5d8ea38e760b37b4a66ec0d50da8e"
Sep 19 06:07:29.611156 ip-10-0-139-144 kubenswrapper[2583]: E0919 06:07:29.611114    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"718372ca6c871216a911c583987ab7a96ce5d8ea38e760b37b4a66ec0d50da8e\": container with ID starting with 718372ca6c871216a911c583987ab7a96ce5d8ea38e760b37b4a66ec0d50da8e not found: ID does not exist" containerID="718372ca6c871216a911c583987ab7a96ce5d8ea38e760b37b4a66ec0d50da8e"
Sep 19 06:07:29.611229 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:29.611164    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"718372ca6c871216a911c583987ab7a96ce5d8ea38e760b37b4a66ec0d50da8e"} err="failed to get container status \"718372ca6c871216a911c583987ab7a96ce5d8ea38e760b37b4a66ec0d50da8e\": rpc error: code = NotFound desc = could not find container \"718372ca6c871216a911c583987ab7a96ce5d8ea38e760b37b4a66ec0d50da8e\": container with ID starting with 718372ca6c871216a911c583987ab7a96ce5d8ea38e760b37b4a66ec0d50da8e not found: ID does not exist"
Sep 19 06:07:29.615425 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:29.615402    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-3182b5-openshift-default-76dd9fb6bb-b5p7s"]
Sep 19 06:07:29.618364 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:29.618340    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-3182b5-openshift-default-76dd9fb6bb-b5p7s"]
Sep 19 06:07:29.652715 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:29.652195    2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b57edf34-4a41-4cb6-bc1f-f73dae8adad1-istiod-ca-cert\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 06:07:29.652715 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:29.652229    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dn57q\" (UniqueName: \"kubernetes.io/projected/b57edf34-4a41-4cb6-bc1f-f73dae8adad1-kube-api-access-dn57q\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 06:07:29.652715 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:29.652243    2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b57edf34-4a41-4cb6-bc1f-f73dae8adad1-istio-podinfo\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 06:07:29.652715 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:29.652257    2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b57edf34-4a41-4cb6-bc1f-f73dae8adad1-istio-token\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 06:07:29.652715 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:29.652270    2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b57edf34-4a41-4cb6-bc1f-f73dae8adad1-workload-socket\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 06:07:29.652715 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:29.652285    2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b57edf34-4a41-4cb6-bc1f-f73dae8adad1-istio-envoy\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 06:07:31.499221 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:31.499187    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b57edf34-4a41-4cb6-bc1f-f73dae8adad1" path="/var/lib/kubelet/pods/b57edf34-4a41-4cb6-bc1f-f73dae8adad1/volumes"
Sep 19 06:07:38.458270 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:38.458229    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-f1aad1/auth-test-model-f1aad1-kserve-57476cc487-jqqzm" podUID="38f09d4e-1855-4f7b-9165-12a2f05a4fa8" containerName="main" containerID="cri-o://1c2ef0e820f7d6bac619eaf73d4c3f8652780bfe7d10a743432195b83b114de0" gracePeriod=45
Sep 19 06:07:38.652701 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:38.652662    2583 generic.go:356] "Generic (PLEG): container finished" podID="38f09d4e-1855-4f7b-9165-12a2f05a4fa8" containerID="1c2ef0e820f7d6bac619eaf73d4c3f8652780bfe7d10a743432195b83b114de0" exitCode=0
Sep 19 06:07:38.652868 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:38.652730    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-f1aad1/auth-test-model-f1aad1-kserve-57476cc487-jqqzm" event={"ID":"38f09d4e-1855-4f7b-9165-12a2f05a4fa8","Type":"ContainerDied","Data":"1c2ef0e820f7d6bac619eaf73d4c3f8652780bfe7d10a743432195b83b114de0"}
Sep 19 06:07:38.708375 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:38.708319    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-f1aad1/auth-test-model-f1aad1-kserve-57476cc487-jqqzm"
Sep 19 06:07:38.732486 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:38.732456    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f229c\" (UniqueName: \"kubernetes.io/projected/38f09d4e-1855-4f7b-9165-12a2f05a4fa8-kube-api-access-f229c\") pod \"38f09d4e-1855-4f7b-9165-12a2f05a4fa8\" (UID: \"38f09d4e-1855-4f7b-9165-12a2f05a4fa8\") "
Sep 19 06:07:38.732599 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:38.732562    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/38f09d4e-1855-4f7b-9165-12a2f05a4fa8-tmp-dir\") pod \"38f09d4e-1855-4f7b-9165-12a2f05a4fa8\" (UID: \"38f09d4e-1855-4f7b-9165-12a2f05a4fa8\") "
Sep 19 06:07:38.732665 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:38.732615    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/38f09d4e-1855-4f7b-9165-12a2f05a4fa8-model-cache\") pod \"38f09d4e-1855-4f7b-9165-12a2f05a4fa8\" (UID: \"38f09d4e-1855-4f7b-9165-12a2f05a4fa8\") "
Sep 19 06:07:38.732665 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:38.732655    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/38f09d4e-1855-4f7b-9165-12a2f05a4fa8-dshm\") pod \"38f09d4e-1855-4f7b-9165-12a2f05a4fa8\" (UID: \"38f09d4e-1855-4f7b-9165-12a2f05a4fa8\") "
Sep 19 06:07:38.732782 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:38.732691    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/38f09d4e-1855-4f7b-9165-12a2f05a4fa8-home\") pod \"38f09d4e-1855-4f7b-9165-12a2f05a4fa8\" (UID: \"38f09d4e-1855-4f7b-9165-12a2f05a4fa8\") "
Sep 19 06:07:38.732782 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:38.732745    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/38f09d4e-1855-4f7b-9165-12a2f05a4fa8-tls-certs\") pod \"38f09d4e-1855-4f7b-9165-12a2f05a4fa8\" (UID: \"38f09d4e-1855-4f7b-9165-12a2f05a4fa8\") "
Sep 19 06:07:38.732904 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:38.732855    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38f09d4e-1855-4f7b-9165-12a2f05a4fa8-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "38f09d4e-1855-4f7b-9165-12a2f05a4fa8" (UID: "38f09d4e-1855-4f7b-9165-12a2f05a4fa8"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 06:07:38.732963 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:38.732926    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38f09d4e-1855-4f7b-9165-12a2f05a4fa8-model-cache" (OuterVolumeSpecName: "model-cache") pod "38f09d4e-1855-4f7b-9165-12a2f05a4fa8" (UID: "38f09d4e-1855-4f7b-9165-12a2f05a4fa8"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 06:07:38.733112 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:38.733087    2583 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/38f09d4e-1855-4f7b-9165-12a2f05a4fa8-tmp-dir\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 06:07:38.733112 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:38.733111    2583 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/38f09d4e-1855-4f7b-9165-12a2f05a4fa8-model-cache\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 06:07:38.733329 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:38.733274    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38f09d4e-1855-4f7b-9165-12a2f05a4fa8-home" (OuterVolumeSpecName: "home") pod "38f09d4e-1855-4f7b-9165-12a2f05a4fa8" (UID: "38f09d4e-1855-4f7b-9165-12a2f05a4fa8"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 06:07:38.734760 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:38.734734    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/38f09d4e-1855-4f7b-9165-12a2f05a4fa8-kube-api-access-f229c" (OuterVolumeSpecName: "kube-api-access-f229c") pod "38f09d4e-1855-4f7b-9165-12a2f05a4fa8" (UID: "38f09d4e-1855-4f7b-9165-12a2f05a4fa8"). InnerVolumeSpecName "kube-api-access-f229c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 06:07:38.735216 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:38.735200    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38f09d4e-1855-4f7b-9165-12a2f05a4fa8-dshm" (OuterVolumeSpecName: "dshm") pod "38f09d4e-1855-4f7b-9165-12a2f05a4fa8" (UID: "38f09d4e-1855-4f7b-9165-12a2f05a4fa8"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 06:07:38.735297 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:38.735254    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/38f09d4e-1855-4f7b-9165-12a2f05a4fa8-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "38f09d4e-1855-4f7b-9165-12a2f05a4fa8" (UID: "38f09d4e-1855-4f7b-9165-12a2f05a4fa8"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 06:07:38.834391 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:38.834361    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f229c\" (UniqueName: \"kubernetes.io/projected/38f09d4e-1855-4f7b-9165-12a2f05a4fa8-kube-api-access-f229c\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 06:07:38.834391 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:38.834387    2583 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/38f09d4e-1855-4f7b-9165-12a2f05a4fa8-dshm\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 06:07:38.834391 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:38.834396    2583 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/38f09d4e-1855-4f7b-9165-12a2f05a4fa8-home\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 06:07:38.834586 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:38.834404    2583 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/38f09d4e-1855-4f7b-9165-12a2f05a4fa8-tls-certs\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 06:07:39.659623 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:39.659596    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-f1aad1/auth-test-model-f1aad1-kserve-57476cc487-jqqzm"
Sep 19 06:07:39.660022 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:39.659644    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-f1aad1/auth-test-model-f1aad1-kserve-57476cc487-jqqzm" event={"ID":"38f09d4e-1855-4f7b-9165-12a2f05a4fa8","Type":"ContainerDied","Data":"90a00dd3869c69c6be63ccbf08601055f672199e216ac98c8ba23d541b0c96b0"}
Sep 19 06:07:39.660022 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:39.659691    2583 scope.go:117] "RemoveContainer" containerID="1c2ef0e820f7d6bac619eaf73d4c3f8652780bfe7d10a743432195b83b114de0"
Sep 19 06:07:39.675834 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:39.675725    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-f1aad1/auth-test-model-f1aad1-kserve-57476cc487-jqqzm"]
Sep 19 06:07:39.678024 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:39.678003    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-f1aad1/auth-test-model-f1aad1-kserve-57476cc487-jqqzm"]
Sep 19 06:07:41.499203 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:07:41.499172    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="38f09d4e-1855-4f7b-9165-12a2f05a4fa8" path="/var/lib/kubelet/pods/38f09d4e-1855-4f7b-9165-12a2f05a4fa8/volumes"
Sep 19 06:11:12.069409 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:11:12.069375    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-7142e3-56bffb46fb-6ggb7"]
Sep 19 06:11:12.071640 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:11:12.069889    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d637e261-4039-40e0-859f-2a105352401a" containerName="maas-api"
Sep 19 06:11:12.071640 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:11:12.069904    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="d637e261-4039-40e0-859f-2a105352401a" containerName="maas-api"
Sep 19 06:11:12.071640 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:11:12.069921    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b57edf34-4a41-4cb6-bc1f-f73dae8adad1" containerName="istio-proxy"
Sep 19 06:11:12.071640 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:11:12.069930    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="b57edf34-4a41-4cb6-bc1f-f73dae8adad1" containerName="istio-proxy"
Sep 19 06:11:12.071640 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:11:12.069965    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="635f4638-2b98-4ff8-b247-cf98593ef0ca" containerName="maas-api"
Sep 19 06:11:12.071640 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:11:12.069975    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="635f4638-2b98-4ff8-b247-cf98593ef0ca" containerName="maas-api"
Sep 19 06:11:12.071640 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:11:12.069989    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1980603d-cfad-4f61-959c-e9b7507da10f" containerName="maas-api"
Sep 19 06:11:12.071640 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:11:12.069997    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="1980603d-cfad-4f61-959c-e9b7507da10f" containerName="maas-api"
Sep 19 06:11:12.071640 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:11:12.070015    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="484a06e7-0464-448b-86aa-1f166f59dfca" containerName="istio-proxy"
Sep 19 06:11:12.071640 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:11:12.070024    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="484a06e7-0464-448b-86aa-1f166f59dfca" containerName="istio-proxy"
Sep 19 06:11:12.071640 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:11:12.070040    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="38f09d4e-1855-4f7b-9165-12a2f05a4fa8" containerName="main"
Sep 19 06:11:12.071640 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:11:12.070048    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="38f09d4e-1855-4f7b-9165-12a2f05a4fa8" containerName="main"
Sep 19 06:11:12.071640 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:11:12.070061    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="484a06e7-0464-448b-86aa-1f166f59dfca" containerName="istio-proxy"
Sep 19 06:11:12.071640 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:11:12.070069    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="484a06e7-0464-448b-86aa-1f166f59dfca" containerName="istio-proxy"
Sep 19 06:11:12.071640 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:11:12.070079    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="912e198c-c64b-487a-b063-c8eb2b2386a4" containerName="maas-api"
Sep 19 06:11:12.071640 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:11:12.070087    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="912e198c-c64b-487a-b063-c8eb2b2386a4" containerName="maas-api"
Sep 19 06:11:12.071640 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:11:12.070191    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="38f09d4e-1855-4f7b-9165-12a2f05a4fa8" containerName="main"
Sep 19 06:11:12.071640 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:11:12.070203    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="484a06e7-0464-448b-86aa-1f166f59dfca" containerName="istio-proxy"
Sep 19 06:11:12.071640 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:11:12.070212    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="1980603d-cfad-4f61-959c-e9b7507da10f" containerName="maas-api"
Sep 19 06:11:12.071640 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:11:12.070220    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="912e198c-c64b-487a-b063-c8eb2b2386a4" containerName="maas-api"
Sep 19 06:11:12.071640 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:11:12.070230    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="d637e261-4039-40e0-859f-2a105352401a" containerName="maas-api"
Sep 19 06:11:12.071640 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:11:12.070239    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="b57edf34-4a41-4cb6-bc1f-f73dae8adad1" containerName="istio-proxy"
Sep 19 06:11:12.071640 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:11:12.070247    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="635f4638-2b98-4ff8-b247-cf98593ef0ca" containerName="maas-api"
Sep 19 06:11:12.071640 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:11:12.070256    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="b57edf34-4a41-4cb6-bc1f-f73dae8adad1" containerName="istio-proxy"
Sep 19 06:11:12.073400 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:11:12.073382    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7142e3-56bffb46fb-6ggb7"
Sep 19 06:11:12.075859 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:11:12.075832    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-7142e3-serving-cert\""
Sep 19 06:11:12.083197 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:11:12.083169    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-7142e3-56bffb46fb-6ggb7"]
Sep 19 06:11:12.121827 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:11:12.121799    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a24953e3-409f-4d7d-8715-a61bb87351e5-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-7142e3-56bffb46fb-6ggb7\" (UID: \"a24953e3-409f-4d7d-8715-a61bb87351e5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7142e3-56bffb46fb-6ggb7"
Sep 19 06:11:12.122039 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:11:12.122016    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a24953e3-409f-4d7d-8715-a61bb87351e5-metrics-certs\") pod \"maas-api-e2e-ipp-a-7142e3-56bffb46fb-6ggb7\" (UID: \"a24953e3-409f-4d7d-8715-a61bb87351e5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7142e3-56bffb46fb-6ggb7"
Sep 19 06:11:12.122171 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:11:12.122054    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a24953e3-409f-4d7d-8715-a61bb87351e5-maas-api-tls\") pod \"maas-api-e2e-ipp-a-7142e3-56bffb46fb-6ggb7\" (UID: \"a24953e3-409f-4d7d-8715-a61bb87351e5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7142e3-56bffb46fb-6ggb7"
Sep 19 06:11:12.122221 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:11:12.122183    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pk5wh\" (UniqueName: \"kubernetes.io/projected/a24953e3-409f-4d7d-8715-a61bb87351e5-kube-api-access-pk5wh\") pod \"maas-api-e2e-ipp-a-7142e3-56bffb46fb-6ggb7\" (UID: \"a24953e3-409f-4d7d-8715-a61bb87351e5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7142e3-56bffb46fb-6ggb7"
Sep 19 06:11:12.222912 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:11:12.222873    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pk5wh\" (UniqueName: \"kubernetes.io/projected/a24953e3-409f-4d7d-8715-a61bb87351e5-kube-api-access-pk5wh\") pod \"maas-api-e2e-ipp-a-7142e3-56bffb46fb-6ggb7\" (UID: \"a24953e3-409f-4d7d-8715-a61bb87351e5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7142e3-56bffb46fb-6ggb7"
Sep 19 06:11:12.223078 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:11:12.222965    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a24953e3-409f-4d7d-8715-a61bb87351e5-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-7142e3-56bffb46fb-6ggb7\" (UID: \"a24953e3-409f-4d7d-8715-a61bb87351e5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7142e3-56bffb46fb-6ggb7"
Sep 19 06:11:12.223078 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:11:12.222989    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a24953e3-409f-4d7d-8715-a61bb87351e5-metrics-certs\") pod \"maas-api-e2e-ipp-a-7142e3-56bffb46fb-6ggb7\" (UID: \"a24953e3-409f-4d7d-8715-a61bb87351e5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7142e3-56bffb46fb-6ggb7"
Sep 19 06:11:12.223078 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:11:12.223010    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a24953e3-409f-4d7d-8715-a61bb87351e5-maas-api-tls\") pod \"maas-api-e2e-ipp-a-7142e3-56bffb46fb-6ggb7\" (UID: \"a24953e3-409f-4d7d-8715-a61bb87351e5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7142e3-56bffb46fb-6ggb7"
Sep 19 06:11:12.223244 ip-10-0-139-144 kubenswrapper[2583]: E0919 06:11:12.223165    2583 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-a-7142e3-serving-cert: secret "maas-api-e2e-ipp-a-7142e3-serving-cert" not found
Sep 19 06:11:12.223244 ip-10-0-139-144 kubenswrapper[2583]: E0919 06:11:12.223236    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a24953e3-409f-4d7d-8715-a61bb87351e5-maas-api-tls podName:a24953e3-409f-4d7d-8715-a61bb87351e5 nodeName:}" failed. No retries permitted until 2026-09-19 06:11:12.723213892 +0000 UTC m=+1831.381668431 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a24953e3-409f-4d7d-8715-a61bb87351e5-maas-api-tls") pod "maas-api-e2e-ipp-a-7142e3-56bffb46fb-6ggb7" (UID: "a24953e3-409f-4d7d-8715-a61bb87351e5") : secret "maas-api-e2e-ipp-a-7142e3-serving-cert" not found
Sep 19 06:11:12.223618 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:11:12.223597    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a24953e3-409f-4d7d-8715-a61bb87351e5-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-7142e3-56bffb46fb-6ggb7\" (UID: \"a24953e3-409f-4d7d-8715-a61bb87351e5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7142e3-56bffb46fb-6ggb7"
Sep 19 06:11:12.225561 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:11:12.225534    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a24953e3-409f-4d7d-8715-a61bb87351e5-metrics-certs\") pod \"maas-api-e2e-ipp-a-7142e3-56bffb46fb-6ggb7\" (UID: \"a24953e3-409f-4d7d-8715-a61bb87351e5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7142e3-56bffb46fb-6ggb7"
Sep 19 06:11:12.232843 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:11:12.232816    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pk5wh\" (UniqueName: \"kubernetes.io/projected/a24953e3-409f-4d7d-8715-a61bb87351e5-kube-api-access-pk5wh\") pod \"maas-api-e2e-ipp-a-7142e3-56bffb46fb-6ggb7\" (UID: \"a24953e3-409f-4d7d-8715-a61bb87351e5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7142e3-56bffb46fb-6ggb7"
Sep 19 06:11:12.728105 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:11:12.728068    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a24953e3-409f-4d7d-8715-a61bb87351e5-maas-api-tls\") pod \"maas-api-e2e-ipp-a-7142e3-56bffb46fb-6ggb7\" (UID: \"a24953e3-409f-4d7d-8715-a61bb87351e5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7142e3-56bffb46fb-6ggb7"
Sep 19 06:11:12.730651 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:11:12.730620    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a24953e3-409f-4d7d-8715-a61bb87351e5-maas-api-tls\") pod \"maas-api-e2e-ipp-a-7142e3-56bffb46fb-6ggb7\" (UID: \"a24953e3-409f-4d7d-8715-a61bb87351e5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7142e3-56bffb46fb-6ggb7"
Sep 19 06:11:12.986499 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:11:12.986411    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7142e3-56bffb46fb-6ggb7"
Sep 19 06:11:13.261179 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:11:13.261145    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-7142e3-56bffb46fb-6ggb7"]
Sep 19 06:11:13.262315 ip-10-0-139-144 kubenswrapper[2583]: W0919 06:11:13.262288    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda24953e3_409f_4d7d_8715_a61bb87351e5.slice/crio-ff1e138136841e189058f271ab4ffac379e9754c7f9b3f2fb6850b394dfa310e WatchSource:0}: Error finding container ff1e138136841e189058f271ab4ffac379e9754c7f9b3f2fb6850b394dfa310e: Status 404 returned error can't find the container with id ff1e138136841e189058f271ab4ffac379e9754c7f9b3f2fb6850b394dfa310e
Sep 19 06:11:13.263636 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:11:13.263615    2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 19 06:11:13.826581 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:11:13.826548    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7142e3-56bffb46fb-6ggb7" event={"ID":"a24953e3-409f-4d7d-8715-a61bb87351e5","Type":"ContainerStarted","Data":"6ceeca16c0e77c23edf786db880486fa413f5b17f048c4015967aa1f5e1a04bc"}
Sep 19 06:11:13.826747 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:11:13.826603    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7142e3-56bffb46fb-6ggb7" event={"ID":"a24953e3-409f-4d7d-8715-a61bb87351e5","Type":"ContainerStarted","Data":"ff1e138136841e189058f271ab4ffac379e9754c7f9b3f2fb6850b394dfa310e"}
Sep 19 06:11:13.826747 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:11:13.826637    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7142e3-56bffb46fb-6ggb7"
Sep 19 06:11:13.854572 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:11:13.854521    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7142e3-56bffb46fb-6ggb7" podStartSLOduration=1.460618444 podStartE2EDuration="1.854506913s" totalImagesPullingTime="393.888469ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 06:11:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 06:11:13.851633567 +0000 UTC m=+1832.510088124" watchObservedRunningTime="2026-09-19 06:11:13.854506913 +0000 UTC m=+1832.512961470"
Sep 19 06:11:19.837404 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:11:19.837371    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7142e3-56bffb46fb-6ggb7"
Sep 19 06:12:52.091515 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:12:52.091480    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-4eb285-openshift-default-7569df8676-xtnsl"]
Sep 19 06:12:52.092181 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:12:52.091887    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b57edf34-4a41-4cb6-bc1f-f73dae8adad1" containerName="istio-proxy"
Sep 19 06:12:52.092181 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:12:52.091903    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="b57edf34-4a41-4cb6-bc1f-f73dae8adad1" containerName="istio-proxy"
Sep 19 06:12:52.092181 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:12:52.091964    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="484a06e7-0464-448b-86aa-1f166f59dfca" containerName="istio-proxy"
Sep 19 06:12:52.095108 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:12:52.095087    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-4eb285-openshift-default-7569df8676-xtnsl"
Sep 19 06:12:52.097722 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:12:52.097700    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-cleanup-4eb285-openshift-default-dockercfg-v59jc\""
Sep 19 06:12:52.104762 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:12:52.104738    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-4eb285-openshift-default-7569df8676-xtnsl"]
Sep 19 06:12:52.240587 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:12:52.240551    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/edb9545e-bb86-4ecc-b0bf-53fabea174fb-istio-token\") pod \"e2e-ipp-cleanup-4eb285-openshift-default-7569df8676-xtnsl\" (UID: \"edb9545e-bb86-4ecc-b0bf-53fabea174fb\") " pod="openshift-ingress/e2e-ipp-cleanup-4eb285-openshift-default-7569df8676-xtnsl"
Sep 19 06:12:52.240587 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:12:52.240592    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/edb9545e-bb86-4ecc-b0bf-53fabea174fb-istiod-ca-cert\") pod \"e2e-ipp-cleanup-4eb285-openshift-default-7569df8676-xtnsl\" (UID: \"edb9545e-bb86-4ecc-b0bf-53fabea174fb\") " pod="openshift-ingress/e2e-ipp-cleanup-4eb285-openshift-default-7569df8676-xtnsl"
Sep 19 06:12:52.240808 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:12:52.240617    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/edb9545e-bb86-4ecc-b0bf-53fabea174fb-workload-socket\") pod \"e2e-ipp-cleanup-4eb285-openshift-default-7569df8676-xtnsl\" (UID: \"edb9545e-bb86-4ecc-b0bf-53fabea174fb\") " pod="openshift-ingress/e2e-ipp-cleanup-4eb285-openshift-default-7569df8676-xtnsl"
Sep 19 06:12:52.240808 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:12:52.240671    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/edb9545e-bb86-4ecc-b0bf-53fabea174fb-istio-envoy\") pod \"e2e-ipp-cleanup-4eb285-openshift-default-7569df8676-xtnsl\" (UID: \"edb9545e-bb86-4ecc-b0bf-53fabea174fb\") " pod="openshift-ingress/e2e-ipp-cleanup-4eb285-openshift-default-7569df8676-xtnsl"
Sep 19 06:12:52.240808 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:12:52.240704    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/edb9545e-bb86-4ecc-b0bf-53fabea174fb-credential-socket\") pod \"e2e-ipp-cleanup-4eb285-openshift-default-7569df8676-xtnsl\" (UID: \"edb9545e-bb86-4ecc-b0bf-53fabea174fb\") " pod="openshift-ingress/e2e-ipp-cleanup-4eb285-openshift-default-7569df8676-xtnsl"
Sep 19 06:12:52.240808 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:12:52.240742    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/edb9545e-bb86-4ecc-b0bf-53fabea174fb-workload-certs\") pod \"e2e-ipp-cleanup-4eb285-openshift-default-7569df8676-xtnsl\" (UID: \"edb9545e-bb86-4ecc-b0bf-53fabea174fb\") " pod="openshift-ingress/e2e-ipp-cleanup-4eb285-openshift-default-7569df8676-xtnsl"
Sep 19 06:12:52.240808 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:12:52.240768    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hxm2k\" (UniqueName: \"kubernetes.io/projected/edb9545e-bb86-4ecc-b0bf-53fabea174fb-kube-api-access-hxm2k\") pod \"e2e-ipp-cleanup-4eb285-openshift-default-7569df8676-xtnsl\" (UID: \"edb9545e-bb86-4ecc-b0bf-53fabea174fb\") " pod="openshift-ingress/e2e-ipp-cleanup-4eb285-openshift-default-7569df8676-xtnsl"
Sep 19 06:12:52.240808 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:12:52.240799    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/edb9545e-bb86-4ecc-b0bf-53fabea174fb-istio-data\") pod \"e2e-ipp-cleanup-4eb285-openshift-default-7569df8676-xtnsl\" (UID: \"edb9545e-bb86-4ecc-b0bf-53fabea174fb\") " pod="openshift-ingress/e2e-ipp-cleanup-4eb285-openshift-default-7569df8676-xtnsl"
Sep 19 06:12:52.241015 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:12:52.240840    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/edb9545e-bb86-4ecc-b0bf-53fabea174fb-istio-podinfo\") pod \"e2e-ipp-cleanup-4eb285-openshift-default-7569df8676-xtnsl\" (UID: \"edb9545e-bb86-4ecc-b0bf-53fabea174fb\") " pod="openshift-ingress/e2e-ipp-cleanup-4eb285-openshift-default-7569df8676-xtnsl"
Sep 19 06:12:52.341581 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:12:52.341497    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/edb9545e-bb86-4ecc-b0bf-53fabea174fb-workload-socket\") pod \"e2e-ipp-cleanup-4eb285-openshift-default-7569df8676-xtnsl\" (UID: \"edb9545e-bb86-4ecc-b0bf-53fabea174fb\") " pod="openshift-ingress/e2e-ipp-cleanup-4eb285-openshift-default-7569df8676-xtnsl"
Sep 19 06:12:52.341581 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:12:52.341535    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/edb9545e-bb86-4ecc-b0bf-53fabea174fb-istio-envoy\") pod \"e2e-ipp-cleanup-4eb285-openshift-default-7569df8676-xtnsl\" (UID: \"edb9545e-bb86-4ecc-b0bf-53fabea174fb\") " pod="openshift-ingress/e2e-ipp-cleanup-4eb285-openshift-default-7569df8676-xtnsl"
Sep 19 06:12:52.341767 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:12:52.341660    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/edb9545e-bb86-4ecc-b0bf-53fabea174fb-credential-socket\") pod \"e2e-ipp-cleanup-4eb285-openshift-default-7569df8676-xtnsl\" (UID: \"edb9545e-bb86-4ecc-b0bf-53fabea174fb\") " pod="openshift-ingress/e2e-ipp-cleanup-4eb285-openshift-default-7569df8676-xtnsl"
Sep 19 06:12:52.341767 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:12:52.341735    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/edb9545e-bb86-4ecc-b0bf-53fabea174fb-workload-certs\") pod \"e2e-ipp-cleanup-4eb285-openshift-default-7569df8676-xtnsl\" (UID: \"edb9545e-bb86-4ecc-b0bf-53fabea174fb\") " pod="openshift-ingress/e2e-ipp-cleanup-4eb285-openshift-default-7569df8676-xtnsl"
Sep 19 06:12:52.341869 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:12:52.341780    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hxm2k\" (UniqueName: \"kubernetes.io/projected/edb9545e-bb86-4ecc-b0bf-53fabea174fb-kube-api-access-hxm2k\") pod \"e2e-ipp-cleanup-4eb285-openshift-default-7569df8676-xtnsl\" (UID: \"edb9545e-bb86-4ecc-b0bf-53fabea174fb\") " pod="openshift-ingress/e2e-ipp-cleanup-4eb285-openshift-default-7569df8676-xtnsl"
Sep 19 06:12:52.341869 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:12:52.341811    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/edb9545e-bb86-4ecc-b0bf-53fabea174fb-istio-data\") pod \"e2e-ipp-cleanup-4eb285-openshift-default-7569df8676-xtnsl\" (UID: \"edb9545e-bb86-4ecc-b0bf-53fabea174fb\") " pod="openshift-ingress/e2e-ipp-cleanup-4eb285-openshift-default-7569df8676-xtnsl"
Sep 19 06:12:52.341965 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:12:52.341882    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/edb9545e-bb86-4ecc-b0bf-53fabea174fb-istio-podinfo\") pod \"e2e-ipp-cleanup-4eb285-openshift-default-7569df8676-xtnsl\" (UID: \"edb9545e-bb86-4ecc-b0bf-53fabea174fb\") " pod="openshift-ingress/e2e-ipp-cleanup-4eb285-openshift-default-7569df8676-xtnsl"
Sep 19 06:12:52.341965 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:12:52.341900    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/edb9545e-bb86-4ecc-b0bf-53fabea174fb-workload-socket\") pod \"e2e-ipp-cleanup-4eb285-openshift-default-7569df8676-xtnsl\" (UID: \"edb9545e-bb86-4ecc-b0bf-53fabea174fb\") " pod="openshift-ingress/e2e-ipp-cleanup-4eb285-openshift-default-7569df8676-xtnsl"
Sep 19 06:12:52.342066 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:12:52.341991    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/edb9545e-bb86-4ecc-b0bf-53fabea174fb-istio-token\") pod \"e2e-ipp-cleanup-4eb285-openshift-default-7569df8676-xtnsl\" (UID: \"edb9545e-bb86-4ecc-b0bf-53fabea174fb\") " pod="openshift-ingress/e2e-ipp-cleanup-4eb285-openshift-default-7569df8676-xtnsl"
Sep 19 06:12:52.342066 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:12:52.342027    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/edb9545e-bb86-4ecc-b0bf-53fabea174fb-istiod-ca-cert\") pod \"e2e-ipp-cleanup-4eb285-openshift-default-7569df8676-xtnsl\" (UID: \"edb9545e-bb86-4ecc-b0bf-53fabea174fb\") " pod="openshift-ingress/e2e-ipp-cleanup-4eb285-openshift-default-7569df8676-xtnsl"
Sep 19 06:12:52.342202 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:12:52.342078    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/edb9545e-bb86-4ecc-b0bf-53fabea174fb-workload-certs\") pod \"e2e-ipp-cleanup-4eb285-openshift-default-7569df8676-xtnsl\" (UID: \"edb9545e-bb86-4ecc-b0bf-53fabea174fb\") " pod="openshift-ingress/e2e-ipp-cleanup-4eb285-openshift-default-7569df8676-xtnsl"
Sep 19 06:12:52.342202 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:12:52.342189    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/edb9545e-bb86-4ecc-b0bf-53fabea174fb-istio-data\") pod \"e2e-ipp-cleanup-4eb285-openshift-default-7569df8676-xtnsl\" (UID: \"edb9545e-bb86-4ecc-b0bf-53fabea174fb\") " pod="openshift-ingress/e2e-ipp-cleanup-4eb285-openshift-default-7569df8676-xtnsl"
Sep 19 06:12:52.342403 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:12:52.342380    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/edb9545e-bb86-4ecc-b0bf-53fabea174fb-credential-socket\") pod \"e2e-ipp-cleanup-4eb285-openshift-default-7569df8676-xtnsl\" (UID: \"edb9545e-bb86-4ecc-b0bf-53fabea174fb\") " pod="openshift-ingress/e2e-ipp-cleanup-4eb285-openshift-default-7569df8676-xtnsl"
Sep 19 06:12:52.342623 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:12:52.342604    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/edb9545e-bb86-4ecc-b0bf-53fabea174fb-istiod-ca-cert\") pod \"e2e-ipp-cleanup-4eb285-openshift-default-7569df8676-xtnsl\" (UID: \"edb9545e-bb86-4ecc-b0bf-53fabea174fb\") " pod="openshift-ingress/e2e-ipp-cleanup-4eb285-openshift-default-7569df8676-xtnsl"
Sep 19 06:12:52.343903 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:12:52.343885    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/edb9545e-bb86-4ecc-b0bf-53fabea174fb-istio-envoy\") pod \"e2e-ipp-cleanup-4eb285-openshift-default-7569df8676-xtnsl\" (UID: \"edb9545e-bb86-4ecc-b0bf-53fabea174fb\") " pod="openshift-ingress/e2e-ipp-cleanup-4eb285-openshift-default-7569df8676-xtnsl"
Sep 19 06:12:52.344279 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:12:52.344263    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/edb9545e-bb86-4ecc-b0bf-53fabea174fb-istio-podinfo\") pod \"e2e-ipp-cleanup-4eb285-openshift-default-7569df8676-xtnsl\" (UID: \"edb9545e-bb86-4ecc-b0bf-53fabea174fb\") " pod="openshift-ingress/e2e-ipp-cleanup-4eb285-openshift-default-7569df8676-xtnsl"
Sep 19 06:12:52.350381 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:12:52.350348    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hxm2k\" (UniqueName: \"kubernetes.io/projected/edb9545e-bb86-4ecc-b0bf-53fabea174fb-kube-api-access-hxm2k\") pod \"e2e-ipp-cleanup-4eb285-openshift-default-7569df8676-xtnsl\" (UID: \"edb9545e-bb86-4ecc-b0bf-53fabea174fb\") " pod="openshift-ingress/e2e-ipp-cleanup-4eb285-openshift-default-7569df8676-xtnsl"
Sep 19 06:12:52.350741 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:12:52.350713    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/edb9545e-bb86-4ecc-b0bf-53fabea174fb-istio-token\") pod \"e2e-ipp-cleanup-4eb285-openshift-default-7569df8676-xtnsl\" (UID: \"edb9545e-bb86-4ecc-b0bf-53fabea174fb\") " pod="openshift-ingress/e2e-ipp-cleanup-4eb285-openshift-default-7569df8676-xtnsl"
Sep 19 06:12:52.407001 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:12:52.406965    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-4eb285-openshift-default-7569df8676-xtnsl"
Sep 19 06:12:52.682602 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:12:52.682574    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-4eb285-openshift-default-7569df8676-xtnsl"]
Sep 19 06:12:52.685426 ip-10-0-139-144 kubenswrapper[2583]: W0919 06:12:52.685396    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podedb9545e_bb86_4ecc_b0bf_53fabea174fb.slice/crio-dc7cf1675128eff6eed4b9ad541e0be606883a8beab26c5b971daad3689659d5 WatchSource:0}: Error finding container dc7cf1675128eff6eed4b9ad541e0be606883a8beab26c5b971daad3689659d5: Status 404 returned error can't find the container with id dc7cf1675128eff6eed4b9ad541e0be606883a8beab26c5b971daad3689659d5
Sep 19 06:12:52.687800 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:12:52.687768    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 19 06:12:52.687906 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:12:52.687833    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 19 06:12:52.687906 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:12:52.687862    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 19 06:12:53.364347 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:12:53.364305    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-4eb285-openshift-default-7569df8676-xtnsl" event={"ID":"edb9545e-bb86-4ecc-b0bf-53fabea174fb","Type":"ContainerStarted","Data":"51f5e501f3e261e8dd66d7eeab62517c11c6e76032527efbb08542f169050cfe"}
Sep 19 06:12:53.364724 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:12:53.364375    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-4eb285-openshift-default-7569df8676-xtnsl" event={"ID":"edb9545e-bb86-4ecc-b0bf-53fabea174fb","Type":"ContainerStarted","Data":"dc7cf1675128eff6eed4b9ad541e0be606883a8beab26c5b971daad3689659d5"}
Sep 19 06:12:53.383749 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:12:53.383695    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-cleanup-4eb285-openshift-default-7569df8676-xtnsl" podStartSLOduration=1.383682525 podStartE2EDuration="1.383682525s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 06:12:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 06:12:53.382263942 +0000 UTC m=+1932.040718501" watchObservedRunningTime="2026-09-19 06:12:53.383682525 +0000 UTC m=+1932.042137163"
Sep 19 06:12:53.407571 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:12:53.407545    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-4eb285-openshift-default-7569df8676-xtnsl"
Sep 19 06:12:53.412983 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:12:53.412956    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-4eb285-openshift-default-7569df8676-xtnsl"
Sep 19 06:12:54.369847 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:12:54.369814    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-4eb285-openshift-default-7569df8676-xtnsl"
Sep 19 06:12:54.370784 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:12:54.370763    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-4eb285-openshift-default-7569df8676-xtnsl"
Sep 19 06:12:57.995484 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:12:57.995452    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4eb285-78bbd7dd5c-x2vjj"]
Sep 19 06:12:57.999385 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:12:57.999365    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4eb285-78bbd7dd5c-x2vjj"
Sep 19 06:12:58.002655 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:12:58.001984    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-4eb285-serving-cert\""
Sep 19 06:12:58.009686 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:12:58.009657    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4eb285-78bbd7dd5c-x2vjj"]
Sep 19 06:12:58.098181 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:12:58.098118    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c9d24b0-a805-4f4f-8c02-60ac818fa42d-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-4eb285-78bbd7dd5c-x2vjj\" (UID: \"4c9d24b0-a805-4f4f-8c02-60ac818fa42d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4eb285-78bbd7dd5c-x2vjj"
Sep 19 06:12:58.098305 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:12:58.098198    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4c9d24b0-a805-4f4f-8c02-60ac818fa42d-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-4eb285-78bbd7dd5c-x2vjj\" (UID: \"4c9d24b0-a805-4f4f-8c02-60ac818fa42d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4eb285-78bbd7dd5c-x2vjj"
Sep 19 06:12:58.098305 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:12:58.098269    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4c9d24b0-a805-4f4f-8c02-60ac818fa42d-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-4eb285-78bbd7dd5c-x2vjj\" (UID: \"4c9d24b0-a805-4f4f-8c02-60ac818fa42d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4eb285-78bbd7dd5c-x2vjj"
Sep 19 06:12:58.098380 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:12:58.098303    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mb4gd\" (UniqueName: \"kubernetes.io/projected/4c9d24b0-a805-4f4f-8c02-60ac818fa42d-kube-api-access-mb4gd\") pod \"maas-api-e2e-ipp-cleanup-4eb285-78bbd7dd5c-x2vjj\" (UID: \"4c9d24b0-a805-4f4f-8c02-60ac818fa42d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4eb285-78bbd7dd5c-x2vjj"
Sep 19 06:12:58.199302 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:12:58.199267    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4c9d24b0-a805-4f4f-8c02-60ac818fa42d-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-4eb285-78bbd7dd5c-x2vjj\" (UID: \"4c9d24b0-a805-4f4f-8c02-60ac818fa42d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4eb285-78bbd7dd5c-x2vjj"
Sep 19 06:12:58.199460 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:12:58.199307    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mb4gd\" (UniqueName: \"kubernetes.io/projected/4c9d24b0-a805-4f4f-8c02-60ac818fa42d-kube-api-access-mb4gd\") pod \"maas-api-e2e-ipp-cleanup-4eb285-78bbd7dd5c-x2vjj\" (UID: \"4c9d24b0-a805-4f4f-8c02-60ac818fa42d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4eb285-78bbd7dd5c-x2vjj"
Sep 19 06:12:58.199460 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:12:58.199378    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c9d24b0-a805-4f4f-8c02-60ac818fa42d-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-4eb285-78bbd7dd5c-x2vjj\" (UID: \"4c9d24b0-a805-4f4f-8c02-60ac818fa42d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4eb285-78bbd7dd5c-x2vjj"
Sep 19 06:12:58.199460 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:12:58.199397    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4c9d24b0-a805-4f4f-8c02-60ac818fa42d-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-4eb285-78bbd7dd5c-x2vjj\" (UID: \"4c9d24b0-a805-4f4f-8c02-60ac818fa42d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4eb285-78bbd7dd5c-x2vjj"
Sep 19 06:12:58.199709 ip-10-0-139-144 kubenswrapper[2583]: E0919 06:12:58.199497    2583 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4eb285-serving-cert: secret "maas-api-e2e-ipp-cleanup-4eb285-serving-cert" not found
Sep 19 06:12:58.199709 ip-10-0-139-144 kubenswrapper[2583]: E0919 06:12:58.199571    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4c9d24b0-a805-4f4f-8c02-60ac818fa42d-maas-api-tls podName:4c9d24b0-a805-4f4f-8c02-60ac818fa42d nodeName:}" failed. No retries permitted until 2026-09-19 06:12:58.699551181 +0000 UTC m=+1937.358005721 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/4c9d24b0-a805-4f4f-8c02-60ac818fa42d-maas-api-tls") pod "maas-api-e2e-ipp-cleanup-4eb285-78bbd7dd5c-x2vjj" (UID: "4c9d24b0-a805-4f4f-8c02-60ac818fa42d") : secret "maas-api-e2e-ipp-cleanup-4eb285-serving-cert" not found
Sep 19 06:12:58.200189 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:12:58.200161    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c9d24b0-a805-4f4f-8c02-60ac818fa42d-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-4eb285-78bbd7dd5c-x2vjj\" (UID: \"4c9d24b0-a805-4f4f-8c02-60ac818fa42d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4eb285-78bbd7dd5c-x2vjj"
Sep 19 06:12:58.201901 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:12:58.201878    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4c9d24b0-a805-4f4f-8c02-60ac818fa42d-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-4eb285-78bbd7dd5c-x2vjj\" (UID: \"4c9d24b0-a805-4f4f-8c02-60ac818fa42d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4eb285-78bbd7dd5c-x2vjj"
Sep 19 06:12:58.209791 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:12:58.209768    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mb4gd\" (UniqueName: \"kubernetes.io/projected/4c9d24b0-a805-4f4f-8c02-60ac818fa42d-kube-api-access-mb4gd\") pod \"maas-api-e2e-ipp-cleanup-4eb285-78bbd7dd5c-x2vjj\" (UID: \"4c9d24b0-a805-4f4f-8c02-60ac818fa42d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4eb285-78bbd7dd5c-x2vjj"
Sep 19 06:12:58.705506 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:12:58.705466    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4c9d24b0-a805-4f4f-8c02-60ac818fa42d-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-4eb285-78bbd7dd5c-x2vjj\" (UID: \"4c9d24b0-a805-4f4f-8c02-60ac818fa42d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4eb285-78bbd7dd5c-x2vjj"
Sep 19 06:12:58.711509 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:12:58.711472    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4c9d24b0-a805-4f4f-8c02-60ac818fa42d-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-4eb285-78bbd7dd5c-x2vjj\" (UID: \"4c9d24b0-a805-4f4f-8c02-60ac818fa42d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4eb285-78bbd7dd5c-x2vjj"
Sep 19 06:12:58.914820 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:12:58.914787    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4eb285-78bbd7dd5c-x2vjj"
Sep 19 06:12:59.185406 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:12:59.185377    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4eb285-78bbd7dd5c-x2vjj"]
Sep 19 06:12:59.188096 ip-10-0-139-144 kubenswrapper[2583]: W0919 06:12:59.188068    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c9d24b0_a805_4f4f_8c02_60ac818fa42d.slice/crio-5d908db1a7cc2515b7f9a3fa6d41327db8e782e11c28b18769a32ca6cac7efa1 WatchSource:0}: Error finding container 5d908db1a7cc2515b7f9a3fa6d41327db8e782e11c28b18769a32ca6cac7efa1: Status 404 returned error can't find the container with id 5d908db1a7cc2515b7f9a3fa6d41327db8e782e11c28b18769a32ca6cac7efa1
Sep 19 06:12:59.398648 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:12:59.398614    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4eb285-78bbd7dd5c-x2vjj" event={"ID":"4c9d24b0-a805-4f4f-8c02-60ac818fa42d","Type":"ContainerStarted","Data":"5d908db1a7cc2515b7f9a3fa6d41327db8e782e11c28b18769a32ca6cac7efa1"}
Sep 19 06:13:00.406196 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:13:00.406154    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4eb285-78bbd7dd5c-x2vjj" event={"ID":"4c9d24b0-a805-4f4f-8c02-60ac818fa42d","Type":"ContainerStarted","Data":"d41765bbd1c9f21610e9e222d4e453855d2fe54902b97f58637d5e50b196d799"}
Sep 19 06:13:00.422191 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:13:00.422124    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4eb285-78bbd7dd5c-x2vjj" podStartSLOduration=2.868547028 podStartE2EDuration="3.422110601s" totalImagesPullingTime="553.563573ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 06:12:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 06:13:00.420873395 +0000 UTC m=+1939.079327955" watchObservedRunningTime="2026-09-19 06:13:00.422110601 +0000 UTC m=+1939.080565158"
Sep 19 06:13:01.412504 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:13:01.412472    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4eb285-78bbd7dd5c-x2vjj"
Sep 19 06:13:07.423955 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:13:07.423918    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4eb285-78bbd7dd5c-x2vjj"
Sep 19 06:13:16.497819 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:13:16.497791    2583 generic.go:356] "Generic (PLEG): container finished" podID="edb9545e-bb86-4ecc-b0bf-53fabea174fb" containerID="51f5e501f3e261e8dd66d7eeab62517c11c6e76032527efbb08542f169050cfe" exitCode=0
Sep 19 06:13:16.498252 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:13:16.497826    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-4eb285-openshift-default-7569df8676-xtnsl" event={"ID":"edb9545e-bb86-4ecc-b0bf-53fabea174fb","Type":"ContainerDied","Data":"51f5e501f3e261e8dd66d7eeab62517c11c6e76032527efbb08542f169050cfe"}
Sep 19 06:13:16.498252 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:13:16.498124    2583 scope.go:117] "RemoveContainer" containerID="51f5e501f3e261e8dd66d7eeab62517c11c6e76032527efbb08542f169050cfe"
Sep 19 06:13:16.499300 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:13:16.499257    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 19 06:13:16.499360 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:13:16.499345    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 19 06:13:16.499397 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:13:16.499378    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 19 06:13:17.504813 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:13:17.504781    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-4eb285-openshift-default-7569df8676-xtnsl" event={"ID":"edb9545e-bb86-4ecc-b0bf-53fabea174fb","Type":"ContainerStarted","Data":"dcfdafdc44c3e2619b770e687507552ef53f544a0386b1dca8f795015cec8e24"}
Sep 19 06:13:18.407288 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:13:18.407253    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-4eb285-openshift-default-7569df8676-xtnsl"
Sep 19 06:13:18.407457 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:13:18.407385    2583 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-4eb285-openshift-default-7569df8676-xtnsl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body=
Sep 19 06:13:18.407457 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:13:18.407432    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-4eb285-openshift-default-7569df8676-xtnsl" podUID="edb9545e-bb86-4ecc-b0bf-53fabea174fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused"
Sep 19 06:13:19.408040 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:13:19.408002    2583 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-4eb285-openshift-default-7569df8676-xtnsl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body=
Sep 19 06:13:19.408484 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:13:19.408059    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-4eb285-openshift-default-7569df8676-xtnsl" podUID="edb9545e-bb86-4ecc-b0bf-53fabea174fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused"
Sep 19 06:13:20.407464 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:13:20.407425    2583 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-4eb285-openshift-default-7569df8676-xtnsl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body=
Sep 19 06:13:20.407623 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:13:20.407486    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-4eb285-openshift-default-7569df8676-xtnsl" podUID="edb9545e-bb86-4ecc-b0bf-53fabea174fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused"
Sep 19 06:13:21.407365 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:13:21.407325    2583 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-4eb285-openshift-default-7569df8676-xtnsl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body=
Sep 19 06:13:21.407784 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:13:21.407391    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-4eb285-openshift-default-7569df8676-xtnsl" podUID="edb9545e-bb86-4ecc-b0bf-53fabea174fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused"
Sep 19 06:13:22.407331 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:13:22.407293    2583 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-4eb285-openshift-default-7569df8676-xtnsl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body=
Sep 19 06:13:22.407534 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:13:22.407349    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-4eb285-openshift-default-7569df8676-xtnsl" podUID="edb9545e-bb86-4ecc-b0bf-53fabea174fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused"
Sep 19 06:13:23.411869 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:13:23.411839    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-4eb285-openshift-default-7569df8676-xtnsl"
Sep 19 06:13:23.412385 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:13:23.412120    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-4eb285-openshift-default-7569df8676-xtnsl"
Sep 19 06:13:23.412871 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:13:23.412853    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-4eb285-openshift-default-7569df8676-xtnsl"
Sep 19 06:13:27.294337 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:13:27.294245    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4eb285-78bbd7dd5c-x2vjj"]
Sep 19 06:13:27.294784 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:13:27.294490    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4eb285-78bbd7dd5c-x2vjj" podUID="4c9d24b0-a805-4f4f-8c02-60ac818fa42d" containerName="maas-api" containerID="cri-o://d41765bbd1c9f21610e9e222d4e453855d2fe54902b97f58637d5e50b196d799" gracePeriod=30
Sep 19 06:13:27.418956 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:13:27.418918    2583 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4eb285-78bbd7dd5c-x2vjj" podUID="4c9d24b0-a805-4f4f-8c02-60ac818fa42d" containerName="maas-api" probeResult="failure" output="Get \"https://10.132.0.67:8443/health\": dial tcp 10.132.0.67:8443: connect: connection refused"
Sep 19 06:13:27.545907 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:13:27.545846    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4eb285-78bbd7dd5c-x2vjj"
Sep 19 06:13:27.586613 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:13:27.586562    2583 generic.go:356] "Generic (PLEG): container finished" podID="4c9d24b0-a805-4f4f-8c02-60ac818fa42d" containerID="d41765bbd1c9f21610e9e222d4e453855d2fe54902b97f58637d5e50b196d799" exitCode=0
Sep 19 06:13:27.586855 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:13:27.586626    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4eb285-78bbd7dd5c-x2vjj" event={"ID":"4c9d24b0-a805-4f4f-8c02-60ac818fa42d","Type":"ContainerDied","Data":"d41765bbd1c9f21610e9e222d4e453855d2fe54902b97f58637d5e50b196d799"}
Sep 19 06:13:27.586957 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:13:27.586865    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4eb285-78bbd7dd5c-x2vjj"
Sep 19 06:13:27.589578 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:13:27.588907    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4eb285-78bbd7dd5c-x2vjj" event={"ID":"4c9d24b0-a805-4f4f-8c02-60ac818fa42d","Type":"ContainerDied","Data":"5d908db1a7cc2515b7f9a3fa6d41327db8e782e11c28b18769a32ca6cac7efa1"}
Sep 19 06:13:27.589578 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:13:27.588966    2583 scope.go:117] "RemoveContainer" containerID="d41765bbd1c9f21610e9e222d4e453855d2fe54902b97f58637d5e50b196d799"
Sep 19 06:13:27.608611 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:13:27.608587    2583 scope.go:117] "RemoveContainer" containerID="d41765bbd1c9f21610e9e222d4e453855d2fe54902b97f58637d5e50b196d799"
Sep 19 06:13:27.609545 ip-10-0-139-144 kubenswrapper[2583]: E0919 06:13:27.609459    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d41765bbd1c9f21610e9e222d4e453855d2fe54902b97f58637d5e50b196d799\": container with ID starting with d41765bbd1c9f21610e9e222d4e453855d2fe54902b97f58637d5e50b196d799 not found: ID does not exist" containerID="d41765bbd1c9f21610e9e222d4e453855d2fe54902b97f58637d5e50b196d799"
Sep 19 06:13:27.609675 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:13:27.609508    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d41765bbd1c9f21610e9e222d4e453855d2fe54902b97f58637d5e50b196d799"} err="failed to get container status \"d41765bbd1c9f21610e9e222d4e453855d2fe54902b97f58637d5e50b196d799\": rpc error: code = NotFound desc = could not find container \"d41765bbd1c9f21610e9e222d4e453855d2fe54902b97f58637d5e50b196d799\": container with ID starting with d41765bbd1c9f21610e9e222d4e453855d2fe54902b97f58637d5e50b196d799 not found: ID does not exist"
Sep 19 06:13:27.698125 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:13:27.698096    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4c9d24b0-a805-4f4f-8c02-60ac818fa42d-maas-api-tls\") pod \"4c9d24b0-a805-4f4f-8c02-60ac818fa42d\" (UID: \"4c9d24b0-a805-4f4f-8c02-60ac818fa42d\") "
Sep 19 06:13:27.698315 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:13:27.698213    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mb4gd\" (UniqueName: \"kubernetes.io/projected/4c9d24b0-a805-4f4f-8c02-60ac818fa42d-kube-api-access-mb4gd\") pod \"4c9d24b0-a805-4f4f-8c02-60ac818fa42d\" (UID: \"4c9d24b0-a805-4f4f-8c02-60ac818fa42d\") "
Sep 19 06:13:27.698315 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:13:27.698235    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c9d24b0-a805-4f4f-8c02-60ac818fa42d-openshift-service-ca\") pod \"4c9d24b0-a805-4f4f-8c02-60ac818fa42d\" (UID: \"4c9d24b0-a805-4f4f-8c02-60ac818fa42d\") "
Sep 19 06:13:27.698315 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:13:27.698306    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4c9d24b0-a805-4f4f-8c02-60ac818fa42d-metrics-certs\") pod \"4c9d24b0-a805-4f4f-8c02-60ac818fa42d\" (UID: \"4c9d24b0-a805-4f4f-8c02-60ac818fa42d\") "
Sep 19 06:13:27.698631 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:13:27.698601    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4c9d24b0-a805-4f4f-8c02-60ac818fa42d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "4c9d24b0-a805-4f4f-8c02-60ac818fa42d" (UID: "4c9d24b0-a805-4f4f-8c02-60ac818fa42d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 06:13:27.700383 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:13:27.700352    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c9d24b0-a805-4f4f-8c02-60ac818fa42d-kube-api-access-mb4gd" (OuterVolumeSpecName: "kube-api-access-mb4gd") pod "4c9d24b0-a805-4f4f-8c02-60ac818fa42d" (UID: "4c9d24b0-a805-4f4f-8c02-60ac818fa42d"). InnerVolumeSpecName "kube-api-access-mb4gd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 06:13:27.700383 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:13:27.700367    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4c9d24b0-a805-4f4f-8c02-60ac818fa42d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "4c9d24b0-a805-4f4f-8c02-60ac818fa42d" (UID: "4c9d24b0-a805-4f4f-8c02-60ac818fa42d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 06:13:27.700535 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:13:27.700385    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4c9d24b0-a805-4f4f-8c02-60ac818fa42d-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "4c9d24b0-a805-4f4f-8c02-60ac818fa42d" (UID: "4c9d24b0-a805-4f4f-8c02-60ac818fa42d"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 06:13:27.799448 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:13:27.799377    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4c9d24b0-a805-4f4f-8c02-60ac818fa42d-maas-api-tls\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 06:13:27.799448 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:13:27.799403    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mb4gd\" (UniqueName: \"kubernetes.io/projected/4c9d24b0-a805-4f4f-8c02-60ac818fa42d-kube-api-access-mb4gd\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 06:13:27.799448 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:13:27.799413    2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c9d24b0-a805-4f4f-8c02-60ac818fa42d-openshift-service-ca\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 06:13:27.799448 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:13:27.799421    2583 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4c9d24b0-a805-4f4f-8c02-60ac818fa42d-metrics-certs\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 06:13:27.910303 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:13:27.910272    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4eb285-78bbd7dd5c-x2vjj"]
Sep 19 06:13:27.914453 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:13:27.914428    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4eb285-78bbd7dd5c-x2vjj"]
Sep 19 06:13:29.498560 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:13:29.498522    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4c9d24b0-a805-4f4f-8c02-60ac818fa42d" path="/var/lib/kubelet/pods/4c9d24b0-a805-4f4f-8c02-60ac818fa42d/volumes"
Sep 19 06:13:33.637652 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:13:33.637615    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-4eb285-openshift-default-7569df8676-xtnsl"]
Sep 19 06:13:33.638146 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:13:33.637839    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-cleanup-4eb285-openshift-default-7569df8676-xtnsl" podUID="edb9545e-bb86-4ecc-b0bf-53fabea174fb" containerName="istio-proxy" containerID="cri-o://dcfdafdc44c3e2619b770e687507552ef53f544a0386b1dca8f795015cec8e24" gracePeriod=30
Sep 19 06:13:38.414466 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:13:38.414427    2583 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-4eb285-openshift-default-7569df8676-xtnsl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 19 06:13:38.414863 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:13:38.414515    2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-cleanup-4eb285-openshift-default-7569df8676-xtnsl" podUID="edb9545e-bb86-4ecc-b0bf-53fabea174fb" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 19 06:13:38.896349 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:13:38.896323    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-4eb285-openshift-default-7569df8676-xtnsl"
Sep 19 06:13:39.001996 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:13:39.001968    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/edb9545e-bb86-4ecc-b0bf-53fabea174fb-istiod-ca-cert\") pod \"edb9545e-bb86-4ecc-b0bf-53fabea174fb\" (UID: \"edb9545e-bb86-4ecc-b0bf-53fabea174fb\") "
Sep 19 06:13:39.002178 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:13:39.002011    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/edb9545e-bb86-4ecc-b0bf-53fabea174fb-istio-podinfo\") pod \"edb9545e-bb86-4ecc-b0bf-53fabea174fb\" (UID: \"edb9545e-bb86-4ecc-b0bf-53fabea174fb\") "
Sep 19 06:13:39.002178 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:13:39.002042    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/edb9545e-bb86-4ecc-b0bf-53fabea174fb-istio-token\") pod \"edb9545e-bb86-4ecc-b0bf-53fabea174fb\" (UID: \"edb9545e-bb86-4ecc-b0bf-53fabea174fb\") "
Sep 19 06:13:39.002178 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:13:39.002069    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/edb9545e-bb86-4ecc-b0bf-53fabea174fb-credential-socket\") pod \"edb9545e-bb86-4ecc-b0bf-53fabea174fb\" (UID: \"edb9545e-bb86-4ecc-b0bf-53fabea174fb\") "
Sep 19 06:13:39.002178 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:13:39.002097    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/edb9545e-bb86-4ecc-b0bf-53fabea174fb-istio-data\") pod \"edb9545e-bb86-4ecc-b0bf-53fabea174fb\" (UID: \"edb9545e-bb86-4ecc-b0bf-53fabea174fb\") "
Sep 19 06:13:39.002178 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:13:39.002163    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hxm2k\" (UniqueName: \"kubernetes.io/projected/edb9545e-bb86-4ecc-b0bf-53fabea174fb-kube-api-access-hxm2k\") pod \"edb9545e-bb86-4ecc-b0bf-53fabea174fb\" (UID: \"edb9545e-bb86-4ecc-b0bf-53fabea174fb\") "
Sep 19 06:13:39.002455 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:13:39.002231    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/edb9545e-bb86-4ecc-b0bf-53fabea174fb-workload-socket\") pod \"edb9545e-bb86-4ecc-b0bf-53fabea174fb\" (UID: \"edb9545e-bb86-4ecc-b0bf-53fabea174fb\") "
Sep 19 06:13:39.002455 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:13:39.002292    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/edb9545e-bb86-4ecc-b0bf-53fabea174fb-workload-certs\") pod \"edb9545e-bb86-4ecc-b0bf-53fabea174fb\" (UID: \"edb9545e-bb86-4ecc-b0bf-53fabea174fb\") "
Sep 19 06:13:39.002455 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:13:39.002336    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/edb9545e-bb86-4ecc-b0bf-53fabea174fb-istio-envoy\") pod \"edb9545e-bb86-4ecc-b0bf-53fabea174fb\" (UID: \"edb9545e-bb86-4ecc-b0bf-53fabea174fb\") "
Sep 19 06:13:39.002634 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:13:39.002487    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/edb9545e-bb86-4ecc-b0bf-53fabea174fb-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "edb9545e-bb86-4ecc-b0bf-53fabea174fb" (UID: "edb9545e-bb86-4ecc-b0bf-53fabea174fb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 06:13:39.002634 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:13:39.002407    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/edb9545e-bb86-4ecc-b0bf-53fabea174fb-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "edb9545e-bb86-4ecc-b0bf-53fabea174fb" (UID: "edb9545e-bb86-4ecc-b0bf-53fabea174fb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 06:13:39.002634 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:13:39.002495    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/edb9545e-bb86-4ecc-b0bf-53fabea174fb-istio-data" (OuterVolumeSpecName: "istio-data") pod "edb9545e-bb86-4ecc-b0bf-53fabea174fb" (UID: "edb9545e-bb86-4ecc-b0bf-53fabea174fb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 06:13:39.002789 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:13:39.002662    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/edb9545e-bb86-4ecc-b0bf-53fabea174fb-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "edb9545e-bb86-4ecc-b0bf-53fabea174fb" (UID: "edb9545e-bb86-4ecc-b0bf-53fabea174fb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 06:13:39.002789 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:13:39.002719    2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/edb9545e-bb86-4ecc-b0bf-53fabea174fb-istiod-ca-cert\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 06:13:39.002789 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:13:39.002736    2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/edb9545e-bb86-4ecc-b0bf-53fabea174fb-credential-socket\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 06:13:39.002789 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:13:39.002751    2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/edb9545e-bb86-4ecc-b0bf-53fabea174fb-istio-data\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 06:13:39.003014 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:13:39.002919    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/edb9545e-bb86-4ecc-b0bf-53fabea174fb-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "edb9545e-bb86-4ecc-b0bf-53fabea174fb" (UID: "edb9545e-bb86-4ecc-b0bf-53fabea174fb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 06:13:39.004393 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:13:39.004357    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/edb9545e-bb86-4ecc-b0bf-53fabea174fb-kube-api-access-hxm2k" (OuterVolumeSpecName: "kube-api-access-hxm2k") pod "edb9545e-bb86-4ecc-b0bf-53fabea174fb" (UID: "edb9545e-bb86-4ecc-b0bf-53fabea174fb"). InnerVolumeSpecName "kube-api-access-hxm2k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 06:13:39.004393 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:13:39.004368    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/edb9545e-bb86-4ecc-b0bf-53fabea174fb-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "edb9545e-bb86-4ecc-b0bf-53fabea174fb" (UID: "edb9545e-bb86-4ecc-b0bf-53fabea174fb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 19 06:13:39.004541 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:13:39.004432    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/edb9545e-bb86-4ecc-b0bf-53fabea174fb-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "edb9545e-bb86-4ecc-b0bf-53fabea174fb" (UID: "edb9545e-bb86-4ecc-b0bf-53fabea174fb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 06:13:39.004541 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:13:39.004468    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/edb9545e-bb86-4ecc-b0bf-53fabea174fb-istio-token" (OuterVolumeSpecName: "istio-token") pod "edb9545e-bb86-4ecc-b0bf-53fabea174fb" (UID: "edb9545e-bb86-4ecc-b0bf-53fabea174fb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 06:13:39.103374 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:13:39.103334    2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/edb9545e-bb86-4ecc-b0bf-53fabea174fb-istio-token\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 06:13:39.103374 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:13:39.103367    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hxm2k\" (UniqueName: \"kubernetes.io/projected/edb9545e-bb86-4ecc-b0bf-53fabea174fb-kube-api-access-hxm2k\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 06:13:39.103374 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:13:39.103380    2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/edb9545e-bb86-4ecc-b0bf-53fabea174fb-workload-socket\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 06:13:39.103612 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:13:39.103392    2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/edb9545e-bb86-4ecc-b0bf-53fabea174fb-workload-certs\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 06:13:39.103612 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:13:39.103405    2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/edb9545e-bb86-4ecc-b0bf-53fabea174fb-istio-envoy\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 06:13:39.103612 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:13:39.103416    2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/edb9545e-bb86-4ecc-b0bf-53fabea174fb-istio-podinfo\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 06:13:39.657219 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:13:39.657188    2583 generic.go:356] "Generic (PLEG): container finished" podID="edb9545e-bb86-4ecc-b0bf-53fabea174fb" containerID="dcfdafdc44c3e2619b770e687507552ef53f544a0386b1dca8f795015cec8e24" exitCode=0
Sep 19 06:13:39.657645 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:13:39.657231    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-4eb285-openshift-default-7569df8676-xtnsl" event={"ID":"edb9545e-bb86-4ecc-b0bf-53fabea174fb","Type":"ContainerDied","Data":"dcfdafdc44c3e2619b770e687507552ef53f544a0386b1dca8f795015cec8e24"}
Sep 19 06:13:39.657645 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:13:39.657235    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-4eb285-openshift-default-7569df8676-xtnsl"
Sep 19 06:13:39.657645 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:13:39.657277    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-4eb285-openshift-default-7569df8676-xtnsl" event={"ID":"edb9545e-bb86-4ecc-b0bf-53fabea174fb","Type":"ContainerDied","Data":"dc7cf1675128eff6eed4b9ad541e0be606883a8beab26c5b971daad3689659d5"}
Sep 19 06:13:39.657645 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:13:39.657303    2583 scope.go:117] "RemoveContainer" containerID="dcfdafdc44c3e2619b770e687507552ef53f544a0386b1dca8f795015cec8e24"
Sep 19 06:13:39.666634 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:13:39.666616    2583 scope.go:117] "RemoveContainer" containerID="51f5e501f3e261e8dd66d7eeab62517c11c6e76032527efbb08542f169050cfe"
Sep 19 06:13:39.675534 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:13:39.675507    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-4eb285-openshift-default-7569df8676-xtnsl"]
Sep 19 06:13:39.676356 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:13:39.676261    2583 scope.go:117] "RemoveContainer" containerID="dcfdafdc44c3e2619b770e687507552ef53f544a0386b1dca8f795015cec8e24"
Sep 19 06:13:39.676601 ip-10-0-139-144 kubenswrapper[2583]: E0919 06:13:39.676577    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dcfdafdc44c3e2619b770e687507552ef53f544a0386b1dca8f795015cec8e24\": container with ID starting with dcfdafdc44c3e2619b770e687507552ef53f544a0386b1dca8f795015cec8e24 not found: ID does not exist" containerID="dcfdafdc44c3e2619b770e687507552ef53f544a0386b1dca8f795015cec8e24"
Sep 19 06:13:39.676679 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:13:39.676609    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dcfdafdc44c3e2619b770e687507552ef53f544a0386b1dca8f795015cec8e24"} err="failed to get container status \"dcfdafdc44c3e2619b770e687507552ef53f544a0386b1dca8f795015cec8e24\": rpc error: code = NotFound desc = could not find container \"dcfdafdc44c3e2619b770e687507552ef53f544a0386b1dca8f795015cec8e24\": container with ID starting with dcfdafdc44c3e2619b770e687507552ef53f544a0386b1dca8f795015cec8e24 not found: ID does not exist"
Sep 19 06:13:39.676679 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:13:39.676631    2583 scope.go:117] "RemoveContainer" containerID="51f5e501f3e261e8dd66d7eeab62517c11c6e76032527efbb08542f169050cfe"
Sep 19 06:13:39.677100 ip-10-0-139-144 kubenswrapper[2583]: E0919 06:13:39.677000    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"51f5e501f3e261e8dd66d7eeab62517c11c6e76032527efbb08542f169050cfe\": container with ID starting with 51f5e501f3e261e8dd66d7eeab62517c11c6e76032527efbb08542f169050cfe not found: ID does not exist" containerID="51f5e501f3e261e8dd66d7eeab62517c11c6e76032527efbb08542f169050cfe"
Sep 19 06:13:39.677100 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:13:39.677040    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"51f5e501f3e261e8dd66d7eeab62517c11c6e76032527efbb08542f169050cfe"} err="failed to get container status \"51f5e501f3e261e8dd66d7eeab62517c11c6e76032527efbb08542f169050cfe\": rpc error: code = NotFound desc = could not find container \"51f5e501f3e261e8dd66d7eeab62517c11c6e76032527efbb08542f169050cfe\": container with ID starting with 51f5e501f3e261e8dd66d7eeab62517c11c6e76032527efbb08542f169050cfe not found: ID does not exist"
Sep 19 06:13:39.679724 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:13:39.679702    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-4eb285-openshift-default-7569df8676-xtnsl"]
Sep 19 06:13:41.502604 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:13:41.502570    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="edb9545e-bb86-4ecc-b0bf-53fabea174fb" path="/var/lib/kubelet/pods/edb9545e-bb86-4ecc-b0bf-53fabea174fb/volumes"
Sep 19 06:13:52.871761 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:13:52.871724    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-7142e3-56bffb46fb-6ggb7"]
Sep 19 06:13:52.872297 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:13:52.871989    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7142e3-56bffb46fb-6ggb7" podUID="a24953e3-409f-4d7d-8715-a61bb87351e5" containerName="maas-api" containerID="cri-o://6ceeca16c0e77c23edf786db880486fa413f5b17f048c4015967aa1f5e1a04bc" gracePeriod=30
Sep 19 06:13:53.131313 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:13:53.131246    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7142e3-56bffb46fb-6ggb7"
Sep 19 06:13:53.240212 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:13:53.240178    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pk5wh\" (UniqueName: \"kubernetes.io/projected/a24953e3-409f-4d7d-8715-a61bb87351e5-kube-api-access-pk5wh\") pod \"a24953e3-409f-4d7d-8715-a61bb87351e5\" (UID: \"a24953e3-409f-4d7d-8715-a61bb87351e5\") "
Sep 19 06:13:53.240380 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:13:53.240246    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a24953e3-409f-4d7d-8715-a61bb87351e5-maas-api-tls\") pod \"a24953e3-409f-4d7d-8715-a61bb87351e5\" (UID: \"a24953e3-409f-4d7d-8715-a61bb87351e5\") "
Sep 19 06:13:53.240380 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:13:53.240347    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a24953e3-409f-4d7d-8715-a61bb87351e5-openshift-service-ca\") pod \"a24953e3-409f-4d7d-8715-a61bb87351e5\" (UID: \"a24953e3-409f-4d7d-8715-a61bb87351e5\") "
Sep 19 06:13:53.240499 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:13:53.240378    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a24953e3-409f-4d7d-8715-a61bb87351e5-metrics-certs\") pod \"a24953e3-409f-4d7d-8715-a61bb87351e5\" (UID: \"a24953e3-409f-4d7d-8715-a61bb87351e5\") "
Sep 19 06:13:53.240792 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:13:53.240763    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a24953e3-409f-4d7d-8715-a61bb87351e5-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a24953e3-409f-4d7d-8715-a61bb87351e5" (UID: "a24953e3-409f-4d7d-8715-a61bb87351e5"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 06:13:53.242485 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:13:53.242452    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a24953e3-409f-4d7d-8715-a61bb87351e5-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a24953e3-409f-4d7d-8715-a61bb87351e5" (UID: "a24953e3-409f-4d7d-8715-a61bb87351e5"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 06:13:53.242591 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:13:53.242497    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a24953e3-409f-4d7d-8715-a61bb87351e5-kube-api-access-pk5wh" (OuterVolumeSpecName: "kube-api-access-pk5wh") pod "a24953e3-409f-4d7d-8715-a61bb87351e5" (UID: "a24953e3-409f-4d7d-8715-a61bb87351e5"). InnerVolumeSpecName "kube-api-access-pk5wh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 06:13:53.242639 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:13:53.242615    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a24953e3-409f-4d7d-8715-a61bb87351e5-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "a24953e3-409f-4d7d-8715-a61bb87351e5" (UID: "a24953e3-409f-4d7d-8715-a61bb87351e5"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 06:13:53.341531 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:13:53.341489    2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a24953e3-409f-4d7d-8715-a61bb87351e5-openshift-service-ca\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 06:13:53.341531 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:13:53.341530    2583 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a24953e3-409f-4d7d-8715-a61bb87351e5-metrics-certs\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 06:13:53.341531 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:13:53.341539    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pk5wh\" (UniqueName: \"kubernetes.io/projected/a24953e3-409f-4d7d-8715-a61bb87351e5-kube-api-access-pk5wh\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 06:13:53.341775 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:13:53.341549    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a24953e3-409f-4d7d-8715-a61bb87351e5-maas-api-tls\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 06:13:53.744468 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:13:53.744427    2583 generic.go:356] "Generic (PLEG): container finished" podID="a24953e3-409f-4d7d-8715-a61bb87351e5" containerID="6ceeca16c0e77c23edf786db880486fa413f5b17f048c4015967aa1f5e1a04bc" exitCode=0
Sep 19 06:13:53.744747 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:13:53.744483    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7142e3-56bffb46fb-6ggb7"
Sep 19 06:13:53.744747 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:13:53.744515    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7142e3-56bffb46fb-6ggb7" event={"ID":"a24953e3-409f-4d7d-8715-a61bb87351e5","Type":"ContainerDied","Data":"6ceeca16c0e77c23edf786db880486fa413f5b17f048c4015967aa1f5e1a04bc"}
Sep 19 06:13:53.744747 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:13:53.744591    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7142e3-56bffb46fb-6ggb7" event={"ID":"a24953e3-409f-4d7d-8715-a61bb87351e5","Type":"ContainerDied","Data":"ff1e138136841e189058f271ab4ffac379e9754c7f9b3f2fb6850b394dfa310e"}
Sep 19 06:13:53.744747 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:13:53.744612    2583 scope.go:117] "RemoveContainer" containerID="6ceeca16c0e77c23edf786db880486fa413f5b17f048c4015967aa1f5e1a04bc"
Sep 19 06:13:53.754032 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:13:53.753996    2583 scope.go:117] "RemoveContainer" containerID="6ceeca16c0e77c23edf786db880486fa413f5b17f048c4015967aa1f5e1a04bc"
Sep 19 06:13:53.754383 ip-10-0-139-144 kubenswrapper[2583]: E0919 06:13:53.754363    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6ceeca16c0e77c23edf786db880486fa413f5b17f048c4015967aa1f5e1a04bc\": container with ID starting with 6ceeca16c0e77c23edf786db880486fa413f5b17f048c4015967aa1f5e1a04bc not found: ID does not exist" containerID="6ceeca16c0e77c23edf786db880486fa413f5b17f048c4015967aa1f5e1a04bc"
Sep 19 06:13:53.754447 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:13:53.754390    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ceeca16c0e77c23edf786db880486fa413f5b17f048c4015967aa1f5e1a04bc"} err="failed to get container status \"6ceeca16c0e77c23edf786db880486fa413f5b17f048c4015967aa1f5e1a04bc\": rpc error: code = NotFound desc = could not find container \"6ceeca16c0e77c23edf786db880486fa413f5b17f048c4015967aa1f5e1a04bc\": container with ID starting with 6ceeca16c0e77c23edf786db880486fa413f5b17f048c4015967aa1f5e1a04bc not found: ID does not exist"
Sep 19 06:13:53.759994 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:13:53.759931    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-7142e3-56bffb46fb-6ggb7"]
Sep 19 06:13:53.765748 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:13:53.765726    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-7142e3-56bffb46fb-6ggb7"]
Sep 19 06:13:55.498482 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:13:55.498439    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a24953e3-409f-4d7d-8715-a61bb87351e5" path="/var/lib/kubelet/pods/a24953e3-409f-4d7d-8715-a61bb87351e5/volumes"
Sep 19 06:15:09.747923 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:15:09.747832    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-888f75-559c558bcc-nczl4"]
Sep 19 06:15:09.748500 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:15:09.748077    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-888f75-559c558bcc-nczl4" podUID="98d89776-8cfe-4f8f-8441-16fd67eedd1f" containerName="maas-api" containerID="cri-o://2014f118f5411ae5bd5763b06c39e714e4316d7b085157b4d31ecfc57d776be9" gracePeriod=30
Sep 19 06:15:09.991001 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:15:09.990968    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-888f75-559c558bcc-nczl4"
Sep 19 06:15:10.040172 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:15:10.040146    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/98d89776-8cfe-4f8f-8441-16fd67eedd1f-openshift-service-ca\") pod \"98d89776-8cfe-4f8f-8441-16fd67eedd1f\" (UID: \"98d89776-8cfe-4f8f-8441-16fd67eedd1f\") "
Sep 19 06:15:10.040325 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:15:10.040180    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/98d89776-8cfe-4f8f-8441-16fd67eedd1f-metrics-certs\") pod \"98d89776-8cfe-4f8f-8441-16fd67eedd1f\" (UID: \"98d89776-8cfe-4f8f-8441-16fd67eedd1f\") "
Sep 19 06:15:10.040325 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:15:10.040220    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-66bbx\" (UniqueName: \"kubernetes.io/projected/98d89776-8cfe-4f8f-8441-16fd67eedd1f-kube-api-access-66bbx\") pod \"98d89776-8cfe-4f8f-8441-16fd67eedd1f\" (UID: \"98d89776-8cfe-4f8f-8441-16fd67eedd1f\") "
Sep 19 06:15:10.040325 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:15:10.040262    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/98d89776-8cfe-4f8f-8441-16fd67eedd1f-maas-api-tls\") pod \"98d89776-8cfe-4f8f-8441-16fd67eedd1f\" (UID: \"98d89776-8cfe-4f8f-8441-16fd67eedd1f\") "
Sep 19 06:15:10.040520 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:15:10.040463    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/98d89776-8cfe-4f8f-8441-16fd67eedd1f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "98d89776-8cfe-4f8f-8441-16fd67eedd1f" (UID: "98d89776-8cfe-4f8f-8441-16fd67eedd1f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 06:15:10.042367 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:15:10.042335    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/98d89776-8cfe-4f8f-8441-16fd67eedd1f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "98d89776-8cfe-4f8f-8441-16fd67eedd1f" (UID: "98d89776-8cfe-4f8f-8441-16fd67eedd1f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 06:15:10.042477 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:15:10.042378    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/98d89776-8cfe-4f8f-8441-16fd67eedd1f-kube-api-access-66bbx" (OuterVolumeSpecName: "kube-api-access-66bbx") pod "98d89776-8cfe-4f8f-8441-16fd67eedd1f" (UID: "98d89776-8cfe-4f8f-8441-16fd67eedd1f"). InnerVolumeSpecName "kube-api-access-66bbx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 06:15:10.042477 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:15:10.042410    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/98d89776-8cfe-4f8f-8441-16fd67eedd1f-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "98d89776-8cfe-4f8f-8441-16fd67eedd1f" (UID: "98d89776-8cfe-4f8f-8441-16fd67eedd1f"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 06:15:10.141504 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:15:10.141432    2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/98d89776-8cfe-4f8f-8441-16fd67eedd1f-openshift-service-ca\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 06:15:10.141504 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:15:10.141498    2583 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/98d89776-8cfe-4f8f-8441-16fd67eedd1f-metrics-certs\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 06:15:10.141807 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:15:10.141520    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-66bbx\" (UniqueName: \"kubernetes.io/projected/98d89776-8cfe-4f8f-8441-16fd67eedd1f-kube-api-access-66bbx\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 06:15:10.141807 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:15:10.141542    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/98d89776-8cfe-4f8f-8441-16fd67eedd1f-maas-api-tls\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 06:15:10.165727 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:15:10.165686    2583 generic.go:356] "Generic (PLEG): container finished" podID="98d89776-8cfe-4f8f-8441-16fd67eedd1f" containerID="2014f118f5411ae5bd5763b06c39e714e4316d7b085157b4d31ecfc57d776be9" exitCode=0
Sep 19 06:15:10.165879 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:15:10.165741    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-888f75-559c558bcc-nczl4"
Sep 19 06:15:10.165879 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:15:10.165769    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-888f75-559c558bcc-nczl4" event={"ID":"98d89776-8cfe-4f8f-8441-16fd67eedd1f","Type":"ContainerDied","Data":"2014f118f5411ae5bd5763b06c39e714e4316d7b085157b4d31ecfc57d776be9"}
Sep 19 06:15:10.165879 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:15:10.165851    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-888f75-559c558bcc-nczl4" event={"ID":"98d89776-8cfe-4f8f-8441-16fd67eedd1f","Type":"ContainerDied","Data":"ff7eca5dbfc7f68bc407527e34b5dc3f202c7999cf87a3f1fdfaa8deb69e70af"}
Sep 19 06:15:10.165879 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:15:10.165874    2583 scope.go:117] "RemoveContainer" containerID="2014f118f5411ae5bd5763b06c39e714e4316d7b085157b4d31ecfc57d776be9"
Sep 19 06:15:10.177334 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:15:10.177318    2583 scope.go:117] "RemoveContainer" containerID="2014f118f5411ae5bd5763b06c39e714e4316d7b085157b4d31ecfc57d776be9"
Sep 19 06:15:10.177608 ip-10-0-139-144 kubenswrapper[2583]: E0919 06:15:10.177586    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2014f118f5411ae5bd5763b06c39e714e4316d7b085157b4d31ecfc57d776be9\": container with ID starting with 2014f118f5411ae5bd5763b06c39e714e4316d7b085157b4d31ecfc57d776be9 not found: ID does not exist" containerID="2014f118f5411ae5bd5763b06c39e714e4316d7b085157b4d31ecfc57d776be9"
Sep 19 06:15:10.177683 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:15:10.177615    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2014f118f5411ae5bd5763b06c39e714e4316d7b085157b4d31ecfc57d776be9"} err="failed to get container status \"2014f118f5411ae5bd5763b06c39e714e4316d7b085157b4d31ecfc57d776be9\": rpc error: code = NotFound desc = could not find container \"2014f118f5411ae5bd5763b06c39e714e4316d7b085157b4d31ecfc57d776be9\": container with ID starting with 2014f118f5411ae5bd5763b06c39e714e4316d7b085157b4d31ecfc57d776be9 not found: ID does not exist"
Sep 19 06:15:10.188091 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:15:10.188068    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-888f75-559c558bcc-nczl4"]
Sep 19 06:15:10.191687 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:15:10.191664    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-888f75-559c558bcc-nczl4"]
Sep 19 06:15:11.499489 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:15:11.499456    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="98d89776-8cfe-4f8f-8441-16fd67eedd1f" path="/var/lib/kubelet/pods/98d89776-8cfe-4f8f-8441-16fd67eedd1f/volumes"
Sep 19 06:15:16.520751 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:15:16.520716    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-888f75-openshift-default-599bd5457b-8mhn2"]
Sep 19 06:15:16.521340 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:15:16.520910    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w3-888f75-openshift-default-599bd5457b-8mhn2" podUID="21ad9ff3-d47f-477a-ac4a-176a0258ec36" containerName="istio-proxy" containerID="cri-o://e7cbd576b7f0b5d941a661fac184636eee6975fcb375ba2dca4a838ce2ff3b26" gracePeriod=30
Sep 19 06:15:21.781010 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:15:21.780982    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-888f75-openshift-default-599bd5457b-8mhn2"
Sep 19 06:15:21.849521 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:15:21.849440    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/21ad9ff3-d47f-477a-ac4a-176a0258ec36-credential-socket\") pod \"21ad9ff3-d47f-477a-ac4a-176a0258ec36\" (UID: \"21ad9ff3-d47f-477a-ac4a-176a0258ec36\") "
Sep 19 06:15:21.849521 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:15:21.849481    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/21ad9ff3-d47f-477a-ac4a-176a0258ec36-istio-envoy\") pod \"21ad9ff3-d47f-477a-ac4a-176a0258ec36\" (UID: \"21ad9ff3-d47f-477a-ac4a-176a0258ec36\") "
Sep 19 06:15:21.849521 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:15:21.849518    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cxk9p\" (UniqueName: \"kubernetes.io/projected/21ad9ff3-d47f-477a-ac4a-176a0258ec36-kube-api-access-cxk9p\") pod \"21ad9ff3-d47f-477a-ac4a-176a0258ec36\" (UID: \"21ad9ff3-d47f-477a-ac4a-176a0258ec36\") "
Sep 19 06:15:21.849782 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:15:21.849574    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/21ad9ff3-d47f-477a-ac4a-176a0258ec36-istio-data\") pod \"21ad9ff3-d47f-477a-ac4a-176a0258ec36\" (UID: \"21ad9ff3-d47f-477a-ac4a-176a0258ec36\") "
Sep 19 06:15:21.849782 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:15:21.849603    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/21ad9ff3-d47f-477a-ac4a-176a0258ec36-workload-socket\") pod \"21ad9ff3-d47f-477a-ac4a-176a0258ec36\" (UID: \"21ad9ff3-d47f-477a-ac4a-176a0258ec36\") "
Sep 19 06:15:21.849782 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:15:21.849631    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/21ad9ff3-d47f-477a-ac4a-176a0258ec36-istiod-ca-cert\") pod \"21ad9ff3-d47f-477a-ac4a-176a0258ec36\" (UID: \"21ad9ff3-d47f-477a-ac4a-176a0258ec36\") "
Sep 19 06:15:21.849782 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:15:21.849687    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/21ad9ff3-d47f-477a-ac4a-176a0258ec36-istio-token\") pod \"21ad9ff3-d47f-477a-ac4a-176a0258ec36\" (UID: \"21ad9ff3-d47f-477a-ac4a-176a0258ec36\") "
Sep 19 06:15:21.849782 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:15:21.849725    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/21ad9ff3-d47f-477a-ac4a-176a0258ec36-workload-certs\") pod \"21ad9ff3-d47f-477a-ac4a-176a0258ec36\" (UID: \"21ad9ff3-d47f-477a-ac4a-176a0258ec36\") "
Sep 19 06:15:21.849782 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:15:21.849731    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/21ad9ff3-d47f-477a-ac4a-176a0258ec36-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "21ad9ff3-d47f-477a-ac4a-176a0258ec36" (UID: "21ad9ff3-d47f-477a-ac4a-176a0258ec36"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 06:15:21.849782 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:15:21.849754    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/21ad9ff3-d47f-477a-ac4a-176a0258ec36-istio-podinfo\") pod \"21ad9ff3-d47f-477a-ac4a-176a0258ec36\" (UID: \"21ad9ff3-d47f-477a-ac4a-176a0258ec36\") "
Sep 19 06:15:21.850184 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:15:21.849850    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/21ad9ff3-d47f-477a-ac4a-176a0258ec36-istio-data" (OuterVolumeSpecName: "istio-data") pod "21ad9ff3-d47f-477a-ac4a-176a0258ec36" (UID: "21ad9ff3-d47f-477a-ac4a-176a0258ec36"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 06:15:21.850184 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:15:21.849892    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/21ad9ff3-d47f-477a-ac4a-176a0258ec36-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "21ad9ff3-d47f-477a-ac4a-176a0258ec36" (UID: "21ad9ff3-d47f-477a-ac4a-176a0258ec36"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 06:15:21.850184 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:15:21.849995    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/21ad9ff3-d47f-477a-ac4a-176a0258ec36-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "21ad9ff3-d47f-477a-ac4a-176a0258ec36" (UID: "21ad9ff3-d47f-477a-ac4a-176a0258ec36"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 06:15:21.850184 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:15:21.850106    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/21ad9ff3-d47f-477a-ac4a-176a0258ec36-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "21ad9ff3-d47f-477a-ac4a-176a0258ec36" (UID: "21ad9ff3-d47f-477a-ac4a-176a0258ec36"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 06:15:21.850184 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:15:21.850123    2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/21ad9ff3-d47f-477a-ac4a-176a0258ec36-credential-socket\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 06:15:21.850184 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:15:21.850175    2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/21ad9ff3-d47f-477a-ac4a-176a0258ec36-istio-data\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 06:15:21.850184 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:15:21.850189    2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/21ad9ff3-d47f-477a-ac4a-176a0258ec36-workload-socket\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 06:15:21.850496 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:15:21.850202    2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/21ad9ff3-d47f-477a-ac4a-176a0258ec36-istiod-ca-cert\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 06:15:21.851826 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:15:21.851802    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/21ad9ff3-d47f-477a-ac4a-176a0258ec36-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "21ad9ff3-d47f-477a-ac4a-176a0258ec36" (UID: "21ad9ff3-d47f-477a-ac4a-176a0258ec36"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 06:15:21.851826 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:15:21.851819    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/21ad9ff3-d47f-477a-ac4a-176a0258ec36-istio-token" (OuterVolumeSpecName: "istio-token") pod "21ad9ff3-d47f-477a-ac4a-176a0258ec36" (UID: "21ad9ff3-d47f-477a-ac4a-176a0258ec36"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 06:15:21.852117 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:15:21.852103    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/21ad9ff3-d47f-477a-ac4a-176a0258ec36-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "21ad9ff3-d47f-477a-ac4a-176a0258ec36" (UID: "21ad9ff3-d47f-477a-ac4a-176a0258ec36"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 19 06:15:21.852206 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:15:21.852117    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/21ad9ff3-d47f-477a-ac4a-176a0258ec36-kube-api-access-cxk9p" (OuterVolumeSpecName: "kube-api-access-cxk9p") pod "21ad9ff3-d47f-477a-ac4a-176a0258ec36" (UID: "21ad9ff3-d47f-477a-ac4a-176a0258ec36"). InnerVolumeSpecName "kube-api-access-cxk9p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 06:15:21.951607 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:15:21.951563    2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/21ad9ff3-d47f-477a-ac4a-176a0258ec36-istio-token\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 06:15:21.951607 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:15:21.951600    2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/21ad9ff3-d47f-477a-ac4a-176a0258ec36-workload-certs\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 06:15:21.951607 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:15:21.951613    2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/21ad9ff3-d47f-477a-ac4a-176a0258ec36-istio-podinfo\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 06:15:21.951837 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:15:21.951624    2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/21ad9ff3-d47f-477a-ac4a-176a0258ec36-istio-envoy\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 06:15:21.951837 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:15:21.951636    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cxk9p\" (UniqueName: \"kubernetes.io/projected/21ad9ff3-d47f-477a-ac4a-176a0258ec36-kube-api-access-cxk9p\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 06:15:22.236902 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:15:22.236807    2583 generic.go:356] "Generic (PLEG): container finished" podID="21ad9ff3-d47f-477a-ac4a-176a0258ec36" containerID="e7cbd576b7f0b5d941a661fac184636eee6975fcb375ba2dca4a838ce2ff3b26" exitCode=0
Sep 19 06:15:22.236902 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:15:22.236863    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-888f75-openshift-default-599bd5457b-8mhn2"
Sep 19 06:15:22.236902 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:15:22.236873    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-888f75-openshift-default-599bd5457b-8mhn2" event={"ID":"21ad9ff3-d47f-477a-ac4a-176a0258ec36","Type":"ContainerDied","Data":"e7cbd576b7f0b5d941a661fac184636eee6975fcb375ba2dca4a838ce2ff3b26"}
Sep 19 06:15:22.237124 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:15:22.236926    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-888f75-openshift-default-599bd5457b-8mhn2" event={"ID":"21ad9ff3-d47f-477a-ac4a-176a0258ec36","Type":"ContainerDied","Data":"33b06c89ec68e40a922033855fc8f6fe41db2aca2577397e9297255978dbace3"}
Sep 19 06:15:22.237124 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:15:22.236956    2583 scope.go:117] "RemoveContainer" containerID="e7cbd576b7f0b5d941a661fac184636eee6975fcb375ba2dca4a838ce2ff3b26"
Sep 19 06:15:22.246883 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:15:22.246862    2583 scope.go:117] "RemoveContainer" containerID="05f733480345e86473ec333947be6bfa9ec4c4d6beeaf41ab4e4952b7e5e298b"
Sep 19 06:15:22.255989 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:15:22.255963    2583 scope.go:117] "RemoveContainer" containerID="e7cbd576b7f0b5d941a661fac184636eee6975fcb375ba2dca4a838ce2ff3b26"
Sep 19 06:15:22.256384 ip-10-0-139-144 kubenswrapper[2583]: E0919 06:15:22.256353    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e7cbd576b7f0b5d941a661fac184636eee6975fcb375ba2dca4a838ce2ff3b26\": container with ID starting with e7cbd576b7f0b5d941a661fac184636eee6975fcb375ba2dca4a838ce2ff3b26 not found: ID does not exist" containerID="e7cbd576b7f0b5d941a661fac184636eee6975fcb375ba2dca4a838ce2ff3b26"
Sep 19 06:15:22.256534 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:15:22.256396    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e7cbd576b7f0b5d941a661fac184636eee6975fcb375ba2dca4a838ce2ff3b26"} err="failed to get container status \"e7cbd576b7f0b5d941a661fac184636eee6975fcb375ba2dca4a838ce2ff3b26\": rpc error: code = NotFound desc = could not find container \"e7cbd576b7f0b5d941a661fac184636eee6975fcb375ba2dca4a838ce2ff3b26\": container with ID starting with e7cbd576b7f0b5d941a661fac184636eee6975fcb375ba2dca4a838ce2ff3b26 not found: ID does not exist"
Sep 19 06:15:22.256534 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:15:22.256424    2583 scope.go:117] "RemoveContainer" containerID="05f733480345e86473ec333947be6bfa9ec4c4d6beeaf41ab4e4952b7e5e298b"
Sep 19 06:15:22.256839 ip-10-0-139-144 kubenswrapper[2583]: E0919 06:15:22.256790    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"05f733480345e86473ec333947be6bfa9ec4c4d6beeaf41ab4e4952b7e5e298b\": container with ID starting with 05f733480345e86473ec333947be6bfa9ec4c4d6beeaf41ab4e4952b7e5e298b not found: ID does not exist" containerID="05f733480345e86473ec333947be6bfa9ec4c4d6beeaf41ab4e4952b7e5e298b"
Sep 19 06:15:22.256921 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:15:22.256846    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"05f733480345e86473ec333947be6bfa9ec4c4d6beeaf41ab4e4952b7e5e298b"} err="failed to get container status \"05f733480345e86473ec333947be6bfa9ec4c4d6beeaf41ab4e4952b7e5e298b\": rpc error: code = NotFound desc = could not find container \"05f733480345e86473ec333947be6bfa9ec4c4d6beeaf41ab4e4952b7e5e298b\": container with ID starting with 05f733480345e86473ec333947be6bfa9ec4c4d6beeaf41ab4e4952b7e5e298b not found: ID does not exist"
Sep 19 06:15:22.259214 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:15:22.259173    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-888f75-openshift-default-599bd5457b-8mhn2"]
Sep 19 06:15:22.262457 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:15:22.262434    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-888f75-openshift-default-599bd5457b-8mhn2"]
Sep 19 06:15:23.498334 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:15:23.498300    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="21ad9ff3-d47f-477a-ac4a-176a0258ec36" path="/var/lib/kubelet/pods/21ad9ff3-d47f-477a-ac4a-176a0258ec36/volumes"
Sep 19 06:15:34.015191 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:15:34.012508    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-f1aad1-openshift-default-c6fd9dd46-l98nd"]
Sep 19 06:15:34.015191 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:15:34.012875    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w3-f1aad1-openshift-default-c6fd9dd46-l98nd" podUID="726c78fe-d902-4d20-a882-1e0df84d795c" containerName="istio-proxy" containerID="cri-o://799c004db5135b82b56ef8d1d32bac6c520f6ba1aa437248c729f2c02fda599b" gracePeriod=30
Sep 19 06:15:37.187852 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:15:37.187817    2583 patch_prober.go:28] interesting pod/e2e-shared-b-w3-f1aad1-openshift-default-c6fd9dd46-l98nd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 19 06:15:37.188296 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:15:37.187873    2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w3-f1aad1-openshift-default-c6fd9dd46-l98nd" podUID="726c78fe-d902-4d20-a882-1e0df84d795c" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 19 06:15:39.257825 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:15:39.257794    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-f1aad1-openshift-default-c6fd9dd46-l98nd"
Sep 19 06:15:39.330844 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:15:39.330764    2583 generic.go:356] "Generic (PLEG): container finished" podID="726c78fe-d902-4d20-a882-1e0df84d795c" containerID="799c004db5135b82b56ef8d1d32bac6c520f6ba1aa437248c729f2c02fda599b" exitCode=0
Sep 19 06:15:39.330844 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:15:39.330816    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-f1aad1-openshift-default-c6fd9dd46-l98nd"
Sep 19 06:15:39.331024 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:15:39.330848    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-f1aad1-openshift-default-c6fd9dd46-l98nd" event={"ID":"726c78fe-d902-4d20-a882-1e0df84d795c","Type":"ContainerDied","Data":"799c004db5135b82b56ef8d1d32bac6c520f6ba1aa437248c729f2c02fda599b"}
Sep 19 06:15:39.331024 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:15:39.330902    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-f1aad1-openshift-default-c6fd9dd46-l98nd" event={"ID":"726c78fe-d902-4d20-a882-1e0df84d795c","Type":"ContainerDied","Data":"7f2b8edd233b81d1dccd10227fc200550ef47e329d1e97158be88f6426e19a67"}
Sep 19 06:15:39.331024 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:15:39.330924    2583 scope.go:117] "RemoveContainer" containerID="799c004db5135b82b56ef8d1d32bac6c520f6ba1aa437248c729f2c02fda599b"
Sep 19 06:15:39.339951 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:15:39.339931    2583 scope.go:117] "RemoveContainer" containerID="4a613c439eae9f8cf1cdfb3d18bc3ff2c84434b736e10afc07ff72f366776189"
Sep 19 06:15:39.347960 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:15:39.347941    2583 scope.go:117] "RemoveContainer" containerID="799c004db5135b82b56ef8d1d32bac6c520f6ba1aa437248c729f2c02fda599b"
Sep 19 06:15:39.348246 ip-10-0-139-144 kubenswrapper[2583]: E0919 06:15:39.348227    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"799c004db5135b82b56ef8d1d32bac6c520f6ba1aa437248c729f2c02fda599b\": container with ID starting with 799c004db5135b82b56ef8d1d32bac6c520f6ba1aa437248c729f2c02fda599b not found: ID does not exist" containerID="799c004db5135b82b56ef8d1d32bac6c520f6ba1aa437248c729f2c02fda599b"
Sep 19 06:15:39.348297 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:15:39.348257    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"799c004db5135b82b56ef8d1d32bac6c520f6ba1aa437248c729f2c02fda599b"} err="failed to get container status \"799c004db5135b82b56ef8d1d32bac6c520f6ba1aa437248c729f2c02fda599b\": rpc error: code = NotFound desc = could not find container \"799c004db5135b82b56ef8d1d32bac6c520f6ba1aa437248c729f2c02fda599b\": container with ID starting with 799c004db5135b82b56ef8d1d32bac6c520f6ba1aa437248c729f2c02fda599b not found: ID does not exist"
Sep 19 06:15:39.348297 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:15:39.348273    2583 scope.go:117] "RemoveContainer" containerID="4a613c439eae9f8cf1cdfb3d18bc3ff2c84434b736e10afc07ff72f366776189"
Sep 19 06:15:39.348515 ip-10-0-139-144 kubenswrapper[2583]: E0919 06:15:39.348499    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4a613c439eae9f8cf1cdfb3d18bc3ff2c84434b736e10afc07ff72f366776189\": container with ID starting with 4a613c439eae9f8cf1cdfb3d18bc3ff2c84434b736e10afc07ff72f366776189 not found: ID does not exist" containerID="4a613c439eae9f8cf1cdfb3d18bc3ff2c84434b736e10afc07ff72f366776189"
Sep 19 06:15:39.348553 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:15:39.348520    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a613c439eae9f8cf1cdfb3d18bc3ff2c84434b736e10afc07ff72f366776189"} err="failed to get container status \"4a613c439eae9f8cf1cdfb3d18bc3ff2c84434b736e10afc07ff72f366776189\": rpc error: code = NotFound desc = could not find container \"4a613c439eae9f8cf1cdfb3d18bc3ff2c84434b736e10afc07ff72f366776189\": container with ID starting with 4a613c439eae9f8cf1cdfb3d18bc3ff2c84434b736e10afc07ff72f366776189 not found: ID does not exist"
Sep 19 06:15:39.406035 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:15:39.406004    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/726c78fe-d902-4d20-a882-1e0df84d795c-workload-certs\") pod \"726c78fe-d902-4d20-a882-1e0df84d795c\" (UID: \"726c78fe-d902-4d20-a882-1e0df84d795c\") "
Sep 19 06:15:39.406229 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:15:39.406044    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4bmkn\" (UniqueName: \"kubernetes.io/projected/726c78fe-d902-4d20-a882-1e0df84d795c-kube-api-access-4bmkn\") pod \"726c78fe-d902-4d20-a882-1e0df84d795c\" (UID: \"726c78fe-d902-4d20-a882-1e0df84d795c\") "
Sep 19 06:15:39.406229 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:15:39.406072    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/726c78fe-d902-4d20-a882-1e0df84d795c-istio-data\") pod \"726c78fe-d902-4d20-a882-1e0df84d795c\" (UID: \"726c78fe-d902-4d20-a882-1e0df84d795c\") "
Sep 19 06:15:39.406229 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:15:39.406088    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/726c78fe-d902-4d20-a882-1e0df84d795c-istio-envoy\") pod \"726c78fe-d902-4d20-a882-1e0df84d795c\" (UID: \"726c78fe-d902-4d20-a882-1e0df84d795c\") "
Sep 19 06:15:39.406229 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:15:39.406138    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/726c78fe-d902-4d20-a882-1e0df84d795c-istio-podinfo\") pod \"726c78fe-d902-4d20-a882-1e0df84d795c\" (UID: \"726c78fe-d902-4d20-a882-1e0df84d795c\") "
Sep 19 06:15:39.406229 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:15:39.406161    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/726c78fe-d902-4d20-a882-1e0df84d795c-istiod-ca-cert\") pod \"726c78fe-d902-4d20-a882-1e0df84d795c\" (UID: \"726c78fe-d902-4d20-a882-1e0df84d795c\") "
Sep 19 06:15:39.406229 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:15:39.406192    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/726c78fe-d902-4d20-a882-1e0df84d795c-credential-socket\") pod \"726c78fe-d902-4d20-a882-1e0df84d795c\" (UID: \"726c78fe-d902-4d20-a882-1e0df84d795c\") "
Sep 19 06:15:39.406534 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:15:39.406280    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/726c78fe-d902-4d20-a882-1e0df84d795c-istio-token\") pod \"726c78fe-d902-4d20-a882-1e0df84d795c\" (UID: \"726c78fe-d902-4d20-a882-1e0df84d795c\") "
Sep 19 06:15:39.406534 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:15:39.406305    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/726c78fe-d902-4d20-a882-1e0df84d795c-workload-socket\") pod \"726c78fe-d902-4d20-a882-1e0df84d795c\" (UID: \"726c78fe-d902-4d20-a882-1e0df84d795c\") "
Sep 19 06:15:39.406534 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:15:39.406355    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/726c78fe-d902-4d20-a882-1e0df84d795c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "726c78fe-d902-4d20-a882-1e0df84d795c" (UID: "726c78fe-d902-4d20-a882-1e0df84d795c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 06:15:39.406534 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:15:39.406448    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/726c78fe-d902-4d20-a882-1e0df84d795c-istio-data" (OuterVolumeSpecName: "istio-data") pod "726c78fe-d902-4d20-a882-1e0df84d795c" (UID: "726c78fe-d902-4d20-a882-1e0df84d795c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 06:15:39.406736 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:15:39.406603    2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/726c78fe-d902-4d20-a882-1e0df84d795c-workload-certs\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 06:15:39.406736 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:15:39.406622    2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/726c78fe-d902-4d20-a882-1e0df84d795c-istio-data\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 06:15:39.406736 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:15:39.406658    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/726c78fe-d902-4d20-a882-1e0df84d795c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "726c78fe-d902-4d20-a882-1e0df84d795c" (UID: "726c78fe-d902-4d20-a882-1e0df84d795c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 06:15:39.406736 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:15:39.406658    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/726c78fe-d902-4d20-a882-1e0df84d795c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "726c78fe-d902-4d20-a882-1e0df84d795c" (UID: "726c78fe-d902-4d20-a882-1e0df84d795c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 06:15:39.406736 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:15:39.406683    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/726c78fe-d902-4d20-a882-1e0df84d795c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "726c78fe-d902-4d20-a882-1e0df84d795c" (UID: "726c78fe-d902-4d20-a882-1e0df84d795c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 06:15:39.408492 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:15:39.408466    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/726c78fe-d902-4d20-a882-1e0df84d795c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "726c78fe-d902-4d20-a882-1e0df84d795c" (UID: "726c78fe-d902-4d20-a882-1e0df84d795c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 19 06:15:39.408492 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:15:39.408476    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/726c78fe-d902-4d20-a882-1e0df84d795c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "726c78fe-d902-4d20-a882-1e0df84d795c" (UID: "726c78fe-d902-4d20-a882-1e0df84d795c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 06:15:39.408726 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:15:39.408708    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/726c78fe-d902-4d20-a882-1e0df84d795c-istio-token" (OuterVolumeSpecName: "istio-token") pod "726c78fe-d902-4d20-a882-1e0df84d795c" (UID: "726c78fe-d902-4d20-a882-1e0df84d795c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 06:15:39.408787 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:15:39.408726    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/726c78fe-d902-4d20-a882-1e0df84d795c-kube-api-access-4bmkn" (OuterVolumeSpecName: "kube-api-access-4bmkn") pod "726c78fe-d902-4d20-a882-1e0df84d795c" (UID: "726c78fe-d902-4d20-a882-1e0df84d795c"). InnerVolumeSpecName "kube-api-access-4bmkn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 06:15:39.507162 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:15:39.507116    2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/726c78fe-d902-4d20-a882-1e0df84d795c-istio-envoy\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 06:15:39.507162 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:15:39.507163    2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/726c78fe-d902-4d20-a882-1e0df84d795c-istio-podinfo\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 06:15:39.507325 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:15:39.507173    2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/726c78fe-d902-4d20-a882-1e0df84d795c-istiod-ca-cert\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 06:15:39.507325 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:15:39.507181    2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/726c78fe-d902-4d20-a882-1e0df84d795c-credential-socket\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 06:15:39.507325 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:15:39.507189    2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/726c78fe-d902-4d20-a882-1e0df84d795c-istio-token\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 06:15:39.507325 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:15:39.507197    2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/726c78fe-d902-4d20-a882-1e0df84d795c-workload-socket\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 06:15:39.507325 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:15:39.507206    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4bmkn\" (UniqueName: \"kubernetes.io/projected/726c78fe-d902-4d20-a882-1e0df84d795c-kube-api-access-4bmkn\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 06:15:39.646900 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:15:39.646821    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-f1aad1-openshift-default-c6fd9dd46-l98nd"]
Sep 19 06:15:39.650402 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:15:39.650372    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-f1aad1-openshift-default-c6fd9dd46-l98nd"]
Sep 19 06:15:41.498974 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:15:41.498937    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="726c78fe-d902-4d20-a882-1e0df84d795c" path="/var/lib/kubelet/pods/726c78fe-d902-4d20-a882-1e0df84d795c/volumes"
Sep 19 06:21:34.087163 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:21:34.087115    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-68c86dfd56-ddqzd"]
Sep 19 06:21:34.087610 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:21:34.087483    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="726c78fe-d902-4d20-a882-1e0df84d795c" containerName="istio-proxy"
Sep 19 06:21:34.087610 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:21:34.087495    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="726c78fe-d902-4d20-a882-1e0df84d795c" containerName="istio-proxy"
Sep 19 06:21:34.087610 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:21:34.087504    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4c9d24b0-a805-4f4f-8c02-60ac818fa42d" containerName="maas-api"
Sep 19 06:21:34.087610 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:21:34.087510    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c9d24b0-a805-4f4f-8c02-60ac818fa42d" containerName="maas-api"
Sep 19 06:21:34.087610 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:21:34.087518    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="edb9545e-bb86-4ecc-b0bf-53fabea174fb" containerName="istio-proxy"
Sep 19 06:21:34.087610 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:21:34.087523    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="edb9545e-bb86-4ecc-b0bf-53fabea174fb" containerName="istio-proxy"
Sep 19 06:21:34.087610 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:21:34.087530    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="98d89776-8cfe-4f8f-8441-16fd67eedd1f" containerName="maas-api"
Sep 19 06:21:34.087610 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:21:34.087537    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="98d89776-8cfe-4f8f-8441-16fd67eedd1f" containerName="maas-api"
Sep 19 06:21:34.087610 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:21:34.087543    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="edb9545e-bb86-4ecc-b0bf-53fabea174fb" containerName="istio-proxy"
Sep 19 06:21:34.087610 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:21:34.087548    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="edb9545e-bb86-4ecc-b0bf-53fabea174fb" containerName="istio-proxy"
Sep 19 06:21:34.087610 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:21:34.087557    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="21ad9ff3-d47f-477a-ac4a-176a0258ec36" containerName="istio-proxy"
Sep 19 06:21:34.087610 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:21:34.087562    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="21ad9ff3-d47f-477a-ac4a-176a0258ec36" containerName="istio-proxy"
Sep 19 06:21:34.087610 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:21:34.087569    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="726c78fe-d902-4d20-a882-1e0df84d795c" containerName="istio-proxy"
Sep 19 06:21:34.087610 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:21:34.087574    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="726c78fe-d902-4d20-a882-1e0df84d795c" containerName="istio-proxy"
Sep 19 06:21:34.087610 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:21:34.087580    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="21ad9ff3-d47f-477a-ac4a-176a0258ec36" containerName="istio-proxy"
Sep 19 06:21:34.087610 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:21:34.087585    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="21ad9ff3-d47f-477a-ac4a-176a0258ec36" containerName="istio-proxy"
Sep 19 06:21:34.087610 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:21:34.087595    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a24953e3-409f-4d7d-8715-a61bb87351e5" containerName="maas-api"
Sep 19 06:21:34.087610 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:21:34.087600    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="a24953e3-409f-4d7d-8715-a61bb87351e5" containerName="maas-api"
Sep 19 06:21:34.088191 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:21:34.087651    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="edb9545e-bb86-4ecc-b0bf-53fabea174fb" containerName="istio-proxy"
Sep 19 06:21:34.088191 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:21:34.087657    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="21ad9ff3-d47f-477a-ac4a-176a0258ec36" containerName="istio-proxy"
Sep 19 06:21:34.088191 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:21:34.087664    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="726c78fe-d902-4d20-a882-1e0df84d795c" containerName="istio-proxy"
Sep 19 06:21:34.088191 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:21:34.087670    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="a24953e3-409f-4d7d-8715-a61bb87351e5" containerName="maas-api"
Sep 19 06:21:34.088191 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:21:34.087676    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="edb9545e-bb86-4ecc-b0bf-53fabea174fb" containerName="istio-proxy"
Sep 19 06:21:34.088191 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:21:34.087682    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="21ad9ff3-d47f-477a-ac4a-176a0258ec36" containerName="istio-proxy"
Sep 19 06:21:34.088191 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:21:34.087689    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="98d89776-8cfe-4f8f-8441-16fd67eedd1f" containerName="maas-api"
Sep 19 06:21:34.088191 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:21:34.087695    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="4c9d24b0-a805-4f4f-8c02-60ac818fa42d" containerName="maas-api"
Sep 19 06:21:34.090538 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:21:34.090521    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-68c86dfd56-ddqzd"
Sep 19 06:21:34.099763 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:21:34.099733    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-68c86dfd56-ddqzd"]
Sep 19 06:21:34.271396 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:21:34.271353    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9f2ns\" (UniqueName: \"kubernetes.io/projected/53c07e81-326c-4f16-8d27-67201aa91541-kube-api-access-9f2ns\") pod \"maas-api-68c86dfd56-ddqzd\" (UID: \"53c07e81-326c-4f16-8d27-67201aa91541\") " pod="odh-ai-gateway-infra/maas-api-68c86dfd56-ddqzd"
Sep 19 06:21:34.271396 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:21:34.271403    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/53c07e81-326c-4f16-8d27-67201aa91541-maas-api-tls\") pod \"maas-api-68c86dfd56-ddqzd\" (UID: \"53c07e81-326c-4f16-8d27-67201aa91541\") " pod="odh-ai-gateway-infra/maas-api-68c86dfd56-ddqzd"
Sep 19 06:21:34.271683 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:21:34.271426    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/53c07e81-326c-4f16-8d27-67201aa91541-openshift-service-ca\") pod \"maas-api-68c86dfd56-ddqzd\" (UID: \"53c07e81-326c-4f16-8d27-67201aa91541\") " pod="odh-ai-gateway-infra/maas-api-68c86dfd56-ddqzd"
Sep 19 06:21:34.271683 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:21:34.271521    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/53c07e81-326c-4f16-8d27-67201aa91541-metrics-certs\") pod \"maas-api-68c86dfd56-ddqzd\" (UID: \"53c07e81-326c-4f16-8d27-67201aa91541\") " pod="odh-ai-gateway-infra/maas-api-68c86dfd56-ddqzd"
Sep 19 06:21:34.372986 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:21:34.372886    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9f2ns\" (UniqueName: \"kubernetes.io/projected/53c07e81-326c-4f16-8d27-67201aa91541-kube-api-access-9f2ns\") pod \"maas-api-68c86dfd56-ddqzd\" (UID: \"53c07e81-326c-4f16-8d27-67201aa91541\") " pod="odh-ai-gateway-infra/maas-api-68c86dfd56-ddqzd"
Sep 19 06:21:34.372986 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:21:34.372931    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/53c07e81-326c-4f16-8d27-67201aa91541-maas-api-tls\") pod \"maas-api-68c86dfd56-ddqzd\" (UID: \"53c07e81-326c-4f16-8d27-67201aa91541\") " pod="odh-ai-gateway-infra/maas-api-68c86dfd56-ddqzd"
Sep 19 06:21:34.372986 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:21:34.372963    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/53c07e81-326c-4f16-8d27-67201aa91541-openshift-service-ca\") pod \"maas-api-68c86dfd56-ddqzd\" (UID: \"53c07e81-326c-4f16-8d27-67201aa91541\") " pod="odh-ai-gateway-infra/maas-api-68c86dfd56-ddqzd"
Sep 19 06:21:34.373334 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:21:34.373024    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/53c07e81-326c-4f16-8d27-67201aa91541-metrics-certs\") pod \"maas-api-68c86dfd56-ddqzd\" (UID: \"53c07e81-326c-4f16-8d27-67201aa91541\") " pod="odh-ai-gateway-infra/maas-api-68c86dfd56-ddqzd"
Sep 19 06:21:34.373753 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:21:34.373727    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/53c07e81-326c-4f16-8d27-67201aa91541-openshift-service-ca\") pod \"maas-api-68c86dfd56-ddqzd\" (UID: \"53c07e81-326c-4f16-8d27-67201aa91541\") " pod="odh-ai-gateway-infra/maas-api-68c86dfd56-ddqzd"
Sep 19 06:21:34.375812 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:21:34.375785    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/53c07e81-326c-4f16-8d27-67201aa91541-metrics-certs\") pod \"maas-api-68c86dfd56-ddqzd\" (UID: \"53c07e81-326c-4f16-8d27-67201aa91541\") " pod="odh-ai-gateway-infra/maas-api-68c86dfd56-ddqzd"
Sep 19 06:21:34.376181 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:21:34.376157    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/53c07e81-326c-4f16-8d27-67201aa91541-maas-api-tls\") pod \"maas-api-68c86dfd56-ddqzd\" (UID: \"53c07e81-326c-4f16-8d27-67201aa91541\") " pod="odh-ai-gateway-infra/maas-api-68c86dfd56-ddqzd"
Sep 19 06:21:34.380948 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:21:34.380912    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9f2ns\" (UniqueName: \"kubernetes.io/projected/53c07e81-326c-4f16-8d27-67201aa91541-kube-api-access-9f2ns\") pod \"maas-api-68c86dfd56-ddqzd\" (UID: \"53c07e81-326c-4f16-8d27-67201aa91541\") " pod="odh-ai-gateway-infra/maas-api-68c86dfd56-ddqzd"
Sep 19 06:21:34.401716 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:21:34.401697    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-68c86dfd56-ddqzd"
Sep 19 06:21:34.677478 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:21:34.677447    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-68c86dfd56-ddqzd"]
Sep 19 06:21:34.678339 ip-10-0-139-144 kubenswrapper[2583]: W0919 06:21:34.678310    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53c07e81_326c_4f16_8d27_67201aa91541.slice/crio-4745f3502a1f3c57507ef191248c53750f2d2e147ea6570ed842d0161622bb4b WatchSource:0}: Error finding container 4745f3502a1f3c57507ef191248c53750f2d2e147ea6570ed842d0161622bb4b: Status 404 returned error can't find the container with id 4745f3502a1f3c57507ef191248c53750f2d2e147ea6570ed842d0161622bb4b
Sep 19 06:21:34.679637 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:21:34.679617    2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 19 06:21:35.186964 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:21:35.186928    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-68c86dfd56-ddqzd" event={"ID":"53c07e81-326c-4f16-8d27-67201aa91541","Type":"ContainerStarted","Data":"acf6dee937316809a6776e12f90fae238bb0f77091bfd12fbb47ba52a2feb323"}
Sep 19 06:21:35.187367 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:21:35.186981    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-68c86dfd56-ddqzd" event={"ID":"53c07e81-326c-4f16-8d27-67201aa91541","Type":"ContainerStarted","Data":"4745f3502a1f3c57507ef191248c53750f2d2e147ea6570ed842d0161622bb4b"}
Sep 19 06:21:35.187367 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:21:35.187043    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-68c86dfd56-ddqzd"
Sep 19 06:21:35.203385 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:21:35.203342    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-68c86dfd56-ddqzd" podStartSLOduration=0.810331125 podStartE2EDuration="1.203330048s" totalImagesPullingTime="392.998923ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 06:21:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 06:21:35.201762272 +0000 UTC m=+2453.860216830" watchObservedRunningTime="2026-09-19 06:21:35.203330048 +0000 UTC m=+2453.861784605"
Sep 19 06:21:41.197909 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:21:41.197879    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-68c86dfd56-ddqzd"
Sep 19 06:21:48.763720 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:21:48.763684    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-5598d8c9fb-wjwvk"]
Sep 19 06:21:48.764176 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:21:48.763890    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-wjwvk" podUID="bfd333e1-9918-4a1b-aa17-a6f889847c46" containerName="maas-api" containerID="cri-o://66fa0cae40cbda3fa074f5be80cf4cc775f35dbb513136f8805df186b4bb5610" gracePeriod=30
Sep 19 06:21:50.001409 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:21:50.001385    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-wjwvk"
Sep 19 06:21:50.109436 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:21:50.109359    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bfd333e1-9918-4a1b-aa17-a6f889847c46-openshift-service-ca\") pod \"bfd333e1-9918-4a1b-aa17-a6f889847c46\" (UID: \"bfd333e1-9918-4a1b-aa17-a6f889847c46\") "
Sep 19 06:21:50.109436 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:21:50.109419    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7nv6t\" (UniqueName: \"kubernetes.io/projected/bfd333e1-9918-4a1b-aa17-a6f889847c46-kube-api-access-7nv6t\") pod \"bfd333e1-9918-4a1b-aa17-a6f889847c46\" (UID: \"bfd333e1-9918-4a1b-aa17-a6f889847c46\") "
Sep 19 06:21:50.109651 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:21:50.109509    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bfd333e1-9918-4a1b-aa17-a6f889847c46-metrics-certs\") pod \"bfd333e1-9918-4a1b-aa17-a6f889847c46\" (UID: \"bfd333e1-9918-4a1b-aa17-a6f889847c46\") "
Sep 19 06:21:50.109651 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:21:50.109535    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bfd333e1-9918-4a1b-aa17-a6f889847c46-maas-api-tls\") pod \"bfd333e1-9918-4a1b-aa17-a6f889847c46\" (UID: \"bfd333e1-9918-4a1b-aa17-a6f889847c46\") "
Sep 19 06:21:50.109774 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:21:50.109751    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bfd333e1-9918-4a1b-aa17-a6f889847c46-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "bfd333e1-9918-4a1b-aa17-a6f889847c46" (UID: "bfd333e1-9918-4a1b-aa17-a6f889847c46"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 06:21:50.111623 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:21:50.111596    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bfd333e1-9918-4a1b-aa17-a6f889847c46-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "bfd333e1-9918-4a1b-aa17-a6f889847c46" (UID: "bfd333e1-9918-4a1b-aa17-a6f889847c46"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 06:21:50.111741 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:21:50.111620    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bfd333e1-9918-4a1b-aa17-a6f889847c46-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "bfd333e1-9918-4a1b-aa17-a6f889847c46" (UID: "bfd333e1-9918-4a1b-aa17-a6f889847c46"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 06:21:50.111741 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:21:50.111667    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bfd333e1-9918-4a1b-aa17-a6f889847c46-kube-api-access-7nv6t" (OuterVolumeSpecName: "kube-api-access-7nv6t") pod "bfd333e1-9918-4a1b-aa17-a6f889847c46" (UID: "bfd333e1-9918-4a1b-aa17-a6f889847c46"). InnerVolumeSpecName "kube-api-access-7nv6t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 06:21:50.211054 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:21:50.211018    2583 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bfd333e1-9918-4a1b-aa17-a6f889847c46-metrics-certs\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 06:21:50.211054 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:21:50.211048    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bfd333e1-9918-4a1b-aa17-a6f889847c46-maas-api-tls\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 06:21:50.211054 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:21:50.211057    2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bfd333e1-9918-4a1b-aa17-a6f889847c46-openshift-service-ca\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 06:21:50.211303 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:21:50.211066    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7nv6t\" (UniqueName: \"kubernetes.io/projected/bfd333e1-9918-4a1b-aa17-a6f889847c46-kube-api-access-7nv6t\") on node \"ip-10-0-139-144.ec2.internal\" DevicePath \"\""
Sep 19 06:21:50.274712 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:21:50.274676    2583 generic.go:356] "Generic (PLEG): container finished" podID="bfd333e1-9918-4a1b-aa17-a6f889847c46" containerID="66fa0cae40cbda3fa074f5be80cf4cc775f35dbb513136f8805df186b4bb5610" exitCode=0
Sep 19 06:21:50.274858 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:21:50.274718    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-wjwvk"
Sep 19 06:21:50.274858 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:21:50.274760    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-wjwvk" event={"ID":"bfd333e1-9918-4a1b-aa17-a6f889847c46","Type":"ContainerDied","Data":"66fa0cae40cbda3fa074f5be80cf4cc775f35dbb513136f8805df186b4bb5610"}
Sep 19 06:21:50.274858 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:21:50.274810    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-wjwvk" event={"ID":"bfd333e1-9918-4a1b-aa17-a6f889847c46","Type":"ContainerDied","Data":"7f73376b256672680ca407bbaf941dd0fcb82e6215c5f11c7004c68a1d3988ad"}
Sep 19 06:21:50.274858 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:21:50.274834    2583 scope.go:117] "RemoveContainer" containerID="66fa0cae40cbda3fa074f5be80cf4cc775f35dbb513136f8805df186b4bb5610"
Sep 19 06:21:50.284594 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:21:50.284578    2583 scope.go:117] "RemoveContainer" containerID="66fa0cae40cbda3fa074f5be80cf4cc775f35dbb513136f8805df186b4bb5610"
Sep 19 06:21:50.284841 ip-10-0-139-144 kubenswrapper[2583]: E0919 06:21:50.284819    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"66fa0cae40cbda3fa074f5be80cf4cc775f35dbb513136f8805df186b4bb5610\": container with ID starting with 66fa0cae40cbda3fa074f5be80cf4cc775f35dbb513136f8805df186b4bb5610 not found: ID does not exist" containerID="66fa0cae40cbda3fa074f5be80cf4cc775f35dbb513136f8805df186b4bb5610"
Sep 19 06:21:50.284892 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:21:50.284850    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"66fa0cae40cbda3fa074f5be80cf4cc775f35dbb513136f8805df186b4bb5610"} err="failed to get container status \"66fa0cae40cbda3fa074f5be80cf4cc775f35dbb513136f8805df186b4bb5610\": rpc error: code = NotFound desc = could not find container \"66fa0cae40cbda3fa074f5be80cf4cc775f35dbb513136f8805df186b4bb5610\": container with ID starting with 66fa0cae40cbda3fa074f5be80cf4cc775f35dbb513136f8805df186b4bb5610 not found: ID does not exist"
Sep 19 06:21:50.296395 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:21:50.296373    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-5598d8c9fb-wjwvk"]
Sep 19 06:21:50.300230 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:21:50.300211    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-5598d8c9fb-wjwvk"]
Sep 19 06:21:51.498286 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:21:51.498251    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bfd333e1-9918-4a1b-aa17-a6f889847c46" path="/var/lib/kubelet/pods/bfd333e1-9918-4a1b-aa17-a6f889847c46/volumes"
Sep 19 06:22:09.590184 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:22:09.590156    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-pqsfk_043ddfe7-5d27-4ec7-a278-3ba5df899d67/server/0.log"
Sep 19 06:22:09.708773 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:22:09.708743    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-djvnv_f188d5a4-6946-4fb2-88ad-c5c3ec50cfa3/manager/0.log"
Sep 19 06:22:09.823912 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:22:09.823886    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-fd6794c5b-bptml_41b05291-5cbe-4649-9cf3-355202e312d1/copy-manifests/0.log"
Sep 19 06:22:09.829959 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:22:09.829941    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-fd6794c5b-bptml_41b05291-5cbe-4649-9cf3-355202e312d1/manager/0.log"
Sep 19 06:22:10.530664 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:22:10.530634    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-68c86dfd56-ddqzd_53c07e81-326c-4f16-8d27-67201aa91541/maas-api/0.log"
Sep 19 06:22:12.751018 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:22:12.750984    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-8whhg_bc5086e2-363c-404c-9b76-b11cf86d9071/istio-proxy/0.log"
Sep 19 06:22:12.872499 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:22:12.872466    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-cgpct_0a797d1e-2b5c-4511-8823-346eeb6c8596/discovery/0.log"
Sep 19 06:22:13.218606 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:22:13.218578    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-5trhr_f43550bd-3864-4d97-90c6-2f2a283b088a/istio-proxy/1.log"
Sep 19 06:22:14.050003 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:22:14.049974    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-qnggd_873e444d-8837-4170-bbbe-ce6ff40e745d/main/0.log"
Sep 19 06:22:14.509928 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:22:14.509848    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-s6jvb_84cf7fac-b867-4a9b-9d68-d74950714d2a/main/0.log"
Sep 19 06:22:23.090782 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:22:23.090753    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-fpnw2_4bd3176a-beea-424d-b4c2-dfc6f27298d8/global-pull-secret-syncer/0.log"
Sep 19 06:22:23.241114 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:22:23.241082    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-7qkz7_ef2336d0-3175-4773-92fd-bafecb4f21f8/konnectivity-agent/0.log"
Sep 19 06:22:23.322541 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:22:23.322514    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-139-144.ec2.internal_96c6a7a003ab3151e3ef151f14cc508f/haproxy/0.log"
Sep 19 06:22:29.971463 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:22:29.971384    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-847b8bd8d7-9nkln_e3671a49-9883-42c5-b0cf-726774c62cd1/cluster-monitoring-operator/0.log"
Sep 19 06:22:30.225722 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:22:30.225647    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-hvzsv_6c4cd1ae-b085-41f9-bf91-6520dee8b101/node-exporter/0.log"
Sep 19 06:22:30.249853 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:22:30.249827    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-hvzsv_6c4cd1ae-b085-41f9-bf91-6520dee8b101/kube-rbac-proxy/0.log"
Sep 19 06:22:30.273072 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:22:30.273050    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-hvzsv_6c4cd1ae-b085-41f9-bf91-6520dee8b101/init-textfile/0.log"
Sep 19 06:22:32.378177 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:22:32.378141    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6446c8d75d-zpk78_d76dc580-506e-47c0-81b7-9fa0e7db353c/networking-console-plugin/0.log"
Sep 19 06:22:33.570376 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:22:33.570344    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-6594f4c75-dnccb_93fe0f3f-a6cc-4238-9492-fe1488f10bfd/download-server/0.log"
Sep 19 06:22:33.697271 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:22:33.697225    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-5n4fs/perf-node-gather-daemonset-bsbdt"]
Sep 19 06:22:33.697605 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:22:33.697591    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bfd333e1-9918-4a1b-aa17-a6f889847c46" containerName="maas-api"
Sep 19 06:22:33.697650 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:22:33.697607    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="bfd333e1-9918-4a1b-aa17-a6f889847c46" containerName="maas-api"
Sep 19 06:22:33.697690 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:22:33.697661    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="bfd333e1-9918-4a1b-aa17-a6f889847c46" containerName="maas-api"
Sep 19 06:22:33.697690 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:22:33.697668    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="726c78fe-d902-4d20-a882-1e0df84d795c" containerName="istio-proxy"
Sep 19 06:22:33.700799 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:22:33.700776    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5n4fs/perf-node-gather-daemonset-bsbdt"
Sep 19 06:22:33.703038 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:22:33.703016    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-5n4fs\"/\"default-dockercfg-96wgx\""
Sep 19 06:22:33.703168 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:22:33.703042    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-5n4fs\"/\"kube-root-ca.crt\""
Sep 19 06:22:33.704149 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:22:33.704114    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-5n4fs\"/\"openshift-service-ca.crt\""
Sep 19 06:22:33.707275 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:22:33.707250    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5n4fs/perf-node-gather-daemonset-bsbdt"]
Sep 19 06:22:33.787057 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:22:33.787023    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/9f7498a3-154c-4a10-8fda-f77b51f82d15-proc\") pod \"perf-node-gather-daemonset-bsbdt\" (UID: \"9f7498a3-154c-4a10-8fda-f77b51f82d15\") " pod="openshift-must-gather-5n4fs/perf-node-gather-daemonset-bsbdt"
Sep 19 06:22:33.787057 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:22:33.787059    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9f7498a3-154c-4a10-8fda-f77b51f82d15-sys\") pod \"perf-node-gather-daemonset-bsbdt\" (UID: \"9f7498a3-154c-4a10-8fda-f77b51f82d15\") " pod="openshift-must-gather-5n4fs/perf-node-gather-daemonset-bsbdt"
Sep 19 06:22:33.787250 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:22:33.787176    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-74tmn\" (UniqueName: \"kubernetes.io/projected/9f7498a3-154c-4a10-8fda-f77b51f82d15-kube-api-access-74tmn\") pod \"perf-node-gather-daemonset-bsbdt\" (UID: \"9f7498a3-154c-4a10-8fda-f77b51f82d15\") " pod="openshift-must-gather-5n4fs/perf-node-gather-daemonset-bsbdt"
Sep 19 06:22:33.787250 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:22:33.787223    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9f7498a3-154c-4a10-8fda-f77b51f82d15-lib-modules\") pod \"perf-node-gather-daemonset-bsbdt\" (UID: \"9f7498a3-154c-4a10-8fda-f77b51f82d15\") " pod="openshift-must-gather-5n4fs/perf-node-gather-daemonset-bsbdt"
Sep 19 06:22:33.787324 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:22:33.787274    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/9f7498a3-154c-4a10-8fda-f77b51f82d15-podres\") pod \"perf-node-gather-daemonset-bsbdt\" (UID: \"9f7498a3-154c-4a10-8fda-f77b51f82d15\") " pod="openshift-must-gather-5n4fs/perf-node-gather-daemonset-bsbdt"
Sep 19 06:22:33.887749 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:22:33.887674    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-74tmn\" (UniqueName: \"kubernetes.io/projected/9f7498a3-154c-4a10-8fda-f77b51f82d15-kube-api-access-74tmn\") pod \"perf-node-gather-daemonset-bsbdt\" (UID: \"9f7498a3-154c-4a10-8fda-f77b51f82d15\") " pod="openshift-must-gather-5n4fs/perf-node-gather-daemonset-bsbdt"
Sep 19 06:22:33.887749 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:22:33.887710    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9f7498a3-154c-4a10-8fda-f77b51f82d15-lib-modules\") pod \"perf-node-gather-daemonset-bsbdt\" (UID: \"9f7498a3-154c-4a10-8fda-f77b51f82d15\") " pod="openshift-must-gather-5n4fs/perf-node-gather-daemonset-bsbdt"
Sep 19 06:22:33.887749 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:22:33.887731    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/9f7498a3-154c-4a10-8fda-f77b51f82d15-podres\") pod \"perf-node-gather-daemonset-bsbdt\" (UID: \"9f7498a3-154c-4a10-8fda-f77b51f82d15\") " pod="openshift-must-gather-5n4fs/perf-node-gather-daemonset-bsbdt"
Sep 19 06:22:33.888009 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:22:33.887778    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/9f7498a3-154c-4a10-8fda-f77b51f82d15-proc\") pod \"perf-node-gather-daemonset-bsbdt\" (UID: \"9f7498a3-154c-4a10-8fda-f77b51f82d15\") " pod="openshift-must-gather-5n4fs/perf-node-gather-daemonset-bsbdt"
Sep 19 06:22:33.888009 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:22:33.887796    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9f7498a3-154c-4a10-8fda-f77b51f82d15-sys\") pod \"perf-node-gather-daemonset-bsbdt\" (UID: \"9f7498a3-154c-4a10-8fda-f77b51f82d15\") " pod="openshift-must-gather-5n4fs/perf-node-gather-daemonset-bsbdt"
Sep 19 06:22:33.888009 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:22:33.887870    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9f7498a3-154c-4a10-8fda-f77b51f82d15-lib-modules\") pod \"perf-node-gather-daemonset-bsbdt\" (UID: \"9f7498a3-154c-4a10-8fda-f77b51f82d15\") " pod="openshift-must-gather-5n4fs/perf-node-gather-daemonset-bsbdt"
Sep 19 06:22:33.888009 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:22:33.887890    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/9f7498a3-154c-4a10-8fda-f77b51f82d15-proc\") pod \"perf-node-gather-daemonset-bsbdt\" (UID: \"9f7498a3-154c-4a10-8fda-f77b51f82d15\") " pod="openshift-must-gather-5n4fs/perf-node-gather-daemonset-bsbdt"
Sep 19 06:22:33.888009 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:22:33.887880    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9f7498a3-154c-4a10-8fda-f77b51f82d15-sys\") pod \"perf-node-gather-daemonset-bsbdt\" (UID: \"9f7498a3-154c-4a10-8fda-f77b51f82d15\") " pod="openshift-must-gather-5n4fs/perf-node-gather-daemonset-bsbdt"
Sep 19 06:22:33.888009 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:22:33.887929    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/9f7498a3-154c-4a10-8fda-f77b51f82d15-podres\") pod \"perf-node-gather-daemonset-bsbdt\" (UID: \"9f7498a3-154c-4a10-8fda-f77b51f82d15\") " pod="openshift-must-gather-5n4fs/perf-node-gather-daemonset-bsbdt"
Sep 19 06:22:33.894956 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:22:33.894933    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-74tmn\" (UniqueName: \"kubernetes.io/projected/9f7498a3-154c-4a10-8fda-f77b51f82d15-kube-api-access-74tmn\") pod \"perf-node-gather-daemonset-bsbdt\" (UID: \"9f7498a3-154c-4a10-8fda-f77b51f82d15\") " pod="openshift-must-gather-5n4fs/perf-node-gather-daemonset-bsbdt"
Sep 19 06:22:34.011853 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:22:34.011830    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5n4fs/perf-node-gather-daemonset-bsbdt"
Sep 19 06:22:34.281244 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:22:34.281220    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5n4fs/perf-node-gather-daemonset-bsbdt"]
Sep 19 06:22:34.283398 ip-10-0-139-144 kubenswrapper[2583]: W0919 06:22:34.283371    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod9f7498a3_154c_4a10_8fda_f77b51f82d15.slice/crio-ef94cdb46c80b494f1cf63c666b94c07f34bba05848f8bab241a60a0acad9dd2 WatchSource:0}: Error finding container ef94cdb46c80b494f1cf63c666b94c07f34bba05848f8bab241a60a0acad9dd2: Status 404 returned error can't find the container with id ef94cdb46c80b494f1cf63c666b94c07f34bba05848f8bab241a60a0acad9dd2
Sep 19 06:22:34.506412 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:22:34.506373    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5n4fs/perf-node-gather-daemonset-bsbdt" event={"ID":"9f7498a3-154c-4a10-8fda-f77b51f82d15","Type":"ContainerStarted","Data":"4dbdf4103c858c538de0b589156041c5cc7958ace3a3b87075c4d9cddc25bd71"}
Sep 19 06:22:34.506589 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:22:34.506436    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5n4fs/perf-node-gather-daemonset-bsbdt" event={"ID":"9f7498a3-154c-4a10-8fda-f77b51f82d15","Type":"ContainerStarted","Data":"ef94cdb46c80b494f1cf63c666b94c07f34bba05848f8bab241a60a0acad9dd2"}
Sep 19 06:22:34.506589 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:22:34.506475    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-5n4fs/perf-node-gather-daemonset-bsbdt"
Sep 19 06:22:34.522920 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:22:34.522872    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-5n4fs/perf-node-gather-daemonset-bsbdt" podStartSLOduration=1.5228599109999998 podStartE2EDuration="1.522859911s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 06:22:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 06:22:34.520026276 +0000 UTC m=+2513.178480835" watchObservedRunningTime="2026-09-19 06:22:34.522859911 +0000 UTC m=+2513.181314469"
Sep 19 06:22:35.413958 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:22:35.413923    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-656cq_6a315987-3770-4424-8864-36e28a3fc43b/dns/0.log"
Sep 19 06:22:35.433532 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:22:35.433507    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-656cq_6a315987-3770-4424-8864-36e28a3fc43b/kube-rbac-proxy/0.log"
Sep 19 06:22:35.542791 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:22:35.542763    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-9pctd_733a5136-d60a-4843-a361-463ac9830695/dns-node-resolver/0.log"
Sep 19 06:22:36.190609 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:22:36.190576    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-xw29h_220708f7-de57-4213-ae89-7193308011cf/node-ca/0.log"
Sep 19 06:22:37.170710 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:22:37.170679    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-8whhg_bc5086e2-363c-404c-9b76-b11cf86d9071/istio-proxy/0.log"
Sep 19 06:22:37.574948 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:22:37.574916    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-cgpct_0a797d1e-2b5c-4511-8823-346eeb6c8596/discovery/0.log"
Sep 19 06:22:37.641912 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:22:37.641881    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-5trhr_f43550bd-3864-4d97-90c6-2f2a283b088a/istio-proxy/0.log"
Sep 19 06:22:37.644714 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:22:37.644691    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-5trhr_f43550bd-3864-4d97-90c6-2f2a283b088a/istio-proxy/1.log"
Sep 19 06:22:38.719363 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:22:38.719338    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-znm4v_b839569f-a0b2-4046-aee5-5e4ba7b0ef9f/serve-healthcheck-canary/0.log"
Sep 19 06:22:39.152145 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:22:39.152098    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-6f9c644dd9-k9mzm_965fac7e-ec07-48bb-8d5b-202ddd313625/insights-operator/0.log"
Sep 19 06:22:39.152531 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:22:39.152499    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-6f9c644dd9-k9mzm_965fac7e-ec07-48bb-8d5b-202ddd313625/insights-operator/1.log"
Sep 19 06:22:39.171696 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:22:39.171669    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-jgkc7_098cb613-a7e2-46de-a6c6-898a7ccb1a14/kube-rbac-proxy/0.log"
Sep 19 06:22:39.191374 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:22:39.191350    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-jgkc7_098cb613-a7e2-46de-a6c6-898a7ccb1a14/exporter/0.log"
Sep 19 06:22:39.210366 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:22:39.210339    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-jgkc7_098cb613-a7e2-46de-a6c6-898a7ccb1a14/extractor/0.log"
Sep 19 06:22:40.520824 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:22:40.520795    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-5n4fs/perf-node-gather-daemonset-bsbdt"
Sep 19 06:22:42.019397 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:22:42.019367    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-pqsfk_043ddfe7-5d27-4ec7-a278-3ba5df899d67/server/0.log"
Sep 19 06:22:42.105803 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:22:42.105771    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-djvnv_f188d5a4-6946-4fb2-88ad-c5c3ec50cfa3/manager/0.log"
Sep 19 06:22:42.130557 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:22:42.130533    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-fd6794c5b-bptml_41b05291-5cbe-4649-9cf3-355202e312d1/manager/0.log"
Sep 19 06:22:42.153033 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:22:42.153010    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-fd6794c5b-bptml_41b05291-5cbe-4649-9cf3-355202e312d1/copy-manifests/0.log"
Sep 19 06:22:43.753198 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:22:43.753166    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-58cf5d9b8c-jsq77_32b98bc1-5450-4de6-a3a4-0bce0a8e9371/manager/0.log"
Sep 19 06:22:44.198315 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:22:44.198207    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-8whhg_bc5086e2-363c-404c-9b76-b11cf86d9071/istio-proxy/0.log"
Sep 19 06:22:44.603090 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:22:44.603044    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-cgpct_0a797d1e-2b5c-4511-8823-346eeb6c8596/discovery/0.log"
Sep 19 06:22:44.649195 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:22:44.649169    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-5trhr_f43550bd-3864-4d97-90c6-2f2a283b088a/istio-proxy/0.log"
Sep 19 06:22:44.651721 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:22:44.651704    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-5trhr_f43550bd-3864-4d97-90c6-2f2a283b088a/istio-proxy/1.log"
Sep 19 06:22:46.228875 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:22:46.228846    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-8whhg_bc5086e2-363c-404c-9b76-b11cf86d9071/istio-proxy/0.log"
Sep 19 06:22:46.640030 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:22:46.639999    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-cgpct_0a797d1e-2b5c-4511-8823-346eeb6c8596/discovery/0.log"
Sep 19 06:22:46.698542 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:22:46.698515    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-5trhr_f43550bd-3864-4d97-90c6-2f2a283b088a/istio-proxy/0.log"
Sep 19 06:22:46.701667 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:22:46.701644    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-5trhr_f43550bd-3864-4d97-90c6-2f2a283b088a/istio-proxy/1.log"
Sep 19 06:22:49.000712 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:22:49.000676    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-76794cbd6c-gsnhm_63c5a37c-f796-4baf-8c1f-e090bcae95d3/migrator/0.log"
Sep 19 06:22:49.020765 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:22:49.020735    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-76794cbd6c-gsnhm_63c5a37c-f796-4baf-8c1f-e090bcae95d3/graceful-termination/0.log"
Sep 19 06:22:50.337340 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:22:50.337309    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9rczd_95fdae8d-ad6b-451b-bc3d-555fd930f300/kube-multus-additional-cni-plugins/0.log"
Sep 19 06:22:50.356532 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:22:50.356506    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9rczd_95fdae8d-ad6b-451b-bc3d-555fd930f300/egress-router-binary-copy/0.log"
Sep 19 06:22:50.375605 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:22:50.375580    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9rczd_95fdae8d-ad6b-451b-bc3d-555fd930f300/cni-plugins/0.log"
Sep 19 06:22:50.395328 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:22:50.395306    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9rczd_95fdae8d-ad6b-451b-bc3d-555fd930f300/bond-cni-plugin/0.log"
Sep 19 06:22:50.415613 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:22:50.415590    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9rczd_95fdae8d-ad6b-451b-bc3d-555fd930f300/routeoverride-cni/0.log"
Sep 19 06:22:50.434968 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:22:50.434948    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9rczd_95fdae8d-ad6b-451b-bc3d-555fd930f300/whereabouts-cni-bincopy/0.log"
Sep 19 06:22:50.453930 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:22:50.453904    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9rczd_95fdae8d-ad6b-451b-bc3d-555fd930f300/whereabouts-cni/0.log"
Sep 19 06:22:50.833488 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:22:50.833462    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-s2vw4_32511ff0-ad60-4238-af22-53ccabc2c35b/kube-multus/0.log"
Sep 19 06:22:50.935436 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:22:50.935398    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-nrtjk_b291ff5e-1667-4066-98d5-21495cd76311/network-metrics-daemon/0.log"
Sep 19 06:22:50.952540 ip-10-0-139-144 kubenswrapper[2583]: I0919 06:22:50.952518    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-nrtjk_b291ff5e-1667-4066-98d5-21495cd76311/kube-rbac-proxy/0.log"